每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-09-01
日期2026-09-01
已评分—
均分—
最高—

Daily Papers — 2026-09-01

20 papers on audio, speech, music, and acoustics.

1. Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Models

Authors: Kunlin Cai, Kaiyuan Zhang, Zihang Xiang, Jinghuai Zhang, Abeer Alwan et al.

Categories: cs.CR, cs.LG, cs.SD, eess.AS | 18 pages Score: 7.70/10 (Obj:9 Id:7 Ind:7 Comp:8 Eff:9 Nov:8 Rep:5)

  • Strength: 首个面向生成式 TTS 的黑盒 MIA 框架，双粒度审计暴露严重隐私泄漏：说话人级 AUC 全面 >0.80 且最强设置接近 1.0（3 条攻击者录音即达 0.96），记录级 AUC 0.80–0.90 且在成员/非成员同为同一说话人的困难设置下仍有效。
  • Weakness: 无代码仓库且无发布承诺（GitHub 检索零命中），影子模型超参与 LSTM 细节披露不全致全部结果无法第三方复跑，查询评估准则为启发式、易感特征为单模型相关性观察、公开英语朗读数据上的结论对真实私有微调场景外推性未测。
  • Full review: Claude Code 全文七公理审稿

Text-to-Speech (TTS) foundation models are increasingly fine-tuned on private datasets to synthesize highly personalized voices, introducing severe privacy risks by exposing both biometric identities and sensitive speech content. Existing black-box membership inference attacks (MIAs) follow a two-stage pipeline of query generation and representation engineering, both of which face unique challenges when adapted to TTS. For query generation, dual conditioning on synthesis text and reference speech creates a large and underexplored query design space with no established criterion for identifying an effective query. For representation engineering, the multi-level speech characteristics and temporal variability of speech make low-level representations and direct comparisons inadequate for capturing membership signals. To address these challenges, we present the first black-box MIA framework explicitly tailored to TTS models at both the speaker and record levels. For query generation, we characterize the feasible query space and establish two criteria, scorable extent and memorization elicitation, for evaluating five representative queries, identifying recitation as the strongest. For representation engineering, we obtain multi-level speech representations from embedding models and temporally align the generated and target audio for fine-grained comparison. Evaluations across three state-of-the-art TTS models (CosyVoice2, F5-TTS, and XTTS-v2) fine-tuned on two benchmark datasets (VCTK and British Dialect) reveal severe privacy leakage: speaker-level AUC remains above 0.80 and approaches 1.0 in the strongest settings, while record-level AUC ranges from 0.80 to 0.90 and remains effective even in challenging scenarios where both members and non-members are of the same speakers. We further identify speech characteristics associated with disproportionate vulnerability to memorization.

2. TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data

Authors: Jianhuai Hu, Yashan Wang, Shangda Wu, Zhancheng Guo, Shijie Liang et al.

Categories: cs.SD, cs.AI Score: 7.16/10 (Obj:8 Id:6 Ind:8 Comp:6 Eff:8 Nov:7 Rep:7)

  • Strength: 363,610 对全合成多乐器音频-乐谱预训练使统一纯注意力 A2S 模型在三个域刷新 SOTA——ASAP 钢琴 76.1（超 Zeng et al. 74.2，多音高 +7.4）、四重奏 95.6（超基线 84.9），并零样本迁移到完全未见的萨克斯（85.3/84.9 vs 基线 60.3/59.9，+25.0）；代码与 TuttiCorpus 数据集已实际公开（GitHub + HF/ModelScope）。
  • Weakness: 基线数字疑为引用而非同条件重跑，缺数据规模缩放与符号生成模型敏感度消融，多乐器增益无机制级分析，转写限于约 14.8 秒片段且神经 EPR 仅覆盖钢琴，仓库缺预训练权重、license 与数据生成管线（config 的 61M 命名与论文 88.9M 参数不一致）。
  • Full review: Claude Code 全文七公理审稿

Generalizable Audio-to-Score (A2S) transcription is fundamentally constrained by the severe scarcity of high-quality, real-world paired data. Relying solely on existing human-annotated datasets often restricts the generalization of A2S models, limiting their efficacy primarily to single-instrumentation domains. To break this dependency on scarce real-world data, we introduce TUTTI (Transformer for Unified audio-To-score Transcription trained on Synthetic multi-Instrumentation Data), a pre-training paradigm driven by a purely synthetic, large-scale dataset. Rather than using human-composed scores, we leverage a symbolic music generation model to generate a massive, highly scalable multi-instrumentation corpus and create audio-score pairs with expressive acoustic characteristics. Capitalizing on the generated data, we employ a standard Transformer encoder-decoder architecture. We empirically demonstrate that pre-training a unified attention-based model on generated, multi-instrumentation data yields a consistently stronger foundational representation than single-instrumentation training. When fine-tuned with downstream real-world datasets, TUTTI outperforms previous approaches, establishing new overall state-of-the-art results across various A2S baselines. Notably, TUTTI shows remarkable cross-instrument transferability, effectively adapting to unseen instruments with highly competitive performance. The source code and the TuttiCorpus dataset will be made publicly available at https://github.com/a-musiclover/TUTTI.

3. From Visual Cues to Spoken Narration: Rethinking Audio Description

Authors: Akshita Gupta, Aditya Arora, Federico Tombari, Marcus Rohrbach, Anna Rohrbach

Categories: cs.CV | Accepted to EMNLP main conference Score: 7.10/10 (Obj:8 Id:8 Ind:6 Comp:8 Eff:6 Nov:8 Rep:5)

  • Strength: 首个在平均 6.5 分钟未剪辑片段上联合预测 AD 视觉窗口与口述窗口的工作，LongLSMDC（83.8K 标注）填补基准空白，定位超基线 5–12 mAP，生成 CIDEr 从 36.0 提至 37.3（三种子均值，增益约为噪声 4 倍），MAD-Eval 零样本 F1 达 90.0。
  • Weakness: 关键缺失证据——代码与数据仓库截至审稿仅一行 README（承诺未兑现），CIDEr 主指标与 DRL 训练目标同向偏置自身风格（Gemini 仅 17.3），端到端插入精确率仅 27.5% 且无盲/低视力用户研究。
  • Full review: Claude Code 全文七公理审稿

Audio Description (AD) provides spoken narration of visual events during dialogue gaps, making movies accessible to visually impaired audiences. The problem requires determining both what (which visual event) and when (position for inserting the AD) to narrate, to achieve the best user experience. Prior work has largely reduced the problem to video captioning of pre-segmented video clips, i.e., what is largely predefined and when is ignored entirely. We propose Cue2Narrate, a two-stage pipeline that jointly predicts what and when to narrate in longer untrimmed movie clips. A dual-head audio-visual localizer predicts two temporally distinct windows per AD utterance: a visual cue window and a spoken narration window. A LoRA-adapted VLM then generates concise ADs from the predicted visual evidence, trained with a Description Ranking Loss that ranks captions (negative samples) of the same frames lower than the GT AD. To benchmark this new problem statement, we introduce the LongLSMDC benchmark with up to 8-min movie clips (~6.5min on average). On LongLSMDC, Cue2Narrate outperforms video-only and audio-only localization baselines by 5–12 points in avg. mAP. Under both predicted- and GT-window evaluation, Cue2Narrate improves AD generation over the corresponding fine-tuned base VLM. These results establish the first benchmark for multi-segment AD generation on long-form clips. Data & Code: https://github.com/multimodal-ai-lab/Cue2Narrate

4. Ready to Speak: Aligning LLMs for TTS-Friendly Text Generation

Authors: Thibaut Thonet, Jos Rozen, Laurent Besacier

Categories: cs.CL | EMNLP 2026 - Main Conference Score: 7.10/10 (Obj:9 Id:7 Ind:8 Comp:9 Eff:7 Nov:6 Rep:5)

  • Strength: 用仅 40 个可学习参数的 FaRM 奖励在 10 样本 regime 下实现 TTS 友好生成对齐，偏好预测准确率 0.960/0.941 反超 4B 参数 RM（0.871/0.946），并以一半延迟（1.6s vs 3.4s）逼近 PolyNorm 的友好度（4.73 vs 4.92）且 helpfulness 更高（4.86 vs 4.72）；启发式指标与 ASR 回转及人工 MUSHRA 强相关（语句级 ρ −0.71/−0.。
  • Weakness: 核心 FaST 框架是同一作者 EMNLP 2025 前作的直接应用且验证仅限两个英语域、3B/4B 模型与单一 TTS 引擎（100 样本下普通 SFT 已高度竞争、存在需手工修权重的冗长偏差），而论文声称的代码库 github.com/naver/tts-friendly-gen 经三重验证（gh api、直接 HTTP、GitHub 搜索）不可访问，合成数据构造无法独立审查。
  • Full review: Claude Code 全文七公理审稿

Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet poorly suited for spoken delivery via Text-to-Speech (TTS). In this work, we study how to make LLMs natively generate TTS-friendly text, which we frame as a preference alignment problem: instead of relying on downstream rewriting modules, we directly align LLMs to generate text optimized for spoken delivery. We introduce two preference datasets spanning different target domains, CORA and Recipe, which contain paired TTS-friendly and TTS-unfriendly responses. We further propose an evaluation suite combining a pattern-based heuristic metric, a TTS$\to$ASR evaluation pipeline, and a MUSHRA listening study with human judges. Our experiments compare the recently proposed Feature-aware Sampling and Tuning (FaST) framework – leveraging interpretable features instead of a black-box reward model – against an array of alignment baselines on the TTS-friendly generation task. Notably, we found that FaST achieves the best overall tradeoff between TTS-friendliness and helpfulness across various settings. We also identified a strong correlation between our different metrics, highlighting the ability to reliably assess TTS-friendliness via an efficient heuristic.

5. Efficient Passive Acoustic Monitoring of Killer Whales Using a Two-Stage Detection and Ecotype Classification Cascade

Authors: Daniela Ruiz, Manuel Castellote, Zhongqi Miao, Carl Chalmers, Bruno Demuro et al.

Categories: cs.SD, cs.CV Score: 6.95/10 (Obj:9 Id:8 Ind:6 Comp:9 Eff:7 Nov:5 Rep:6)

  • Strength: 在 1,065.8 小时 DCLDE 2027 公开数据集上，两阶段 ResNet-18 级联使七类 macro-F1 从 0.919 提升到 0.933（稀有 OKW 类 0.842→0.930），Stage 1/2 分别达 0.960/0.958 并超过 Perch 2.0 冻结嵌入（0.933），单窗推理 1.4 ms（约 700 窗/秒），Puget Sound 域适应把检测 F1 从 0.405 提到 0.755。
  • Weakness: 弃权阈值校准与验证同分布（34.5%–65.0% 未标注代理样本仍通过）、域适应仅覆盖 Stage 1 且 Puget Sound 评估未测连续流召回、边缘设备性能未验证，且无官方代码发布，方法为已有组件（级联/弃权/主动学习）的应用级组合。
  • Full review: Claude Code 全文七公理审稿

Passive acoustic monitoring of killer whales is particularly important for conservation of the endangered Southern Resident killer whale population, but requires accurate models that can operate in real time under severe class imbalance and deployment shift. We propose a lightweight ResNet-based two-stage cascade that first detects killer whale vocalizations and then classifies confident detections into five eastern North Pacific ecotypes, abstaining on ambiguous calls. We train and evaluate the pipeline on the DCLDE 2027 dataset, where the detector achieves 0.960 macro-F1 and the classifier 0.958, outperforming frozen Perch 2.0 embeddings on the five-ecotype benchmark. By separating detection from ecotype recognition, the end-to-end cascade improves seven-class macro-F1 from 0.919 for a single-stage model to 0.933, with the largest gain on the rare OKW ecotype. To assess transfer beyond the benchmark, we use active learning to adapt the Stage 1 to the acoustic environment of Puget Sound, WA, increasing killer whale detection F1 from 0.405 to 0.755 on manually verified detection windows. Finally, each stage processes a 3 s window in approximately 1.4 ms on an NVIDIA H100, enabling faster than real time inference. These results demonstrate that the proposed two-stage cascade pipeline enables reliable killer whale detection and classification, adaptation to new acoustic domains, and real-time monitoring for conservation applications.

6. AVERT: Audio-Verified Adjudication for Spoken Dialogue State Tracking

Authors: Chunggi Lee, Hanspeter Pfister

Categories: cs.CL Score: 6.95/10 (Obj:8 Id:6 Ind:8 Comp:6 Eff:8 Nov:8 Rep:3)

  • Strength: 不重训基础模型，用音频验证器加权投票 + 三个受限算子把 SpokenWOZ 上 1B 级 Spoken DST 从编辑器 38.34 JGA 提到 40.13，超过最强 1B 端到端系统（39.32），长对话（轮 31+）相对提升达 +19.4%，验证器推理仅占约 2% 延迟。
  • Weakness: 仅在 SpokenWOZ 单数据集验证、槽子集与阈值均同源调优且未测迁移，1B 级胜出用约两倍解码参数且未报显著性，音频验证器实际只贡献总增益的约 1/5，且全文无代码/权重发布、Fisher 数据需 LDC 许可证。
  • Full review: Claude Code 全文七公理审稿

Spoken dialogue state tracking recovers slot-value pairs from speech, where ASR errors concentrate in entity values and persist across turns, making it both a generation and an editing problem. A strong per-turn text editor corrects much of this but, operating on the transcript alone, leaves three recoverable errors: a value predicted inconsistently across turns, an omitted slot, and a value the audio does not support. We present AVERT, which scores each candidate value by combining cross-turn agreement with a trained audio-conditioned verifier and resolves the three error types with three operators, vote, add, and swap, each restricted to the slots where its error is common. On SpokenWOZ, a base speech-LLM reaches 33.04 JGA, a text editor 38.34, and AVERT 40.13, without retraining either. This is in the range of a 1B end-to-end system that consumes the full spoken history (39.32), though AVERT uses two 1B decoders rather than one. The audio verifier contributes a statistically significant gain, and restricting each operator to a selected slot subset matters: removing it lets unrestricted voting overwrite correct categorical values and fall below the editor.

7. Heard but Not Heeded: Paralinguistic Information Encoding and Loss in Audio-Language Models

Authors: Bhuvan Koduru, Dareen Safar B Alharthi, Rita Singh, Bhiksha Raj

Categories: cs.SD, cs.AI Score: 6.87/10 (Obj:8 Id:8 Ind:6 Comp:6 Eff:6 Nov:6 Rep:6)

  • Strength: 在 4 个开源 ALM 上用受控风格数据+LOSO 探针+泄漏率完成全管线机制定位：后三分之一编码器以 82–85% 探针准确率强编码风格（机会 14.3%），输出端泄漏率两簇分化 0.977/0.844 vs 0.272，并给出训练目标主导归因与 GRL 负结果（风格头 93.3% 但泄漏率不动 0.977→0.979）。
  • Weakness: 仅 4 说话人 7 表演风格单一数据集，「训练目标决定论」靠 4 模型相关性支撑而无控制实验，未演示任何有效干预（GRL 失败），核心问题与 VoxParadox/ParaPairAudioBench/反事实审计等 2026 年 5-8 月工作重叠且无官方代码发布。
  • Full review: Claude Code 全文七公理审稿

Audio language models are designed to understand speech, yet it remains unclear whether they capture how something is said beyond what is said. We present a mechanistic analysis of paralinguistic information in four open source models, Whisper-large-v2, Qwen2-Audio-7B Instruct, Qwen2.5-Omni-7B, and Chroma-4B, using the Expresso dataset with controlled speaking styles. We combine centered kernel alignment, linear probing with leave one speaker out evaluation, open ended tone prediction, and a content prosody leakage metric to trace how style information moves from the audio encoder to the final output. All models strongly encode speaking style in the late encoder, that is, the top third of the audio encoder’s layers, but this information is consistently degraded before reaching the output. The projector reshapes representation geometry without removing information, while decoders differ in how much style they preserve depending on architecture and training objective. At the output level, models fall into two behaviors. Some are content driven, where predictions depend mainly on text. Others are acoustic driven, where predictions vary with speaking style. The leakage metric quantifies this difference, and qualitative results confirm it. Overall, we identify a gap between what models encode and what they use, highlighting a key limitation in current audio language models.

8. Phrase-Localized Language-Contrastive Guidance: Training-Free Localized Accent Control for Code-Switching Text-to-Speech

Authors: Che Hyun Lee, Sangkwon Park, Donghun Kang, Dongwook Lee, Youngho Cho et al.

Categories: cs.CL, cs.SD | Accepted to EMNLP 2026 (Main Conference). Demo: https://saga1214.github.io/PhraseLocalizedLCG/ Score: 6.80/10 (Obj:8 Id:6 Ind:6 Comp:8 Eff:6 Nov:7 Rep:8)

  • Strength: 免训练三支路 CFG + 自注意力短语定位使 12 方向码转宏平均 LAe 从 0.233 升至 0.518、MER 从 0.564 降至 0.445，人耳盲测偏好率 75.5%（p<0.001）且全局 MOS 统计不掉，代码与 1200 句基准全量开源。
  • Weakness: 掩码边界质量在码转数据上无直接量化、方向间效果方差极大（KO→FR 的 LAe 仅 0.022→0.064）且无归因，框架耦合单一 DLM 骨干、未验证自回归 TTS，评测限于 LLM 合成码转文本与单一参考声音，外部索引检索失败使新颖性查重未完成。
  • Full review: Claude Code 全文七公理审稿

Current speech synthesis struggles with code-switching, which mixes a foreign language phrase into a primary language utterance, causing the phrase to be spoken with the primary language’s accent rather than its native one. We propose Phrase-Localized Language-Contrastive Guidance (LCG), a training-free inference framework that restores a native accent to code-switched phrases in cross-lingual text-to-speech. LCG replaces the single language guidance applied across the whole utterance with a separate guidance for each region, so each part is guided by its own language. To choose where to apply this localized guidance, we propose a self-attention probing technique that finds the phrase boundaries without external alignments. Together, these components generate speech in which each region carries the accent of its own language, requiring no fine-tuning or auxiliary models. Across diverse language pairs, LCG robustly increases the nativeness of the code-switched phrase while suppressing accent leakage, and preserving overall speaker identity and naturalness.

9. BiMTokenizer: Preserving Semantic-Acoustic Balance in Low-Bitrate Speech Tokenization via Bidirectional State-Space Modeling

Authors: Xin Zhang, Lin Li, Chuanbo Liu, Jianquan Liu, Kong Aik Lee

Categories: cs.SD | Accepted to EMNLP 2026 Main Conference; 19 pages, 5 figures Score: 6.74/10 (Obj:8 Id:6 Ind:7 Comp:8 Eff:8 Nov:5 Rep:6)

  • Strength: 1.1 kbps 单塔 codec 在 test-clean 全面领先双塔基线（WER 2.44%、PESQ-WB 3.03、SIM 0.87），参数量 253M 不到 DualCodec（664M）一半，MACs 降 63.8-81.9%，并开源 4 个 checkpoint。
  • Weakness: 单塔 vs 双塔的核心结论依赖训练配方混杂的第三方横向比较（WER 优势仅 0.02 点、近噪声），方法组件借自 ExtBiMamba 与 npq-vit 的 Leech 格量化，且训练代码未开源、主评测限于同分布 test-clean。
  • Full review: Claude Code 全文七公理审稿

Speech codecs serve as bridges between continuous speech signals and large language models, yet face an inherent conflict between acoustic fidelity and semantic preservation. To mitigate this conflict, recent works increasingly adopt dual-tower architectures to decouple semantic and acoustic modeling with separate encoders. However, these dual-tower designs incur substantial architectural overhead. To avoid such complexity, we revisit the single-tower paradigm and propose BiMTokenizer, a low-bitrate speech codec (around 1.1 kbps) combining a bidirectional state-space backbone with Residual Spherical Leech Quantization (RSLQ). The bidirectional backbone strengthens temporal modeling, while RSLQ offers a fixed, well-separated lattice bottleneck for robust semantic and acoustic tokenization without learned-codebook collapse. Experiments show that BiMTokenizer achieves superior acoustic reconstruction and the lowest WER among low-bitrate codec baselines across both clean and noisy environments, while using less than half the parameters of recent dual-tower baselines. Furthermore, its robust semantic representations yield strong performance on downstream speech understanding tasks, confirming that a well-designed single-tower codec can preserve the semantic-acoustic balance at low bitrates. The code and model weights are available at https://github.com/ZhangXinWhut/BiMTokenizer.

10. On the Human and Computer Alignment of Attribute-Based Music Matches

Authors: Roser Batlle-Roca, Woosung Choi, Joan Serrà, Fabio Morreale, Wei-Hsiang Liao et al.

Categories: cs.SD, cs.AI Score: 6.70/10 (Obj:8 Id:5 Ind:6 Comp:8 Eff:6 Nov:8 Rep:6)

  • Strength: 首个属性级音乐匹配感知数据集 MATCHA（300 三元组、1105 条专家评估、83 名专家）原始标注全部公开，旋律属性人类一致率 >85%、Fleiss’ κ≈0.60，证实属性级分解显著优于全局相似度的低一致性，并量化出 CoverID/CLAP/DEfNet/KL 四指标与五属性的互补对齐模式。
  • Weakness: 平均每案例仅 3.7 次评估、剔除 AS 后有效样本 270，无下游基准任务定义且受版权音频不公开，数据集复用受限；AS 子集”生成伪影”归因无直接检验，人机对齐为纯相关分析且 MiRA 指标出自同一团队，存在自我确认风险。
  • Full review: Claude Code 全文七公理审稿

Recent advances in generative AI are raising ethical concerns regarding the originality of generated content and the potential replication of training data, with further implications for transparency, attribution, and intellectual property. In music, several computational approaches have been proposed to identify potential replication, using audio-based similarity metrics. Yet, their alignment with human judgments across distinct musical attributes remains underexplored. To address this gap, we conduct a perceptual experiment on music matches, defined as strongly similar musical excerpts. We focus on five musical attributes: melody, harmony, rhythm, voice, and timbre. We design a triplet-based forced-choice task comprising 300 cases, including plagiarism examples, cover songs, and AI-generated music. From this experiment, we introduce the MATCHA (Musical Attribute-based Triplet Comparison with Human Annotations) dataset: a collection of 1105 perceptual assessments of attribute-based music matches from 83 expert participants. Our findings reveal measurable agreement among participants in identifying matches across attributes. We further observe partial alignment between human judgments and computational similarity measures. Overall, this work underscores the importance of domain-specific and perceptually grounded evaluation frameworks for generative AI in creative practice.

11. A Unified Uncertainty-Aware Back-End for Speaker Verification: Scoring, Normalization, and Calibration

Authors: Junjie Li, Kong Aik Lee

Categories: cs.SD Score: 6.63/10 (Obj:8 Id:6 Ind:8 Comp:6 Eff:7 Nov:5 Rep:8)

  • Strength: 把嵌入协方差统一贯穿余弦打分、AS-Norm 与 QMF 校准全流程，零可学习参数、免重训编码器，双骨干平均相对 EER 改善 28.01%（ECAPA）/27.15%（ResNet），Vox1-O EER 1.069→0.750 / 0.867→0.745，代码全流水线开源。
  • Weakness: 消融显示 UAS-Norm 三组件边际增益仅 24.59%→25.86% 且未跨骨干验证，个别 minDCF 退化无归因，缺跨域评测，未见对协方差估计误差的敏感度分析。
  • Full review: Claude Code 全文七公理审稿

Speaker verification back-ends commonly combine similarity scoring, score normalization, and calibration. However, speaker embeddings extracted from real-world utterances have trial-dependent reliability because of factors such as duration, noise, and channel variation. Existing uncertainty-aware methods primarily improve the speaker encoder or the initial similarity score, while the estimated uncertainty is typically not propagated through subsequent normalization and calibration. We represent each utterance by a speaker embedding, interpreted as a posterior mean, together with its covariance as an uncertainty estimate. We present a unified uncertainty-aware back-end comprising uncertainty-aware cosine scoring, uncertainty-aware AS-Norm (UAS-Norm), and uncertainty-aware Quality Measure Function calibration (UQMF). Covariance information is incorporated throughout this pipeline to adjust score scaling, cohort statistics, normalized-score combination, and calibration features. Experiments with ECAPA-TDNN and ResNet show consistent EER reductions and improved target–non-target separation across both architectures.

12. Ontology-based Target Sound Extraction

Authors: Carlos Hernandez-Olivan, Marc Delcroix, Tsubasa Ochiai, Naohiro Tawara, Shoko Araki

Categories: eess.AS | Accepted for publication at IWAENC 2026 Score: 6.58/10 (Obj:8 Id:7 Ind:6 Comp:8 Eff:6 Nov:7 Rep:4)

  • Strength: 提出本体层级 TSE 新任务（arXiv 检索无先例），multi-hot 条件+CPCC 正则单次前向在 Root/L1/Leaves 全面最优（SNRi 7.72/8.36/10.21 dB），叶级较基线 +2.4 dB，根级较最强基线 +2.15 dB。
  • Weakness: 对比全部为内部消融、无任何已发表 TSE 系统 head-to-head，评测仅限作者自建合成混合（无真实录音、无听评），CPCC 仅对 multi-hot 有效但机制未解释，且论文与第一作者 GitHub 均无代码发布。
  • Full review: Claude Code 全文七公理审稿

Target sound extraction (TSE) aims to isolate a sound source of interest from a mixture, given a semantic query. Existing TSE systems are conditioned on fixed class representations tied to individual sound categories, limiting their ability to handle the hierarchical relationships that naturally organize environmental sounds. In this paper, we introduce ontology-based TSE, a new task formulation in which a single model extracts sounds queried at any level of a sound ontology, from fine-grained leaf classes such as cat and dog to high-level categories such as animal. We propose a learnable class embedding table defined over all nodes of an AudioSet-derived ontology, regularized with a Cophenetic Correlation Coefficient (CPCC) loss that aligns embedding distances with shortest-path distances in the ontology tree. Our experiments across different approaches show the benefit of considering the ontology structure when training TSE systems.

13. ABSE-NET: A Lightweight Neural Model for Active Binaural Speech Enhancement in Open-Fit Hearing Aids

Authors: De Hu, Xue Du, Qingying Zhao, Qintuya Si

Categories: cs.SD, eess.AS | Accepted to INTERSPEECH 2026. 10 pages, 3 figures Score: 6.32/10 (Obj:8 Id:6 Ind:6 Comp:9 Eff:5 Nov:7 Rep:4)

  • Strength: 以 0.112M 参数 / 0.184G FLOPs（比 DeepANC 低约 176 倍参数、比 ASE-TM 低约 78 倍 FLOPs）取得 PESQ 3.626 / STOI 0.955 全场最优，并首次在仿真中实现免耳内误差麦克风的开式助听器声泄漏抵消（15° DOA 失配下仍保持 6.434 dB SI-SDR）。
  • Weakness: 全部实验为窄条件仿真（SNR 仅 -5~0 dB、单噪声源、无混响、无真机、无延迟/实时率报告、无主观测试），官方代码仓库为空壳（size=0、仅 10 字节 README、零提交），且缺少区分泄漏抵消与失真补偿贡献的专门证据。
  • Full review: Claude Code 全文七公理审稿

Open-fit hearing aids have attracted growing attention due to their superior wearing comfort. However, the open-fit design inevitably causes acoustic leakage into the ear canal, degrading the performance of existing binaural speech enhancement (BSE). To this end, we propose ABSE-NET, an active BSE framework integrating active noise control (ANC) with BSE to jointly enhance target speech and suppress acoustic leakage. The ABSE-NET pipeline cascades a binaural MVDR (BMVDR) with a lightweight neural network (LNN). The former achieves a coarse BSE, whereas the latter simultaneously cancels acoustic leakage and compensates for BMVDR-induced distortion. The LNN uses an encoder-decoder with a feature fusion module, which includes frequency-time dependency learning and convolutional attention blocks. Unlike traditional BSE+ANC solutions via adaptive filtering, ABSE-NET needs no in-ear microphone in practical deployment. Experiments validate its superiority over state-of-the-art methods. Code repository: https://github.com/Bream101/ABSE-NET.

14. Perceptible or Not? Diagnosing Passive Fingerprints for Speech Deepfake Attribution

Authors: Yupei Li, Qiyang Sun, Emmanouil Benetos, Berrak Sisman, Björn Schuller

Categories: cs.SD Score: 6.21/10 (Obj:8 Id:6 Ind:6 Comp:5 Eff:6 Nov:7 Rep:5)

  • Strength: 提出首个区分可感知与不可感知被动指纹的诊断协议 PIPDP，十生成器×三检测器实验证明感知透明扰动（94.4% TR）可致归因准确率最多下降 48.2%，而情感提示变化在域内仅致约 1.0% 波动，量化确立不可感知指纹的归因可靠性。
  • Weakness: “情感基本不影响归因”的结论与 ESD 域外 HiggsAudioV3 angry 下降 18.45% 的数据存在张力且未被正面解释，同时协议只在情感单一维度、2 个生成器、闭集 10 路与自训检测器框架内验证，无代码与数据发布，缺少与既有归因系统的基准对比。
  • Full review: Claude Code 全文七公理审稿

Passive fingerprints (intrinsic traces naturally left by generators) have been shown to enable attribution in speech deepfake detection, yet their persistence, reproducibility, and content-independence remain unverified. Moreover, no prior work distinguishes perceptible from imperceptible fingerprints, although the two have very different implications for attribution reliability. Perceptible fingerprints, such as emotional expression, are shaped by perceptual quality objectives and may change across model updates, whereas imperceptible fingerprints are not explicitly optimised by current training objectives and are rarely considered in existing dataset design or training strategies, as they have limited influence on downstream applications. We therefore propose a Perceptible-Imperceptible Passive-fingerprint Diagnostic Protocol (PIPDP) to define and separately analyze these two fingerprint types. PIPDP comprises three complementary analyses: multi-evidence fingerprint verification through residual-energy, reproducibility, and saliency analyses, perceptually transparent perturbations preserving audio quality, and prompt-driven emotion change that modifies perceptible fingerprints without model retraining. Experiments across ten speech generators and three attribution detectors show that imperceptible fingerprints provide persistent attribution cues. Perceptually transparent perturbations reduce attribution accuracy by up to 48.2\% on HiggsAudioV3, whereas emotion-driven changes leave attribution largely unchanged, with only about a 1.0\% accuracy variation across emotions on CosyVoice2 using w2v-bert-MLP. These results suggest that imperceptible fingerprints are more reliable for trustworthy attribution.

15. TimeSteer: Inference-Time Speech Scheduling in Joint Audio-Visual Diffusion Models

Authors: Chao Zhou, Yiling Chen, Qi Chu, Tao Gong, Nenghai Yu et al.

Categories: cs.CV, cs.AI, cs.MM Score: 6.20/10 (Obj:8 Id:6 Ind:6 Comp:6 Eff:7 Nov:6 Rep:4)

  • Strength: 免训练源区间定位 + 区域感知潜重映射使 LTX-2 骨干 HR₀.₂ 从 0.21 升至 0.73、IoU 从 0.63 升至 0.87（daVinci 骨干 0.09→0.53），起止误差降至 0.11/0.15s，且 WER 与音视频同步 LSE-C 与无干预采样基本持平。
  • Weakness: 论文核心贡献之一的 SpeechShift 基准与代码截至发布 4 天未开源（GitHub 检索无果），定位阈值 τ 与时序敏感注意力头的筛选准则缺失、主表单种子，全部主结果暂不可独立验证，每步 DiT 回放的延迟开销无量化数字。
  • Full review: Claude Code 全文七公理审稿

Although pretrained joint audio-visual diffusion models offer rich control over \emph{what} to generate, they provide no explicit control over \emph{when} an utterance should occur. To address this, we study \emph{inference-time speech scheduling}, a novel task that places coupled speech and visual articulation within user-specified begin–end intervals without finetuning the backbone model. We uncover two intrinsic properties of the denoising process that enable this task. First, a timing-sensitive text-to-audio cross-attention head exposes each utterance’s model-implied source span along the latent timeline. Second, the predicted clean latent already organizes coupled speech and visual articulation, allowing their temporal placement to be edited without regenerating the content. Building on these discoveries, we propose \textbf{TimeSteer}, a training-free framework that localizes each utterance’s source span through \textbf{Source Span Localization} and transfers the associated audio-visual latent content from the source interval to the specified target interval through \textbf{Region-Aware Latent Remapping}. We further introduce \textbf{SpeechShift}, the first benchmark for interval-level speech scheduling in joint audio-visual generation. Experiments across two representative backbones show that TimeSteer substantially improves interval controllability over training-free baselines while maintaining competitive overall generation quality.

16. TAG-Bench: Benchmarking Temporal Audio Grounding in Large Audio Language Models

Authors: Yuhang Dai, Xin Shu, Zengxi Li, Lei Xie, Xiangang Li et al.

Categories: eess.AS | 13 pages, 7 figures, 9 tables Score: 6.00/10 (Obj:8 Id:6 Ind:6 Comp:6 Eff:6 Nov:6 Rep:4)

  • Strength: 首个 LALM 时序音频定位基准（1,750 人工验证对、149.5 小时、7 秒至 20 分钟、22.1% 一对多），21 系统统一评测暴露核心短板：最佳仅 31.2 mIoU、R@0.7 21.5%，一对多欠报率 79.3%–100%、计数准确率上限 13.2%。
  • Weakness: 作者团队与第一名 FireRedAudio 研制方高度重合且无利益冲突披露，数据与评测代码截至评审日未公开（GitHub 检索零命中）致全部结果无法第三方复跑，跨子集比较自认为描述性、无归因控制，无人类天花板与缓解干预实验。
  • Full review: Claude Code 全文七公理审稿

Large audio language models (LALMs) can describe what is heard, but their ability to localize when queried content occurs remains less systematically evaluated. We present TAG-Bench, a benchmark for temporal audio grounding in which a model returns every time interval that matches a natural-language query. TAG-Bench contains 1,750 human-verified query-recording pairs covering 149.5 hours, with eight source-dependent subsets spanning query categories and audio durations from 7 s to 20 min; 22.1% of the queries have multiple ground-truth intervals. Across 21 evaluated systems, the best-performing model achieves 31.2 mIoU and is the only system above 20 mIoU on the two long subsets, yet even this top performer reaches only 21.5% recall at IoU >= 0.7. Moreover, 9 of 21 systems fall below 5 mIoU, and every model under-reports the number of occurrences on one-to-many queries, with none exceeding 13.2% count accuracy. Because responses are free-form, we report parsing-failure rate and MAE coverage: parsing failures remain in mIoU, Recall, gIoU, and count metrics as empty predictions but do not enter MAE. The results separate precise localization, occurrence enumeration, and output-format reliability within a benchmark whose cross-subset comparisons are descriptive rather than controlled estimates of query abstraction or duration. We will release the TAG-Bench data and evaluation code to support future research.

17. Soft Posterior Speaker Injection for Multi-Talker Speech Recognition

Authors: Jian Zhu, Cheng Luo

Categories: cs.SD, cs.MM | This paper is submitted to ICASSP2027 Score: 5.89/10 (Obj:8 Id:6 Ind:6 Comp:7 Eff:5 Nov:6 Rep:4)

  • Strength: 提出软 simplex 说话人占比经多层 FiLM+解码器 prompt 注入 Whisper，消融证明软后验联合训练优于硬 mask 与 oracle 能量比（硬 mask 0.526/oracle 0.537 vs 软 0.496 cpWER），held-out LibriCSS 经 freeze-posterior+OV-heavy 续训降 5.1 点（37.5%→32.4%，p<0.001）。
  • Weakness: 全部对比仅限自建 SOT 变体、无任何已发表第三方方法，LibriCSS 用 GT 切窗非官方协议且说话人互斥性未声明，zero-shot 与 SOT 统计持平（CI 含 0），主增益依赖特殊适配协议且三个变体无法复现，全文无代码或数据发布。
  • Full review: Claude Code 全文七公理审稿

Multi-talker automatic speech recognition (MT-ASR) remains challenging under overlapping speech. Hard diarization-based segmentation introduces irreversible errors, whereas serialized output training (SOT) avoids explicit segmentation but does not condition a pretrained encoder on speaker activity. We propose Soft Posterior Speaker Injection (SPSI): a lightweight head predicts frame-level speaker posteriors $\hat{\mathbf{P}}$ and injects them into Whisper through multi-layer feature-wise linear modulation (FiLM) and decoder speaker-memory prompts. On controlled two-speaker LibriSpeech overlap, SPSI reduces utterance-mean constrained permutation word error rate (cpWER) from 50.7\% (SOT) to 49.6\% (one-sided paired bootstrap $p{\approx}0.006$), with a larger reduction in the high-overlap bin (60.4\%$\to$58.8\%). Same-backbone speaker-auxiliary objectives and voice activity detection (VAD) pipelines do not outperform SOT; zero-shot (ZS) LibriCSS is comparable. Freeze-posterior adaptation with overlap-heavy (OV-heavy) continuation reduces held-out LibriCSS cpWER (sessions 8–9) to 32.4\% (versus 37.5\% for SOT). Ablations indicate complementary encoder FiLM and decoder prompts, and that the effective signal is a \emph{soft} simplex-valued speaker share.

18. SpeakPay: Domain-Adaptive LoRA Fine-Tuning of Whisper for Low-Resource Nepali Financial Speech Recognition

Authors: Biraj Subedi

Categories: cs.CL | 12 pages, 2 figures. Code, dataset, and model weights: https://github.com/subedibiraj/speakpay Score: 5.70/10 (Obj:8 Id:5 Ind:5 Comp:6 Eff:6 Nov:4 Rep:8)

  • Strength: 首个尼泊尔语金融语音数据集 NepFinSpeech-403 + LoRA 微调 Whisper large-v2，WER 129.95%→42.58%（相对降 67.2%）、数字准确率 0%→73.9%、交易成功率 1.67%→33.33%（约 20 倍，符号检验 p=3.5×10⁻¹⁸），100 条领域语句即可使零样本 WER 减半；代码/数据/结果 JSON/部署全部当日兑现公开。
  • Weakness: 通用基线用 Whisper small 与本文 large-v2 比较不公平、0% 数字准确率部分归因转录格式差异，且训练/测试可能说话人重叠（仓库有 speakerdisjointsplit.py 但正文未报告说话人不重叠结果）；目标视障用户从未被正式可用性评估，0dB SNR 下 WER 崩溃至 103.85%。
  • Full review: Claude Code 全文七公理审稿

Mobile payment applications in Nepal are graphically mediated and largely inaccessible to visually impaired users. This paper presents SpeakPay, a voice-first digital wallet, and documents the central technical contribution: a controlled study of domain adaptation for low-resource financial speech recognition. We introduce NepFinSpeech-403, a 403-utterance dataset of Nepali financial voice commands (send, load, and balance operations spanning 237 unique numerals), and fine-tune Whisper large-v2 with LoRA. On the held-out test set, the domain-adapted model reduces Word Error Rate from 129.95% (zero-shot baseline) to 42.58% — a 67.2% relative reduction — and improves Devanagari numeral recognition accuracy from 0.0% to 73.9%. We find that word-level metrics understate the practical task-level impact: domain adaptation improves the Transaction Success Rate from 1.67% to 33.33%, a roughly 20x gain. The improvement is consistent at the individual-utterance level (sign test, $p < 10^{-17}$) and across all command types. A data efficiency analysis shows that as few as 100 domain-specific utterances are sufficient to halve the zero-shot WER, with performance plateauing around 300 examples. Error analysis reveals systematic numeral confusion patterns (zero insertion/deletion, prefix hallucination) that account for the majority of remaining transaction failures. The trained system is deployed as a publicly accessible voice-first web application. All code, dataset, model weights, and this paper are released at https://github.com/subedibiraj/speakpay.

19. MADS: A Multiview Acoustic Descriptor Set Beyond Standard Spectral Summaries

Authors: Utsab Ghosh, Roshni Chakraborty

Categories: cs.SD, cs.AI, eess.AS Score: 5.53/10 (Obj:8 Id:5 Ind:6 Comp:8 Eff:4 Nov:5 Rep:5)

  • Strength: 19 维物理启发描述子在 ESC-10/ESC-50/MSoS 上以约一半维度超越 38D 基线，峰值准确率 81.00%/52.78%/67.48%，全部描述子公式化、无需可学习参数。
  • Weakness: 对最强基线领先仅 1.00–1.55 个百分点且无显著性检验，无任何深度模型或 openSMILE 等第三方特征基线对照（ESC-50 低于 2015 年 CNN 约 12 点），缺 leave-one-view-out 消融，代码未发布。
  • Full review: Claude Code 全文七公理审稿

Dominant audio classification pipelines rely either on compact handcrafted summaries or on fixed time-frequency frontends such as log-mel representations prior to deep modeling. While highly successful, these representations do not explicitly expose the physical dynamics of the underlying sound-generating event. We introduce MADS (Multi-view Acoustic Descriptor Set), a compact 19-dimensional physics-informed descriptor set de- signed to capture complementary spectral, temporal, mechanical, and stochastic structure in audio signals. Rather than treating sound only as a spectral pattern, MADS encodes properties related to excitation, damping, periodicity, impulsiveness, and structural consistency within a unified multi-view representation. We evaluate MADS using standard classical machine learning models on ESC-10, ESC-50, and MSoS, and compare it against two conventional handcrafted baselines: a compact 26D MFCC- based baseline and an expanded 38D spectral-summary baseline. Across ESC-10 and ESC-50, MADS achieves the strongest peak results overall, reaching 81.00% and 52.78%, respectively, while using roughly half the dimensionality of the 38D baseline. On MSoS, MADS again delivers the strongest top-end performance, reaching 67.48%. These results establish MADS not merely as a competitive standalone descriptor set, but as the foundational descriptor layer of a broader acoustically grounded representation program for future frame-level and deep-learning-compatible audio modeling.

20. Artificial Rosetta Stone: Constrained Maximum A Posteriori (MAP) Reconstruction of Symbolic Raga Sequences via Order-k Markov Models

Authors: Saanvi Raghavendran, Abhishek Bhattacharjee

Categories: cs.SD, cs.LG, stat.ML | 37 pages,5 tables,11 Graphical Representations Score: 4.84/10 (Obj:6 Id:4 Ind:4 Comp:8 Eff:4 Nov:4 Rep:6)

  • Strength: 约束 MAP 重构形式化严谨（精确 DP 复杂度 O(TN^{k+1})、参数量 N^k(N−1) 闭式、集中界显式），真实 Yaman 数据 30 条序列上九个条件全部超均匀基线（最佳 k=2@10% 掩码 0.470 vs 0.143），且三主张拆分与限制声明是同类论文中最诚实的。
  • Weakness: 唯一对比基线是均匀猜测、真实实验未启用语法约束且参考序列由同一自动转写管线生成（仅 6 条测试序列的自洽循环），方法组件均为教科书标准件而 2018 年的 arXiv:1808.01603 已用马尔可夫链建模 raga 音符，消融大部分停留在设计规范而非已执行实验。
  • Full review: Claude Code 全文七公理审稿

Reconstructing a damaged musical fragment is an inverse problem: the observed sequence contains partial information, while a raga encodes constraints limiting allowable completions. This paper formalizes a mathematical framework for this, proposing the Artificial Rosetta Stone (ARS). We separate three claims often conflated: a symbolic sequence can be reconstructed probabilistically; a sequence can be consistent with an explicit grammar; and a historical performance can be authenticated. We only support the first two. We model a raga via a finite alphabet and constraint system, using an order-k Markov model for melodic probabilities. A symmetric Dirichlet prior yields a tractable posterior. We pose missing-note reconstruction as a constrained MAP problem. For fixed-length sequences and finite-order constraints, optimization admits an exact dynamic-programming solution with worst-case time complexity $O(TN^{k+1})$. We derive the parameter count $N^k(N - 1)$, prove a concentration bound under explicit mixing assumptions, and analyze estimation error propagation. A reproducible synthetic experiment uses six raga-inspired alphabets, orders $k \in {1, 2, 3}$, and masking rates up to 50%. This is a proof of concept, not historical reconstruction. A real-audio feasibility pilot evaluates 30 usable sequences from 42 Yaman clips via automated pitch extraction, segmentation, and quantization. Lacking documented provenance and relying on automated transcription, this is not expert-validated archival reconstruction. Claims are tied to stated conditions, not universal properties of Hindustani music. Code: https://github.com/mathacker23/ArtificialRosettaStone.