Daily Papers — 2026-08-13

6 papers on audio, speech, music, and acoustics.

Authors: Koen P. de Vries, Xavier Alameda-Pineda, Estefanía Talavera, Stéphane Lathuilière

Categories: cs.CV | Accepted at HCMIW ECCV workshop. Code available here: https://github.com/koenv759/VanillaSFT Score: 7.10/10 (Obj:8 Id:8 Ind:6 Comp:9 Eff:8 Nov:6 Rep:8)

Strength: Vanilla SFT (LoRA, <4.5h training) 在三个 benchmark 上超越 HumanOmniV2 等 CoT reasoning 方法，推理延迟快 6.7×，训练成本低 11×；且 question-independent caption 达到与完整视频输入可比的性能（68.6% vs 69.5%），直接质疑 MLLM 的视频信息提取能力。
Weakness: 所有实验仅基于单一基座模型 Qwen2.5-Omni-7B，结论泛化性受限；IntentBench-Prime 清洗和 Caption SFT 均使用同源模型（Qwen2.5-Omni 系列），存在轻微循环依赖。
Full review: Claude Code 全文七公理审稿

Training Multimodal Large Language Models for audio-visual social understanding is a crucial step toward embodied social intelligence. Chain-of-thought (CoT) reasoning has become the dominant approach, with HumanOmniV2 and its IntentBench benchmark as a prominent reference point. In this context, we report three findings. First, IntentBench is highly noisy: $\sim$7% of questions are broken and $\sim$23% are trivially answerable without the video input. We remove the affected questions and release Intentbench-Prime. Second, current reasoning approaches are expensive and surprisingly ineffective. A simple Vanilla SFT baseline matches or outperforms existing reasoning methods across three benchmarks at a fraction of the cost, establishing it as an essential baseline for evaluating novel fine-tuning techniques. Third, our analysis reveals that substantial priors can be learned solely from the text modality and that using a textual caption instead of the video yields performance on par with Vanilla SFT. These surprising findings reveal the limitations of current MLLMs when it comes to social understanding. IntentBench-Prime, Vanilla SFT model, and code are publicly available.

2. Alignment Drift in Single-Model Speculative Decoding for ASR: Mechanism, Correction, and Cost

Authors: Xinyu Wang, Huapeng Zhou, Ziyu Zhao, Silin Meng, Ke Bai et al.

Categories: cs.SD, cs.MM Score: 7.00/10 (Obj:9 Id:9 Ind:9 Comp:8 Eff:6 Nov:6 Rep:6)

Strength: 通过固定窗口宽度、仅改变中心位置的因果干预实验，证明了ASR单模型推测解码中alignment drift是continuation失败的原因之一（正确位置窗口在depth-2恢复+0.254条件接受率，95% CI [+0.241, +0.268]），并提出AnchorDraft训练修正实现+3.9–+7.1%端到端加速且WER变化≤0.01pp。
Weakness: 实用加速仅限于batch-one短音频场景（batch size增大或音频时长增至~104秒时加速衰减），运行时修正1.7B置信区间包含零，Voxtral控制组显示负效果（-0.011），且未提供公开代码仓库。
Full review: Claude Code 全文七公理审稿

Speculative decoding speeds up generation by letting a cheap draft propose several tokens that a target model checks in one pass. In the single-model form, the draft is a lightweight module attached to the target rather than a separate model. Applying this design to Automatic Speech Recognition (ASR) introduces an extra problem. The draft can read the whole audio at every step, yet its proposals get worse as it runs on its own. Access is not localization. The accepted text keeps the transcript position explicit, but the draft must also track the changing audio position. In the primary matched comparison, per-step audio access changes the first proposal modestly but roughly doubles later-proposal acceptance. Fixed-width windows show that the audio position explains part of this gap. A correctly placed window recovers continuation, while an equally narrow window at the wrong position reduces it. Late-draft median error reaches 21 frames in the hardest reported condition, while target attention during verification stays within a 2-frame median. We test two ways to reduce this drift. The first reads the audio position from verification attention and uses it to guide the next draft round. It saves time only when the extra accepted tokens offset the readout cost. The second is AnchorDraft, which teaches the draft to track the audio position during training without changing the inference graph. The trained draft improves end-to-end speed at both tested target scales. These results show that ASR self-speculation depends on token prediction, audio-position tracking, and draft cost.

3. CASA: Content-Acoustic Speaking Assessment with Speech Encoder and Large Language Model

Authors: Nhan Phan, Ilona Lähteenmäki, Anna von Zansen, Olli-Pekka Pauna, Yaroslav Getman et al.

Categories: cs.CL, eess.AS | To be submitted to ICASSP 2027. Code is available at https://github.com/aalto-speech/casa Score: 5.74/10 (Obj:8 Id:5 Ind:9 Comp:5 Eff:5 Nov:5 Rep:9)

Strength: 双分支架构在 S&I 2025 上达到 RMSE 0.358 / PCC 0.829，以 3.13B 参数实现与 6.24B NTNU 系统可比的性能，代码完全开源并报告 10 次重复实验的 95% CI。
Weakness: 报告的 0.358 落在 10 次重复运行 95% CI [0.359, 0.367] 之外，SOTA 声称缺乏统计支撑；核心分析目标（声学 vs 内容各自贡献）缺乏 acoustic-only 和 content-only 最终预测的直接消融，3 个手工流利度特征未被消融。
Full review: Claude Code 全文七公理审稿

Research on automatic speaking assessment (ASA) has increasingly adopted multimodal speech large language models to assess learners’ speaking performance. However, existing studies provide limited analysis of how acoustic and content information contribute to predictions and how stable the resulting performance is. We propose CASA, a simpler architecture combining Whisper-medium and Qwen3.5-2B that achieves state-of-the-art performance while providing a more interpretable separation between speech delivery and content. On the Speak & Improve Corpus 2025, CASA achieves a root mean square error (RMSE) of 0.358, improving on the previous best RMSE while using approximately half the estimated inference parameters. The general-purpose architecture is designed for adaptation to other ASA corpora without structural changes and relies on three handcrafted fluency features. Through ablations and repeated runs, we analyze the individual and complementary contributions of acoustic and content information, examine performance variability, and demonstrate the potential of large language model reasoning for training-free content validation.

4. VoxAudio: Vocalized Audio Synthesis via Multi-Reward Autoregressive Flow Matching

Authors: Wenxiang Guo, Changhao Pan, Ziyue Jiang, Fei Wu, Zhou Zhao

Categories: cs.SD Score: 5.42/10 (Obj:6 Id:5 Ind:3 Comp:5 Eff:6 Nov:5 Rep:8)

Strength: Chunk-agnostic pretraining（随机边界 + 几何分布采样）以简洁技巧实现推理粒度自由化，在 VoxBench 上 WER 0.038 vs Dasheng-AudioGen 0.264 大幅领先，且 234M 参数 + 0.32 RTF 的效率优势明显。
Weakness: 训练 reward（Whisper WER、Audiobox Aesthetics、PE-A-Frame TG-IoU、PEAV）与评测指标高度重叠构成循环论证，且最直接可比的 VoiceLDM 和 UmbraTTS 未被比较或引用，同组 SwanSphere 也未被引用。
Full review: Claude Code 全文七公理审稿

Vocalized audio synthesis, the task of generating audio in which intelligible speech is embedded within an environmental soundscape, underpins applications such as podcast production and video dubbing. Existing Text-to-Audio (T2A) systems either reduce quoted speech to unintelligible vocal murmur or delegate it to a separate TTS model with post-hoc mixing, which forfeits control over when speech occurs and how it interacts with the scene. We present VoxAudio, a causal autoregressive flow matching model that addresses this problem from three complementary aspects. At the architecture level, chunk-wise causal factorization with independent per-chunk noise levels lets audio be emitted through sliding-window streaming inference with KV caching at variable target durations; to enable inference at arbitrary chunk granularities, we further pretrain the model with randomized chunk boundaries. At the preference level, multi-reward Negative-aware FineTuning (NFT) jointly optimizes semantic fidelity, linguistic accuracy, aesthetic quality, and temporal grounding At the data level, to supply the missing supervision for vocal content, we build VoxCorpus, a large-scale corpus whose captions quote the verbatim transcript of embedded speech with time intervals, and VoxBench, an interval-annotated benchmark with a temporal-grounding metric. Experiments on four benchmarks spanning general audio, speech, and unified vocalized audio validate the effectiveness and efficiency of VoxAudio. Our code and demos are available at https://voxaudio.github.io.

5. HybridSB-MoE: Dual-Domain Schrödinger Bridges with Scene-Adaptive Expert Routing for Speech Enhancement

Authors: Zhengyi Lu, Aswini Sivakumar, Jie Hu, Yao Qiang

Categories: cs.SD, cs.AI Score: 5.40/10 (Obj:6 Id:5 Ind:6 Comp:5 Eff:5 Nov:6 Rep:5)

Strength: 非对称不确定性融合（频谱路径的认知信号，波形路径的偶然信号）是一个真正新颖的概念，在 VoiceBank+DEMAND 数据集上实现了 3.88 的 PESQ，同时在 K=8 时 RTF 达到 0.28（4-5 倍速度提升），且消融实验（图 3a）显示各组件必要性梯度清晰（-0.63/-0.43/-0.17 PESQ）。
Weakness: 仅在过时的单数据集 VoiceBank+DEMAND 上评估，缺失强判别式基线 (TF-GridNet)，未直接测试同构 vs. 异构 MoE（关键识别缺失），无代码发布，定理 1 被作者明确承认为非紧的“设计论证不等式”。
Full review: Claude Code 全文七公理审稿

Generative speech enhancement faces three gaps: spectral models capture harmonic structure but often disrupt phase, waveform models preserve phase but miss harmonics, and Schrödinger Bridges (SB) shorten transport from noise to clean speech but leave inference cost only loosely tied to training. We propose HybridSB-MoE, a dual-domain framework that fills these gaps through three contributions unified by a single asymmetric design principle. (i) Asymmetric uncertainty fusion: The spectral path captures epistemic uncertainty via expert disagreement, while the waveform bridge models aleatoric variance through stochastic dynamics. We fuse them asymmetrically, allowing the mixing weight to adapt to distinct error regimes rather than average predictions. (ii) Heterogeneous MoE with top-k=2 routing across five distinct architectural archetypes, where architectural diversity makes the epistemic signal indicate which inductive bias fails rather than small perturbations among similar experts. (iii) Discretization bound (Theorem 1): path-consistency and trajectory regularizers together bound the K-step bridge sampling error in 2-Wasserstein distance at rate K-alpha, making small-K inference an objective-level guarantee rather than an empirical claim. On VoiceBank+DEMAND, HybridSB-MoE outperforms diffusion- and SB-based baselines at their step budgets while remaining competitive with consistency-distilled few-step methods.

6. FastThaiG2P: Lightning-fast Thai Grapheme-to-phoneme Conversion for Voice Agent Pipelines

Authors: Charin Polpanumas

Categories: cs.CL Score: 5.20/10 (Obj:8 Id:5 Ind:5 Comp:5 Eff:5 Nov:3 Rep:8)

Strength: 亚毫秒级泰语 G2P (0.15ms/话语，比 TLTK 快 15 倍)，采用完全开放的流水线（代码 Apache-2.0、开放数据、命名脚本），针对 CPU 部署场景有明确延迟预算。
Weakness: 无客观正确性评估（未报告任何独立测试集上的 PER/WER/CER），TTS 质量仅由作者主观判断为“可理解”，且 79% 的字典条目由 LLM 生成，仅有 78.8% 的精确匹配验证——最终音素准确率仍未量化。
Full review: Claude Code 全文七公理审稿

FastThaiG2P provides sub-millisecond Thai grapheme-to-phoneme conversion for text-to-speech pipelines (International Phonetic Alphabet and Kokoro-TTS conventions) using a PyThaiNLP-tokenized, extensible dictionary and normalization rules for common Central Thai speech. The approach achieves an average latency of 0.15 ms per utterance on a benchmark of 27,242 synthetically generated utterances, of which 30\% is spent on tokenization, 12\% on normalization, and 58\% on out-of-vocabulary fallbacks (0.5\% OOV rate). To demonstrate its effectiveness, we used FastThaiG2P to phonemize Som-TTS, an open dataset containing 20 hours of grapheme-and-audio pairs, then trained an 82M-parameter StyleTTS 2 model based on a Kokoro-TTS recipe. The resulting model vocalizes intelligible Thai speech suitable for prototyping and development at 0.25 real-time factor (4x real-time) with ONNX inference on CPU.