Daily Papers — 2026-06-28

10 papers on audio, speech, music, and acoustics.

1. AMR: Adaptive Modality Routing for Multimodal Polyglot Speaker Identification

Authors: Chuxiao Zuo, Yao Zhu, Minqiang Xu, Manhong Wang, Yunke Zhang et al.

Categories: cs.LG, cs.AI, cs.SD Score: 8.5/10 (Obj:9 Id:8 Ind:9 Comp:7 Eff:10 Nov:6)

• Strength: 在缺失模态和跨语言场景下效果极为出色，将纯音频识别准确率从约30%大幅提升至近98%，彻底解决了基线方法的崩溃问题。
• Weakness: 核心方法属于动态路由与适配器的有效组合，在多模态融合范式中新颖性有限，偏重于针对特定挑战赛的工程优化。

中文摘要： 本文针对多模态说话人识别在真实场景下面临的模态缺失与训练-测试语言不匹配两大挑战，提出了一种面向 POLY-SIM 2026 大赛的多模态多语种说话人识别系统。系统核心为自适应模态路由（AMR）模块，通过两个模态适配器分别处理来自 W2V-BERT 2.0 音频编码器和 IResNet-18 人脸编码器的嵌入，并由可训练的路由器根据输入质量动态估计模态权重，进而聚合各模态的预测 logits 得到最终结果。为优化路由机制，作者采用模态感知训练策略，构造四类样本对以模拟多样化输入条件，并以 KL 散度作为权重分配的显式监督。在 POLY-SIM 2026 评测集上，该系统在四个协议上的平均准确率达 99.07%，较 FOP 基线提升了 32.73%，并在多种模态与语言条件下均取得接近饱和的识别精度。> Multimodal speaker identification systems face two key challenges in real-world deployment: missing modalities and language mismatch between training and testing conditions. In practical scenarios, background multi-speaker conversations, ambient noise, and overlapping speech further degrade identification accuracy. To address these challenges, we propose a multimodal polyglot speaker identification system for the POLY-SIM 2026 Grand Challenge. The system is fundamentally built upon Adaptive Moda

2. TF-MoE: Time-Frequency Mixture-of-Experts for Efficient Speech Separation

Authors: Qinzhe Hu, Chenda Li, Wangyou Zhang, Shujie Liu, Yan Lu et al.

Categories: cs.SD, cs.AI | Accepted to Interspeech 2026 Score: 8.4/10 (Obj:9 Id:7 Ind:9 Comp:7 Eff:9 Nov:8)

• Strength: 在同等低算力（4.1 GMACs/s）下SDR大幅超越BSRNN 3.8 dB，效果极其显著；提出契合语音时频特征的TF双维度交替MoE路由机制，物理意义清晰。
• Weakness: MoE框架本身并非全新范式，属于将已有范式在语音分离领域的成功定制化迁移；稀疏路由可能带来的实际推理延迟和训练不稳定性未在摘要中体现。

中文摘要： 本文针对语音分离模型参数虽小但计算成本高、难以在边缘设备部署的问题，提出了一种稀疏混合专家框架 TF-MoE。该方法通过交替的时域与频域 MoE 模块，在时间和频率两个维度引入动态专家特化，在不显著增加推理开销的前提下提升模型容量。其核心思想是利用稀疏激活机制，使模型只在相关专家被触发时才产生计算，从而实现容量增长与推理成本近乎解耦。该方法为在资源受限设备上实现高效语音分离提供了一条可扩展的技术路径。

由于您提供的英文摘要在末尾被截断，以上概括基于已有内容，可能未涵盖完整的实验结果部分。若您能补充完整摘要，我可以进一步补充主要实验结论。

Recent advances in speech separation (SS) have led to compact front-end models with small parameter sizes, yet their high computational cost remains a major barrier for deployment on edge devices. To address this, we propose TF-MoE, a sparse Mixture-of-Experts (MoE) framework that enhances model capacity with almost no increase in inference cost. Our method introduces dynamic expert specialization in time and frequency dimensions through alternating time-wise and frequency-wise MoE modules, each

3. Preference-ASR: A Preference-Aware Test Set for Benchmarking ASR in the Era of Speech LLMs

Authors: Nithin Rao Koluguri, Sasha Meister, Nikolay Karpov, Piotr Zelasko, Desh Raj et al.

Categories: cs.CL, eess.AS | Accepted at Interspeech 2026 Score: 7.9/10 (Obj:9 Id:7 Ind:7 Comp:8 Eff:8 Nov:8)

• Strength: Timely and elegant solution to the long-standing convention inconsistency problem in ASR evaluation by introducing instruction-conditioned benchmarking and selective normalization.
• Weakness: Base audio data likely overlaps with evaluated models’ training sets, and reliance on LLM-assisted reference generation might inherit LLM biases despite human verification.

中文摘要： 现有ASR测试集在数字、不流利词、实体和大小写等方面采用不一致的惯例，而标准归一化方法又抹去了用户真正关心的格式区分，导致当前基准无法衡量模型是否遵循用户对输出风格的偏好。针对这一问题，作者提出了PreferenceASR，一个用于评测ASR系统遵循自然语言偏好指令能力的测试集，覆盖归一化、实体、不流利词和大小写四个类别，并基于七个（已有/公开）数据源构建。该测试集将偏好以自然语言指令形式给出，使得评测更贴近真实用户的多样化需求。PreferenceASR填补了语音大模型时代下”用户偏好感知评测”的空白，为衡量ASR系统是否按用户指定风格输出提供了统一、可量化的基准。

Popular ASR test sets adopt inconsistent conventions for numbers, disfluencies, entities, and casing, while standard normalizers erase the format distinctions users care about. Current benchmarks therefore cannot measure whether a model follows user preferences for output style. We introduce PreferenceASR, a test set evaluating ASR systems on their ability to follow natural-language preference instructions across four categories: normalization, entities, disfluencies, and case. Built from seven

4. Position-Aware Target Speaker Extraction for Long-Form Multi-Party Conversations: A Diarization-Free Framework for ASR

Authors: Yichi Wang, Junzhe Chen, Wangjin Zhou, Tatsuya Kawahara

Categories: cs.SD, cs.MM | 5 pages, 2 figures, Accept by Interspeech 2026 Score: 7.9/10 (Obj:9 Id:8 Ind:8 Comp:8 Eff:8 Nov:7)

• Strength: 利用物理先验（DOA）的稳定性优雅地绕过了易错的级联语音分离-说话人日志管线，有效解决了长录音中的跨窗口说话人不一致问题。
• Weakness: 方法强依赖说话人位置固定且DOA估计准确的假设，在说话人移动或DOA估计受混响干扰严重的场景下鲁棒性存疑；网络架构层面的新颖性相对有限。

中文摘要： 长多人对话场景下说话人活动极度不平衡且频繁重叠，传统滑动窗CSS方法存在跨窗说话人不一致和残留串扰，需依赖diarization才能可靠归属说话人。本文提出PATSE——一种多通道位置感知目标说话人提取前端，利用会议中说话人到达方向（DOA）的稳定性作为空间先验，通过DOA引导的空间编码器和条件器直接生成按说话人归属的语音流，从而省去显式diarization，仅靠VAD等简单后处理即可推断说话人活动。在LibriReplay-DOA（新发布的真实房间回放数据集）和真实三方对话语料TEIDAN上的实验表明，PATSE持续优于CSS及基于diarization的流水线，在LibriReplay-DOA上将Whisper Large-v3的总体WER从CSS的32.8%降至14.0%。该框架产生排列无关、按说话人归属的输出，在各类角度配置和重叠区间均显著降低WER。

In long-form multi-party conversations, highly imbalanced speaker activity and frequent overlap make it difficult to identify “who spoke when and what”. Sliding-window continuous speech separation (CSS) mitigates sparse supervision, but often suffers from cross-window speaker inconsistency and residual crosstalk, which in practice requires diarization for reliable speaker attribution. Motivated by the stability of speakers’ directions of arrival (DOAs) in meetings, we propose PATSE, a multi-chan

5. DTM-Codec: Dynamic Token Masking for VFR Speech Coding with Efficient Boundary Selection

Authors: Hoyeol Sohn, Juhan Nam

Categories: eess.AS, cs.SD | 10 pages, 2 figures, accepted to INTERSPEECH 2026 Score: 7.9/10 (Obj:9 Id:8 Ind:8 Comp:8 Eff:8 Nov:7)

• Strength: 在严格匹配总比特率（含边信息开销）的公平协议下，通过简洁的动态Token掩码和线性时间边界选择机制，有效解决了VFR语音编码的痛点，效果优于固定帧率基线且媲美更大模型。
• Weakness: VFR范式本身并非全新，方法更多是现有概念（Token掩码、动态帧率）在语音编码中的优雅组合与工程实现，范式突破性一般。

中文摘要： 这篇论文针对可变帧率（VFR）神经语音编解码器中长期存在的难题展开研究：VFR通过为冗余区域分配更少帧、为快速变化语音分配更多帧来降低比特率，但必须额外传输保留时间步的边信息，导致先前方法在计入开销后增益往往不明显。作者提出了动态令牌掩码编解码器（DTM-Codec），其核心方法是采用动态令牌掩码机制配合高效的边界选择策略，在严格比特率匹配条件下精确控制保留帧的位置与开销。实验结果表明，DTM-Codec在将边信息开销纳入总比特率的严格评估下，相比固定帧率基线仍能展现出清晰且一致的编码增益。该方法有效解决了VFR编解码器中边信息开销侵蚀主要增益的瓶颈问题，为实用化低比特率神经语音编码提供了一条可行路径。

Variable frame rate (VFR) coding has recently emerged in neural speech codecs, allocating fewer frames to redundant regions and more frames to rapidly changing speech. VFR must transmit side information about retained time steps, but prior gains are either not rigorously addressed or often minor once these overhead bits are included in total bitrate. We present Dynamic Token Masking (DTM)-Codec, a neural speech codec that demonstrates clear gains over fixed-frame-rate baselines under a strict ma

6. VIB-AVSR: Variational Information Bottleneck for Noise-Robust LLM-Based Audio-Visual Speech Recognition

Authors: Piyush Arora, Navlika Singh, Umberto Cappellazzo, Stavros Petridis, Maja Pantic

Categories: eess.AS, cs.CV, cs.SD | Accepted to INTERSPEECH 2026. Our code is available at https://github.com/PiyushArora1010/VIB-AVSR Score: 7.6/10 (Obj:8 Id:7 Ind:8 Comp:7 Eff:8 Nov:7)

• Strength: 精准定位了LLM-based AVSR在噪声下退化的痛点，提出轻量即插即用的VIB正则化方案，无需额外数据或架构修改即显著提升多噪声条件下的鲁棒性。
• Weakness: VIB本身是已有机制，本质上仍是一种正则化手段，虽应用场景新颖但范式突破有限，且未见在干净条件下性能不下降的绝对保证。

中文摘要： 针对LLM-based音视频语音识别(AVSR)在噪声条件下鲁棒性不足的问题,本文提出VIB-AVSR框架,将变分信息瓶颈(VIB)融入多模态LLM架构,通过对音频特征施加信息瓶颈约束,迫使模型学习对噪声不敏感的压缩表示,从而抑制音频流中的噪声相关信息并增强对视觉信息的依赖。方法上,在连接预训练音视频编码器与LLM的投影层中引入可学习的VIB模块,对音频表征进行概率压缩,同时保留与语音内容相关的判别性信息。实验表明,该方法在噪声环境下的音视频语音识别任务中显著优于现有LLM-based AVSR方法,在保持干净条件性能的同时大幅提升了噪声鲁棒性,验证了信息瓶颈机制在多模态融合中的有效性。

Audio-Visual Speech Recognition takes two input modalities, acoustic and visual streams, where visual information from lip movements aids recognition when audio is noisy. Recently, LLM-based AVSR models have emerged as a promising paradigm by connecting pre-trained audio-visual encoders to an LLM, achieving strong results in clean conditions. However, these models are predominantly optimized for clean acoustic conditions, with limited attention to making the LLM backbone robust to noise. No expl

7. MAVIN: Multi-Shot Audio-Visual Generation with Narrative Control

Authors: Kaiqi Liu, Yunyao Mao, Ziqi Cai, Zheng Geng, Jing Wang et al.

Categories: cs.CV Score: 7.3/10 (Obj:8 Id:7 Ind:7 Comp:5 Eff:8 Nov:7)

• Strength: 首次系统性地解决了多镜头音视频联合生成与叙事控制问题，提出了完整的边界感知与身份保持机制，实际效果与工程落地价值高。
• Weakness: 方法属于典型的工程堆叠（多智能体脚本+注意力路由+ID传播），缺乏统一且优雅的理论洞察，系统复杂度较高。

中文摘要： MAVIN针对多镜头音视频生成中叙事控制复杂、时间对齐困难的问题，提出了首个支持自定义叙事控制的统一框架。该框架通过边界感知注意力机制（boundary-aware attention）利用层次化因果信息来解决时间错位，并引入了多镜头音视频联合建模方法以实现连贯的镜头间过渡与音画同步。实验表明，MAVIN在叙事连贯性、时间对齐和音画质量上均显著优于现有方法，是首个端到端完成多镜头音视频叙事生成的系统。

While recent generative models produce high-fidelity videos, they struggle with the complex narrative control required for coherent multi-shot audio-visual generation. Existing methods suffer from temporal misalignment, limited controllability, and incomplete scripting. In this paper, we propose MAVIN, the first framework for multi-shot audio-visual generation with customized narrative control. To resolve temporal misalignment, we propose boundary-aware attention, which leverages hierarchical ca

8. Proteus: Automated Adversarial Robustness Testing for Audio Deepfake Detectors

Authors: Nicolas M. Müller, Aditya Tirumala Bukkapatnam, Zohaib Ahmed

Categories: cs.SD, cs.AI Score: 7.2/10 (Obj:9 Id:6 Ind:8 Comp:5 Eff:8 Nov:6)

• Strength: 针对音频深度伪造检测器在真实信号处理下的脆弱性，提出了实用的自动化黑盒测试框架，能有效发现复合增强链漏洞并加固生产模型。
• Weakness: 方法本质是BFS和Q-learning在手工设计的增强空间上的工程堆叠，缺乏对检测器失效因果机制的深层理论洞察。

中文摘要： Proteus是Resemble AI提出的针对音频深度伪造检测器的自动化鲁棒性测试框架，核心问题在于现实环境中常见的音频处理操作（如编解码转码、加性噪声、混响、动态范围压缩和VoIP仿真）可能使检测器失效。给定一个检测器，Proteus系统地搜索这些日常音频变换的序列组合，在保持语音质量的前提下找到能欺骗检测器的变换链路。该方法提出两种互补的搜索策略：广度优先搜索可穷举地探索变换组合空间，另一种互补策略则针对更高效的搜索路径进行优化。通过将多种良性音频处理串联叠加，Proteus能够揭示检测器在真实通信链路下的脆弱性边界。该框架为音频伪造检测系统的部署前鲁棒性评估提供了系统化、可复用的测试手段。

We present Proteus, a framework developed at Resemble AI for automated robustness testing of our audio deepfake detection system. Given a detector, Proteus systematically searches over sequences of everyday audio transformations (codec transcoding, additive noise, reverberation, dynamic-range compression, and VoIP simulation) to find combinations that fool the detector while preserving speech quality. We propose two complementary search strategies: (1) a breadth-first search that exhaustively ma

9. EchoHawk: A Reproducible Acoustic Pipeline for Drone Detection, Classification, and Direction-Finding, with a Cautionary Study of Session-Level Data Leakage

Authors: David Shulman

Categories: cs.SD, physics.app-ph Score: 7.0/10 (Obj:9 Id:8 Ind:9 Comp:8 Eff:6 Nov:6)

• Strength: 精准识别并量化了公共无人机音频数据集中严重的会话级数据泄漏问题，强调了评测的独立性与诚实性，工程复现性极佳。
• Weakness: 核心方法依赖经典信号处理与传统机器学习的组合，模型层面创新性有限；绝对性能指标未超越现有SOTA，主要贡献为基准纠偏而非效果突破。

中文摘要： 针对反无人机防御中被动声学感知的应用，本文提出了EchoHawk——一个开放且完全可复现的参考流水线，能够通过旋翼谐波检测无人机、估计叶片通过频率，并利用麦克风阵列结合经典宽带波束成形方法（延时求和、MVDR、MUSIC）与时延估计实现目标定位。该工作强调可复现性与基线方法的系统对比，为反无人机声学感知提供了开放参考基准。此外，论文揭示了机器学习中常见的会话级数据泄露问题，警告在声学无人机检测任务中若按会话划分训练/测试集而非严格按实例划分，会导致性能指标被显著高估。这一警示性研究提醒研究者在评估模型时需谨慎处理数据泄露，以确保结果的真实性与可比性。

Passive acoustic sensing is an attractive modality for counter-unmanned aerial system (counter-UAS) defence: it is covert, low-cost, and effective against drones with small radar cross-sections or minimal radio emissions. We present EchoHawk, an open and fully reproducible reference pipeline that detects a drone from its rotor harmonics, estimates its blade-passing frequency, and localises it with a microphone array via classical wideband beamforming (delay-and-sum, MVDR, MUSIC) and time-delay p

10. VeRe-Flow: Guiding Flow Matching toward Clean Speech via Velocity Contrastive Regularization and Representation Alignment for Noise-Robust Bandwidth Expansion

Authors: Sujin Koo, Sangyoon Kim, Ji Sub Um, Hoirin Kim

Categories: eess.AS | Accepted to Interspeech 2026 Score: 6.8/10 (Obj:9 Id:7 Ind:8 Comp:4 Eff:8 Nov:4)

• Strength: 在噪声鲁棒带宽扩展任务上取得了出色的实证效果，LSD和DNSMOS等核心指标全面超越基线。
• Weakness: 核心组件均直接借用已有工作（VeCoR和表征对齐），属于在特定任务上的工程性堆叠，缺乏本质的方法论创新与洞察。

中文摘要： 本研究针对噪声环境下低分辨率语音带宽扩展中速度估计模糊的问题，提出VeRe-Flow框架，通过引入多层级干净语音监督来引导流匹配生成过程。该方法结合速度对比正则化与表征对齐机制，利用干净语音的速度场作为参考信号约束生成轨迹，从而提升噪声鲁棒性。实验表明，VeRe-Flow在多项客观指标和主观听感上均显著优于现有方法，实现了从含噪窄带输入重建高保真宽带语音的目标。

Noise-robust bandwidth expansion aims to reconstruct high-fidelity wideband speech from noisy low-resolution inputs. While flow matching has shown strong performance in speech generation, accurately recovering clean speech from noisy inputs remains challenging due to the ambiguity of velocity estimation under noise. In this work, we propose VeRe-Flow, a clean-guided flow matching framework that introduces multi-level clean supervision to guide the generative process toward clean speech. At the v