Daily Papers — 2026-07-08

10 papers on audio, speech, music, and acoustics.

1. Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning

Authors: Yuxin Li, Donghang Wu, Guan-Ting Lin, Hung-yi Lee, Chengwei Qin et al.

Categories: eess.AS Score: 6.60/10 (Obj:8 Id:6 Ind:5 Comp:8 Eff:8 Nov:6 Rep:6)

Strength: DuplexPO 提出了一个清晰的 RL 框架来解耦全双工语音对话中的 timing 决策与语义生成，FCDR 的 factorized reward 设计和 dynamics-critical window sampling 是有解释力的方法贡献。实验在 dynamics 和 intelligence 两个维度上都有说服力，FDB-v3 上打破 latency-VIR trade-off 是亮点。
Weakness: (1) 缺少与最直接同类工作 ASPIRin 的实验对比；(2) “user experience”声称仅依赖 LLM judge，无真实人类评测，存在循环论证风险；(3) 基础模型 Nemotron-VoiceChat 不可获取，影响复现。
Full review: Claude Code 全文七公理审稿

Recent full-duplex spoken dialogue models have demonstrated compelling progress toward human-like interaction, enabling agents to respond with low latency, produce backchannels, and handle user barge-ins. Yet these improvements in conversational dynamics often come with weaker reasoning and instruction-following abilities, revealing a potential tension between interactive dynamics and intelligence capability. In this paper, we argue that such an intelligence–dynamics trade-off is not fundamental: conversational dynamics can instead be learned as a separate real-time decision policy from human dialogue data. To this end, we propose DuplexPO, a reinforcement learning (RL) framework that decouples when to speak from what to say. It preserves the semantic response capability of an instruction-tuned assistant, while optimizing its temporal interaction behavior over selected high-impact windows from long human conversations. To quantitatively optimize these dynamics, we formulate the Factorized Conversational Dynamics Reward (FCDR) to enable fine-grained temporal credit assignment for turn initiation, backchanneling, yielding, and regularized participation. The policy is then optimized with a GRPO-style objective. Experiments show that DuplexPO substantially improves full-duplex behaviors, including timely backchannels, smooth turn-taking, and barge-in handling, while maintaining strong reasoning and instruction-following performance. Moreover, improvements in dynamics-oriented metrics are reflected in better user experience, suggesting that optimizing conversational timing as a standalone objective can promote more natural full-duplex interaction.

2. MADB: A Large-Scale Music Aesthetics Dataset with Professional and Multi-Dimensional Annotations

Authors: Sirui Zhang, Tianle Wang, Xinyi Tong, Peiyang Yu, Jishang Chen et al.

Categories: cs.SD, cs.AI Score: 6.30/10 (Obj:8 Id:6 Ind:5 Comp:5 Eff:5 Nov:6 Rep:8)

Strength: 低 — 评测框架是标准方法的应用，无新方法贡献。CLAP 语义适应是标准对比学习的直接应用。
Weakness: 数据标注和评测之间存在部分循环。30 名标注者同时提供数值评分、文本评论和标签——这些信号高度相关。在 CLAP+C 和 CLAP+C&T 实验中，文本评论被用于预适应阶段的对比学习，而下游评测的目标正是同一批标注者给出的美学评分。虽然论文声称”适应阶段不使用美学评分以避免标签泄漏”，但评论本身就编码了美学判断（Table 3 中 Comment&Tag 配置的 LCC 达 0.736），实质上是同一信息源的不同表示。
Full review: Claude Code 全文七公理审稿

Music aesthetic assessment is a challenging yet underexplored problem, requiring models to capture fine-grained, multi-dimensional human perceptual judgments. Progress in this area has been limited by the lack of large-scale datasets with structured aesthetic annotations. We introduce MADB, a large-scale dataset and benchmark comprising 9,999 tracks annotated by 30 trained annotators. Each track is rated by around 10 annotators across 10 perceptual dimensions and one overall score, with additional textual comments for multimodal analysis. We establish a unified evaluation framework over multiple pretrained models. Results reveal substantial gaps between model predictions and human judgments, exposing key limitations of current approaches. MADB provides a new benchmark for human-aligned music understanding. Project page: https://github.com/knownree/madb

3. Text-Independent Speaker Verification Using Discrete Audio Tokens

Authors: Zheng Liang, Junjie Li, Kong Aik Lee

Categories: eess.AS | This paper has been accepted by Interspeech 2026 Score: 6.20/10 (Obj:8 Id:8 Ind:8 Comp:6 Eff:6 Nov:5 Rep:6)

Strength: CFKD 通过 embedding-level cosine KD 让 EnCodec token-based ASV 在 VoxCeleb2 上达到 EER 1.05%（24kbps），相比 Codec-ASV (EER 2.08%) 有 49.5% 相对改善，且诊断实验（E1-E3）有效证明了”信息保留但训练困难”的核心假设。
Weakness: 核心方法（cosine-based embedding-level KD）在 ASV 中已有多篇先前工作（Jung 2019, Wang 2019），跨特征蒸馏仅为配置增量；token-based student 追平但未超越 Fbank teacher，且未与 VoxCeleb SOTA (<1% EER) 竞争。
Full review: Claude Code 全文七公理审稿

Neural audio codecs (NACs) enable efficient audio compression and have achieved success in downstream tasks such as speech synthesis. However, their discrete representations consistently underperform traditional spectral features in automatic speaker verification (ASV). We empirically demonstrate that speaker cues are implicitly preserved in discrete tokens but remain underutilized by conventional ASV training paradigms. To address this, we propose a Cross-Feature Knowledge Distillation (CFKD) framework. By guiding the codec-based student to mimic the embedding space of a strong Fbank-based teacher, CFKD provides structured supervision for effective utilization of speaker information in tokens. Experiments on the VoxCeleb benchmarks show that CFKD substantially improves the ASV performance of codec-based systems, allowing them to approach the accuracy of Fbank-based teacher models and highlighting the potential of discrete audio tokens for diverse speech tasks.

4. A Reliability Assessment of LALM Audio Judges for Full-Duplex Voice Agents

Authors: A. Sayyad, J. Emmons, S. Jones, T. Lin, H. Krishnan

Categories: cs.CL, cs.AI, cs.SD, eess.AS | 28 pages total (12 main body, 1 reference, 15 appendix). In main body: 2 diagrams, 3 table, 2 charts Score: 6.00/10 (Obj:8 Id:6 Ind:5 Comp:8 Eff:6 Nov:5 Rep:8)

Strength: 首次对全双工语音代理的 LALM 音频 judge 进行多评分者人类验证，在 5/8 维度上 LALM-human Spearman ρ 与 human-human ρ 差距 ≤0.07，跨 3 个 Gemini 模型复制了 rank-ordering 能力，并发现 rank-correlation parity 不等于 calibration parity 这一可迁移经验规律。
Weakness: 证据基础狭窄——仅 3 名人类评分者（rater-mean spread 超 1 分）、单一模型家族（Gemini）跨模型验证不构成真正独立性、对抗性分析 41/48 “无差异” cells 实为 underpowered nulls 而非 demonstrated parity，且 2/8 维度 LALM 一致性低于 50% 无法部署。
Full review: Claude Code 全文七公理审稿

We report the empirical reliability of Gemini models as audio judges that score full-duplex agent conversations directly from the raw stereo waveform, tested across three models in the Gemini family: 2.5 Flash, 3.5 Flash, and 3.1 Pro. Our primary evidence base uses Gemini 2.5 Flash as the ground-truth model, validated against three calibrated human raters on 209 stereo sessions, scored on 8 production dimensions: 152 full-duplex conversations across 13 accent-and-condition strata, together with 57 adversarial defect-injected clips. The evidence for Gemini 2.5 Flash is consistent across three tests. (i) On 5 of 8 dimensions the LALM-human Spearman rho departs from the pairwise human-human rho by at most 0.07, and on 7 of 8 dimensions the two quantities 95 percent bootstrap confidence intervals overlap. (ii) The LALM agrees with the three-rater human mean within 1 point on 60 to 92 percent of sessions on 6 of 8 dimensions. (iii) On 45 of 48 (defect, dimension) cells the LALM is as sensitive as humans or better under Newcombe-Wilson 95 percent confidence intervals, though most of these are underpowered nulls rather than demonstrated parity. Rank-ordering ability transfers across the Gemini family: 3.5 Flash improves simple agreement to 8 of 8 dimensions, while 3.1 Pro rates several dimensions markedly lower than humans despite comparable rank correlation. A model swap should be re-validated on calibration specifically, not assumed from rank-correlation alone. We identify four areas where deployment requires care, and we estimate that human rating alone for our current evaluation cadence costs roughly two orders of magnitude more than the equivalent LALM workload. The data presented here provides a defensible empirical basis for deploying the LALM as a substitute or fourth rater on the dimensions where the evidence supports it.

5. MMGenre: Benchmarking Singing Voice Synthesis across Multiple Musical Genres

Authors: Wenhao Feng, Yuxun Tang, Jiatong Shi, Qin Jin

Categories: cs.SD | Accepted by Interspeech 2026. Camera-ready version. 4 pages, 5 figures.Project page: https://fengjin1117.github.io/mmgenre-demo/ Score: 6.00/10 (Obj:8 Id:6 Ind:5 Comp:6 Eff:5 Nov:7 Rep:5)

Strength: 无新方法提出，pipeline 是已有工具的模块化组装，fine-tuning 实验设计简单且仅覆盖单一流派。
Weakness: 存在多重独立性隐患：(1) 数据生成与评测闭环：benchmark 数据由 Suno V4.5（闭源 T2M 系统）生成，评测由 Gemini 2.5 Pro（闭源 LLM）打分。两个系统均为闭源商业模型，无法排除它们共享某些训练数据分布或偏好的可能。
Full review: Claude Code 全文七公理审稿

Singing voice synthesis (SVS) has progressed rapidly, yet its ability to generalize across diverse musical genres remains underexplored. Existing benchmarks are heavily biased toward pop music, limiting systematic analysis of genre-dependent behavior. We introduce MMGenre, a benchmark for multi-genre SVS diagnosis, supported by an automatic pipeline for constructing genre-aligned music scores. MMGenre spans 10 major genres and 26 subgenres, enabling comprehensive analysis of genre-aware synthesis. Extensive evaluation of representative SVS models reveals limited genre discrimination: synthesized vocals across genres exhibit highly similar acoustic characteristics and weak separability. While zero-shot genre adaptation yields only marginal improvements, lightweight genre-specific continued training leads to substantial gains. MMGenre provides a standardized framework for multi-genre SVS evaluation and exposes critical challenges in achieving genre-aware singing voice synthesis.

Authors: Antonio Cano, Guillermo Pérez, Luis Merino, Randy Gomez

Categories: cs.RO, cs.AI, cs.CL | Accepted for presentation at the 35th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2026). Acceptance notification date: 30 May 2026. Final published version pending Score: 5.60/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:5 Nov:4 Rep:8)

Strength: 使用 VA 相关预训练 backbone (TalkNet/WhisperFlamingo) + LoRA 适配，在 NoXi All 上 S/H F1=0.97、S-pred F1=0.97，显著超越 Onishi et al. 的 S/H=0.90、S-pred=0.72，且代码开源、多语言验证 (5种语言) 覆盖全面。
Weakness: BC-pred F1 在多数语言子集上 (0.43-0.52) 全面落后于 Onishi et al. (0.71)，且缺少 semantic consistency loss 和 inter-speaker attention 的消融实验，无法确认这两个新增组件的贡献；与 Saga et al. (2506.03980) 的核心 idea 高度重叠 (预训练 encoder + VAP)，真正的增量有限。
Full review: Claude Code 全文七公理审稿

Turn-taking prediction is a key requirement for social robots involved in human-human interaction, particularly in mediator settings, where the robot must anticipate conversational dynamics rather than merely react to pauses. This work presents a Multimodal Voice Activity Projection (MM-VAP) framework that extends the original audio-only VAP formulation to synchronized audio-visual inputs while preserving its self-supervised future-projection objective. The proposed approach builds on pretrained audio-visual backbones originally optimized for speech-related tasks and adapts them through Low-Rank Adaptation to the multimodal turn-taking problem. After independent speaker encoding, an inter-speaker attention stage models the relational dynamics required to project future voice activity. In addition, a semantic consistency loss is introduced to regularize the 256-state output space according to higher-level dialogue activity patterns. Experiments on NoXi and NoXi+J showed improvements over the current baselines, particularly for some turn-taking events. Additional evaluation on the Haru EDR corpus further supported the suitability of this direction for mediation-oriented human-robot interaction.

7. Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks

Authors: Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu, Vaidehi Patil

Categories: cs.LG, cs.AI, cs.CL, cs.CR, cs.MM | Accepted to ACL Findings 2026 Score: 5.26/10 (Obj:8 Id:5 Ind:6 Comp:5 Eff:5 Nov:4 Rep:8)

Strength: 首篇明确覆盖 vision+language+video+audio 四模态的多模态遗忘综述，被 ACL Findings 2026 接收，附有社区资源库，各模态遗忘需求经外部证据确认真实存在。
Weakness: 相对于已有 MU 综述（ACM Computing Surveys 2023, IEEE TNNLS 2025）和 MU-Bench (2024) 的增量有限，跨模态深度综合分析不足，缺乏系统性方法比较表格和 benchmark 质量评估。
Full review: Claude Code 全文七公理审稿

With the growing adoption of VLMs, DMs, LLMs, and AFMs, these multimodal foundation models can inadvertently encode sensitive, copyrighted, biased, or unsafe cross-modal associations that originate from their training data. Retraining after deletion requests or policy updates is often impractical, and targeted forgetting remains difficult because knowledge is distributed across shared representations. Multimodal unlearning addresses this challenge by enabling selective removal across modalities while retaining overall utility. This survey offers a unified, system-oriented view of multimodal unlearning across vision, language, audio, and video, grounded in recent advances, emerging applications, and open problems. Our taxonomy enables systematic comparison across model architectures and modalities, clarifying trade-offs among deletion strength, retention, efficiency, reversibility, and robustness. This survey highlights open problems and practical considerations to support future research and deployment of multimodal unlearning. We release a curated repository: https://smsnobin77.github.io/Awesome-Multimodal-Unlearning/

8. UBG-Net: An Uncertainty-aware Bayesian Gating Network for Robust Audio-Visual Speech Recognition

Authors: Jinjie Fu, Hang Chen, Wu Guo, Zhijun Zhang, Kuiliang Li et al.

Categories: eess.AS Score: 5.05/10 (Obj:8 Id:5 Ind:7 Comp:5 Eff:5 Nov:4 Rep:5)

Strength: 在 AVCocktail Fixed Chunk 设置下实现 7.9% 相对 WER 降低（39.2→36.1），MUBF 机制在低 SNR（-5 dB）场景下改善明显（6.4→4.2 WER）。
Weakness: 仅对比一条专用 baseline [8]，跳过 LRS3，高 SNR/clean 条件下多处退化，DoRA 贡献未隔离消融，联合 aleatoric+epistemic 建模的 novelty 相对 Depeweg et al. (2018) BNN+LV 有限，代码未开源。
Full review: Claude Code 全文七公理审稿

Audio-Visual speech recognition systems often degrade in real-world scenarios due to signal corruption and distribution shifts. To address this, we propose a unified uncertainty-modeling framework, namely the uncertainty-aware Bayesian gating network (UBG-Net). UBG-Net features a Modality Uncertainty-aware Bayesian Fusion (MUBF) mechanism that injects signal-level aleatoric uncertainty into a Bayesian network to model epistemic uncertainty, thereby ensuring robust fusion of pre-trained backbone features. For inference, we introduce Distribution Uncertainty-aware Hierarchical Voting (DUHV) to select transcripts from Monte Carlo samples, prioritizing frequency and using inference scores in case of a tie. Experiments on the AVCocktail and LRS2 datasets demonstrate the overall superiority of UBG-Net compared to SOTA baselines. Ablation studies confirm that MUBF and DUHV effectively filter noise, enhancing fusion and decoding robustness.

9. Rag Classification of Tagore Songs using Symbolic Music Notation and Novel Weighted Distance Measures

Authors: Chandan Misra, Swarup Chattopadhyay

Categories: cs.SD Score: 4.26/10 (Obj:6 Id:5 Ind:6 Comp:5 Eff:3 Nov:3 Rep:4)

Strength: 低 — “Novel Weighted Distance Measures” 实为标准加权欧氏距离 + 领域知识权重，命名膨胀严重。未提供新机制解释或问题重构。
Weakness: 效果评估存在严重不足
Full review: Claude Code 全文七公理审稿

Rabindra Sangeet, the body of songs written and composed by Rabindranath Tagore, occupies a distinctive position in Indian music by combining poetic expression with melodic ideas drawn from Hindustani rags, Bengali folk traditions, tappa, kırtan, Baul music, and Western tunes. Although many Tagore songs are associated with rag labels provided by Tagore himself or preserved in authoritative notational traditions, rag identification remains challenging because the songs often reflect creative freedom rather than strict adherence to classical rag grammar. This paper formulates rag identification in Rabindra Sangeet as a supervised classification problem using symbolic music-sheet notations from Swarabitan. Since large-scale annotated audio or music datasets for Rabindra Sangeet are not readily available, this study constructs a rag-labelled symbolic dataset from notated Tagore songs. The work investigates Euclidean distance and cosine similarity for rag classification and introduces a weighted Euclidean distance measure that assigns greater importance to notes belonging to characteristic rag sequences such as arohana and avarohana. Applied within a k-nearest-neighbour framework, the proposed measure improves rag classification by better capturing rag-specific melodic identity.

10. EscFOA: Enhancing Spatial Learning for Visually Impaired Learners via Generative Spatial Audio in 360-Degree Educational Environments

Authors: Ziyu Luo, Xiaowei Dai, Siying Zhu, Xiaoming Chen

Categories: cs.SD Score: 2.30/10 (Obj:5 Id:2 Ind:5 Comp:2 Eff:2 Nov:1 Rep:2)

Strength: 针对真实的社会需求（360度教育环境中视障学习者空间认知），在32名蒙眼被试的用户研究中，EscFOA在4项5分制MOS指标上全面优于Mono/Stereo（Overall Preference: 4.25 vs 2.86/3.58），定性轨迹分析显示更平滑的导航路径。
Weakness: 方法完全来自同一作者前序工作DynFOA (arXiv:2602.06846)，无新模块/ablation/因果识别；baseline仅比较Mono/Stereo，未与OmniAudio/Diff-SAGe/DynFOA等SOTA空间音频方法对比；声称测量learning outcomes和cognitive load但实际未测量任何认知负荷或空间知识获取指标；无代码/模型/checkpoint开源，3页正文信息不足以复现。
Full review: Claude Code 全文七公理审稿

Immersive 360-degree educational environments often lack accessible spatial structure, limiting visually impaired learners’ ability to orient, explore, and construct mental representations. This paper proposes EscFOA, a geometry-aware spatial audio generation framework designed as an \emph{acoustic scaffolding} to support spatial cognition. By integrating 3D Gaussian Splatting (3DGS) with conditional diffusion models, EscFOA reconstructs scene geometry from 360-degree videos to synthesize high-fidelity spatial audio consistent with the environmental structure. Explicitly targeting learning outcomes like independent spatial orientation and reduced cognitive load, EscFOA significantly outperforms conventional monaural and stereo audio in supporting spatial learning behaviors among blindfolded sighted participants (simulating visually impaired learners). These findings demonstrate that geometry-consistent generative audio can effectively enable inclusive access to complex spatial learning materials.