每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-07-29
日期2026-07-29
已评分—
均分—
最高—

Daily Papers — 2026-07-29

17 papers on audio, speech, music, and acoustics.

1. Prosody-driven Jailbreaks in Audio LLMs: A Controlled Study and Mechanistic Analysis

Authors: Jiachen Qian, Junyu Li

Categories: cs.SD, cs.CL | Accepted at ACM Multimedia 2026 (ACM MM ‘26). 9 pages, 3 figures. Supplementary material included Score: 6.90/10 (Obj:8 Id:7 Ind:5 Comp:8 Eff:7 Nov:7 Rep:5)

  • Strength: 在固定 transcript 的受控设计下，首次量化证明 prosody alone（44/95 ASR）远超 emotional text alone（11/95），且在 matched-budget 下显著超越 StyleBreak（27/95, p<0.001），将 speech delivery 确立为 Audio LLM 安全的一等风险因素。
  • Weakness: Benchmark 和代码不公开、Llama Guard 3 在 judge 和 defense 中双重使用导致独立性缺陷、95 seeds 规模偏小且仅 Qwen2-Audio 有 exact-count audited 结果，限制了社区可复用性和结论泛化性。
  • Full review: Claude Code 全文七公理审稿

Audio-capable foundation models enable end-to-end spoken interaction, but they also introduce safety risks beyond transcript content. It remains unclear how much jailbreak capability can arise from matched-text variation in speech delivery rather than from lexical rewriting or broader style transfer. We study this question by holding transcript content fixed and varying six speech-delivery presets whose acoustic attributes may co-vary. We present PJ-Break, a black-box evaluation protocol with presets targeting arousal, authority, and speaking rate, together with AdvAudio-Prosody, a 600-sample benchmark with acoustically verified attributes. On the exact post-QC Qwen2-Audio panel, the Q=1 Panic (38/95), Anger (35/95), and Fast (32/95) presets are all well above Neutral (4/95). The fixed six-query pool covers 44/95 Qwen2-Audio seeds and 15/95 GPT-4o seeds and exceeds a matched-budget StyleBreak reimplementation (27/95) on Qwen2-Audio. A same-voice pool excluding the confounded Commanding condition still reaches 40/95, and a retained-panel ablation shows emotional-delivery audio alone (44/95) is far more effective than emotional text alone (11/95). Exploratory surrogate diagnostics and pilot mitigation observations are secondary, non-core analyses. Overall, matched-text speech delivery should be treated as a first-class factor in Audio LLM safety evaluation

2. ThinkOmni: A Reasoning-Driven Omni-Modal LLM Framework for Audio Forgery Detection and Localization

Authors: Yuxiong Xu, Kaiqing Lin, Bin Li, Haodong Li, Sheng Li

Categories: cs.SD | Accepted by ACM MM 2026, 21 pages, 12 figures Score: 6.40/10 (Obj:8 Id:6 Ind:5 Comp:5 Eff:8 Nov:5 Rep:8)

  • Strength: 跨数据集泛化显著优于 13 个 SSL/ALLM baseline（检测 ACC 80.74% vs 最强 76.17%，定位 mAP 74.67% vs 最强 59.35%），FACoT 100K 数据集和 9 维取证标注 schema 为音频取证社区提供了新基础设施。
  • Weakness: CoT 标注 pipeline 是 label-aware 的（Gemini 生成 rationale 时已知标签和边界），引入循环论证隐患；方法本质是已有技术（CoT + 多模态 LLM + 渐进式训练 + 加权损失）的系统性组合，缺少最简 CoT prompting baseline 隔离数据贡献 vs 方法贡献。
  • Full review: Claude Code 全文七公理审稿

Existing audio forgery detection and localization (AFDL) methods often overfit dataset-specific low-level artifacts, limiting their generalization to subtle, localized, and unseen manipulations. Recent audio large language model (ALLM)-based approaches cast AFDL as question answering but still model forensic evidence implicitly, without linking manipulation cues to predictions. To bridge this gap, we propose ThinkOmni, a reasoning-driven omni-modal large language model that jointly performs explicit forensic reasoning, spoofing detection, and temporal manipulation localization. To enable explicit reasoning supervision, we construct Forensic-Aware Chain-of-Thought (FACoT), a 100K-sample dataset with structured forensic evidence and reasoning annotations. Leveraging FACoT, we introduce Forensic-Aware Modality-Incremental Learning (FMIL), which progressively aligns semantic, acoustic, and spectral-visual representations with the LLM backbone to capture complementary forensic cues. We further propose Forensic-Consistent Multi-task Loss (FCML), which combines weighted cross-entropy with an adaptive localization loss to coordinate reasoning generation, spoofing detection, and temporal localization. Extensive experiments show that ThinkOmni achieves strong cross-dataset generalization in both detection and localization. Code, models, data, and inference examples are available at https://beyond0814.github.io/ThinkOmni/.

3. Voice Memory for Agentic Speech Recognition

Authors: Chao-Han Huck Yang, Zih-Ching Chen, Piotr Zelasko, Zhehuai Chen, Jagadeesh Balam et al.

Categories: cs.CL, cs.AI, cs.SD, eess.AS | Preprint. Technical report and open source: https://huggingface.co/huckiyang/voice-memory Score: 6.32/10 (Obj:8 Id:6 Ind:8 Comp:8 Eff:6 Nov:5 Rep:8)

  • Strength: Voice Memory 用 <10KB 文本记忆实现 inference-only ASR correction，在 HyPoradise 10 域上将 weighted WER 从 8.36% 降至 7.52%，ATIS 从 8.40% 降至 3.40%，且无任何数据集回归低于 1-best baseline；核心发现”restraint is the operative skill”由 HER 从 0.64 降至 0.35 量化支。
  • Weakness: 核心机制是 text-prompt optimization + GER cascade 的组合，与同一作者群的 “Evolutionary Prompt Design for Post-ASR Error Correction” 高度重叠且未引用；缺少 confidence-threshold abstain 最简 baseline 和 contextual biasing 强 baseline 的直接比较；翻译实验仅 40 utte。
  • Full review: Claude Code 全文七公理审稿

We present Voice Memory, a inference-only scheme for agentic speech recognition: at stream time, a frozen corrector reads a single per-domain memory.md and decides per utterance whether to act on the hypothesis or abstain and keep the 1-best. Asynchronously, a score-gated optimizer revises that file through bounded edits, accepting an edit only when it strictly improves a held-out score. Extended from classical ASR-LM framework, we refer this split the listener-thinker architecture; the two roles are coupled only through the memory, so no weights change and the learned skill stays auditable and portable. Restraint turns out to be the operative skill this loop discovers: unconstrained generative error correction (GER) over-corrects, breaking correct tokens on up to 64% of its edits on financial news, and Voice Memory, reduces this rate to 35%. Across ten HyPoradise domains with an open corrector, Voice Memory, lowers weighted word error rate from 8.36% to 7.52% (7.47% with three added in-context examples) without regressing any dataset below its 1-best baseline; gains concentrate where recoverable headroom is largest, including air-travel commands (8.40% to 3.40%) and noisy far-field speech (CHiME-4, 12.69% to 10.46%). The memory transfers across corrector families and adds zero parameters to the inference path. A demo and example code are provided for future studies.

4. MPEcho: A Melody and Phoneme-Aware Generative Framework for Controllable Cover Song Generation

Authors: Wei-Jaw Lee, Hsuan-Yu Yeh, Ting-Yi Hu, Chih-Pin Tan, Fang-Duo Tsai et al.

Categories: cs.SD, cs.AI, eess.AS | Accepted by the 27th International Society for Music Information Retrieval (ISMIR) Score: 6.32/10 (Obj:8 Id:6 Ind:6 Comp:5 Eff:8 Nov:5 Rep:7)

  • Strength: 在 SongEcho 框架中引入 SVS-style phoneme encoder + length regulator，将 CSG 的 PER 从 45.62% 降至 18.65%，同时保持旋律一致性 (RPA 0.5764 vs 0.5779)，消融实验清晰地隔离了 melody 和 phoneme 条件的互补贡献。
  • Weakness: 核心 phoneme encoder + LR 组合直接来自 FastSpeech (2019)，方法 novelty 有限；未与 JAM/SegTune 做模型级直接对比；训练用内部 1,427 小时数据集不公开，影响独立验证。
  • Full review: Claude Code 全文七公理审稿

Cover song generation (CSG) should preserve the melodic and linguistic content of a reference song while recreating the remaining musical components. The state-of-the-art model SongEcho utilizes $F_0$ sequences and voiced/unvoiced (V/UV) tags for conditioning; however, implicit linguistic information from V/UV tags cannot guarantee lyric accuracy, leading to a high phoneme error rate (PER). Inspired by singing voice synthesis (SVS), we propose MPEcho, which integrates a phoneme encoder and a length regulator (LR) into the SongEcho framework. By providing explicit phoneme-level conditioning and precise temporal boundaries, MPEcho significantly reduces PER. To enable this, we developed Phonsa, a Whisper-based automatic transcription model that provides high-precision phoneme-level annotations for singing voices, overcoming the scarcity of high-quality audio-phoneme pairs. Experimental results validate the effectiveness of Phonsa for alignment and MPEcho for end-to-end CSG. The audio samples, code and weights can be accessed from https://lonian6.github.io/MPEcho.github.io/.

5. Detection of AI-generated stems within hybrid human-AI music

Authors: François Rigaud, Gabriel Meseguer-Brocal, Benjamin Martin, Romain Hennequin

Categories: cs.SD | Accepted at ISMIR 2026 Score: 6.26/10 (Obj:8 Id:5 Ind:5 Comp:8 Eff:5 Nov:6 Rep:8)

  • Strength: 提出 stem-level AI 生成检测新任务，用极简 MLP（17 维输入）将 mix-level 检测分数 + stem 能量比重校准为 stem-level 后验，在伴奏检测 0dB 场景下将 naive baseline 的 FPR 从 94.7% 降至 26.2%。
  • Weakness: 仅用 encodec 模拟评测，vocals 检测在低 gain 场景 TPR<40%/FPR>26%，缺少 ft 与 SNR 各自贡献的消融实验，且 Demucs 在 MUSDB18-HQ 上训练构成数据泄露风险。
  • Full review: Claude Code 全文七公理审稿

This paper presents, to the best of our knowledge, the first study on detecting human-AI hybrid music tracks created by mixing human-produced and AI-generated stems. Building on recent work showing that AI music detectors can identify decoder-related artifacts in fully generated music, we investigate whether such artifacts remain detectable at the stem level after mixing. Using MUSDB18-HQ database in a two-stem vocals + accompaniment setting, we simulate hybrid mixtures by autoencoding individual stems with a neural codec. We compare two strategies combining AI-generated mix detection and source separation. A naive sequential pipeline, where source separation is followed by detection on separated sources, confirms that artifacts associated with an AI-generated stem are not reliably recovered by generic source separation systems. We therefore propose a parallel architecture in which source separation is only used to estimate source-relative energy within the mixture. We then train simple stem-specific binary classifiers that take as input the generated mix prediction together with the relative energy of the target stem on short audio chunks. Averaging chunk-level predictions yields encouraging track-level results, highlighting the potential of such approaches for detecting AI-generated stems in hybrid music.

6. Ripple: Real-Time Streaming Audio-Video Generation With Cross-Modal Recurrent Memory

Authors: Yanbo Ding, Zhizhi Guo, Quanyue Song, Yishan He, Zhixiang He et al.

Categories: cs.CV Score: 6.10/10 (Obj:8 Id:6 Ind:5 Comp:8 Eff:6 Nov:7 Rep:5)

  • Strength: Ripple 通过 cross-modal recurrent memory 实现固定成本流式音视频生成，28 FPS/480P、15× 加速，在长视频 AV Quality 和 ID Consistency 上超过 19B 教师模型，并首次将在线 RL 后训练应用于流式音视频生成。
  • Weakness: RL 奖励函数与评测指标全面重叠导致循环论证风险，长视频比较仅对照教师模型而排除竞品，Speech Align 和 AV Synchronize 在长视频上输给教师，代码和部分数据未开源。
  • Full review: Claude Code 全文七公理审稿

Audio-video generative models achieve impressive quality but suffer from high latency, making them unsuitable for real-time applications. Although several streaming audio-video generation methods have been proposed, they remain costly and fail to support long-form generation. To address this, we propose \textbf{Ripple}, a real-time joint audio-video generation system with a cross-modal recurrent memory mechanism. To enable efficient streaming inference while preserving long-term context, Ripple combines a fixed-length sliding-window attention with modality-specific memory states that continuously summarize audio and video context. Cross-modal memory interaction is further introduced to enhance audio-visual synchronization. To learn this memory-augmented model effectively, we devise a three-stage training recipe: (1) adapting a bidirectional audio-video teacher to block-wise causal attention with simulated memory, (2) optimizing the memory construction and interaction pipeline through end-to-end distillation, and (3) applying online reinforcement post-training tailored for streaming audio-video generation. As a result, Ripple achieves ~28 FPS at 480P resolution, over faster than the teacher, while capable of coherent long-form generation. Extensive experiments on both short-video and long-video benchmarks demonstrate our superior performance over existing offline and online joint audio-video generation methods.

7. Latent-IM: Latent Interaction Management for Speech LLMs

Authors: Adar Avsian, Atahan Dokme, Tony Woo, Larry Heck

Categories: cs.CL Score: 5.90/10 (Obj:8 Id:8 Ind:6 Comp:6 Eff:6 Nov:5 Rep:5)

  • Strength: 将经典对话管理的状态-动作分解在冻结语音 LLM 残差流中恢复，通过 streaming controller + activation steering 实现 selection→realization 管线，oracle realization 在 3 个数据集 × 3 个模型上平均 60.0%，超越最强 baseline 10.4 个百分点，端到端 move accuracy 46.9% 与 LoRA fine-tuning 持平。
  • Weakness: 核心方法是 probing + Fisher-LDA activation steering 的组合应用，5 类动作中 reply 的 controllability 仅 0.14 导致必须 gating 回避；主数据集 MapTask 上 selection accuracy 0.62 低于 Markov-1 baseline 0.65；自动评估器与部分生成模型同属 Qwen2.5 家族存在循环风险；无代码开源。
  • Full review: Claude Code 全文七公理审稿

Classical spoken dialogue systems often separated dialogue management from response realization: a policy selected the next dialogue action, and a generation component expressed that action. As dialogue systems shift toward LLMs, this decomposition has largely disappeared into the model’s hidden representations. We ask whether an LLM-internal analogue of state estimation and action control can be recovered for conversational moves such as acknowledging, checking, querying, explaining, and replying. We formulate move control as two coupled problems: selection, predicting the appropriate next move from the dialogue context, and realization, causally producing a chosen move at generation time. We introduce Latent-IM, an internal dialogue-management framework that provides a general interface for choosing and deploying conversational moves under different objectives. Here, we use this control to reproduce human move choices, improving average end-to-end move accuracy by 12.5 points over the unsteered backbone while performing comparably to fine-tuning.

8. A Study on Online Mask-based Beamforming Using Per-channel Masking for Spatially Distributed Microphones

Authors: Wiebke Middelberg, Svantje Void, Simon Doclo, Ryan Corey

Categories: eess.AS | Accepted for publication at IWAENC 2026 Score: 5.70/10 (Obj:8 Id:6 Ind:8 Comp:5 Eff:6 Nov:5 Rep:5)

  • Strength: 系统验证了 per-channel masking 在分布式麦克风+低 SNR+nearby noise 场景下相比 single mask 的鲁棒性优势（DNN-multi 在低 SNR 下持续优于 DNN-ref/mean），实验设计控制变量合理、评测独立。
  • Weakness: 核心方法增量有限（per-channel pre-filtering 已有先例 [3,15,16]），缺少最简 baseline 和 non-mask-based beamformer 比较，IRM-ref 与 IRM-multi 表现相当暗示收益来自 mask 误差鲁棒性而非方法本身，无代码发布。
  • Full review: Claude Code 全文七公理审稿

Mask-based beamforming is a popular geometry-agnostic approach for speech enhancement, typically applying a single mask across all microphones to estimate the required covariance matrices. While effective for compact arrays, this strategy may be suboptimal for spatially distributed microphones, where signal characteristics may vary strongly across microphones. To effectively capture the spatial diversity across microphones, we extend the mask-based beamformer to a multi-channel formulation, where each microphone is pre-filtered by a separate mask before covariance estimation. To address time-varying acoustic scenes, caused by spectro-temporal nonstationarity, we adopt a frame-causal online implementation with a sliding window. Experiments with simulated compact arrays and distributed microphones show that multi-channel masking yields a benefit over using a single mask when microphone signals differ substantially, while retaining similar performance in compact arrays. We further demonstrate the robustness of the multi-channel masking approach by comparing oracle ideal ratio masks to blind DNN-based mask estimation.

9. Explicit Note-Event Tokenization and Pitch-Validity Constrained Decoding for MIDI-to-Tablature Transcription

Authors: Ting-Kai Hsu, Wei-Chin Wang, Kai-Xi Hong, Yu-Hua Chen

Categories: cs.SD Score: 5.42/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:5 Nov:4 Rep:5)

  • Strength: 在 DadaGP 上相对 Fretting Transformer 提升 tab accuracy 5.32pp（71.78%→77.10%），pitch-validity constrained decoding 将 tab accuracy 进一步提升至 85.05%，并提供 error decomposition 分析。
  • Weakness: 未隔离 token 表示、架构和正则化的各自贡献（无消融），缺少 MIDI-to-Tab (2024) baseline，Leduc 上 FT 0.49% 疑似训练失败导致对比失真，代码尚未发布。
  • Full review: Claude Code 全文七公理审稿

Guitar tablature transcription predicts the string and fret position for each note so that the resulting tablature reproduces the target musical part. Prior sequence-to-sequence approaches have shown promising results on large-scale datasets, but their generalization behavior across different dataset scales remains less explored. In this work, we propose a guitar tablature transcription framework with explicit note-event tokenization and regularized training. The proposed decoder token representation incorporates note-event tokens together with TAB tokens, allowing note boundaries, pitch-related events, and string-fret positions to be represented more explicitly. We evaluate the proposed framework on DadaGP, a large-scale dataset, and Francois Leduc, a small-scale dataset. Our method improves tablature accuracy over the Fretting Transformer baseline on DadaGP, with especially strong gains when trained directly on the small-scale Leduc dataset. We further introduce a pitch-validity constrained decoding strategy that masks pitch-invalid TAB candidates during generation rather than correcting them after decoding and simultaneously preserves the original timing and note structure from the input. This constraint improves tablature accuracy and provides a controlled setting for measuring how much error remains after pitch-invalid predictions are removed. Our code will be released at:https://github.com/MusicGuitarTab/GuitarTab

10. Speech2Grasp: Data-Efficient Transfer of Text-Conditioned Grasp Detection to Speech in Humanoid Robots

Authors: Hung Nguyen, Kim Nhat Minh Nguyen, Van Duc Vu, Van-Danh Le, Hoang Huy Le et al.

Categories: cs.RO, cs.CV Score: 5.40/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:5 Nov:4 Rep:5)

  • Strength: 轻量 MLP projector 仅用 15K 样本即可将 ALBEF text-conditioned grasp detector 迁移到语音，real-world 成功率从 59% 提升至 70%，推理延迟降低 3 倍。
  • Weakness: 核心方法组件均有明确已有来源（AFD-SLU projector + 作者自身 DWT），缺少对 projector vs DWT 贡献的隔离消融，且 noise 实验中 baseline 未使用 DWT 导致比较不公平。
  • Full review: Claude Code 全文七公理审稿

Humanoid robots increasingly require multi-modal understanding for natural interaction with humans. Despite the prominence of vision-language models, they generally assume textual rather than the more natural speech inputs. In this paper, we investigate whether a well-established text-conditioned model can be transferred to speech in a data-efficient manner. Using ALBEF as a case study, we conduct diagnostic analyses showing that a lightweight MLP-based projector effectively adapts it to speech, while preserving semantic discrimination and robustness. Motivated by these findings, we introduce Speech2Grasp, a framework for data-efficient transfer of text-conditioned grasp detection to speech. Real-world humanoid robot experiments show that Speech2Grasp outperforms cascaded ASR-based pipeline, while reducing inference latency. Our findings suggest a practical paradigm for extending established text-conditioned systems to speech.

11. Audio-Anchored Fusion of Multi-Ratio DiT Reconstruction Residuals for Cross-Domain Audio Deepfake Detection

Authors: Haotian Mo, Jie Liu, Siqi Shen, Songzhu Mei, Xinhai Chen et al.

Categories: cs.SD, cs.AI | 10 pages, 5 figures. Submitted to speech security conference. This work proposes a cross-domain audio deepfake detection framework based on bona-fide trained DiT multi-ratio reconstruction residuals and audio-anchored additive fusion, evaluated on ASVspoof 5 and real-world ITW datasets Score: 5.26/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:5 Nov:4 Rep:6)

  • Strength: 诚实的系统级实验设计：三种子验证、共享协议下 audio-anchored vs. dynamic competitive fusion 的对照表明 auxiliary supervision 在所有三种子上损害 competitive fusion 的 ITW 性能（18.40% → 25.30% EER），且论文严格声明不做超出证据范围的因果归因。
  • Weakness: 核心方法创新极薄（加性融合公式 zf = z̃a + g·z̃v 是已有组件的简单拼装），与直接先行工作 Cheng et al. 2026 (diffusion reconstruction for ADD) 概念重叠度高，且 ITW 性能（15.33% 三种子均值）未达已知最优（SLIM 10.8%），关键对比无法隔离 anchoring 的因果效应。
  • Full review: Claude Code 全文七公理审稿

Audio deepfake detectors often degrade when generators, corpora, or recording conditions change. We use a Diffusion Transformer (DiT), trained only on bona fide speech, as a frozen reconstruction probe. Reconstructions at masking ratios 0.5, 0.75, and 0.9 yield explicit multi-ratio residual maps. Because these residuals are domain sensitive, our audio-anchored detector passes the projected frozen-WavLM auditory representation into the fusion sum without gate-based attenuation and uses residuals only as a scalar-gated additive correction. The pre-specified seed-42 run obtains 6.5442% EER / 0.18456 min-DCF on ASVspoof 5 Eval and 13.8372% / 0.36921 on ITW Full; three-seed means are 6.8885 (0.3308)% and 15.3328 (2.0719)%. The latter is below a separately optimized WavLM-ResNet18 reference under both supervision settings. Auxiliary supervision raises dynamic competitive fusion from 18.4007% to 25.2968% mean ITW EER, worsening all three seeds. The results support reconstruction residuals as complementary evidence and motivate a non-competitive auditory path for ASVspoof 5-to-ITW transfer, without claiming a componentwise causal ablation of anchoring alone.

12. Phoneme- vs. Character-Level Targets and Selective State-Space Models for Intracortical Brain-to-Text

Authors: Lucas Zamora Vera, Jose A. Gonzalez-Lopez

Categories: cs.CL, cs.AI, eess.SP | 6 pages, 1 figure, 6 tables, submitted to IberSPEECH 2026 Score: 4.84/10 (Obj:6 Id:8 Ind:5 Comp:5 Eff:4 Nov:2 Rep:8)

  • Strength: 严格 2×2 受控实验（5 seeds, Wilcoxon p<10⁻¹⁰）在公开 Brain-to-Text ‘25 benchmark 上明确回答了 Mamba/GRU × 音素/字符两个设计问题，音素 GRU 达 12.62% PER / 21.19% WER 为本设置最佳。
  • Weakness: 核心发现（Mamba 不优于 GRU）与 Benchmark ‘24 已有结论（Transformer 不优于 RNN）结构相同，ConvMambaGRU 为已有模块直接迁移且 WER 显著更差（24.11% vs. 21.19%），所有结果仅在验证集报告、测试集不公开。
  • Full review: Claude Code 全文七公理审稿

State-of-the-art intracortical brain-to-text systems pair a neural-sequence phone decoder with an external language model. Two design axes remain underexplored: whether selective state-space models (Mamba) improve on recurrent decoders, and how the output target (phonetic vs.\ character) interacts with that choice. On the public Brain-to-Text ‘25 benchmark, we study a controlled 2x2 grid (GRU vs.\ hybrid Mamba decoder; phonetic vs.\ character targets) trained with a CTC objective under one reproducible protocol. The recurrent baseline remains strongest: the best phonetic GRU reaches 12.62\% PER and 21.19\% WER, while the best textual GRU after LM rescoring reaches 13.39\% CER and 26.28\% WER. The Mamba hybrid is competitive but does not surpass it. Ablations isolate architectural contributions, and error analysis shows representation-dependent failures: articulatory-like phoneme confusions vs.\ lexical and word-boundary errors.

13. Zero-Shot Face-to-Speech Synthesis via Latent Space Adaptation of a Style-Diffusion TTS Model

Authors: Carlos Muñoz-Romero, Jose A. Gonzalez-Lopez

Categories: eess.AS, cs.AI, cs.SD | 5 pages, 1 figure, 5 tables, submitted to IberSPEECH 2026 Score: 4.74/10 (Obj:6 Id:5 Ind:5 Comp:5 Eff:5 Nov:4 Rep:3)

  • Strength: 低 — 四种损失配置无法相互区分（McNemar p>0.1），因此无法确定任何单一损失项的必要性；缺少协议匹配的端到端基线和简单启发式基线，使得该方法相对于现有 F2S 或琐碎替代方案的效用无法确定。
  • Weakness: 代码不可用 —— “代码仓库和音频样本链接将在论文录用后提供。” 西班牙语主干是在“私有纯音频多说话人语料库”（48 个说话人，53K 条话语）上微调的 —— 这个私有数据集从根本上无法复现。训练细节部分充分（学习率、批量缩放、轮次、硬件），但私有西班牙语数据阻塞了跨语言主张的复现。没有模型检查点发布。评估使用 Resemblyzer（公开）、UTMOS（公开）、NISQA（公开）——这些是可复现的。24 个身份的测试集划分未发布。
  • Full review: Claude Code 全文七公理审稿

Zero-shot text-to-speech (TTS) clones a voice from a short audio prompt, but this reliance on reference audio is a barrier when only visual information is available, e.g. for historical figures or video-game characters. In this work, we propose a Face-to-Speech (F2S) framework that predicts a plausible voice from a static facial image. A lightweight Face Adapter, together with soft-tuning of the face encoder’s upper blocks, aligns face-recognition features with the style space of a frozen StyleTTS 2 model, kept frozen during training. We evaluate on held-out identities from LRS3, a large-scale audiovisual corpus of English TED-talk videos. The synthesized speech is highly natural (UTMOS 3.7-4.0, matching or exceeding the 3.61 of ground truth), face-to-voice retrieval is consistently above chance, and the generated voice is consistent with the target speaker. Without any retraining, an English-trained adapter also produces fluent Spanish speech, indicating that the face-to-style mapping is largely language-agnostic.

14. MMAC: A Massive Multi-dimensional Benchmark for Audio Captioning

Authors: Weijie Wu, Junbo Li, Lin Li, Jun Fang, Qingyang Hong

Categories: cs.SD, cs.AI Score: 4.68/10 (Obj:6 Id:5 Ind:2 Comp:5 Eff:5 Nov:4 Rep:6)

  • Strength: 提出 coverage/precision/accuracy 三指标分离的自由形式音频字幕诊断框架，覆盖 15 个细粒度维度（含 implication 和 dynamic 两个较少见维度），5638 clips 评测 8 个 AudioLLM 揭示了明确的 capability gap（最高 Accuracy 仅 46.85%）。
  • Weakness: 评测 judge (Qwen3.6-27B) 与多个被评测模型同族，且数据标注使用了被评测模型 (Gemini 3.1 Pro, Qwen3-Omni)，构成标注-评测闭环污染；未引用直接竞品 MECAT (2507.23511) 和 AudioCapBench (2602.23649)，核心评测框架新颖性受限。
  • Full review: Claude Code 全文七公理审稿

With the development of audio large language models (AudioLLMs), audio captioning needs to move from brief descriptions toward open-ended and fine-grained free-form descriptions. Existing evaluations often focus on generation quality or task performance, making it difficult to diagnose information coverage and description reliability. We propose MMAC, a \textbf{M}assive \textbf{M}ulti-dimensional benchmark for \textbf{A}udio \textbf{C}aptioning. MMAC contains 5,638 audio clips from more than 20 data sources, covering 6 capability categories and 15 evaluation dimensions. Given a model-generated caption, MMAC checks whether it mentions relevant information in the target dimension and whether the mentioned content is consistent with the reference label. We evaluate representative open-source and proprietary AudioLLMs. Results show clear differences across evaluation dimensions, information coverage, and description reliability. We will release the MMAC benchmark and evaluation code.

15. Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement

Authors: Tianyan Deng, Yanxiong Li, Rui Gao, Jiahao Du

Categories: cs.SD, cs.LG | 6 pages, 1 figures Score: 4.68/10 (Obj:6 Id:5 Ind:8 Comp:3 Eff:5 Nov:2 Rep:8)

  • Strength: 在三个音频数据集、18 个实验设置中，ROLE 在中高 outlier ratio（50%/80%）下通过 inlierness-gated 原型精修有效抑制了未知类污染，macro AUROC 达 85.88%/92.22%（1-shot/5-shot），较强 baseline MET 提升 4.73%/1.96%。
  • Weakness: ROLE 本质是 OSLO（Phase 1 inlierness 加权）+ EOL（解耦评分）+ TIM（直推式 loss）的串联，真正新增内容仅一行 prior-adaptive shift；ablation 显示 Phase 2 贡献仅 −1.29% AUROC，且在 20% outlier 下全面落后 OSLO，8/18 设置未达最优。
  • Full review: Claude Code 全文七公理审稿

Few-shot Open-set audio classification requires classifying query samples from known classes with a few labeled support samples while rejecting query samples from unknown classes. Transductive inference jointly observes the full unlabeled query set to improve prototype estimation, yet standard transductive updates do not distinguish known from unknown query samples, leaving prototypes vulnerable to open-set contamination. Drawing on latent-inlierness weighting and decoupled scoring for unknown-class samples, we propose a two-phase transductive method operating over a frozen audio encoder. First, each query sample is assigned a latent inlierness score that down-weights likely unknown-class samples, so that prototype refinement is driven primarily by known-class evidence. The refined prototypes are then directly optimized on a transductive loss combining support cross-entropy, inlierness-weighted conditional entropy minimization, and inlierness-weighted marginal entropy maximization, while open-set rejection uses a prior-adaptive free-energy score that adjusts its threshold with the prior proportion of unknown-class samples, decoupling detection from classification. Experiments on three audio datasets show our method achieves state-of-the-art results for few-shot open-set audio classification under multiple experimental conditions.

16. Unfolded Recursive Expectation-Maximization Neural Network For Speaker Tracking

Authors: Rina Veler, Sharon Gannot

Categories: eess.AS, eess.SP | proceedings of IWAENC 2026 Score: 3.95/10 (Obj:8 Id:4 Ind:8 Comp:5 Eff:3 Nov:5 Rep:5)

  • Strength: 将递归 EM 展开为具有 FiLM+PE 条件作用的可学习步长网络，在概念上桥接了经典在线 EM 与用于声学追踪的深度学习，并在消声条件下取得了 0.25m RMSE（匹配最佳固定步长 CREM）。
  • Weakness: 仅与单一弱基准（采用网格搜索的 CREM）比较，无消融实验，无现代深度学习基准；在混响条件下 0.55m RMSE，且 56% 追踪超过 0.5m 误差阈值，使该方法在目标条件下无法使用。
  • Full review: Claude Code 全文七公理审稿

We propose a deep unfolded REM network for robust tracking of a single moving speaker in mild reverberant environments. Unlike classical REM algorithms, which rely on fixed-step-size decay schedules, the proposed architecture learns an adaptive update policy by unfolding the iterative procedure into differentiable layers. We introduce a Step Size Network that leverages FiLM and PE to dynamically adjust the recursion weights based on temporal context and convergence state. Experimental results for tracking a single speaker under reverberant conditions demonstrate that the proposed unfolded network outperforms the classical CREM baseline, which employs a spatial grid search to map the estimated centroids to physical positions. In the single-speaker tracking task, the proposed method achieves a lower RMSE than the CREM baseline, highlighting its potential for dynamic acoustic scenarios.

17. Qwen-Audio-3.0-Gen-Preview Technical Report

Authors: Junyu Dai, Xiaoyue Duan, Xinyue Fan, Yihan Feng, Xiangang Li et al.

Categories: eess.AS Score: 3.80/10 (Obj:6 Id:2 Ind:5 Comp:5 Eff:5 Nov:4 Rep:1)

  • Strength: 在 Seed-TTS-Eval 三子集上取得最高 WavLM 说话人相似度（0.805/0.819/0.808），跨轮一致性 CONS 在中英文均高于 Seed-Audio-1.0，且用 0.1x 音乐数据在 SongBench 七项中三项领先专用模型，证明统一 NAR DiT+共享 VAE 架构在 scene-level 音频组织上的可行性。
  • Weakness: 系统层面零消融实验，8+ 个同时引入的组件（两阶段课程、PE、semantic views、CFG、synthetic data 等）无一被隔离验证；自建 benchmark（多说话人 100 例、时间控制 100 例、SongBench 仅 20 例）均不公开且样本量极小；评测使用同属阿里的 Qwen 系列 LALM 作为 judge 存在循环风险；无代码、无数据、无 checkpoint、无关键训练超参数，复现几乎不可能。
  • Full review: Claude Code 全文七公理审稿

Existing single-domain and multi-task audio systems remain limited in directly organizing speech, music, sound effects, ambience, and multiple roles into long-form temporal scenes. We present Qwen-Audio-3.0-Gen-Preview, a unified non-autoregressive framework that uses a Diffusion Transformer (DiT) and a shared variational autoencoder (VAE) to generate the complete mixed waveform. Prompt enhancement converts free-form requests into structured temporal records that are rendered as textual conditions, while a two-stage data curriculum and semantic conditional views train the proposed model to use these conditions across domains. A shared continuous VAE compresses 48kHz stereo waveforms into 25Hz latent sequences and incorporates semantic supervision, providing one representation for speech, music, sound effects, and their mixtures. On Seed-TTS-Eval, speaker similarity is the proposed model’s clearest strength across all three subsets, and on the multi-speaker benchmark, the proposed model shows higher cross-turn consistency than Seed-Audio-1.0 in both languages. On AudioCaps, its advantages are concentrated in evaluations using large audio-language models and AudioBox. Relative to Seed-Audio-1.0, it achieves stronger temporal localization. Using approximately 10% music data of a dedicated in-house model, the proposed model remains close across all seven SongBench components and leads in three while retaining speech and general-audio capabilities. These results demonstrate the potential of unified generation for temporally structured, multi-domain audio.