每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-07-17
日期2026-07-17
已评分—
均分—
最高—

Daily Papers — 2026-07-17

8 papers on audio, speech, music, and acoustics.

1. A Geometry-Limited Identification Floor and Its Consequences for Voice-Clone Attribution in Professional Voice Actors

Authors: Shuhei Kato

Categories: eess.AS, cs.SD | 24 pages, 6 figures, 8 tables. Submitted to IEEE Access Score: 9.05/10 (Obj:9 Id:9 Ind:9 Comp:9 Eff:9 Nov:8 Rep:7)

  • Strength: 在 1,168 日本声优 / 56,568 段大规模语料上，通过 8 编码器 + 2 集成 + 完整后端套件（cosine→AS-norm→LDA→WCCN→PLDA，moment+EM，linear+nonlinear）+ 全量混杂控制矩阵（codec/channel/vocoder/content/BGM/duplicate/loudness/session/fusion），证明 misidentification floor 是几。
  • Weakness: 克隆-参考/注册重叠使 attribution-recall 数字 same-session-optimistic；raw embeddings 因生物识别/法律原因受限访问（DUA）增加独立复现摩擦；animeva 训练数据非同意收集且不透明（21% 重叠已审计但 source-level leakage 未排除）；mastering transfer function（EQ/limiting）无法在无 pre-master 下决定性。
  • Full review: Claude Code 全文七公理审稿

A voice actor’s voice is their asset, and AI cloning directly threatens it. The natural defense flags the enrolled actor whose embedding similarity to a suspect recording crosses a threshold. We show it fails where it is most needed: trained voices crowd the embedding space, and each actor performs many styles. On 1,168 Japanese voice actors (56,568 segments, ~63 h), a misidentification floor survives calibration, score normalization, and discriminative re-ranking (linear and nonlinear, including PLDA): the residual is a limit of the embedding geometry, not of the back-ends we evaluate. The best ensemble still leaves ~2.6% closed-set misidentification, several-fold above matched controls; session-disjoint, re-ranking lowers the floor only to 13.0%. The same crowding drives false attribution: on a generic English encoder, roughly half the clones of non-enrolled people falsely accuse an enrolled actor, while – by a separate real-vs-synthetic shift – 32% of Seed-VC clones of enrolled targets are missed at the same threshold; one operating point couples the two, and none escapes both. A domain-matched, voice-actor-trained encoder mitigates substantially (a four-fold gender gap vanishes; wrongful misattribution falls to 1.5-10%), but does not remove the floor. Controls (codec, channel, vocoder, content) support reading the miss rate as a real-versus-synthetic covariate shift, not missing speaker information. Fixed-threshold clone attribution is thus unreliable here, and on a generic encoder unfair. Robust attribution must extend spoofing-aware speaker verification to open-set 1:N (anti-spoofing gate, domain-matched encoder, per-speaker calibration, abstain option), and even then supports detection, not autonomous enforcement.

2. StemFX: Learning Mixing Style Representations via Autoregressive FX Chain Prediction on Source-Separated Stems

Authors: Yuan-Chiao Cheng, Jui-Te Wu, Brian Chen, Yen-Tung Yeh, Yu-Hua Chen et al.

Categories: cs.SD | Accepted to ISMIR 2026. 8 pages, 4 figures Score: 7.16/10 (Obj:7 Id:8 Ind:8 Comp:6 Eff:7 Nov:6 Rep:9)

  • Strength: 将 FX 链预测重构为变长序列生成，在 controlled 实验下（BSFiLM-CL）检索 86.8% vs 77.8%，推理 0.24s vs 1033s（4000×加速），代码与权重完全开源。
  • Weakness: 相对 LLM2Fx-Tools (ICLR 2026) 的核心增量仅为 joint encoder training，且 Real Mix MUSHRA 60.6 vs 目标 96.6 的绝对质量差距表明方法远未达到专业混音可用水平。
  • Full review: Claude Code 全文七公理审稿

Audio mixing style encompasses the artistic and technical decisions a mix engineer makes, including level balancing, spatialization, and the choice, ordering, and parameterization of audio effects (FX) on each stem. FX chains are a key determinant of this style, yet existing approaches to modeling them remain limited. Some operate on stereo mixtures without explicit per-stem FX chain modeling, others fix the number or type of effects per track, and many require differentiable effect implementations or scarce multitrack datasets. We present StemFX, a framework that learns mixing style representations by autoregressively predicting variable-length FX chains on source-separated stems. A Transformer decoder predicts tokenized FX chains autoregressively, while a band-split multi-band CNN encoder with FiLM conditioning captures per-stem spectral structure. To enable large-scale paired training, we extract pseudo-stems from about 105K songs via source separation and augment them using MultiAFx, a toolkit unifying 85 audio effects from 7 Python libraries. Evaluated on mixing style retrieval, StemFX outperforms all baseline models across all tested chain lengths. On paired mixing style transfer, StemFX achieves the best spectral fidelity and the highest listener preference, over 4000 times faster than iterative optimization.

3. Controlling Implicit Shortcut Reliance in L2 Spoken English Auto-markers

Authors: Shilin Gao, Mark J. F. Gales, Kate M. Knill

Categories: cs.CL, eess.AS Score: 6.84/10 (Obj:9 Id:6 Ind:8 Comp:8 Eff:6 Nov:8 Rep:6)

  • Strength: 提出输出级秩相关惩罚（式 8），首次将捷径缓解扩展到不要求捷径特征为显式可微输入的端到端编码器评分器，在 ModernBERT 文本评分器上以拟人化模式 (λ≈0.13) 将字数相关性从 0.874 降至 0.659（人类水平），同时保持评分 ρ≈0.761。
  • Weakness: 未与同一 S&I 2025 语料库的 Challenge 2025 官方基线 (arXiv 2412.11985) 及同期强系统 (arXiv 2507.17918) 比较；音频评分器基线绝对 ρ 仅 0.612 且拟人化模式进一步下降，效用证据在音频模态偏弱；未发布训练/评测代码。
  • Full review: Claude Code 全文七公理审稿

Increasingly, speech and language processing tasks take either audio or text directly rather than extracting features from these as the input to the classifier or regressor. Often these systems make use of complex, for example transformer-based, processes that have the ability to derive highly non-linear mappings between the input and the output. Unfortunately these systems can also learn ‘‘shortcuts’’ where the classifier is overly reliant on particular aspects of the input to yield the output. For the task of language proficiency assessment, this over-reliance can enable learners to increase their score by exploiting the shortcut rather than improving their ability. This paper introduces a novel training criterion that is able to reduce the classifier’s reliance on shortcuts, thus for example limiting this option for malpractice in language assessment. This process is illustrated on two forms of assessment system, one based on the audio the other on the speech recognition text. The results show that, for both systems, there is higher correlations with features that could be exploited for malpractice than expected from the human reference, indicating an over-reliance on these features. By introducing the modified training criterion, this correlation can be reduced to be closer to the reference correlation.

4. Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos

Authors: Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar, Lasha Koroshinadze, Nishit Anand et al.

Categories: eess.AS, cs.CV | Project Page: https://avflamingo.pages.dev/ Score: 5.74/10 (Obj:6 Id:5 Ind:5 Comp:5 Eff:6 Nov:5 Rep:8)

  • Strength: 完全开源 8B AV-LLM，配套 7M 实例 AV-Skills 数据集与三阶段课程，在自建长视频 benchmark LONG-ALAV 上相对次优开源模型提升 +13.2 分，填补开源 AV 长视频生态空白。
  • Weakness: 核心组件（Q-Former、Whisper、CoT、temporal grounding、课程学习）均为已有技术的跨模态组合，ablation 未隔离 interleaved vs. sequential CoT，且最大提升集中在作者自建 benchmark（训练与评测同源 LLM 生成），在外部独立 benchmark（MUSIC-AVQA、AudioScope）上绝对分数仍 <75 且提升 <3 分，漏比 Audio Flaming。
  • Full review: Claude Code 全文七公理审稿

We present Audio-Visual Flamingo (AV-Flamingo), a fully open state-of-the-art audio-visual large language model (AV-LLM) for joint understanding and reasoning over audio, images, and long-form videos. Unlike prior AV-LLMs that primarily focus on short clips, AV-Flamingo is designed for understanding and reasoning over long and complex real-world (audio-visual) videos. To support this, we make three key contributions: (i) Audio-Visual-Skills, a large-scale collection of real-world videos with ~7M caption and question-answer training instances designed to emphasize temporal, compositional, and cross-modal audio-visual reasoning; (ii) a novel three-stage curriculum that progressively trains the model from short-range perception to long-horizon multi-event reasoning; and (iii) Temporal Audio-Visual Interleaved Chain-of-Thought, a reasoning framework that explicitly grounds intermediate reasoning steps to timestamps in long audio-visual streams, improving temporal alignment and interpretability. Extensive experiments across 15+ audio-visual, omni-modal, audio, and vision benchmarks show that AV-Flamingo outperforms similarly sized open models by clear margins and remains highly competitive with, and in some cases surpasses, much larger open-weight and closed models, particularly on long and complex real-world audio-visual understanding and reasoning tasks. Beyond benchmark performance, AV-Flamingo exhibits strong real-world utility and transfers well to unseen tasks, highlighting its robustness and generalization ability.

5. Natural Backdoor Attacks on Speech Recognition Models

Authors: Jinwen Xin, Xixiang Lyu, Jing Ma

Categories: cs.CR, cs.LG, cs.SD | This is the authors’ manuscript of a chapter published in Machine Learning for Cyber Security, Lecture Notes in Computer Science, vol. 13655, pp. 597-610 (2023) Score: 5.21/10 (Obj:7 Id:5 Ind:8 Comp:3 Eff:5 Nov:3 Rep:5)

  • Strength: 在 SCDv2/ESC 两个数据集和三个模型上验证了自然声音（雨声/口哨/鸟鸣）作为后门触发器的有效性，5% 投毒率下 CNN 模型 ASR 达 99%+，BA 无显著下降，并扩展至 Clean-label 和真实物理场景。
  • Weakness: 核心方法仅为时域逐元素加法（Algorithm 1），新颖性依赖从 Liu et al. 2020 (CV reflection backdoor) 的跨领域迁移且未做领域适配创新；未测试现代 SR 模型（Whisper/Wav2Vec2），无防御评估，无代码开源，机制原因未识别。
  • Full review: Claude Code 全文七公理审稿

With the rapid development of deep learning, its vulnerability has gradually emerged in recent years. This work focuses on backdoor attacks on speech recognition systems. We adopt sounds that are ordinary in nature or in our daily life as triggers for natural backdoor attacks. We conduct experiments on two datasets and three models to validate the performance of natural backdoor attacks and explore the effects of poisoning rate, trigger duration and blend ratio on the performance of natural backdoor attacks. Our results show that natural backdoor attacks have a high attack success rate without compromising model performance on benign samples, even with short or low-amplitude triggers. It requires only 5% of poisoned samples to achieve a near 100% attack success rate. In addition, the backdoor will be automatically activated by the corresponding sound in nature, which is not easy to be detected and will bring severer harm.

6. Contextual Semantic Relevance Tracks fMRI BOLD Responses During Naturalistic Speech Comprehension

Authors: Kun Sun, Rong Wang

Categories: cs.CL Score: 4.95/10 (Obj:6 Id:5 Ind:5 Comp:5 Eff:5 Nov:4 Rep:5)

  • Strength: 在两个公开 fMRI 数据集（Alice 12 ROI, Moth 30 ROI）上系统比较了 contextual semantic relevance 与 surprisal 对 BOLD 响应的预测能力，发现 semantic relevance 在 FIR/deconvolution 分析中跨所有 ROI 显著（Alice 12/12, Moth 30/30），而 surprisal 在 FDR 校正后无 ROI 显著，且两指标。
  • Weakness: 核心指标为作者先前多篇论文中已提出的同一指标迁移到 fMRI；Moth 的关键分析窗口（4-12s HRF-weighted directional contrast）为 Alice-informed 选择而非独立验证；保守的 surrogate max-t检验未能拒绝时间结构 null；缺少声学包络等关键 nuisance 控制；代码仓库链接为空。
  • Full review: Claude Code 全文七公理审稿

Naturalistic language comprehension requires listeners to process both local probabilistic expectations and contextual semantic relations. Surprisal has been widely used to quantify local word unexpectedness, but evidence that it robustly predicts fMRI BOLD responses during continuous comprehension has been mixed. This study investigates whether contextual semantic relevance, defined as how strongly an incoming word relates to its recent semantic context, predicts BOLD responses during naturalistic speech comprehension. We analyzed two public fMRI datasets, the Alice dataset and the Moth dataset, treating them as complementary rather than identical replications. Transformed BOLD responses were modeled with generalized additive mixed models (GAMMs) and original continuous BOLD time series were tested with FIR/deconvolution analyses. In Alice, semantic relevance was significant across all 12 ROIs (region of interest), whereas surprisal was not significant after FDR correction. In Moth, semantic relevance showed consistent negative effects across all 30 ROIs, while surprisal showed no comparable pattern. These findings suggest that semantic relevance is a promising BOLD-sensitive metric of contextual semantic fit. More broadly, our findings support the view that slow hemodynamic responses during naturalistic speech comprehension may be especially sensitive to contextual semantic integration, whereas local probabilistic prediction error may be more difficult to detect reliably with fMRI. In this sense, semantic relevance extends computational models of language comprehension from prediction alone toward context-sensitive semantic integration.

7. AuEmoChat: Authentic Emotion Understanding and Rendering for Conversational Speech Synthesis

Authors: Zhenqi Jia, Yuan Zhao, Aruukhan, Rui Liu, Haizhou Li

Categories: cs.SD, cs.AI | Accepted by ACM MM 2026 Score: 4.90/10 (Obj:6 Id:5 Ind:3 Comp:5 Eff:6 Nov:4 Rep:5)

  • Strength: 在 NCSSD-EmCap 上全面超越 Chain-Talker 等 SOTA（WER 9.14 vs 15.07, E-DMOS 3.979 vs 3.756），8 组消融覆盖全部三个组件，问题动机真实。
  • Weakness: 关键指标 AuEmoACC 由训练好的 AuEmoCodec 计算（训练标注→系统组件→评测指标自闭环），且所有 baseline 被改造注入 AuEmo token 导致主实验无法隔离 AuEmoCodec 贡献，仅在单一数据集评测。
  • Full review: Claude Code 全文七公理审稿

Conversational Speech Synthesis (CSS) aims to synthesize speech with human-like emotional expression and contextual consistency in user-agent interactions. Existing CSS methods struggle to render authentic human emotions due to limited predefined emotion label spaces (e.g., seven emotion categories), while redundant multimodal tokens in multi-turn dialogue history interfere with context understanding. To address these issues, we propose AuEmoChat, a CSS framework for authentic emotion understanding and rendering. First, we develop AuEmoCodec, which learns a discrete authentic emotion token space from large-scale emotional speech via finite scalar quantization, enabling a more authentic emotion representation than limited basic emotion categories. We further propose AuEmoToMe, an authentic-emotion-guided token merging algorithm that merges redundant tokens in multimodal dialogue history while preserving emotion-relevant context. We integrate it into an autoregressive text-speech model to predict the target authentic emotion token and speech tokens. Finally, we propose Authentic Emotion Flow Matching, which renders speech by jointly conditioning on merged dialogue context, target authentic emotion, and acoustic priors. Extensive experiments on the NCSSD-EmCap dataset demonstrate that AuEmoChat outperforms state-of-the-art CSS baselines and generates more expressive and authentic emotional speech.

8. SpeechGuard: Online Defense against Backdoor Attacks on Speech Recognition Models

Authors: Jinwen Xin, Xixiang Lv

Categories: cs.SD, cs.CR, cs.LG | 8 pages Score: 3.45/10 (Obj:5 Id:3 Ind:2 Comp:4 Eff:5 Nov:2 Rep:5)

  • Strength: 两阶段 pipeline 在 environmental noise 和 ultrasonic pulse 触发器上能将 ASR 从 ~100% 降至 <10%，同时 PA 保持 ~86–95%，证明 T-F masking 对频带稀疏的触发器有效。
  • Weakness: S-STRIP 未与原版 STRIP/STRIP-ViTA 直接对比以隔离 SNR 自适应的增量贡献；autoencoder 训练依赖 oracle 配对标签（poisoned+同标签 clean 计算 IBM），与威胁模型中”defender 无法获得 poisoned 样本”的假设矛盾；random noise 触发器下 PA 仅 60–73%、误检 clean 样本 BA 下降 28–36 个百分点，可用性严重不足。
  • Full review: Claude Code 全文七公理审稿

Backdoor attacks pose a critical threat to neural network models, allowing attackers to implant a backdoor during the training phase by manipulating a small portion of the training data. In security-sensitive applications such as voice interaction for autonomous driving, the presence of backdoor attacks introduces substantial security risks. This study focuses on implementing backdoor defense measures for speech recognition models in run-time, taking into account the characteristics of audio signals. We propose SpeechGuard, the first online backdoor defense pipeline designed to identify and purify poisoned audio samples. Specifically, we improve STRIP method to perform adaptive perturbation injection to detect and filter poisoned samples, named as S-STRIP. More importantly, we further consider the purification of poisoned samples. We utilize time-frequency (T-F) masking to suppress the expression of trigger signals and autonomously generate masks based on an autoencoder. The two-stage processing prevents the backdoor in the model from being triggered, and even input speech carrying triggers can be accurately predicted. Extensive experimental demonstrate that SpeechGuard can accurately filter out poisoned samples. Through purification, it can significantly mitigate the backdoor threat while maintaining a certain prediction accuracy.