每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-08-08
日期2026-08-08
已评分—
均分—
最高—

Daily Papers — 2026-08-08

9 papers on audio, speech, music, and acoustics.

1. The Voiceprint Fallacy: Why Voices Are Not Unique Biometric Imprints

Authors: Tianle Yang, Cuiling Zhang, Chengzhe Sun, Siwei Lyu, Phil Rose

Categories: eess.AS, cs.CL, cs.SD Score: 7.42/10 (Obj:9 Id:7 Ind:8 Comp:8 Eff:8 Nov:6 Rep:6)

  • Strength: 跨学科整合了历史声纹争议、多尺度语音变异证据、自动系统条件依赖和深度伪造挑战，提出了可操作的术语和报告建议，覆盖中美欧澳四个司法管辖区。
  • Weakness: 核心论点（声纹≠指纹）在 Bolt 1970、NRC 1979、Rose 2002、OSAC 2024 中已被反复论证，新颖性增量主要来自深度伪造整合（占全文约15%）和中国实践覆盖（约3段）；文献检索方法未达系统综述标准。
  • Full review: Claude Code 全文七公理审稿

In recent years, the term voiceprint has regained attention, particularly in technological applications and policy-making contexts, often carrying the assumption that a person’s voice constitutes a stable and unique biometric trace analogous to a fingerprint. Yet this conception has been repeatedly criticized and rejected by forensic voice experts throughout the decades since its introduction. Although voices undoubtedly contain speaker-related information, this simplified conception obscures the highly dynamic and context-dependent nature of speech. This article revisits the voiceprint fallacy and reconsiders what can count as evidence of speaker identity by reviewing the historical development of voiceprint identification, evidence on human voice variability, developments in forensic voice comparison, research on human and automatic speaker recognition, and the recent challenge posed by deepfake speech to speaker identity. We point out that the voiceprint metaphor and its underlying implications are scientifically misleading because they transform a probabilistic source of speaker information into an imagined stable object of identity. To avoid treating voices as imprint-like traces, we recommend that voice evidence be interpreted through validated and calibrated probabilistic frameworks that explicitly account for variability, uncertainty, and alternative explanations.

2. SAMOT: State-Aware Step Modulation and Optimal Transport Matching for Audio-Visual Instance Segmentation

Authors: Kai Peng, Yunzhe Shen, Miao Zhang, Leiye Liu, Wei Ji et al.

Categories: cs.MM | Accepted by ACM MM 2026 Score: 7.05/10 (Obj:8 Id:6 Ind:8 Comp:5 Eff:8 Nov:6 Rep:9)

  • Strength: 在 AVISeg 基准上全面超越现有 AVIS 方法（+3.76 FSLA, +2.75 HOTA, +2.58 mAP），ADSM 的跨模态步长调制是真实原创贡献，消融实验充分验证了各模块贡献，代码开源可复现。
  • Weakness: 框架由多个已有技术组合而成（Mamba+OT+MMD），整体复杂度较高（5个新模块/缩写），消融基线与主表基线不一致导致增量贡献难以精确归因，缺乏对组件最简替代方案的对比验证。
  • Full review: Claude Code 全文七公理审稿

Audio-Visual Instance Segmentation (AVIS) aims to simultaneously classify, segment, and track sounding objects within video sequences. Unlike Audio-Visual Semantic Segmentation (AVS), AVIS involves instance-level modeling across longer video sequences, introducing two key challenges: (1) complex modality-state changes disrupt long-range modeling, and (2) substantial structural and distributional discrepancies between modalities hinder precise instance-level association. Existing methods rely on fixed-step Transformers and recursive Mamba models, lacking adaptability to modality-state changes. In addition, methods performing implicit matching ignore the inherent distributional inconsistencies. To address these issues, we propose a framework with Adaptive Dynamic Step Modulation (ADSM) and Optimal Transport-based Matching Modulation (OT-MM). ADSM adaptively modulates Mamba step sizes using temporal variation, cross-modal discrepancy, and historical context, balancing rapid response to modality-state changes with stable long-range modeling. OT-MM explicitly formulates instance-level cross-modal matching as an entropy-regularized optimal transport problem solved via log-domain Sinkhorn iterations, and further enforces distribution-level coherence with an MMD regularizer. Extensive experiments demonstrate state-of-the-art performance on the AVIS benchmark (+3.76 FSLA, +2.75 HOTA, +2.58 mAP), verified through comprehensive qualitative visualizations. The code and model are available at https://github.com/happylife-pk/SAMOT.

3. ReLMCodec: Designing Predictable Speech Tokens from Pre-Quantization Phoneme Structure

Authors: Zixiang Wan, Xusheng Yang, Zheng Wang, Peiji Yang

Categories: eess.AS, cs.SD | 18 pages, 9 figures, 12 tables. Project page: https://github.com/ggiggit/ReLMCodec Score: 7.05/10 (Obj:8 Id:8 Ind:6 Comp:8 Eff:8 Nov:6 Rep:5)

  • Strength: 受控诊断（24种表示，matched P-VQ+Qwen2-1.5B）建立预量化音素结构与token可预测性的强关联（Spearman ρ=0.911），据此设计的ReLMCodec@64K在800bps下WER=3.96/SIM=0.804/PESQ=2.40全面领先14个baseline，下游TTS WER从X-Codec2的9.49降至4.93。
  • Weakness: 评测存在循环依赖风险（WavLM同时用作teacher和SIM评估器），代码/checkpoint尚未开源（仅承诺upon acceptance），且未与采用类似语义锚定思路的SACodec(2512.20944)进行直接比较。
  • Full review: Claude Code 全文七公理审稿

Neural speech codecs face a fundamental tension in the language-model era: tokens that support high-fidelity reconstruction are not necessarily easy for autoregressive models to predict. Our controlled analysis of diverse codec and self-supervised speech representations shows that clearer phoneme structure before discrete code assignment is consistently associated with easier autoregressive token prediction. Yet phoneme structure alone is insufficient for high-fidelity reconstruction, which also requires reconstruction-relevant acoustic detail. Guided by this observation, we introduce ReLMCodec, a low-bitrate single-codebook speech codec built upon a preserve–control–refine principle: it preserves the linguistic organization of frozen self-supervised learning (SSL) features at the quantizer input, controls reconstruction-driven drift through Pre-quantization Anchor-Preserving Adaptation (PAPA), and refines the quantized latent space with a training-only WavLM-Large L24 teacher to reduce phoneme-level token fragmentation. Together, these components allow acoustic detail to support waveform reconstruction while keeping the resulting token sequence predictable for autoregressive models. At 650 and 800 bps, ReLMCodec moves the empirical single-stream predictability–reconstruction frontier in our evaluations, with gains that carry over to downstream text-to-speech (TTS) synthesis in both intelligibility and speaker similarity.

4. SCoPE: Training-Free Audio-Visual Event Perception via Sparse Cross-Modal Prior Exchange

Authors: Jaemo Jeong, Junho Yoon, Hyunju Kim, Dongman Lee

Categories: cs.CV, cs.MM, cs.SD | 17 pages, 6 figures Score: 6.95/10 (Obj:6 Id:8 Ind:8 Comp:6 Eff:8 Nov:6 Rep:5)

  • Strength: SCoPE 通过非负 Lasso 稀疏竞争和跨模态固定预算成本重分配 + target re-selection，在三个 AVEP benchmark 上以 training-free 方式实现大幅提升（LLP Type@seg +7.45, OV-AVEBench Avg. +17.52, VGGSound Seg F1 +21.16），且推理仅需 3.16 ms/video。
  • Weakness: 核心组件（稀疏编码、mean centering、gap split readout）均为已有技术的组合，新颖性集中在跨模态 prior exchange 机制；无公开代码，Lemma 1 仅对 two-label 精确，FCA 作为 score-level 诊断与 final prediction error 的定量关系未充分论证。
  • Full review: Claude Code 全文七公理审稿

Audio-visual event perception (AVEP) determines which events occur in a video, when they occur, and whether they are audible, visible, or both. Training-free methods query new event vocabularies by matching frozen audio and visual features with text-encoded event names. However, related labels share evidence. An incorrect label can then score at least as high as a correct one. We call this a false co-activation (FCA). No scalar cutoff can reject the incorrect label while keeping every correct one. Class-specific thresholds may prevent that label from becoming a final prediction, but the FCA remains in the underlying score vector. We introduce SCoPE, a training-free framework in which all queried labels compete for shared evidence and each modality guides event selection in the other. We derive an exact condition for when this competition removes an FCA in a two-label fit. With identical frozen CLIP+CLAP backbones on LLP, SCoPE improves Type@seg by 7.45 points and Event@seg by 5.04 points compared with the reported AV$^2$A values. The same fixed configuration transfers unchanged to OV-AVEBench and VGGSound-AVEL100k.

5. VIOLET: High-Fidelity Violin Synthesis with Techniques and Dynamics

Authors: Baotong Tian, Cynthia Lu, Vincent K. M. Cheung, Ting-Kang Wang, Jonathan Churchill et al.

Categories: eess.AS, cs.SD | Accepted at ISMIR 2026; Code and Demo available at https://github.com/User-tian/VIOLET Score: 6.26/10 (Obj:8 Id:5 Ind:5 Comp:5 Eff:8 Nov:5 Rep:8)

  • Strength: VIOLET 在多个客观指标（FAD 0.513 vs 0.668，动力学 ρ 0.631 vs 0.036）和主观评估（技巧清晰度/音频质量 p < 0.001）上显著优于 SOTA 神经网络基线 ViolinDiff，同时接近商业 VI 质量，并作为首个支持显式技巧和动力学控制的神经小提琴合成器。
  • Weakness: 核心客观评估在由训练数据生成所使用的同一商业 VI 创建的合成测试集上存在循环依赖，且架构消融实验缺失（无 DiT-vs-UNet、修正流-vs-DDPM 或组件交互消融），无法隔离哪些设计选择真正推动了改进。
  • Full review: Claude Code 全文七公理审稿

Neural synthesis for musical instruments has the potential to revolutionize current practices that use concatenative synthesis and a sample library. However, most research focused on piano synthesis and expressive performance generation; little work has been done on continuously articulated instruments like the violin, let alone rendering them with playing techniques and dynamics. We present VIOLET, a latent-diffusion framework for controllable violin synthesis, which uses a Diffusion Transformer (DiT) with rectified flow to synthesize high-fidelity audio from MIDI notes, playing techniques, and continuous dynamics. To train VIOLET, in addition to using a few existing datasets, we curate a new dataset named CSV-TD, which contains 39 h of 48 kHz synthetic audio and time-aligned annotations of MIDI notes, note-level techniques, and continuous dynamics curves. Objective and subjective evaluations show that VIOLET synthesizes violin performances with high technique adherence, accurate pitch and timing alignment, and good dynamics control. It outperforms the current state-of-the-art neural violin synthesis system and approaches a top commercial virtual instrument in terms of technique clarity, naturalness, and dynamics following.

6. Dramarrator: Object-Based Audio Editing for Audio Drama Production from Books

Authors: Karim Benharrak, Oriol Nieto, Bryan Wang, Zeyu Jin, Amy Pavel

Categories: cs.HC, cs.AI, cs.MM | Accepted to UIST 2026 Score: 5.84/10 (Obj:8 Id:5 Ind:6 Comp:5 Eff:7 Nov:6 Rep:3)

  • Strength: 完整的端到端系统，引入基于对象的音频编辑范式，用户研究 (N=8) 显示 4× 时间缩减和显著任务负载降低，听众研究 (N=300) 显示精修输出在 7/11 指标上与专业工具产物持平。
  • Weakness: 核心效果未隔离因果变量（自动流水线与对象编辑混淆），且系统完全依赖闭源 API（Gemini 3 Pro, ElevenLabs, Adobe 专有模型），无代码开源，不可独立复现。
  • Full review: Claude Code 全文七公理审稿

Audio dramas weave dialogue, sound effects, and music into immersive stories. Creators often adapt books into audio dramas, but this process remains labor-intensive, requiring them to interpret source material, author scripts, generate audio assets, and assemble them on a timeline. Because story elements like characters and scenes manifest across many interdependent assets, a single change can ripple into manual updates across the entire project. We present Dramarrator, an audio drama authoring tool built around object-based audio editing, where these story elements are represented as editable objects. Dramarrator extracts these objects from a book, generates linked audio assets (speech, sound effects, and music), and composes a multi-track audio drama. Edits to any object (e.g., a character’s voice) automatically propagate to all dependent assets. In a user study with professionals (N=8), Dramarrator significantly lowered task load when creating audio dramas. A listener study (N=300) shows that creator-refined output from Dramarrator approaches the quality of productions made with existing professional tools, and an exploratory study (N=3) suggests object-based editing lowers entry barriers and generalizes beyond audio dramas.

7. DialectS2S: End-to-End Speech Dialogue Modeling for Low-Resource Chinese Dialects

Authors: Yi Shu, Tianyu Peng, Yingzhuo Deng, Wen Yang, Jun Lin et al.

Categories: cs.CL, cs.AI Score: 5.74/10 (Obj:8 Id:5 Ind:5 Comp:6 Eff:5 Nov:5 Rep:8)

  • Strength: 首个开源端到端中文方言语音对话模型，覆盖四川话/粤语/天津话，语言匹配准确率达 91-96%，自对齐监督在公平消融中使 CER 从 10.15 降至 6.98
  • Weakness: 主要 baseline 对比不公平（baseline 未在方言数据上训练），消融未隔离 TTS 质量混淆变量，LLM 评测器与训练数据生成器同源（Qwen3），无人工评测
  • Full review: Claude Code 全文七公理审稿

Current end-to-end speech dialogue models are primarily optimized for mainstream languages and remain limited in low-resource dialect scenarios due to the scarcity of dialect speech data. Moreover, during dialect adaptation, the semantic representation space of speech dialogue models continuously evolves, while conventional speech supervision remains unchanged, leading to semantic inconsistency between hidden representations and speech targets and degrading speech stability and naturalness. To address these issues, we propose DialectS2S, an end-to-end speech dialogue model for Chinese dialects. We first develop a scalable dialect speech dialogue synthesis pipeline for efficient data construction. We further introduce a two-stage post-training strategy with self-aligned speech supervision, which aligns the semantic content of speech supervision with the evolved semantic representations of the model to improve dialect speech generation quality. Experimental results show that DialectS2S consistently outperforms existing baselines across multiple Chinese dialects in speech dialogue, achieving substantial improvements in dialect consistency, response quality, and speech intelligibility. Our work provides an efficient and scalable solution for end-to-end speech dialogue modeling in low-resource dialect scenarios. To facilitate future research and practical applications, we fully open-source the DialectS2S framework, including model checkpoints, training datasets, and fine-tuning code.

8. CtrlSpeech: Coarse-to-Fine Control for Expressive Speech Synthesis

Authors: Zhisheng Zheng, Xiaohang Sun, Zhu Liu, Caren Chen, Rohith Kumar et al.

Categories: eess.AS, cs.HC, cs.SD | Interspeech 2026. Demo, code and model weights are available at https://zhishengzheng.com/ctrlspeech Score: 5.42/10 (Obj:6 Id:5 Ind:8 Comp:4 Eff:5 Nov:4 Rep:8)

  • Strength: 在 DiTAR backbone 上实现 phone-aligned pitch/loudness/duration 显式控制，控制信号启用后 duration MAE 从 28.08 降至 11.86、pitch RMSE 从 67.86 降至 38.39 Hz，并提供开源 code 和 model weights。
  • Weakness: 核心 idea（显式 prosody conditioning）在 Raitio 2020、2211.16307、DrawSpeech 等先验工作中已有不同形式实现；zero-shot TTS 仅与作者自行复现的 DiTAR 比较（CMOS -0.16 仍为负），未与 F5-TTS/MaskGCT/NaturalSpeech 3/CosyVoice 2/PRESENT 等关键 SOTA 实验对比，且无逐项控制信号消融。
  • Full review: Claude Code 全文七公理审稿

Recent Text-To-Speech (TTS) systems have achieved strong naturalness and zero-shot voice cloning performance, but fine-grained control of expressive speech at the word or phoneme level remains challenging. We propose CtrlSpeech, a controllable, expressive TTS framework with coarse-to-fine control. Built on the DiTAR architecture, CtrlSpeech combines global speaker conditioning with phone-aligned pitch, loudness, and duration signals, enabling localized prosodic control while preserving the target speaker’s timbre. This design allows users to adjust expressive attributes at a fine temporal granularity, making speech refinement more flexible and controllable. Experimental results show that CtrlSpeech achieves competitive zero-shot TTS performance and improves controllability over expressive attributes, demonstrating its effectiveness for flexible and practical expressive speech synthesis.

9. SraVaani 1.0: Scaling Inclusive Speech Recognition for Indic Languages

Authors: Sujith Pulikodan, Agneedh Basu, Pavan Kumar J, Pranav D Bhat, Suryansh Shukla et al.

Categories: eess.AS Score: 5.26/10 (Obj:8 Id:5 Ind:5 Comp:5 Eff:6 Nov:4 Rep:7)

  • Strength: 覆盖 65 种印度语言/方言（含部落语言），基于 62K 小时数据训练三阶段 pipeline，在多个 benchmark 上达到 SOTA，且为部分低资源语言唯一可用开源 ASR
  • Weakness: 三阶段各组件均直接采用已有工作（SSL、arXiv:2606.24080 音频-图像对齐、NeMo TDT-CTC），缺乏逐阶段消融隔离贡献，且低资源语言评测依赖与训练数据同源的 VAANI 基准，存在循环论证风险
  • Full review: Claude Code 全文七公理审稿

India’s linguistic landscape spans over 700 languages and thousands of dialects, yet the vast majority of automatic speech recognition (ASR) systems support only a small fraction of this diversity. We present SraVaani-1.0, a multilingual ASR model covering 65 Indian languages and dialects, many of which currently have no publicly available or competing ASR system. SraVaani-1.0 is built on a FastConformer architecture and trained from scratch through a three-stage pipeline.In the first stage, we perform self-supervised pretraining on 31,255 hours of unlabelled speech from the VAANI corpus using a contrastive learning objective. In the second stage, we introduce an audio-image representation alignment stage that leverages the paired images and speech available in the VAANI corpus. This multimodal alignment encourages the speech encoder to learn semantically richer representations by exploiting the relationship between visual context and spoken content, thereby improving downstream recognition, particularly for low-resource languages.In the final stage, the aligned encoder is fine-tuned end-to-end using a Hybrid Token-and-Duration Transducer (TDT)-CTC decoder on 31,263 hours of labelled multilingual Indian speech compiled from 24 public datasets spanning 65 languages and dialects. We evaluate SraVaani-1.0 against three state-of-the-art multilingual ASR systems across eight benchmarks. SraVaani-1.0 achieves the lowest word error rate (WER) on a large number of language-dataset pairs while remaining competitive with the best-performing systems on high-resource languages.Most importantly, it is the only open-source evaluated model that provides transcription capability for multiple low-resource and tribal Indian languages, which are assessed exclusively on the VAANI benchmark.