每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-07-23
日期2026-07-23
已评分—
均分—
最高—

Daily Papers — 2026-07-23

12 papers on audio, speech, music, and acoustics.

1. Designed Vocalizations Dataset: Sound-Designed Human and Animal Voices for Non-human Voice Conversion

Authors: Seolhee Lee, Minsu Kang, Yangsun Lee, Woosun Min, Choonghyeon Lee et al.

Categories: eess.AS, cs.SD | Accepted at InterSpeech 2026 Score: 6.60/10 (Obj:8 Id:5 Ind:5 Comp:8 Eff:5 Nov:8 Rep:6)

  • Strength: 首个公开的 non-human designed vocalization 数据集（237,574 samples, 47 presets, 5,774 raw sources），填补了 H2NH-VC 领域的数据空白，seen/unseen 二维 split 框架支持系统化泛化评测。
  • Weakness: 仅一个自研 baseline（无第三方方法对比），baseline 模型与数据集同团队导致评测独立性不足，MOS 绝对值偏低（3.49-3.81/5.0），References 列表为空且核心处理工具为商业软件影响复现。
  • Full review: Claude Code 全文七公理审稿

Advances in AI-based voice conversion have enabled a wide range of media applications, including films, audiobooks, and games. However, most research and public benchmarks still focus on natural human speech, leaving designed vocalizations, such as monster growls and robotic voices, underexplored, partly due to the lack of publicly available resources. To address this gap, we introduce the Designed Vocalizations Dataset, constructed by curating diverse raw vocal sources, including speech and animal vocalizations, and applying professional vocal effects processing to produce corresponding effect modified variants. We further provide a standardized test set with explicit seen/unseen splits over source timbre groups and preset styles to assess generalization under controlled conditions. Finally, we report baseline benchmark results to support reproducible evaluation and future research. The dataset and demo samples are available at https://ncai-official.github.io/speech/publications/designed-vocalizations-dataset/.

2. VibeVoice-ASR-BitNet Technical Report

Authors: Songchen Xu, Ting Song, Shaohan Huang, Zhiliang Peng, Yan Xia et al.

Categories: cs.SD, cs.CL, eess.AS | Technical Report Score: 6.16/10 (Obj:8 Id:5 Ind:9 Comp:6 Eff:5 Nov:5 Rep:8)

  • Strength: 通过异构量化（VAE 用 I8S、LM 用 BitNet I2S）和 progressive QAT，将 VibeVoice-ASR 从 4.62GB 压缩到 1.58GB（2.9×），在 AMD EPYC 上以 3 线程实现 RTF<1，比 Whisper.cpp 快 1.6–2.3×，代码和模型开源。
  • Weakness: 实验仅在 24 核服务器 CPU 上进行却声称 “edge CPUs”，缺少模型大小（7B→1.5B）与量化各自精度影响的消融，缺少异构 vs 均匀量化的对比，核心组件（BitNet ternary、SIMD kernels、ggml）均来自同一团队前作，新颖性有限。
  • Full review: Claude Code 全文七公理审稿

We present VibeVoice-ASR-BitNet, a compressed variant of VibeVoice-ASR optimized for real-time inference on edge CPUs. We apply heterogeneous quantization tailored to the computational characteristics of each stage: the VAE acoustic tokenizer uses full-pipeline INT8 quantization (I8_S) with kernel fusion and SIMD optimization, while the autoregressive language model adopts BitNet-style ternary weights (I2_S). To preserve accuracy under aggressive compression, we employ a progressive quantization-aware training strategy. For inference, we implement custom SIMD kernels and fused operators within the ggml framework targeting both ARM and x86 platforms, achieving real-time recognition with RTF < 1 using as few as 3 CPU threads. VibeVoice-ASR-BitNet is 1.6-2.3x faster than Whisper.cpp at comparable model sizes (~1.6 GB), with only modest accuracy degradation compared to the FP16 baseline.

3. Faster IndexTTS-2: Accelerating and Streaming Autoregressive Zero-Shot Text-to-Speech Synthesis on GPUs

Authors: Muyang Du, Shuang Yu, Junjie Lai

Categories: cs.AI | 4 pages, 2 figures, 3 tables Score: 5.90/10 (Obj:9 Id:5 Ind:9 Comp:5 Eff:8 Nov:5 Rep:5)

  • Strength: 在 A100 上将 IndexTTS-2 端到端推理加速 3.6x（RTF 0.84→0.24），GPT 组件 5.0x，同时 WER/SIM-o/UTMOS 质量损失极小，并首次实现 TensorRT-LLM 对自回归 TTS 的全组件加速与流式/批处理支持。
  • Weakness: 唯一 baseline 为 PyTorch 原版实现，未对比 vLLM/SGLang/ONNX Runtime 等替代加速方案，且四个核心 TensorRT-LLM adaptation 缺乏组件级消融，代码未明确开源。
  • Full review: Claude Code 全文七公理审稿

Autoregressive text-to-speech models achieve strong naturalness but suffer from slow inference due to sequential token generation, limiting their deployment in production applications that require low latency. IndexTTS-2 is a state-of-the-art autoregressive TTS model consisting of a GPT, a flow-matching Diffusion Transformer, and a vocoder. Despite its high synthesis quality, its inference speed barely reaches real-time without streaming or batching support. We present Faster IndexTTS-2, which accelerates all neural network components of IndexTTS-2 for production deployment on GPUs using NVIDIA TensorRT and TensorRT-LLM. Faster IndexTTS-2 also enables streaming synthesis for latency-sensitive interactive applications, and batched inference across all components to maximize GPU utilization. Experiments on the Seed-TTS benchmark for both English and Chinese demonstrate up to 5.0$\times$ speedup on the autoregressive GPT and 3.6$\times$ end-to-end, with minimal degradation in word error rate, speaker similarity, and naturalness. Our methodology provides a practical reference for efficiently accelerating similar autoregressive speech models on GPUs.

4. Latent Variable-Mediated Cross-Learning for Few-Shot Acoustic Impedance Imaging

Authors: Junheng Peng, Yong Li, Mingwei Wang, Yi Bao

Categories: cs.CV, physics.geo-ph | The manuscript is currently under review Score: 5.79/10 (Obj:8 Id:6 Ind:8 Comp:5 Eff:6 Nov:5 Rep:6)

  • Strength: 提出闭式一阶 Tikhonov 去卷积算子替代显式 wavelet 提取网络，仅 56.5k 参数即在前序 PICL 基础上进一步降低复杂度，在 SEAM 和 Marmousi 2 基准上保持竞争力。
  • Weakness: 相对于同作者前序工作 PICL (arXiv 2512.11521) 的核心增量仅为闭式算子替代网络模块，属工程优化而非机制创新；且未与 2025 年最新半监督 baseline（Swin Transformer, KAN）对齐标签率比较。
  • Full review: Claude Code 全文七公理审稿

Acoustic impedance imaging is a fundamental yet severely ill-posed problem in subsurface analysis: the seismic wavelet is unknown, observations are band-limited, and labeled well-log samples are extremely scarce (typically <1% of all traces). Existing semi-supervised deep learning methods mitigate few-shot problem by incorporating forward modeling, yet they either rely on inaccurate prior wavelet assumptions or introduce auxiliary networks, leading to unstable optimization and degraded performance. We propose RD-SCL, a novel framework that integrates regularized deconvolution with semi-supervised cross-learning. At its core lies a differentiable, closed-form first-order Tikhonov deconvolution operator that dynamically estimates the latent wavelet in the frequency domain during training, providing stable physics-guided feedback without explicit auxiliary networks and fixed wavelet priors. Building on this operator, we design a symmetric cross-learning that enforces consistency between predictions on labeled and unlabeled data, thereby effectively exploiting abundant unlabeled traces. Extensive experiments on the SEAM and Marmousi 2 benchmarks demonstrate that RD-SCL consistently outperforms state-of-the-art supervised and semi-supervised methods, achieving substantial gains with lower computational cost. With only 56.5k learnable parameters and competitive runtime, RD-SCL offers a practical, physically consistent, and efficient solution for acoustic impedance imaging.

5. X$^3$-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment

Authors: Dongjie Fu, Di Cao, Xize Cheng, Zihan Zhang, Wenxu Jia et al.

Categories: cs.LG Score: 5.60/10 (Obj:8 Id:6 Ind:5 Comp:5 Eff:7 Nov:5 Rep:5)

  • Strength: 三层对称语料将跨模态蒸馏监督从文本可转写推理扩展到声学事件和副语言对话推理，BIG Bench Audio 达 93.6（超 GPT-4o-Audio +5.1），且 OOD 退化仅 -1.2/-2.3（vs SFT -5~-6）。
  • Weakness: 核心算法是同团队前作 X-OPD 的增量扩展（teacher 加 answer 条件 + PPO ratio），未隔离算法贡献与数据贡献的交叉 ablation，且未声明代码/数据/checkpoint 开源，Tier-3 meta-caption 生成 prompt 缺失。
  • Full review: Claude Code 全文七公理审稿

While large audio-language models have achieved remarkable progress in auditory perception, they still lag behind text-based large language models in deep logical reasoning, primarily due to the scarcity of high-quality audio reasoning data. To bridge this gap, we propose X$^3$-OPD, a cross-modal on-policy distillation framework that transfers reasoning capabilities from a powerful text teacher to an audio-language student. During training, the student generates reasoning trajectories conditioned on its own acoustic perception, while the teacher provides token-level guidance using matched textual inputs and verified answers. We further construct a three-tier symmetric corpus covering textual reasoning rendered into speech, audio-event reasoning grounded in complex acoustic scenes, and spoken-dialogue reasoning involving paralinguistic cues. This design extends cross-modal distillation beyond textually recoverable content to reasoning grounded in non-linguistic events, prosody, and conversational context. Experiments on MMSU, MMAU, BIG Bench Audio, and MMAR demonstrate that X$^3$-OPD substantially improves audio-grounded reasoning and chain-of-thought quality while largely preserving the model’s existing capabilities under domain shift.

6. From Read Speech to Spoken Digits: A Task-Specific Evaluation of Speech Privacy With Informed Attackers

Authors: Jule Pohlhausen, Anjana Rajasekhar, Anna Leschanowsky, Joerg Bitzer

Categories: eess.AS Score: 4.79/10 (Obj:5 Id:5 Ind:5 Comp:5 Eff:4 Nov:4 Rep:9)

  • Strength: 首次将语音隐私评测从通用 WER 转向针对数字这一封闭词表敏感信息类的任务特定评测，发现 shredding 对 bag-of-features 攻击者完全失效（时间置换不变性导致 MFCC 均值/方差不变）这一有机制解释价值的结论，且代码与模型全开源。
  • Weakness: 缺乏与 2025 年 VoicePrivacy Attacker Challenge (55 个攻击系统) 和 Williams et al. 2021 等最直接相关工作的比较与讨论，”first” 主张不严谨；DNN 攻击者使用 oracle 边界分割使结果为上界而非现实威胁估计；实验仅覆盖英文 0-9 朗读数字，外延远小于 “linguistic speech content protection” 的 claim。
  • Full review: Claude Code 全文七公理审稿

Protecting speech privacy in real-life audio recordings is a growing concern. This contribution evaluates the effectiveness of three obfuscation techniques in protecting linguistic speech content, using digit recognition as a task-specific and practically motivated evaluation scenario. As a first baseline, a general-purpose speech recognition model and a digit-specific classifier were applied as informed attackers to recognise both single digits and concatenated digit sequences. Our experimental results demonstrate significant differences in recognition performance across digit modality, speech rate, and attack model. These findings emphasize the need for more comprehensive and application-oriented evaluation methods to ensure speech privacy.

7. Toward Interpretable Speech Deepfake Detection using Artifact-Specific Experts and Calibrated Detection Scores

Authors: Viola Negroni, Xin Wang, Wanying Ge, Paolo Bestagini, Junichi Yamagishi et al.

Categories: cs.SD | In peer review Score: 4.70/10 (Obj:6 Id:3 Ind:5 Comp:5 Eff:2 Nov:5 Rep:5)

  • Strength: 提出了一个理论清晰的可解释语音 deepfake 检测框架（5 个 artifact-specific experts + LLR 校准 + sum aggregation），在 ASVspoof 2019 上 EER 19.76%/Cllr 0.638，展示了 per-attack 的可解释 LLR 映射，并演示了通过添加 silence-duration 专家诊断 dataset shortcut 的应用。
  • Weakness: 性能远低于 SOTA AASIST（EER 0.83%），无任何外部 baseline 对比，校准集与评测集同源存在循环论证风险，代码未开源，ASVspoof 5/SpoofCeleb 上未报告整体 EER/Cllr，专家间无 synergy 证明。
  • Full review: Claude Code 全文七公理审稿

In this work, we propose an interpretable framework for speech deepfake detection based on artifact-specific expert models. Rather than relying on black-box decisions, the framework provides human-understandable evidence, which is critical in high-stakes settings. Each expert is trained to detect a specific speech synthesis artifact, and its output is calibrated into a log-likelihood ratio that serves as an interpretable evidence score. We evaluate five artifact-specific experts and show that, with proper calibration, they can capture their target artifacts and produce meaningful evidence. Importantly, each expert estimates only the presence of its assigned artifact rather than directly performing the final decision. Their outputs are aggregated into an ensemble to produce the actual real-versus-fake classification, while maintaining interpretability by indicating how strongly each expert supports or contradicts a fake classification. Results show that artifact-specific experts capture interpretable signals of synthetic speech across multiple generation pipelines.

8. An Evaluation Framework for Structured Audio Captions Validated by Controlled Perturbations

Authors: Liang-Yuan Wu, Sripathi Sridhar, Mark Cartwright, Magdalena Fuentes

Categories: cs.CL, cs.SD | submitted to DCASE 2026 Score: 4.70/10 (Obj:6 Id:5 Ind:4 Comp:5 Eff:3 Nov:4 Rep:8)

  • Strength: 提出首个面向结构化音频 caption 的五轴评测框架，受控扰动实验（4 类型 × 3 严重度）有效验证了 LLM judge 对 paraphrase 的鲁棒性（保持高分）和对真实语义/声学错误的单调敏感性。
  • Weakness: 框架仅在 GT vs 扰动 GT 上验证，未评测任何真实 AAC 模型输出，且缺乏人类评测锚点来校准 LLM judge 的评分可靠性。
  • Full review: Claude Code 全文七公理审稿

Recent advancements in automated audio captioning (AAC) have shifted from monolithic sentence generation toward structured formats that explicitly disentangle distinct acoustic and semantic properties. However, evaluating this heterogeneous data remains a significant challenge. Existing caption metrics focus on flat textual outputs and fail to reliably assess multimodal attributes. To bridge this gap, we propose a multi-axis evaluation framework tailored for structured audio descriptions. Building on the AudioCards dataset, we evaluate outputs across five orthogonal axes: tag-sets, descriptions, logical reasoning, numeric measurements, and spectral profiles. Our approach combines Large Language Model (LLM) judges to capture semantic nuance with deterministic computational metrics to precisely measure acoustic deviations. To rigorously validate the reliability of this framework, we introduce a controlled perturbation testing protocol that injects typed, graded errors into groundtruth annotations. Our results demonstrate that this framework successfully distinguishes meaning-preserving paraphrases from genuine semantic and acoustic corruptions.

9. Investigating Codec-Internal Latent Audio Watermarking for Neural Codec Robustness

Authors: Zi Hu, Houmin Sun, Linxi Li, Yechen Wang, Liwei Jin et al.

Categories: cs.SD | 7 pages. Submitted to IEEE Spoken Language Technology Workshop (SLT) Score: 4.70/10 (Obj:8 Id:5 Ind:6 Comp:5 Eff:5 Nov:3 Rep:4)

  • Strength: 在 codec-like autoencoder 连续潜在空间嵌入 32-bit watermark，通过 EnCodec-aware 训练将 EnCodec-24k bit accuracy 从 78.8% 提升至 95.6%/97.1%，并通过 backbone probe 实验揭示 reconstruction capacity 与 watermarking 兼容性不完全正相关。
  • Weakness: 无任何外部 baseline (WavMark/AudioSeal/Latent-Mark) 对比，核心 idea “codec latent space watermarking” 已被 Latent-Mark (Interspeech 2026, 2026-03) 先行发表且未被引用，EnCodec-16k 仍接近随机，代码未开源且训练超参数不完整。
  • Full review: Claude Code 全文七公理审稿

Neural audio codecs are challenging transformations for audio watermarking because they re-encode, quantize, and resynthesize speech. This paper investigates continuous latent-space watermarking for codec robustness. Instead of adding a watermark only to the waveform or spectrogram, we embed a 32-bit message into the continuous latent representation of a codec-like speech autoencoder. The pipeline uses a SEANet-style encoder-decoder, a Conformer-based message embedder, RVQ-guided latent decomposition, and a latent-domain detector trained under signal-processing and neural-codec transformations. Rather than proposing a final universal watermarking baseline, we characterize the trade-offs that appear when the watermark carrier is moved before neural decoding. On 48 kHz speech, EnCodec-aware training improves EnCodec-24k bit accuracy from 78.8% to 95.6% and 97.1%, while PESQ decreases from 3.727 to 3.514 and 3.427.

10. Spectrogram-Based Joint Detection, Localization, and Classification of Events in Continuously Recorded IBR Waveforms

Authors: Shivanshu Tripathi, Maziar Raissi, Hamed Mohsenian-Rad

Categories: cs.SD, eess.SY Score: 4.70/10 (Obj:6 Id:5 Ind:8 Comp:4 Eff:5 Nov:2 Rep:8)

  • Strength: 在小规模真实 IBR 波形数据上，spectrogram + U-Net 在事件检测和定位上达到 AP@0.5=0.985/IoU=0.926（三相），显著超过 raw time-series baseline，且代码数据开源。
  • Weakness: 方法是 STFT+U-Net+YOLO-style detection 的直接组合，核心 idea 与 MTTNet (IEEE TSG 2025) 高度重叠；三相故障分类准确率仅 0.38（4 类，接近随机），且未与领域内关键 baseline 对比。
  • Full review: Claude Code 全文七公理审稿

Continuously recorded high-resolution waveform measurements provide rich information about fast power system dynamics. However, they require automated methods to identify events. This problem is addressed by developing a spectrogram-based framework to jointly detect, localize, and classify events in real-world continuously recorded waveforms at the terminal of an Inverter-Based Resource. We recast this problem as a temporal object detection problem on spectrogram images, as they capture the transient and harmonic signatures more explicitly than in raw waveform data. Each time-series waveform is transformed using the short-time Fourier transform, and the resulting per-channel spectrograms are stacked as a tensor for event detection. We benchmark this method against a detector operating directly on raw time-series measurements. Experiments on single-phase disturbances and three-phase faults demonstrate that the proposed spectrogram method consistently improves event detection, localization, and classification over the raw waveform baseline.

11. TF-MossFormer: Integrating Convolution Gated Local-Global Attentions for Enhanced Time-Frequency Domain Monaural Speech Separation

Authors: Shengkui Zhao, Zexu Pan, Haoxu Wang, Biao Tian, Bin Ma et al.

Categories: cs.SD | 5 pages, 3 figures, 5 tables, Interspeech 2026 Score: 4.60/10 (Obj:6 Id:4 Ind:8 Comp:4 Eff:3 Nov:4 Rep:6)

  • Strength: 在 WSJ0-2Mix 上三个参数规模（5.9M/16.9M/25.4M）均取得 SI-SDRi 22.6/24.0/24.4 dB，在非 DM baseline 中表现一致，消融实验覆盖窗口大小和注意力排序。
  • Weakness: 虚假 SOTA 声明——实际落后 TF-Locoformer(L)+DM（25.1 dB）0.7 dB，遗漏 SepReformer 和所有 DM 变体 baseline；组件均为已有方法拼装，与同一作者前作 SPGM 的 local-only 结论存在未解释的矛盾。
  • Full review: Claude Code 全文七公理审稿

Transformers with global attention capture long-range dependencies but can miss the fine-grained local continuity crucial for speech separation. We propose TF-MossFormer, a time-frequency transformer that combines local and global attention to jointly model short- and long-range contexts for monaural speech separation. At its core is a content-aware sliding-window attention mechanism that dynamically adapts receptive fields for stronger local interactions, avoiding the rigidity of static convolutions. Unlike time-domain chunk-based methods, TF-MossFormer leverages the 2D spectrogram to model structure along both time and frequency axes. Convolutional gating between attention layers further improves feature selection and information flow. TF-MossFormer achieves SI-SDRi of 22.6, 24.0, and 24.4 dB on WSJ0-2Mix with 5.9M, 16.9M, and 25.4M parameters, respectively, outperforming prior approaches.

12. DONDO: Open w2v-BERT Speech-Recognition Base Models for African Languages

Authors: Paul Azunre

Categories: cs.CL Score: 4.05/10 (Obj:6 Id:3 Ind:5 Comp:4 Eff:3 Nov:4 Rep:5)

  • Strength: 覆盖 27 个非洲语言变体（约 1.15 亿 L1 说话人）的 26 个 ASR 模型以 Apache-2.0 开源发布，多语 annealed 模型 in-domain 平均 WER 达 10–13%，部分语言（French、Fante）超过自训单语 baseline。
  • Weakness: 无任何外部基线对比（Whisper、MMS、Omnilingual ASR 等均未比较），两个方法贡献（两步 LR 退火、prefix-frame 语言条件）均无消融实验验证必要性和有效性，评测仅在训练域内进行，训练数据和评测代码未公开。
  • Full review: Claude Code 全文七公理审稿

We present DONDO, a family of open, permissively licensed automatic speech recognition (ASR) base models for African languages, built on the w2v-BERT 2.0 self-supervised speech encoder. DONDO comprises twenty-one monolingual models and five multilingual models spanning twenty-seven language varieties across Ghana, Sierra Leone, Nigeria, Senegal, Kenya and Zimbabwe. Models are fine-tuned primarily on read speech drawn from religious texts, which offer broad, license-clear and orthographically consistent coverage for languages that otherwise lack transcribed audio. We describe a two-step (and, for one family, three-step) learning-rate-annealed fine-tuning procedure that first adapts a shared multilingual model at a high learning rate and then anneals it to recover, and in several cases surpass, strong monolingual baselines. We further describe a lightweight language-conditioning mechanism that injects a one-hot language identity as a sequence of prefix frames prepended to the acoustic features, allowing a single multilingual checkpoint to be steered to a target language at inference. Across the five multilingual families the annealed models reach average word error rates (WER) of 10-13%, closing most of the gap to monolingual models while covering many languages in a single checkpoint. All models are released on the Hugging Face KhayaAI organisation under the Apache-2.0 license (attribution only) so that others may fine-tune them freely, including for commercial use. We provide a conservative estimate that the languages covered are spoken by on the order of one hundred million first-language speakers, and by substantially more when second-language use is included.