每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-07-27
日期2026-07-27
已评分—
均分—
最高—

Daily Papers — 2026-07-27

10 papers on audio, speech, music, and acoustics.

1. Revisiting Vocos: That Phasiness Business in Time-Frequency Neural Vocoding

Authors: Ünal Ege Gaznepoğlu, Frank Zalkow, Mohammad Joshaghani, Emanuël A. P. Habets, Nils Peters et al.

Categories: eess.AS | Accepted at IWAENC 2026 Score: 7.60/10 (Obj:9 Id:9 Ind:8 Comp:8 Eff:7 Nov:7 Rep:6)

  • Strength: 通过 oracle magnitude/phase 消融和相位差预测子任务，精确定位 Vocos 的 1D 卷积为相位建模瓶颈，并发现”不一致 magnitude 是 feature 不是 bug”这一反直觉结论，用 37.1K 参数的 Conv2D 在相位差预测上超越 15.0M 的 Conv1D（Lwa 0.112 vs 0.646）。
  • Weakness: 论文诊断了问题但未解决——Conv2D 在相位差子任务上的优势无法直接迁移到端到端 vocoding（harmonic structure 建模挑战），代码开源情况不明确，D2 评测数据为 proprietary。
  • Full review: Claude Code 全文七公理审稿

Recently, time-frequency neural vocoders have been approaching the state-of-the-art quality of time-domain neural vocoders. Vocos is a notable example due to its efficiency, but its audio quality lags behind the time-domain vocoders and the reasons remain debated. Thus, in this study, we revisit Vocos from a phase reconstruction perspective. First, we quantify the gap between time-domain and time-frequency domain vocoders using bandlimited mel spectrograms as inputs. Later, via an ablation study, we verify the Vocos architecture is effective for magnitude modeling, but less so for phase. We then adapt the Vocos backbone to predict phase differences, a precursor for phase reconstruction, and identify 1D convolutional layers are hindering their accurate prediction. Our findings indicate that future research needs to focus on inductive biases that allow the architecture to better model the time-frequency structure of speech signals, without sacrificing the support for arbitrary input representations.

2. AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars

Authors: Hengyuan Zhang, Jingna Sun, Meiguang Jin, Junfeng Ma

Categories: cs.CV | 9 pages, 5 figures Score: 6.42/10 (Obj:8 Id:6 Ind:7 Comp:6 Eff:7 Nov:6 Rep:5)

  • Strength: 2-NFE 实现 720p 长视频数字人生成，对比自家 120-NFE backbone 达到 60x 加速且全指标最优，EADD 的 bridge-endpoint anchor 机制是 DMD 蒸馏的真实变体。
  • Weakness: 缺失直接竞品 TurboTalk（1-step avatar distillation）的比较，长视频 benchmark 缺失 FID 核心指标，400h 训练数据和测试集均私有未公开，复现性严重不足。
  • Full review: Claude Code 全文七公理审稿

Production-ready audio-driven avatar generation requires efficient inference without sacrificing fidelity or motion expressiveness. However, existing acceleration methods often compromise quality through restrictive architectural choices, such as causal attention and short temporal horizons, or by reducing model capacity and resolution. Without such compromises, we propose AptAvatar, a 14B-parameter long-form audio-driven avatar generation framework that delivers fast and expressive inference. For efficiency in production-level applications, AptAvatar addresses the extreme two-step generation challenge. To bridge the gap between the multi-step teacher model and the two-step student model, we introduce Endpoint-Anchored Distribution Distillation. It augments vanilla distribution matching with a dedicated Anchor Score Estimator trained on the trajectory-endpoint distribution defined from a frozen pretrained 4-step bridge generator. This provides an attainable endpoint-level anchor for the evolving two-step student. To improve long-horizon consistency, we further introduce Self-Generated History Replay, which reuses cached outputs from earlier generator checkpoints as history conditions during chunk-wise training. This approximates inference-time conditioning on self-generated histories without costly online rollouts, mitigating quality degradation from accumulated history errors. Extensive experiments demonstrate that AptAvatar generates vivid 720p long-form avatar videos with only 2 NFEs, achieving a 60x speedup while preserving visual fidelity and long-horizon identity. Code is available at https://github.com/TaoLiveAIGC/AptAvatar

3. Leveraging Gradient Reversal Loss and Multitask Learning for Datasets-Aware Audio Deepfake Detection

Authors: Mingrui Liang, Thomas Thebaud, Lukasz Wojciak, Laureano Moro Velazquez, Yishay Carmiel et al.

Categories: eess.AS | Accepted by SPSC 2026. Camera-ready version pending Score: 5.95/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:6 Nov:4 Rep:8)

  • Strength: 使用数据集身份作为辅助标签的 MT 和 GRL 方法在 Speech DF Arena 基准上将 Average EER 相对降低 13.14%、Pooled EER 相对降低 5.32%，以 315.4M 参数在开源系统中排名第 3，且模型权重公开可复现。
  • Weakness: 核心技术 (GRL + 多任务用于反欺骗检测) 已被 Dao et al. (ICASSP 2026) 发表，本文未引用或比较该最直接相关工作；新颖性主要限于辅助标签选择 (数据集 ID vs 说话人 ID)，属增量贡献；多个数据集上性能劣于基线。
  • Full review: Claude Code 全文七公理审稿

Recent advances in speech synthesis and voice conversion, which pose threats to security and privacy, have underscored the need for deepfake detection technology. Although existing detection systems achieve strong performance on individual datasets, they often fail to generalize across diverse datasets. Prior methods for improving generalization, including data augmentation, adversarial training on auxiliary factors such as language or codec types, and Mixture-of-Experts (MoE), are limited by predefined augmentation coverage, difficulties in obtaining auxiliary factors, and substantial model complexity. In this work, we propose a practical dataset-aware framework for deepfake detection. Our method targets heterogeneous datasets for which auxiliary annotations such as language, codec, or spoofing method may not be consistently available. We therefore rely only on dataset identity as a naturally available supervisory signal for multitask (MT) and gradient reversal layer (GRL) training, allowing the model to investigate both dataset-aware multitask supervision and adversarial suppression of dataset-specific information. We conduct experiments following the 2025 Speech DeepFake Arena benchmark protocol, evaluating our model across multiple evaluation datasets and reporting aggregate performance in terms of Equal Error Rate (EER), including Average EER and Pooled EER. Compared with the baseline, MT reduces Average EER by 13.14% relatively, while GRL reduces Pooled EER by 5.32% relatively. These results demonstrate that our method can improve aggregate detection performance across heterogeneous evaluation datasets, offering a practical solution for deploying reliable deepfake detection systems on diverse and unseen real-world data.

4. Qwen-Audio-3.0-TTS: Freely Controllable and Highly Robust Speech Synthesis with Multi-Stage Training Paradigm

Authors: Bajian Xiang, Cheng Wen, Han Zhao, Hao Wang, Haoxu Wang et al.

Categories: eess.AS | 19 pages Score: 5.95/10 (Obj:8 Id:5 Ind:6 Comp:5 Eff:8 Nov:5 Rep:2)

  • Strength: 在 SEED-TTS-Eval、CV3-Eval 16 语言、cross-lingual 12 方向和噪声鲁棒性上取得最强 aggregate 结果，并获独立 Artificial Analysis leaderboard 第一（Elo 1237），baseline 覆盖四类 TTS 范式的 13 个代表系统。
  • Weakness: 五阶段训练范式无逐阶段消融，各模块贡献无法因果识别；代码/模型/数据/评测脚本均未公开，自建 benchmark 2909 条评测数据未发布，核心结果不可独立复现。
  • Full review: Claude Code 全文七公理审稿

In this report, we present Qwen-Audio-3.0-TTS, a production-oriented speech synthesis system that jointly advances content consistency, speaker similarity, prosodic naturalness, audio quality, controllability, multilingual coverage, efficiency, and robustness. It combines a 12.5~Hz low-frame-rate speech tokenizer for reduced inference latency with a five-stage progressive training paradigm for coordinated language model (LM) and flow-matching model (FM) optimization. The model provides production-level control through free-style natural-language instructions and fine-grained inline tags, while supporting 16 languages, 20 Chinese dialect regions, one-pass long-form synthesis up to 3 minutes, and robust generation from noisy, reverberant, or unclear reference speech. Across SEED-TTS-Eval, CV3-Eval, instruction-following, long-form, and acoustic-robustness evaluations, Qwen-Audio-3.0-TTS achieves state-of-the-art performance on many reported dimensions or the strongest aggregate results. It also ranks first on the independent Artificial Analysis Text-to-Speech Leaderboard. These results establish Qwen-Audio-3.0-TTS as a strong foundation for production-level speech synthesis.

5. TaoMate: Anchor-Guided Memory Bridging Evolving and Reference States for Real-Time Audio-Video Digital Human Generation

Authors: Qijun Gan, Chenwei Zhang, Meiguang Jin, Junfeng Ma, Qiu Shen

Categories: cs.CV Score: 5.95/10 (Obj:8 Id:5 Ind:5 Comp:5 Eff:8 Nov:5 Rep:5)

  • Strength: 在 long-form 音视频数字人 benchmark 上全面取胜（LipSync 5.918, Long Consistency 0.9755, Color Δ 0.00260, 35 FPS/3GPU），anchor agreement gate 控制 memory update 使 Color Δ 相比无 gate 降低 20.5%。
  • Weakness: 消融实验仅覆盖 3 个粗粒度组件（Memory/FiLM/Gate），持久记忆内部分解、蒸馏三策略、stage-parallel 加速均无独立消融；训练-评测-初始化均依赖 LTX-2.3 teacher 形成部分闭环；代码开源状态未确认。
  • Full review: Claude Code 全文七公理审稿

Real-time long-form digital-human generation relies on causal models to extend audio-visual content while preserving subject appearance and audio-video synchronization across successive segments. A bounded cache retains local motion and phonetic context but discards older evidence, whereas attending to the complete generated history is computationally expensive and can propagate accumulated errors. We present \method, an anchor-guided persistent-memory framework for few-step joint audio-video generation. The framework preserves an immutable visual anchor, compresses completed video and audio blocks into fixed-capacity dynamic states, and retrieves those states through modality-specific residual attention without extending the active cache. A reference-aware modulation method additionally conditions video features on dynamic and anchor appearance statistics. Anchor-preserving causal-context distillation varies rollout horizon, prefix provenance, and cache-history reliability while keeping the immutable visual anchor unperturbed. By separating persistent memory from stage-local denoising dependencies, \method further admits stage-parallel execution across blocks, accelerating autoregressive inference without pipeline-specific retraining. We evaluate long-form video continuations with appearance, temporal, synchronization, facial, and speech diagnostics. Results show that \method preserves stable appearance across prompt-conditioned segments and strong audio-visual synchronization under autoregressive generation. Our project page is https://taoliveaigc.github.io/TaoMate.

6. MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognition

Authors: Sangmin Lee, Woojin Chung, Woongjib Choi, Hong-Goo Kang

Categories: cs.CL, cs.SD | Accepted to COLM 2026, Github: https://github.com/sanghyang00/molge Score: 5.47/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:5 Nov:5 Rep:6)

  • Strength: 在 495 种语言上系统性地比较了六种语言分组策略，发现显式语言先验在更复杂的正字法领域带来更大的收益（CER 相对降低 19%），并伴随低资源语言 CER 从 42% 降至 30%。
  • Weakness: 未与最直接的已发表基线（Kwon & Chung 2023 的 MoLE, Wang et al. 2023 的 Language-Routing MoE）进行比较；相对于稠密/随机基线的绝对改进适中（CER 22-24%），且在简化音系领域策略间的差异可以忽略不计。
  • Full review: Claude Code 全文七公理审稿

Massively multilingual automatic speech recognition (ASR) models covering hundreds of languages must maintain robust performance across diverse linguistic and acoustic conditions. However, these models often encounter the curse of multilinguality, where model capacity is diluted across languages. To address this challenge, we propose Mixture of Language Group Experts (MoLGE), built upon speech self-supervised models (S3Ms). MoLGE assigns dedicated expert modules to clusters of similar languages, reducing the number of required submodules compared to conventional language-specific Mixture-of-Experts (MoE) schemes. It further integrates a hierarchical Low-Rank Adaptation (LoRA) strategy into the disentangled acoustic and linguistic components of the S3M architecture, enabling efficient modeling of language-specific characteristics while maintaining parameter efficiency. Further, we investigate the impact of language grouping strategies based on both linguistic and data-driven criteria on overall performance, providing an interpretable perspective on how language structure influences scalability in multilingual speech systems. In experiments, we evaluate MoLGE on a multilingual benchmark encompassing 495 languages. Results demonstrate that MoLGE consistently outperforms dense multilingual baselines with a minimal increase in trainable parameters. Notably, these language grouping strategies yield substantial improvements for both phonetic and orthographic aspects of ASR modeling. Our findings suggest that structured language specialization provides an effective pathway for massively scaling language coverage of multilingual ASR.

7. Mind the Microphone Gap: Benchmarking Array Upsampling Strategies for Latent Acoustic Mapping

Authors: Philipp Schmidt, Huw Cheston, Juan Azcarreta, Adrian Stepien, Çağdaş Bilen et al.

Categories: cs.SD | IWAENC 2026 Score: 5.47/10 (Obj:6 Id:5 Ind:8 Comp:5 Eff:5 Nov:4 Rep:8)

  • Strength: 系统性的 6×3 基准测试产生了两个反直觉发现——SRCNN (63K 参数) 与 GAN (4.3M 参数) 匹配，且训练策略比架构更重要——并得到新颖的分阶段 CMD 诊断支持，揭示了端到端训练漂移。
  • Weakness: 所有上采样架构均为导入的现成图像/HRTF SR 模型，没有任何针对 CSM 的设计；在 STARSS23 上的绝对定位误差为 54-64° (对实际 SELD 不可用)；未与任何外部 DoA 基准 (DeepWave, MUSIC) 进行比较；结果缺乏统计不确定性。
  • Full review: Claude Code 全文七公理审稿

Latent Acoustic Mapping (LAM) is a self-supervised learning method that generates high-resolution spherical acoustic maps from multichannel recordings without labelled data, matching supervised baselines on direction-of-arrival benchmarks. However, LAM degrades significantly with sparse 4-channel arrays, as the low-resolution cross-spectral matrix captures far less spatial information than the 32-channel inputs LAM was designed for. We benchmark a diverse set of upsampling architectures, spanning lightweight convolutional networks, iterative back-projection models, physics-informed networks, and generative adversarial approaches. We also study whether aligning these upsamplers with LAM by training them jointly or in different stages helps preserve the spatial structure that LAM depends on. Results show that the original full-resolution LAM is the strongest, that separately trained lightweight models are the most competitive learned approaches, and that representation alignment between the upsampler and LAM matters more than model complexity.

8. Let Me Look at You: Advanced Facial Expression Modeling for Conversational Speech Synthesis

Authors: Yifan Hu, Shuwei He, Rui Liu, Haizhou Li

Categories: cs.HC, cs.CL, eess.AS | 10 pages, 5 figures, 5 tables. Accepted by ACM MM 2026 Score: 5.26/10 (Obj:8 Id:5 Ind:5 Comp:5 Eff:6 Nov:4 Rep:7)

  • Strength: 构建 VSDD-1K（1033 小时 face-to-face 对话语料，>85% 有效面部帧），规模显著超出现有同类数据集；AUTokenizer 将面部表情压缩为单 token 用于 LLM-based CSS，在三个数据集上全面优于前作 UniTalker（SIM +0.02, ACCE +0.05 on MultiDialog）。
  • Weakness: 方法由已有组件拼装（CosyVoice2 基座 + FSQ 量化 + ConvNeXt + DPO 扩展），缺少机制解释；VSDD-1K 同时作为训练和评测数据存在独立性隐患；DualDPO 的 chosen/rejected 构造是 self-referential；AU classification F1 仅 0.65/0.75 说明单 token 压缩有损。
  • Full review: Claude Code 全文七公理审稿

Conversational Speech Synthesis is a fundamental component of human-computer interaction, aiming to generate contextually appropriate, expressive, and empathetic speech. However, facial expressions encode subtle and rich affective cues that are crucial for empathetic speech interaction, whereas existing approaches often overlook this important modality. In addition, the lack of large-scale natural conversational datasets with both speech and visual modalities also limits the development of visual affect understanding in conversational settings.To address these limitations, we propose FacialTalker, a facial-expression-aware CSS framework built upon a large language model backbone. To efficiently encode facial expressions, we propose AUTokenizer, a single-codebook visual tokenizer that discretizes each frame-level facial expression into a compact token, trained with supervision from combinations of facial Action Units. We further introduce a dual direct preference optimization (DualDPO) strategy, which extends the DPO by jointly imposing preference constraints on both visual and speech token sequences, to enhance the model’s understanding of facial expressions and speech semantics in multimodal conversational contexts. Moreover, we construct VSDD-1K, a large-scale multimodal dialogue dataset collected through a fully automated pipeline from real-world Internet conversations, comprising over 1,033 hours of synchronized speaker videos and speech, with more than 85\% of frames containing valid faces. Extensive objective and subjective experiments demonstrate that FacialTalker consistently outperforms strong baselines in facial-expression perception and speech synthesis quality, generating speech that is more natural, expressive, and better aligned with the conversational context. The results also validate the effectiveness of our training strategy and dataset construction pipeline.

9. Modeling Stylistic Co-evolution in Symbolic Music Heritage Collections

Authors: Yulong He, Ivan Smirnov, Yanming Li

Categories: cs.SD, cs.GT Score: 4.10/10 (Obj:5 Id:2 Ind:5 Comp:4 Eff:4 Nov:5 Rep:6)

  • Strength: 构建了从符号和弦序列到跨国影响网络的可复现流水线（和弦嵌入→卡尔曼滤波轨迹→DeGroot/FJ意见动力学），在480部作品语料库上的估计结果与音乐史学叙述定性一致。
  • Weakness: 缺少任何外部基线比较（未与Lahnala 2021、Weiss 2019等最直接同类工作对比），无组件级消融实验，480作品/5国的小样本粒度与”国家级风格”声明单元不匹配，且音乐学验证为确认性叙事而非独立测试。
  • Full review: Claude Code 全文七公理审稿

Digitized musical heritage collections offer new opportunities to examine how stylistic traditions change over historical time, but computational analyses often reduce musical works to static classifications or similarity scores. This article proposes a representation-to-dynamics framework for studying cross-cultural harmonic change in Western art music. Starting from symbolic chord sequences, we derive contextual chord embeddings, project work-level representations into a shared harmonic space, and reconstruct country-level trajectories through temporally causal Kalman filtering. These trajectories are then modeled with DeGroot and Friedkin–Johnsen dynamics, yielding interpretable influence-like networks and estimates of stylistic anchoring. We apply the framework to a curated corpus of 480 dated works from 1875 to 1940 across Russia, France, Germany, Austria, and a heterogeneous ‘‘Others’’ group. Models fitted on 1875–1925 are evaluated through recursive forecasts for 1930–1940, testing whether the estimated dependency structure remains informative across a potentially changing historical and stylistic regime. The estimated trajectories and influence patterns are broadly consistent with established music-historical accounts of late Romantic and early modernist exchange, including the close relationship between German and Austrian traditions and historically plausible cross-currents between Russian and French traditions. PCA-variance-weighted estimation provides modest improvements while preserving a single interpretable influence network. Rather than treating the estimated matrices as direct causal evidence, the framework offers a reproducible quantitative layer for cultural heritage research, complementing archival and musicological interpretation.

10. Looking for Affect in Spontaneous Finnish Speech through Linguistic Interpretability

Authors: Kalle Lahtinen, Liisa Mustanoja, Okko Räsänen

Categories: cs.CL | Accepted for publication at Interspeech 2026, Sydney, Australia Score: 3.95/10 (Obj:5 Id:5 Ind:5 Comp:3 Eff:5 Nov:2 Rep:5)

  • Strength: 在首个芬兰语自发语音情感语料 FinnAffect 上系统验证了 127 种文本-音频特征组合，确认 valence 由文本/音频互补驱动（CCC 0.270→0.428）、arousal 主要由声学驱动（CCC 0.688），与 Wagner 2023 等英语结论一致。
  • Weakness: 核心结论是已知现象在新语言上的复现而非新知识，方法为现成模型 mean-pooling+MLP 的特征枚举无新机制，承诺的”linguistic interpretability”分析失败，valence 最佳结果含人类 arousal 标注作为输入（识别污染），未对比主流多模态 SER 强 baseline，代码未发布。
  • Full review: Claude Code 全文七公理审稿

Existing research on affect in speech has shown how acoustic surface characteristics and content-related linguistic aspects of speech both relate to perceived emotional arousal and valence. However, it is not clear what the relative contributions of these two factors are in the perceptual process. This is especially true for Finnish, for which most existing studies focus on either acoustic-phonetic or text analysis. This paper presents a study where we systematically explore the combinatory role of text- and audio-based features in modeling the human perception of valence and arousal using a newly released affective speech corpus for spontaneous Finnish. We show that the combination of text- and audio-based features improves valence regression results over the individual modalities, whereas for arousal regression the complementary effect is not substantial. The results support prior findings from other languages, providing new data and knowledge on spontaneous Finnish speech.