每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-08-19
日期2026-08-19
已评分—
均分—
最高—

Daily Papers — 2026-08-19

11 papers on audio, speech, music, and acoustics.

1. Introducing the Privacy-HSD Trade-off: Hate Speech Detection, but not at the Cost of Privacy

Authors: Stephen Meisenbacher, Vlad Garbuz, Chirill Donos, Maxim Dnestreanschii, Gabriel Creanga et al.

Categories: cs.CL | 13 pages, 1 figure, 3 tables. Accepted to WOAH 2026 Score: 7.50/10 (Obj:8 Id:7 Ind:7 Comp:8 Eff:8 Nov:8 Rep:6)

  • Strength: AgnoSpeech 在 Reddit-25/50 上取得 0.59/0.61 的最高 PrivHSD 权衡，同时保持 HSD micro-F1 87-90%，优于全部 13 个通用私有化配置（实体法最高 0.52，DP 法在 Twitter 上为负值）。
  • Weakness: 自建权衡指标缺少合理性验证与显著性检验，且 Twitter 数据须经 X API 重水合、数据集待发表后发布，关键证据不可直接复现。
  • Full review: Claude Code 全文七公理审稿

Hate speech is a real and timely threat that affects a large portion of online users, especially youth and minority groups. While building reliable and robust automatic hate speech detection (HSD) systems is paramount, we argue that this must also be balanced with the individual right to privacy. Exploring the intersection of HSD and privacy, we demonstrate that HSD systems might unintentionally achieve performance at the cost of encoding authorship, posing a threat to privacy. Building on these findings, we establish the notion of a privacy-HSD trade-off, which demands a careful balance. We benchmark a series of text privatization methods, as well as our newly proposed domain-specific AgnoSpeech technique, showing that balancing privacy and HSD is difficult but feasible. The findings make a strong case for more research on the trade-offs between privacy and HSD, both of which have tangible implications for the safeguarding of online participation.

2. X2Streaming-TTS: Causal Token-Level Text-to-Speech from Streaming Text with Speech-State Inheritance

Authors: Rime Wen, Zehan Liu, Shawn Qin, Lights Shi, Roy Gan et al.

Categories: cs.CL | 11 pages, 3 figures, 4 tables. Equal contribution by Rime Wen and Zehan Liu. Corresponding author: Hao Wang. Code: https://github.com/X-Square-Robot/X2Streaming-TTS Score: 7.40/10 (Obj:9 Id:7 Ind:8 Comp:5 Eff:9 Nov:8 Rep:4)

  • Strength: 严格零 lookahead token 级 TTS，单请求中位 TTFT 15.8 ms、128 并发 260.8 ms，PBD 0.1092、符号 CER 2.00%，在同一骨干受控对比下与离线基线质量相当。
  • Weakness: 对比池仅含 chunk 级伪流式系统且仓库不发布模型权重/评测数据/实验结果，零 lookahead 相对竞争与完整复现均缺关键证据。
  • Full review: Claude Code 全文七公理审稿

Streaming text-to-speech is essential for low-latency spoken dialogue systems, yet many systems wait for sentence-level text and are therefore only pseudo-streaming. True token-level synthesis must generate speech from uncertain prefixes while maintaining perceptual continuity over an unbounded stream with bounded context. We present X2Streaming-TTS, a causal TTS framework that consumes asynchronously arriving text tokens and emits speech without accessing future input. To handle uncertain prefixes, we introduce causal commitment, which keeps ambiguous expressions provisional through uncertainty-aware buffering and performs capacity-adaptive, punctuation-aware segmentation. To preserve acoustic continuity, we further introduce causal speech-state inheritance, which carries the complete Code2Wav state and selected historical Talker states across segment boundaries. Together with an attention prior constraint, it blocks access to future positions while retaining bounded acoustic context. Experiments show that X2Streaming-TTS outperforms existing pseudo-streaming models on most subjective and objective metrics. Further analysis shows that causal commitment stabilizes online segmentation and reduces failures caused by insufficient context, while speech-state inheritance improves boundary continuity without degrading naturalness or speaker identity. X2Streaming-TTS thus achieves strict token-level synthesis with quality comparable to the evaluated offline baselines, a median time to first audio token (TTFT) of 15.8 ms for a single request, and a median TTFT of 260.8 ms at 128 concurrent requests. Our implementation is publicly available at https://github.com/X-Square-Robot/X2Streaming-TTS .

3. VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation

Authors: Yinming Huang, Shuyuan Tu, Xi Yan, Zihan Yang, Jianhua Han et al.

Categories: cs.CV | 19 pages, 7 figures, 8 tables. Code: https://github.com/ShareLab-SII/VA-Judger Score: 7.37/10 (Obj:8 Id:8 Ind:5 Comp:8 Eff:6 Nov:8 Rep:9)

  • Strength: VA-Judger 以 68.43% 的整体偏好预测准确率超越最强单维指标 11.55 个点，用于后训练 LTX-2 后人评偏好 62.30% 对 OmniNFT 的 27.63%，获得超过两倍的人类偏好率。
  • Weakness: 下游收益全部用论文自评为”不可靠”的窄指标报告且同步指标 DeSync 反而恶化（0.226→0.592），OOD 评测与硬对监督均含与训练同源的流程偏差，RL 只在单一 policy 上验证。
  • Full review: Claude Code 全文七公理审稿

Using reinforcement learning to post-train joint video-audio generation models requires a reward signal. Existing methods construct this reward by combining metrics for individual quality dimensions, including audio quality, visual fidelity, and synchronization. However, these metrics evaluate perceptual dimensions separately and fail to capture the overall semantic and temporal coherence among the text prompt, video, and audio that shapes human preferences. Optimizing models against these metrics encourages reward hacking, generating video-audio content that achieves high scores on these metrics yet appears incoherent or unfaithful to human viewers. To address this problem, we first construct a large-scale human-preference dataset VAPref-10K for joint video-audio generation, comprising 9K prompts and 10.3K fine-grained paired comparisons from open-source generation models. We also introduce the VA-Judger-Bench benchmark with both in-domain and out-of-domain model comparisons to evaluate whether reward models truly align with human preferences. We further propose VA-Judger, a chain-of-thought omni-reward model for joint video-audio generation. In particular, VA-Judger first learns from pairs with clear quality gaps to establish structured output and coarse preference discrimination, then distills reliable preference explanations for harder near-quality comparisons via rejection sampling verified against human annotations, and finally performs dimension-wise reinforcement learning that decomposes human feedback into individual quality dimensions for denser reward signals than a single binary preference label. Experiments show that VA-Judger outperforms metric baselines in predicting human preferences on both in-domain and out-of-domain evaluations. Using its human-aligned rewards for post-training audio-video generation model also yields significant improvements in generation quality.

4. Computational Features for Symbolic Melody Analysis

Authors: David M. Whyatt, Peter M. C. Harrison

Categories: cs.SD Score: 7.10/10 (Obj:8 Id:7 Ind:7 Comp:7 Eff:7 Nov:6 Rep:9)

  • Strength: 统一 7 个工具箱、282 个特征于单一 pip 安装的 Python 包（melody-features），在 4,365 首 Essen 民歌上全特征集 logistic 回归达 0.9920 测试准确率、八因子 EFA 解保留 0.9256 准确率且解释 46.27% 方差。
  • Weakness: 演示任务（欧洲 vs 中国）近饱和（测试集仅错 7/873），缺少最简基线与特征无关方法对照，特征集效用与跨工具箱差异缺乏更有区分度任务的验证。
  • Full review: Claude Code 全文七公理审稿

This paper addresses the general problem of extracting music-theoretic and psychological features from symbolically encoded melodies. We review existing melodic feature extraction toolboxes, enumerate their features, and organise them into a common taxonomy. We then describe a new software library that provides implementations of all of these features in a straightforward Python package. We then demonstrate the combined feature set on the Essen Folksong Collection, using the dataset to produce a series of style classification models. These models help us answer key questions about the interpretability and dimensionality of the feature set. Our results show excellent classification accuracy using the full feature set, and promising performance for an eight-dimensional factor-analytic solution that improves the interpretability of the classifier. We distribute our new toolbox as an open-source Python package, $\mathtt{melody-features}$, which can easily be used in various applications within music analysis, music psychology, and music information retrieval.

5. Geometric Iterative Retrieval for Neural Audio Codec Resynthesis

Authors: Leo Schmidt-Traub, Frédéric Berdoz, Luca A. Lanzendörfer, Roger Wattenhofer

Categories: cs.SD, cs.LG Score: 6.95/10 (Obj:8 Id:8 Ind:8 Comp:6 Eff:5 Nov:8 Rep:6)

  • Strength: 在 DAC 9 层 codec 修复任务上，本方法 LSD 10.61±0.07 优于全部学习基线（CE 12.42、MSE 12.86、OSR 11.03），双盲评测 54.0/100 显著高于最强基线 OSR 43.4（Δ+11.2, p=3.4×10⁻⁸），代码与完整评测管线已开源。
  • Weakness: 客观收益集中在 LSD 单一指标且幅度小（SI-SDR/FAD 上一步回归 OSR 反超），layer-progression 显示客观指标 K=3 即达峰后单调恶化，且未与同格的 diffusion/Schrödinger Bridge 方法做任何直接对比。
  • Full review: Claude Code 全文七公理审稿

Neural audio codecs based on Residual Vector Quantization (RVQ) have become the dominant discrete representation for token-based general audio generation, yet resynthesizing high-quality audio from coarse codec tokens remains an open problem and bounds the fidelity of every system that generates them. Prior work has framed resynthesis as a choice between discrete token prediction and continuous regression. We argue that this dichotomy is incomplete and introduce geometric iterative retrieval, a paradigm that uses the RVQ layer hierarchy itself as a natural iterative decomposition in continuous codebook space. Rather than classifying over discrete vocabularies or regressing to a single target vector, our method performs contrastive retrieval in the codebook’s geometric space. We evaluate our method on codec restoration tasks across speech and music, and show improvements over both single-pass token prediction and one-step regression baselines.

6. Generalized Audio-Driven Synthesis of Precise Drummer Motion

Authors: Álvaro G. Iñesta, Mattia Ryffel, Amit H. Bermano, Robert W. Sumner, Martin Guay

Categories: cs.CV, cs.GR, cs.SD | Best Paper Award at the 25th ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA 2026). For Supplementary Video, see https://studios.disneyresearch.com/2026/08/18/generalized-audio-driven-synthesis-of-precise-drummer-motion/ Score: 6.47/10 (Obj:8 Id:5 Ind:6 Comp:6 Eff:8 Nov:8 Rep:2)

  • Strength: 双目标损失（身体旋转+鼓棒笛卡尔监督）将平均冲击点偏差从 8.4 cm 降至 1.9 cm，PAS 时间对齐达 0.82，用户研究显示与真实动捕无显著差异（GT 偏好 58.7%，p=0.08）。
  • Weakness: 无代码/数据/权重发布，且全部定量评估基于自家指标与自家测试集，缺少与外部方法（如 MIDI 驱动的 SBR25、KCA25）在统一指标下的定量对比。
  • Full review: Claude Code 全文七公理审稿

Music-driven character animation enables and enhances transformative applications in entertainment and interactive education. However, synthesizing realistic drumming motion from audio remains challenging due to the inherent tension between high-acceleration dynamics and the need for extreme spatial-temporal precision. Existing approaches, often reliant on motion matching or MIDI input, struggle with generalizing to diverse real-world audio. Moreover, the field lacks standardized evaluation metrics capable of distinguishing precise drumming from noisy motion. In this paper, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics. Additionally, leveraging our own dataset and data augmentation strategy, the model generalizes to non-curated, in-the-wild audio. To rigorously evaluate performance, we propose two novel metrics: an impact-to-target distance to quantify spatial precision and an audio-motion correlation score to assess temporal alignment. Our quantitative analysis and user studies demonstrate that our system generates high-quality motion that is often indistinguishable from ground-truth performances.

7. A Speech Corpus for Mizo Automatic Speech Recognition: Whisper and SraVaani 1.0 Fine-Tuning with Morphology-Aware Evaluation

Authors: Priyankoo Sarmah, Sanasam Ranbir Singh, Lalhmingmawia

Categories: cs.CL, eess.AS Score: 6.40/10 (Obj:9 Id:6 Ind:7 Comp:5 Eff:7 Nov:6 Rep:5)

  • Strength: Whisper-large-v3 微调后在 17.62 小时 Mizo 语料上达到常规 WER 18.08%、形态感知 WER 7.22%，并随模型容量一致改善；SraVaani 微调使 WER 从 58.27% 降至 29.45%。
  • Weakness: 论文未在自有测试集上对比已发表且更优的 wav2vec2/XLS-R Mizo 基线（11.84% WER），且语料仅部分发布、模型权重需申请、演示页面 404，缺少可直接复现的完整资源。
  • Full review: Claude Code 全文七公理审稿

This study reports the development of an Automatic Speech Recognition (ASR) system in Mizo, a low-resource language. The development included collecting 17.62 hours of speech data, curating it, and fine-tuning the Mizo ASR system with three Whisper multilingual models and with the SraVaani 1.0 Indic multilingual model. Whisper-large-v3 achieved the lowest conventional WER (18.08%), while morphology-aware evaluation yielded a WER of 7.22%. Zero-shot evaluation of the SraVaani 1.0 Indic multilingual model yielded a WER of 58.27%, while Mizo-specific fine-tuning reduced the conventional WER to 29.45% and the morphology-aware WER to 17.93%. The results demonstrate that the Whisper model can achieve a substantially low WER, even when adapted to an unseen language. In contrast, SraVaani 1.0 supports the Mizo language in its multilingual model; however, fine-tuning with carefully curated Mizo speech data substantially improves its performance.

8. Does Mapping Non-Maximal Probabilities to GMM Components Matter for S-JEPA Encoder Representations?

Authors: Wenxuan He, Yunpeng Li, Shan Liang

Categories: cs.LG, cs.SD | 6 pages, 4 figures, 2 tables Score: 6.30/10 (Obj:8 Id:8 Ind:8 Comp:8 Eff:5 Nov:8 Rep:4)

  • Strength: 匹配反事实设计（FIXED-RANDPERM/UNIFORM-TAIL）在三个种子、12 个点估计上全部一致显示真实软 GMM 目标显著优于对照（动态 R² 增益 0.008–0.026，6 个 speaker-cluster 区间排除零）。
  • Weakness: 全部证据来自冻结线性探针且效应量较小，未验证下游任务传导，且无代码/权重发布（GitHub 仅 3 个社区复现，无官方仓库）。
  • Full review: Claude Code 全文七公理审稿

S-JEPA uses soft Gaussian mixture model (GMM) posteriors instead of hard cluster labels to preserve uncertainty. It remains unclear whether the probability values alone are sufficient, or whether it also matters which GMM components receive the non-maximal probabilities. We test this with two matched controls. FIXED-RANDPERM keeps the top-1 component and probability together with the multiset of non-maximal probability values, but reassigns those non-maximal values using a mapping fixed for each physical frame. UNIFORM-TAIL keeps the top-1 component, its probability, and total non-maximal mass but distributes that mass uniformly. Across three independent seeds, REAL SOFT outperforms both controls on two frozen Encoder readouts. It provides better recovery of the original GMM tail and greater accessibility of spectral dynamics over short time scales after controlling for the complete spectrum of the current frame. In two exposure experiments, both readouts improved overall as more frames retained the original mapping. We also descriptively follow one Phase 2 trajectory after the switch to the online GMM. These results show that the numerical probability structure of the soft target does not fully determine the learned Encoder representation. The mapping of non-maximal probabilities to GMM components also matters.

9. Finetuning Strategies for Querying Sounds by Vocal Imitation

Authors: Aditya Bhattacharjee, Christos Plachouras, Sungkyun Chang, Emmanouil Benetos

Categories: cs.SD, cs.AI, cs.IR Score: 5.89/10 (Obj:8 Id:6 Ind:5 Comp:6 Eff:7 Nov:4 Rep:6)

  • Strength: 获胜方案（MRR 0.2932，相对 Greif 基线 0.2726 提升约 7.6%）通过冻结 CED + 对比微调与共享 MobileNetV3 + 半难负样本三元组两条互补策略实现，且已部署于 thatssoundslike.me。
  • Weakness: 报告缺少两 submission 间的消融与主观评测结果，无法归因各组件贡献，且未公开代码与权重。
  • Full review: Claude Code 全文七公理审稿

This technical report describes our winning submission to the AES AIMLA 2025 Challenge on querying sound effects by vocal imitation. We investigate two complementary fine-tuning strategies: contrastive learning with a frozen, pretrained CED encoder, and joint contrastive-triplet learning with semi-hard negatives using a MobileNetV3 encoder. This report has been updated for posterity to include details released after the challenge.

10. MultiVerse: A Creator-Centered Approach to Steering Context-Adaptive Lyrics

Authors: Alexander Wang, Chris Donahue, David Lindlbauer

Categories: cs.HC, cs.SD Score: 5.68/10 (Obj:8 Id:6 Ind:6 Comp:5 Eff:5 Nov:7 Rep:2)

  • Strength: MultiVerse 以 creator/content/context 三维显式引导实现自适应歌词创作，对照研究（N=10 词曲作者）显示 8/10 认为其比纯提示工作流更有掌控力、9/10 主动采用上下文查询机制。
  • Weakness: 所有主观量表差异均 p>.05 无显著性，缺客观歌词质量与约束满足率的量化验证，且未发布代码或数据（生成后端为闭源 Gemini 3 Flash API）。
  • Full review: Claude Code 全文七公理审稿

Generative AI may enable new forms of context-aware creative expression by dynamically tailoring media content to its consumption context. For instance, AI systems could adapt song lyrics to the listener and their current activity. However, existing media adaptation systems primarily optimize for audience experience, often neglecting artists’ intent, style, and preference. We address this challenge by introducing a novel creator-centered approach to adaptive media authoring and present MultiVerse, a system that instantiates this approach for steering adaptive lyrics. Our approach allows creators to explicitly author controls based on their intent, lyric structure, and audience context, and uses rule-based validations to ensure controls are followed. We conducted a study with 10 songwriters, comparing MultiVerse with a prompting-based workflow for composing adaptive lyrics. The comparison revealed that creators preferred to author how lyrics adapt by explicitly specifying relevant context and adaptation constraints, while recognizing trade-offs in flexibility and iteration speed. Interviews further revealed that creators viewed adaptive media as enabling new forms of audience connection, introducing a distinct creative process, favoring new compositional strategies, and reshaping notions of authorship.

11. Sounds Uncertain: Exploring the Affective Aspects of Sonification for Uncertainty Visualization

Authors: Marcel-Simon Dutt, Sita A. Vriend, Elias Elmquist, Daniel Weiskopf

Categories: cs.HC, cs.SD | Accepted to IEEE Workshop on Uncertainty Visualization Score: 5.60/10 (Obj:8 Id:5 Ind:6 Comp:5 Eff:4 Nov:6 Rep:7)

  • Strength: 21 名非专家参与者通过共设计研究将 20 个代码压缩为 4 个主题（波浪/不祥对应不确定性，清晰/放松对应中性），并观察到 8 名女性参与者贡献了 10 个不祥主题声化中的 6 个。
  • Weakness: 论文自述结论为假设生成性质，缺少独立听者组的量化评定，未报告任何效用指标或评分者间一致性系数，声化-情感映射未经验证。
  • Full review: Claude Code 全文七公理审稿

Affective visualization can influence how users perceive, interpret, and engage with data by embedding and conveying emotion through visual design. While sound is widely used in media to evoke emotions, little is known about how sonification can support affective visualization. In this work, we investigate how sonification can communicate emotion in uncertainty visualizations through a co-design study. Participants created two sonifications to accompany a visualization: one conveying the affective component of uncertainty and one conveying neutrality. Our findings show that uncertainty was commonly associated with wavy auditory qualities related to an ominous sentiment. On the other hand, neutrality was associated with clear and relaxing auditory qualities. These results provide insights for the design of visualizations that integrate sonification to communicate the affective component of uncertainty.