每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-08-20
日期2026-08-20
已评分—
均分—
最高—

Daily Papers — 2026-08-20

11 papers on audio, speech, music, and acoustics.

1. Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners

Authors: Umberto Cappellazzo, Xubo Liu, Stavros Petridis, Maja Pantic

Categories: eess.AS, cs.AI, cs.SD | Project website: https://umbertocappellazzo.github.io/nape Score: 8.20/10 (Obj:9 Id:8 Ind:7 Comp:9 Eff:8 Nov:8 Rep:9)

  • Strength: 以仅含因果掩码与 stop-gradient 的极简目标，NAPE-L（303M）在 IEMOCAP 达 68.0%（较最强基线 BEATs +3.5）并在 AS-2M 与最强 SSLAM 打平（50.2 mAP），六基准上三规模单调缩放、每档一致优于 Audio-MAE。
  • Weakness: AS-2M/AS-20K 评测与预训练数据同源（预训练含 AudioSet eval split）削弱独立性，且各基线微调数字均引自原论文未统一复跑；此两任务上 NAPE 均未超越 SSLAM（AS-20K 40.5 vs 40.9）。
  • Full review: Claude Code 全文七公理审稿

Self-supervised learning (SSL) has driven substantial progress in audio representation learning, though existing methods have increasingly relied on elaborate pre-training recipes to reach competitive performance. A markedly different pre-training philosophy underpins the most influential progress in language modeling and, more recently, in visual representation learning: rather than train encoders as static feature extractors, models are trained to predict the next element, a discrete token or a continuous embedding, from the preceding context. Autoregressive prediction thereby provides a unified pre-training interface that transfers across modalities, compelling the model to learn the underlying data distribution. We ask whether such a simple causal paradigm can yield strong audio learners, given that audio’s temporal structure makes autoregressive prediction of patch embeddings a natural fit. We introduce NAPE (Next-Audio-Patch-Embedding prediction), a self-supervised framework in which a causal Transformer predicts each next patch embedding of a log-mel spectrogram from the previous ones, using causal masking and stop-gradient as its sole training signal. The design is intentionally minimalist, avoiding reconstruction decoders, acoustic tokenizers, student-teacher setups, and auxiliary regularization losses. Across six audio and speech benchmarks, NAPE achieves state-of-the-art fine-tuning performance on several tasks, scales consistently across encoder sizes, and yields strong linear-probing results. NAPE also produces structured attention patterns without explicit supervision.

2. Unified Music Identification for Tracks and Versions

Authors: R. Oguz Araz, Joan Serrà, Yuki Mitsufuji, Xavier Serra, Dmitry Bogdanov

Categories: cs.SD | Accepted to ISMIR2026 Score: 8.00/10 (Obj:9 Id:8 Ind:8 Comp:8 Eff:8 Nov:8 Rep:9)

  • Strength: Fish 统一模型在 Discogs-VI 上 VI clean MAP@10k 达 0.687（超 CLEWS 0.628）、TI clean hit rate@1 83.7 / hit rate@10 93.1，且消除信息失配后 track hit rate@1 达 99.9，证明 10 s 查询下统一识别可行，评测覆盖 100 万+ 查询。
  • Weakness: 缺少经典非学习指纹基线（如 Chromaprint）与 1–3 s 短查询评测，且 Fish 的 clean TI hit rate@1（83.7）明显落后专用模型 NMFP（97.4），”统一以鲁棒性换精度”的代价未在短查询场景中量化。
  • Full review: Claude Code 全文七公理审稿

Given a music database, track identification (TI) retrieves the exact track matching an audio excerpt, whereas version identification (VI) retrieves its musical versions. Traditionally, the two tasks have been addressed separately. However, as every track is its own closest version, we investigate whether VI can subsume TI. This requires VI systems to be robust to both signal manipulation and audio degradation. We therefore propose a unified benchmark that evaluates accuracy and robustness on each task. Comparing seven existing models on this benchmark, we show that none of them are both accurate and robust on both tasks. We then train a baseline model targeting both tasks and show that a unified system is possible with 10 s TI queries. Lastly, we characterize the two retrieval constraints that limit our model’s TI performance. We envision extending this unification to other music identification tasks.

3. Fourier is Frontier: Frequency-Aware Autoencoding for High-Fidelity Music Reconstruction

Authors: Kangdi Wang, Yusheng Dai, Jin Xu

Categories: cs.SD, cs.MM Score: 7.79/10 (Obj:8 Id:8 Ind:8 Comp:6 Eff:8 Nov:8 Rep:8)

  • Strength: ear-VAE2 在 Song Describer 546 轨上取得 7 项重构指标中 5 项最佳点估计（Mel 距离 0.461，Duplex-Aware Refiner 使其相对降低 19.4% 且比 Unconstrained Refiner 少约 45% 输出维度），下游生成在全部 12 项自动指标上胜过 LeVo 2 VAE，代码与 Apache-2.0 权重均已公开。
  • Weakness: 主表为非匹配预算比较且 SI-SDR 11.3 dB 弱于 ear-VAE 12.4 dB 与 SAME-L 12.5 dB，全部结果为点估计而无置信区间或显著性检验，训练数据含约 1 万小时私有数据使全量训练不可独立复现。
  • Full review: Claude Code 全文七公理审稿

Continuous-latent audio autoencoders form the backbone of latent music generators, yet decoders at high compression rates commonly exhibit three failure modes: high-frequency loss, phase incoherence, and stereo-image collapse. These share a structural root: waveform autoencoders lack an explicit frequency axis, leaving no handle for targeted per-band correction. Among five matched-budget representations, the complex STFT achieves the lowest full-band and high-frequency spectral distances, providing direct access to magnitude and phase at every bin. Building on this, we present ear-VAE2, a complex-spectral autoencoder with cross-channel interaction. Spec-SnakeBeta learns a periodic activation per frequency bin with frequency-dependent initialization, outperforming other activation variants while using fewer parameters than the fully independent variant. Duplex-Aware Refiner applies band-specific corrections to magnitude and phase following duplex theory of sound localization. On the 546-track Song Describer Dataset, ear-VAE2 achieves the best point estimates on five of seven reconstruction metrics. The Duplex-Aware Refiner reduces Mel Distance by 19.4% and uses ~45% fewer residual-output dimensions than the Unconstrained Refiner, while also lowering spectral distances, spatial-cue errors, and receiving higher ratings from professional engineers. The downstream generator using ear-VAE2 latents achieves better point estimates on all 12 automatic metrics.Demo page is available at https://eps-acoustic-revolution-lab.github.io/EAR_VAE2/.

4. Towards Quantifying Benchmark Optimization in ASR Models

Authors: Theo Lebryk, David Ayllon, Alice Baird, Jakub Piotr Cłapa, Jens Madsen et al.

Categories: cs.SD, cs.AI Score: 7.50/10 (Obj:7 Id:6 Ind:8 Comp:5 Eff:9 Nov:8 Rep:8)

  • Strength: 在 VoxPopuli 上 6 个 WER 最优（5.4–5.8%）模型恰为 accept-ref 最高（0.18–0.30），且拼接基准音频或单层低秩 steering（k=1 恢复 65–80%）可双向翻转该行为、ablation 使 accept-ref 降 82–92%。
  • Weakness: 缺少对”基准分数被高估多少”的总体端到端量化，且机制定位仅在 4/11 模型上成立、DaiKon/ep-fresh/libri-fresh 数据不公开导致关键验证不可独立复现。
  • Full review: Claude Code 全文七公理审稿

Public benchmarks are important measures of Automatic Speech Recognition (ASR) model capabilities. However, by nature of being public, there is risk of models being optimized for these benchmarks in ways that do not generalize well to real-world data. We present a methodology for quantifying benchmark optimization, focusing on cases where the audio underdetermines the reference transcript. We identify three families of behavioral probes that reveal models’ capabilities of reproducing benchmark reference spans despite underdetermined audio: reference disagreement, masked-number recovery, and orthographic switching. We find that the highest-scoring open source models output verbatim reference transcript spans even when the relevant audio is contradictory, masked, or ambiguous. Using a variety of mechanistic probes, we show that models respond to narrow acoustic cues to override the faithful representation of the audio in favor of a benchmark-optimized policy. We show the benchmark-optimized behavior can be causally manipulated via low-rank linear steering or simply appending audio to the end of a segment in some cases. Overall, our results indicate that high-performing models exhibit benchmark-conditioned behaviors that can inflate benchmark performance without reflecting improved general-purpose transcription ability.

5. $TCP_α$: Margin-Controlled Confidence estimation for reliable Music Information Retrieval

Authors: Parampreet Singh, Anushka Singh, Sumit Kumar, Vipul Arora

Categories: eess.AS, cs.LG Score: 7.40/10 (Obj:8 Id:7 Ind:7 Comp:8 Eff:9 Nov:7 Rep:5)

  • Strength: 拒绝最低置信 8% 预测即把 rāga 识别保留样本 macro-F1 从 0.89 提升至 0.98，TCPα 在 PIM 上达到 AUPR-E 95.96、FPR@95%TPR 1.60，全面优于能量分数（56.87）等全部基线。
  • Weakness: 正文宣称可用的代码仓库（tinyurl.com/tcp-alpha → GitHub 2026confidenceTASLP）经实测仅有占位 README、无任何代码，复现性无法验证。
  • Full review: Claude Code 全文七公理审稿

Deep neural networks are often overconfident, assigning high confidence even to incorrect predictions. Consequently, users lack a reliable signal for deciding when a prediction can be trusted. Post-hoc confidence estimation addresses this by training a lightweight auxiliary head over a frozen classifier. Existing targets, however, suffer from inherent ambiguity: they assign overlapping confidence values to correct and incorrect predictions, while errors near the decision boundary receive confidence scores indistinguishable from correct predictions. In this work, we propose $TCP_α$, a novel confidence target that resolves these limitations by introducing a margin-controlled penalty for misclassified samples. We prove that $TCP_α$ guarantees complete separation between the target values of correct and incorrect predictions, with a separation margin that is independent of the number of classes and increases monotonically with the penalty parameter. Since accurate classifiers naturally produce very few errors, learning these targets results in a severely imbalanced regression problem. We therefore present a systematic study of training strategies for learning under this imbalance and identify an effective training configuration through extensive ablation studies. We evaluate the proposed approach on rāga identification, investigate its robustness under domain shift, and further validate it on frame-wise ornamentation detection without modifying the selected configuration. Across all settings, $TCP_α$ consistently outperforms existing confidence targets for failure prediction. Rejecting only the least-confident 8\% of predictions improves the base model’s macro-F1 from 0.89 to 0.98, while fine-tuning the confidence head with only 5\% labeled samples from a new corpus effectively restores performance under domain shift.

6. Hear2Act: Benchmarking When Prosody Should Change What an Assistant Does

Authors: Xinyi Liu, Hooshang Nayyeri, Dilek Hakkani-Tur, Emine Yilmaz, JK Kim et al.

Categories: cs.CL Score: 7.16/10 (Obj:8 Id:6 Ind:8 Comp:8 Eff:8 Nov:8 Rep:3)

  • Strength: Prosody-mediated 反馈下，仅加音频将两个音频 LLM 的平均最优解率从 14.6% 仅提升至 15.3%，而把音频推断出的顾虑状态显式写入文本后再决策可升至 39.6%，接近 ground-truth 状态的 40.7%，文本侧五模型加状态后提升 +36.7 个百分点。
  • Weakness: 基准本体（480 场景、确定性用户引擎、TTS 渲染指令与评测脚本）未公开任何代码或数据链接，真人听感校验仅 2 名标注者，完整复现不可行。
  • Full review: Claude Code 全文七公理审稿

Prosodic cues can convey task-relevant information that alters the trajectory and outcome of a task-oriented dialogue, even when the words themselves remain unchanged. Yet existing benchmarks typically evaluate prosodic perception, response appropriateness, and task-oriented dialogue in isolation, making it difficult to test whether prosodic evidence changes downstream decisions. We introduce Hear2Act, a unified evaluation protocol for text and spoken assistants with 480 persona-grounded scenarios, hidden user concerns, and objectively verifiable outcomes. For each scenario, we keep the task and user needs fixed while varying whether the same concern is conveyed explicitly in words or primarily through prosody, and evaluate decisions under transcript, audio, and concern-state access. Using Hear2Act, we evaluate two audio-capable LLMs. Under Prosody-mediated feedback, adding audio to the transcript changes the average optimal-solution rate only from 14.6% to 15.3%. In contrast, when models infer the concern status from audio, represent it in text, and use it for next-action selection, the rate rises to 39.6%, close to 40.7% with the ground-truth state. This contrast, however, largely disappears under Explicit lexical feedback, where the concern is verbally mentioned in the utterance. Together, these results show that prosody matters when lexical evidence is insufficient, and that audio-capable LLMs can recover information from speech but do not reliably carry it into action without an explicit intermediate representation.

7. DAVSS: Distilled Audio-Visual State Space Models

Authors: Saurabhchand Bhati, Mrudula Athi, Amit S. Chhetri, James Glass

Categories: eess.AS Score: 6.53/10 (Obj:8 Id:6 Ind:8 Comp:8 Eff:6 Nov:6 Rep:5)

  • Strength: DAVSS-Tiny 以 14.4M 参数（CAV-MAE 的约 1/12）在 AudioSet 上达到 51.6 mAP 反超 teacher CAV-MAE 的 50.5，且推理更快。
  • Weakness: 缺少同规模/等 FLOPs 的 transformer 对照来隔离 patch 分辨率混淆，且 EquiAV（54.6 mAP）仍高出 DAVSS-Medium（52.7）1.9 mAP，SOTA 领先性证据不足。
  • Full review: Claude Code 全文七公理审稿

State-space models (SSMs) distilled from transformer teachers combine the performance of transformers with the efficiency of SSMs. We extend the Transformer-SSM knowledge distillation to a multimodal setting and propose the Distilled Audio-visual State-Space (DAVSS) model. The DAVSS model, 14M parameters, is 12 times smaller compared to transformer-based models such as CAV-MAE, and still outperforms them. DAVSS improves over the existing audio-visual models by: 1) Finer input resolution: using smaller patch sizes process the input, compensating for the smaller model size by increasing input sequence lengths. This is supported by the observation that a larger patch size results in lower performance. 2) Deeper joint modeling: utilizing a larger portion of the model (30%) for joint audio-visual processing, compared to <5% in CAV-MAE, enabling deeper cross-modal interaction without significantly increasing the computational cost associated with the concatenated audio-visual tokens.

8. Explainability by Design: Structured Kolmogorov-Arnold Networks over Probabilistic Attributes for Speech Deepfake Source Tracing

Authors: Hoang H. Pham, Manasi Chhibber, Tomi H. Kinnunen

Categories: eess.AS | 26 pages, submitted to Elsevier Computer Speech & Language journal Score: 6.42/10 (Obj:8 Id:6 Ind:5 Comp:5 Eff:8 Nov:6 Rep:6)

  • Strength: 在 ASVspoof2019-attr-17 上，多任务概率属性提取 + 结构化 KAN 将 17 类攻击分类推到 99.64% 平衡精度 / 0.11% EER，较作者先前两阶段基线（84.37%/3.35%）实现约 20 倍的 EER 下降，且 KAN 内置特征重要性与 SHAP 的模型级 Spearman 相关达 0.72。
  • Weakness: 关键反证被弱化处理——攻击 A18 的 KAN-SHAP 排名相关性为 -0.71、另有 5 个攻击类低于 0.5，且无独立外部基线与跨数据集验证，结构化拓扑消融（99.61% vs 全连接 99.65%）显示”可解释性由构造保证”缺乏可量化的解释质量增益。
  • Full review: Claude Code 全文七公理审稿

Modern speech synthesizers can produce highly realistic speech, making source tracing (i.e. identifying the generator behind a spoofed utterance) increasingly important for forensics, online content provenance, and platform accountability. Building on our prior work on transparent probabilistic attributes, which represent utterances as probability distributions over synthesizer sub-components, we extend speech deepfake source tracing with two key ingredients: multi-task training of the probabilistic attribute extractors and a structured Kolmogorov–Arnold Network (KAN) for attack classification. The probabilistic features are estimated jointly with a multi-task learning module built on a shared AASIST or SSL-AASIST countermeasure backbone. The resulting probabilistic feature embedding is classified by a structured KAN whose topology follows known attribute-to-attack relationships. This provides interpretability by construction: the architecture reflects the generative hierarchy of attacks, while KAN feature-importance scores quantify each probabilistic feature’s contribution without post-hoc explainers such as SHAP. On ASVspoof2019-attr-17, the extended framework achieves balanced accuracies above 99% for all seven probabilistic feature extractors, with EERs of 0.16% to 0.07%, and 99.64% balanced accuracy with 0.11% EER for 17-class attack classification. Our revised model outperforms the earlier two-stage baselines, in addition to demonstrating reliable interpretability, with importance scores consistent with SHAP values, and stable results across batch sizes. These findings highlight the potential of structured KAN for speech deepfake source tracing that is both accurate and interpretable by design. For transparency and reproducibility, our codebase is publicly available: https://github.com/HoangHPham/KAN-Probabilistic-Deepfake-Attribution.

9. Tracking the Trend in How Speech Synthesizers Deceive People

Authors: Milan Šalko, Anton Firc, Kamil Malinka, Vojtěch Staněk, Martin Perešini et al.

Categories: cs.CR, cs.SD, eess.AS | Accepted at the 6th Symposium on Security and Privacy in Speech Communication (SPSC 2026) Score: 6.16/10 (Obj:8 Id:5 Ind:7 Comp:7 Eff:7 Nov:6 Rep:3)

  • Strength: 82 名 IT 专业人士上，ElevenLabs 整段伪造句级识别准确率跌至 0.43（95% CI [0.37,0.50]，低于 0.5 随机基线），单句替换的部分伪造严格 All OK 仅 9%、伪造句被 77% 判定为真音，且同料人机对比显示检测器在整段 ElevenLabs 上 F1 仅 0.09。
  • Weakness: 遗漏了早于其提交的 Müller & Choong 2026 大规模研究（1,768 被试、138 系统，真实音频信任侵蚀发现高度重合），且评测数据集与代码均未公开、独立复现不可行。
  • Full review: Claude Code 全文七公理审稿

Advances in speech synthesis have made deepfake audio highly realistic. Earlier studies reported 70-80% human detection accuracy, but relied primarily on older synthesizers. We compare human detection for three selected voice synthesis tools released in 2019, 2022, and 2024 with 82 IT professionals, and benchmark humans against six pretrained detectors on the same material. For fully synthetic speech (full spoofs), the F1 score drops from about 90% for RTVC and YourTTS to 48% for ElevenLabs, although listeners were explicitly warned that deepfakes were present. For partial spoofing, where only one sentence of an utterance is altered, strict accuracy falls to 9%, and listeners classify the synthetic sentence as bona fide 77% of the time. Humans and detectors fail in complementary ways, and neither reliably localizes short manipulations. Additionally, listeners increasingly mislabel bona fide speech as fake, eroding trust in unmanipulated audio. These findings show that human perception alone is unreliable for the selected modern and partial-spoof conditions and motivate procedural verification, provenance, watermarking, and segment-level detection.

10. Does Listening Matter? Backchanneling and Nodding in AI Clone

Authors: Koji Inoue, Kazushi Kato, Tatsuya Kawahara, Shunichi Kasahara

Categories: cs.HC, cs.CL, cs.SD | This paper has been accepted to the Late-Breaking Results (LBR) track of the 28th International Conference on Multimodal Interaction (ICMI 2026) Score: 5.74/10 (Obj:8 Id:5 Ind:8 Comp:6 Eff:5 Nov:5 Rep:5)

  • Strength: 35 名日语被试的被试内实验中，加入 VAP/MaAI 驱动的回馈与点头后，AI clone 的专注度（5.29 vs 4.80，p=.018）、真实感（4.77 vs 4.29，p=.006）与共在感（4.60 vs 3.94，p=.002）显著提升，且回馈量与专注度呈倒 U 型关系（二次项 p=.009）。
  • Weakness: 缺少对真实人类倾听或规则式反馈的对照、回馈与点头捆绑未分解，且系统集成代码与被克隆人音色未公开，无法支撑”clone 因倾听更像原人”的机制结论与端到端复现。
  • Full review: Claude Code 全文七公理审稿

AI clones that imitate a specific person typically reproduce what the person says and how they sound, but not how they listen. We investigate whether adding multimodal listening behaviors gives such a clone more presence and authenticity. We integrated verbal backchannels and head nodding, driven by real-time prediction models, into an AI clone equipped with voice cloning and LLM-based responses. In a within-subjects study (N=35), adding these behaviors significantly improved the perceived attentiveness of the avatar, the sense of talking with the real person, and the feeling of co-presence. These results indicate that AI clone fidelity should extend beyond voice and response content to include interactive listening behavior.

11. Dancing Through Soundscapes: Designing a Low-Cost, Sound-Based Device for Sensing and Interpreting Movement and Dance

Authors: Swen E. Gaudl, Silvia Carderelli-Gronau

Categories: cs.HC, cs.SD | full paper on technical artifact for TEI2027 conference Score: 5.20/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:4 Nov:5 Rep:8)

  • Strength: 开源可复现的 ≈160 欧元声源定位生成声景装置（ODAS + Pure Data），经 4 年 10 场工作坊 37 名参与者的真实舞蹈实践验证了无相机、无穿戴的本地与远程共舞模式。
  • Weakness: 全文无任何量化评测（无用户问卷/评级/延迟实测），远程效用仅在低环境噪声下成立，且与 Sensitiv、OtoKin、WAVE 等相邻系统缺少同条件基线对比。
  • Full review: Claude Code 全文七公理审稿

When we move through space, we often rely on multiple senses beyond vision to perceive and act in that environment: we ``feel’’ the presence of others; we build internal representations and models and recall them to navigate the environment. We also leave traces and impressions that others pick up on. The traces include echoes, heat, the displacement of objects such as furniture or footprints, air movement close to the face of another, smells such as perfume, but also the immediate sounds we make when we move and breathe. Movement is a spatial and temporal activity, and dance as a form of movement practice requires coordination of oneself in relation to others, the space and a potential score. When rehearsing dance, dancers have to relate to others often not just by looking but more often by feeling and imagining or remembering where others are based on experience and shared practice. So how can we approach technology-mediated movement and dance? Why should we explore it? How can we support spatial and temporal practice meaningfully and joyfully? In this work, we focus on sound traces; we present the design and rationale for a sound-based artefact that translates movement-based sound into layered, explorable, generative soundscapes. The work contributes a novel artefact for exploring movement-based activities with a audio-first approach, with a focus on the spatial performative experience. The paper further reflects on observations from workshops and public sharings, including how participants used repetition, stillness, environmental sound, and call-and-response to understand and improvise with the soundscape.