Daily Papers — 2026-08-05

12 papers on audio, speech, music, and acoustics.

1. A Dual Evaluation for Music Transcription

Authors: Ping Wang, Guang Yang, Nazif Can Tamer, Victoria Ebert, Noah A. Smith

Categories: cs.SD Score: 7.58/10 (Obj:9 Id:8 Ind:6 Comp:8 Eff:8 Nov:6 Rep:9)

Strength: 通过 106 人 ABX 听力研究验证 7 类 playback 指标，发现 CLEWS 同时达到最高人类相关性 (Spearman ρ=0.971) 和最低成本 ($0.37/1,000 units)，并在 25 个 AMT 系统的 8×3 因子设计中揭示了 MIDI-to-score converter 系统性决定 notation vs playback 维度偏好。
Weakness: 核心评测指标 (OMR-NED, DTW, TWED, CLEWS, CLaMP 3) 全部为外部已有工作，本文无新指标设计；Rubato case study 的作者与本文作者团队重叠，缺乏完全独立的外部端到端系统验证；未讨论 learned embedding 模型 (CLEWS/CLaMP 3) 训练数据与 ATEPP 的潜在重叠风险。
Full review: Claude Code 全文七公理审稿

Automatic music transcription systems produce sheet music that can be read and played back. We argue that these two targets call for complementary evaluations of notation similarity to a reference score and playback similarity to the original performance, respectively. Our study considers notation similarity metrics from the optical music recognition literature and a wide range of playback-similarity methods validated through a listening study across over 100 participants and 230 piano recordings covering 23 works, 30 performers, and six composers. We find, fortuitously, that the playback similarity metric that correlates best with human judgments, CLEWS, is also the cheapest to run. We also find that the two evaluation dimensions favor different systems among a collection of 24 pipelines formed by pairing eight audio-to-MIDI models with three MIDI-to-score converters, with the latter component systematically determining the favored objective. The complementarity between metrics also holds when adding to the pool Rubato, a new end-to-end system that offers substantially improved notation similarity while remaining competitive, though not the best, on playback similarity.

2. Masked diffusion enables coherent beat tracking

Authors: Francesco Foscarin, Filip Korzeniowski, Richard Vogl

Categories: cs.SD, cs.AI | Accepted at the 27th International Society for Music Information Retrieval Conference (ISMIR), 2026 Score: 6.95/10 (Obj:8 Id:6 Ind:8 Comp:8 Eff:6 Nov:7 Rep:8)

Strength: 在相同数据和 backbone 下，MDM 迭代推理将 downbeat CMLt 从 67.3 提升至 76.4（+9.1pp），consecutive downbeats 从 0.25 降至 0.02/track，无需 DBN 后处理即达到或超越使用 DBN 的 10x 参数量 SOTA。
Weakness: 仅在 GTZAN 单一数据集评测，且 MDM 贡献与同时引入的 SwiGLU、独立 output head、翻倍 epochs 和多项训练超参数调整未做完全隔离消融，single-step MDM 的提升无法排除非 MDM 因素。
Full review: Claude Code 全文七公理审稿

Current neural networks for beat tracking generate invalid outputs, such as consecutive downbeats and erratic tempo changes, even when these are not present in the training data. Heavy post-processing techniques can alleviate these problems, but the original cause of this inconsistent behaviour remains unknown. We hypothesise that it stems from inadequate modelling of multiple plausible output beat grids, resulting in an invalid mixture of competing interpretations. We propose a masked diffusion approach that properly models multiple outputs and enables the model to build coherent predictions through iterative inference. We devise three modifications to standard masked diffusion that enable its application to beat tracking: independent masking of beats and downbeats during training and inference, a balanced masking scheduler for inference, and peak-picking across inference steps. Our approach reduces erratic behaviours and improves beat-tracking performance.

3. Towards Robust Version Identification in the Wild: A Dataset, Benchmark, and Fine-Tuning Study

Authors: Simon Hachmeier, R. Oguz Araz, Dmitry Bogdanov, Robert Jäschke, Xavier Serra

Categories: cs.SD, cs.IR | Accepted to the Proceedings of the 27th International Society for Music Information Retrieval Conference (ISMIR 2026) Score: 6.20/10 (Obj:8 Id:6 Ind:8 Comp:5 Eff:7 Nov:5 Rep:8)

Strength: DiVers 是首个百万级 in-the-wild VI 数据集（1.1M+ versions），人工验证匹配精度 96.25%，在 DiVers-L 上训练的 DVINetX 在所有非 DVI 评测集上全面优于 DVI 训练，CLEWSFT+L2 在 YVI-S 上达 MAP 0.857。
Weakness: 方法新颖性有限——构建 pipeline 全由标准组件（Rapidfuzz/Chromaprint/PANN/GPT-5）组成，DVINetX 仅是 DVINet+ 的维度放大，核心 idea 已在作者 prior work SHS-YT 中提出，本文主要是规模化整合。
Full review: Claude Code 全文七公理审稿

Existing datasets for musical version identification (VI) are primarily derived from curated metadata sources such as SecondHandSongs and Discogs, and are therefore dominated by professionally recorded tracks. This leads to a domain mismatch with real-world scenarios, where amateur and user-generated content is prevalent. To address this limitation, we introduce DiVers, a large-scale VI dataset comprising over 1.1 million musical versions, with train-validation-test splits compatible with established datasets such as Discogs-VI-YT, SHS100K, and Da-TACOS. In addition to standard version-level annotations, DiVers provides automatically assigned tags (e.g., instrumental, live) and segment-level predictions indicating the presence or absence of music. We evaluate the proposed dataset by training state-of-the-art VI systems. Our results show that models trained on DiVers achieve substantially improved robustness to acoustically diverse and noisy inputs, while maintaining a stable performance on cleaner, studio-quality benchmarks. We release the dataset metadata, code for its construction, and all experimental pipelines to support reproducibility.

4. Visual Representation Matters: Exploiting Temporal Differences in Video-to-Audio Generation

Authors: Zehua Chen, Junyou Wang, Yuxuan Jiang, Zhenying Fang, Yusheng Dai et al.

Categories: cs.CV, cs.LG, cs.SD Score: 6.16/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:8 Nov:5 Rep:5)

Strength: 帧级时间差分（FTD）作为 V2A 条件表示简洁有效，在 VGGSound 上 FAD 0.53 / IS 16.9 / IBS 33.8 全面超越 11 个 baseline（含 MMAudio, AudioX），主观评分接近 GT。
Weakness: 缺少”拼接多帧 CLIP”最简 baseline 和 HCL/ATDG 交叉消融，ATDG 引入 4 个超参数但增益有限（FAD 0.57→0.53），代码未声明开源，评测仅限单一 test set。
Full review: Claude Code 全文七公理审稿

Video-to-audio (V2A) generation extends image-to-audio generation (I2A) by introducing consecutive frames that provide essential temporal cues for audio synthesis. However, existing conditional diffusion-based V2A methods typically enhance visual conditioning with additional audio-visual supervision, acoustic structure prediction, or reasoning from large multimodal models, requiring extra networks or strong inductive biases. Inspired by recent advances in visual representation learning, we introduce TD-V2A, which leverages temporal differences (TD) as the key representation that distinguishes V2A from I2A, enriching visual conditioning with minimal architectural modification. We first investigate TD at both the frame and feature levels to identify the most effective representation level at which TD complements visual representations. Based on these findings, we develop a hierarchically continual learning strategy and an annealed temporal differences guidance method to progressively learn and exploit TD information during diffusion training and sampling process, respectively. Extensive experiments on benchmark datasets demonstrate that effectively exploiting TD through our proposed framework significantly improves end-to-end V2A generation quality, even outperforming dedicated V2A representations such as contrastive audio-visual pretraining.

5. Breaking the Curse ofMultilinguality inMany-to-Many Speech-to-Text Translation via a Resource-AwareMixture of Speech Encoders

Authors: Yexing Du, Kaiyuan Liu, Youcheng Pan, Bo Yang, Chengpeng Fu et al.

Categories: cs.CL, cs.AI Score: 5.80/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:8 Nov:4 Rep:8)

Strength: 4B 参数模型在 45×44=1980 个翻译方向上取得 COMET 83.3 均分，1552 方向 ≥80，超过 27B/30B 基线，参数效率和覆盖广度突出。
Weakness: 核心 MoSE 机制是语言级硬路由（frozen+trainable Whisper 编码器），与 PaM、Language-Routing MoE 等已有工作高度重叠，消融实验未隔离各组件独立贡献且仅覆盖 6 个低性能语言。
Full review: Claude Code 全文七公理审稿

Multimodal large language models (MLLMs) have achieved significant success in speech-to-text translation (S2TT). However, when processing multilingual speech inputs, a single speech encoder shared across all languages suffers from the curse of multilinguality: languages at different resource levels compete for limited representation capacity, leading to strong high-resource performance but substantial degradation on low-resource speech. To address this problem and improve multilingual consistency, we propose MSRT, a novel framework built around a resource-aware Mixture of Speech Encoders (MoSE). MoSE uses an explicit language router to assign each utterance to an appropriate expert encoder. A frozen expert preserves high-resource language capabilities, while a trainable expert adapts to and specializes in medium- and low-resource languages. We further introduce a five-stage curriculum learning strategy that substantially reduces data dependence, requiring only 10 hours of paired S2TT data per language for effective alignment. We conduct extensive experiments on 45 languages, systematically evaluating all $45 \times 44$ translation directions. Our 4B-parameter model achieves state-of-the-art performance, outperforming substantially larger baselines. Empirical analyses show that MoSE improves high-, medium-, and low-resource languages simultaneously, with the largest gains on low-resource speech, thereby breaking the curse of multilinguality without compromising high-resource performance. To support future multilingual S2TT research, we release our code and models.

6. AudioScape-TTA: A Structured Soundscape Benchmark for Fine-Grained Text-to-Audio Evaluation

Authors: Jinting Wang, Yuguang Yang, Shengyu Li, Yan Rong, Shan Yang et al.

Categories: cs.SD, cs.AI Score: 5.79/10 (Obj:8 Id:6 Ind:5 Comp:8 Eff:6 Nov:5 Rep:3)

Strength: 结构化 soundscape 表示 + complexity-aware 分区是真实增量，人类对齐验证强（Overall SR Spearman 0.879 vs CLAP 0.312），13 模型评测覆盖度充分且揭示了 speech-content 生成这一关键失败模式（11/13 模型 SCCA=0%）。
Weakness: 未声明代码/数据发布，benchmark 可复现性严重不足；rubric-based 评估的核心概念与同期 AnyAudio-Judge（2026.06，存在共同作者）高度重叠；语音评估样本量极小（163 speech-content rubrics），统计功效不足。
Full review: Claude Code 全文七公理审稿

Text-to-audio (TTA) generation has recently achieved remarkable progress in synthesizing realistic audio from natural language descriptions. However, determining whether generated audio faithfully satisfies complex textual instructions remains challenging. Existing benchmarks mainly rely on global similarity metrics, providing limited insight into fine-grained semantic failures. To address this limitation, we introduce \textbf{AudioScape-TTA}, a structured and complexity-aware benchmark for fine-grained TTA evaluation. AudioScape-TTA represents realistic soundscapes through modality-aware semantic structures and characterizes generation complexity using event density and structural complexity. Based on these annotations, we propose a rubric-based audio-grounded evaluation framework that verifies event realization, acoustic attributes, and speech content through fine-grained semantic criteria. The benchmark contains 2,258 audio-text pairs with 25,707 binary QA rubrics, enabling scalable and interpretable analysis of TTA systems. Experiments on 13 representative open-source TTA models reveal persistent limitations in fine-grained attribute control, speech-content preservation, and compositional soundscape generation. Human validation further demonstrates that our rubric-based evaluation achieves stronger alignment with human semantic judgments than conventional global similarity metrics.

Authors: Zisen Shao, Zihao Wei, Derong Jin, Ruohan Gao

Categories: cs.CV | ECCV 2026, Project page: $\href{https://zisenshao.github.io/AV-MSF/}{\text{this https URL}}$ Score: 5.79/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:6 Nov:5 Rep:5)

Strength: 把物体冲击声场重构为”全局模态参数 + 视觉先验条件化的空间 neural gain field”，在 20% 训练数据下 L1/CDPAM 较 DiffSound 改善 47-58%，并支持接触定位和材质编辑两个下游应用。
Weakness: KNN baseline 在对称物体主导的主表上几乎追平 AV-MSF（ENV 打平、L1 仅差 7%），且超参数在评测指标 ENV 上做模型选择，代码开源未明确声明，削弱了核心贡献的因果归因和独立可复现性。
Full review: Claude Code 全文七公理审稿

While modern 3D reconstruction excels at modeling object geometry and appearance, it largely ignores the rich acoustic cues revealed through physical interaction. Object impact sounds convey material, stiffness, and structural properties that complement vision, yet existing impact sound modeling approaches either rely on expensive physics-based simulation or require large datasets to generalize in a purely data-driven manner. We introduce Audio-Visual Modal Sound Field (AV-MSF), a novel object-level acoustic representation reconstructed from multi-view images and only a few impact sound recordings. AV-MSF builds on 3D Gaussian Splatting integrated with dense 3D visual feature to provide a strong geometry-aware prior, and represents the impact sound field using compact, physically meaningful modal parameters, enabling robust few-shot reconstruction. Experiments on two real-world datasets show that AV-MSF achieves state-of-the-art impact sound rendering, outperforming both physics-based and data-driven baselines. Furthermore, we demonstrate downstream applications enabled by our representation, including contact localization and object sound editing.

8. HyPASE: Hyperbolic Geometry for Parameter-Efficient Speech Emotion Fine-Tuning Framework for Large Audio-Language Models

Authors: Tian Jin, Ruikang Zhang, Zefeng Zhao, Ding Luo, Jin Zeng

Categories: cs.SD, cs.AI Score: 5.70/10 (Obj:6 Id:5 Ind:9 Comp:5 Eff:5 Nov:5 Rep:9)

Strength: 在 MELD 上以 0.12% 参数预算全面超越 Euclidean PEFT（WA +3.54, F1 +5.48 vs LoRA），零样本跨数据集泛化显著（SAVEE WA +35.8 pp vs zero-shot baseline），Gromov δ 分析提供了音频表示具有树状结构的经验证据。
Weakness: HGA 简化后等价于 element-wise weight scaling（Corollary A.5），双曲框架的理论复杂度与实际操作的简单性之间存在未解释的 gap；IEMOCAP 上 WA/F1 低于 LoRA baseline，超参数 τ/λradius 存在崩溃边缘（~30% WA），核心因果链缺乏严格验证。
Full review: Claude Code 全文七公理审稿

Large Audio-Language Models (LALMs) excel at general speech understanding; however, adapting them to fine-grained tasks like Speech Emotion Recognition (SER) remains a significant bottleneck. Current Parameter-Efficient Fine-Tuning (PEFT) methods typically operate in flat Euclidean space, and this geometry fails to capture the multi-granularity nature of emotion cues, which range from low-level prosody to high-level semantics. To address this, we propose HyPASE, a hyperbolic PEFT framework for LALM-based SER. HyPASE leverages the Poincare ball model, using the hyperbolic radius as an explicit proxy for representational granularity. The framework consists of two core components: a Hyperbolic Geometric Adapter (HGA) for layer-adaptive weight modulation, and an Emotion-aware Multi-capacity Cross-modal Aggregator (EMCA) that compresses multi-scale features into compact audio prefixes. Empirical results on standard benchmarks show that HyPASE outperforms Euclidean PEFT baselines across all metrics on MELD and achieves a notable Unweighted Accuracy gain on IEMOCAP, particularly in class-imbalanced emotion recognition, with the accompanying slight Weighted Accuracy trade-off reflecting hyperbolic space’s geometric prioritization of minority-class representations; furthermore, HyPASE achieves robust zero-shot cross-dataset generalization within a constrained parameter budget. By grounding the adaptation process in hyperbolic geometry, HyPASE offers a highly efficient path for LALM fine-tuning.

9. Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models

Authors: Yuezhang Peng, Yuxin Liu, Changfeng Gao, Zhifu Gao, Xiangang Li et al.

Categories: cs.CL, cs.MM | ACM Multimedia 2026 Score: 5.70/10 (Obj:5 Id:5 Ind:5 Comp:5 Eff:6 Nov:4 Rep:9)

Strength: SFC reframing 在 Level 1 使 slot accuracy 提升 5–18%（Table 3, Qwen2.5-Omni ICL SFC 63.86 vs SLU 53.88 Overall），且 SpokenFC-7B 用 GRPO + fine-grained reward 在 SFC Test-ID Overall 达 74.15%，超过 GPT-4o-Audio (68.30%)，同时保持 WER 8.30。
Weakness: 核心 idea 是已有 function calling 范式向 speech modality 的迁移，且 Audio2Tool (2604.22821, 2026-04) 和 BFCL Audio (ICML 2026) 已先发研究 spoken/audio function calling——论文未引用对比这两个直接竞争基准，”first large-scale SFC dataset” 的 claim 不成立。
Full review: Claude Code 全文七公理审稿

Spoken Language Understanding (SLU) is the core component of task-oriented dialogue systems and a pivotal link in achieving seamless human-agent interaction. While traditional SLU can effectively extract user semantics for closed-set tasks after in-domain supervised fine-tuning, it faces significant challenges in leveraging in-context learning for open-domain tasks due to its ambiguous rule definitions. This work proposes Spoken Function Calling (SFC), a novel semantic understanding perspective that optimizes semantic understanding with structured rule definitions, to evolve beyond traditional closed-set SLU. Specifically, we curate and extend a suite of spoken functions based on traditional SLU datasets, construct a multi-agent system to synthesize the SFC-Bench dataset, evaluate the performance of Large Language Models (LLMs) and Large Audio Language Models (LALMs), and enhance the SFC capabilities of LALMs through post-training. Experiments demonstrate that SFC outperforms traditional SLU, substantially enhancing the semantic extraction accuracy for LLMs and LALMs.

10. Helping Music Co-Creation Agents ‘Listen’ Well: Hierarchical Self-Supervised World Models for Understanding and Generation

Authors: Scott H. Hawley

Categories: cs.SD, cs.LG, eess.AS | 20 pages, 14 figures. A 6-page version was submitted to the NeurIPS 2026 Creative AI Track. Supplemental website with listening examples: https://drscotthawley.github.io/midi-rae-jepa-son/. Live demo: https://drscotthawley-midi-rae-jepa-son.hf.space/ Score: 4.26/10 (Obj:6 Id:3 Ind:5 Comp:5 Eff:3 Nov:5 Rep:6)

Strength: Per-level conditioning dropout 在 pixel-space flow-matching 上统一了 variation 与 inpainting 接口，无需 inpainting-specific sampler，CPU 端到端 2.8 s 实现了 live demo 级交互。
Weakness: 唯一生成质量指标 pixel F1=0.996 实为自重建（PCA 90% 方差近乎可逆），mask 区域 note density 仅恢复 53–76%，且无任何同类 inpainting baseline（Polyffusion、Pictures of MIDI、Music SketchNet）的并排对照。
Full review: Claude Code 全文七公理审稿

Collaborative music agents need internal representations rich enough to support both understanding and generation, yet flexible enough for a workflow where the human retains agency. We present a hierarchical self-supervised ``world model’’ for symbolic music: a 2.55M-parameter Swin V2 encoder trained on MIDI piano-roll images with JEPA-style objectives (pitch- and time-shift equivariance, masked embedding prediction, and a distributional regularizer), using no labels and no music-theory vocabulary. Probing the frozen embeddings shows that the level at which a musical property becomes decodable tracks its musical time scale: phrase boundaries are read off the coarsest levels, note density and harmonic detail off the finest. Temporal and phrase structure emerge from the self-supervised objectives alone, while harmonic content must be asked for; a small chord-supervision head raises joint chord recovery from .18 to .54, and key detection, which is never supervised, from .16 to .70. Following the Representation AutoEncoder paradigm, a conditional flow-matching model stands in for a trained decoder, flowing in pixel space from PCA-reduced conditioning: it reproduces a target window at pixel F1 $0.996$, and the same per-level conditioning dropout that controls how far variations stray also enables graphical prompting for masked inpainting with no inpainting-specific sampler. The pipeline runs on CPU producing a suggestion in $2.8$ s, or $0.6$ s on Apple MPS, which we demonstrate in a live interactive demo. In concert with an LLM-based brain, these capabilities supply the core of a collaborative music creation agent in service of, rather than in place of, human agency.

11. A Modular Part-of-Speech Tagger for Scottish Gaelic using spaCy

Authors: Peter Stefan, Peter J Barclay, Alistair Lawson

Categories: cs.CL | A revised version of this paper has been accepted for presentation at UKCI 2026 (https://ukci2026.coventry.ac.uk/home/) and will be published by Springer Score: 4.00/10 (Obj:5 Id:2 Ind:8 Comp:2 Eff:4 Nov:2 Rep:8)

Strength: 代码与数据完全开源，在 Gàidhlig 粗粒度标签集上达到 93.7% 准确率，与最强前序工作 Boizou & Lamb (2020) 的 94.7% 差距较小，且无需特征工程或外部嵌入。
Weakness: 无消融实验、无最简 baseline、无错误分析；两个标签集上均输给最强 baseline；未引用 calamanCy (Tagalog) 和 Partalidou et al. (Greek) 等直接先例，novelty 实际仅为”将 spaCy 默认配置应用于新语言”。
Full review: Claude Code 全文七公理审稿

Part-of-speech tagging for low-resource languages remains challenging due to limited annotated data, especially for linguistically complex languages. Gaidhlig (Scottish Gaelic) is a morphologically rich and endangered language with limited digital resources, making it suitable for examining a lightweight language processing approach. This paper describes using the modular spaCy Natural Language Processing framework to build part-of-speech taggers for Gaidhlig using the Annotated Reference Corpus of Scottish Gaelic. We train two models with minimal pre-processing and configuration: one using a fine-grained tagset and another using a reduced coarse-grained tagset. Both models are trained without external embeddings or pre-trained language models, using only supervised learning from the available corpus. The fine-grained model achieves 88.6% tagging accuracy, while the coarse-grained model achieves 93.7%. The results are comparable to those of the two previously published Gaidhlig taggers, indicating that simple, off-the-shelf language processing pipelines can demonstrate good performance in low-resource and morphologically complex linguistic settings.

12. Searching for Sound-Meaning Collisions: Graph-Based Affordance Retrieval and Multi-Evaluator Ranking for Pun Translation at CLEF 2026 JOKER Task 2

Authors: Russell Taylor, Adam Brikman, Prateek Awate

Categories: cs.CL | CLEF 2026 Working Notes, 21-24 September 2026, Jena, Germany Score: 3.40/10 (Obj:5 Id:2 Ind:2 Comp:4 Eff:5 Nov:4 Rep:2)

Strength: 该系统在 CLEF 2026 JOKER Task 2 leaderboard 上排名第一（37.783），并提供了一项详细的分析，表明同音 affordances 在选定的候选者中被过度呈现（占检索 affordances 的 10.8%，却在最终获胜结果中占 27.5%），且 affordance 质量在各个流水线阶段单调递增。
Weakness: 该系统完全依赖于闭源 LLM 评判者且缺乏人类评估，没有消融实验来分离组件贡献，最强的单一模型（gpt-5.5, 37.119）几乎匹配了完整流水线，且 affordances 仅出现在 3.5% 的最终获胜者中，这意味着检索几乎未对选定输出做出贡献。
Full review: Claude Code 全文七公理审稿

Fifteen years ago, Low proposed that pun translators should stop searching for equivalent words and instead search for new points of contact between sound and meaning. In this paper, we investigate that idea computationally. We model pun translation as a process of discovery, exploration, and selection. A retrieval system searches semantic and phonological neighborhoods for target-language affordances: sound-meaning bridges that may support new wordplay. Multiple language models then explore these opportunities by generating competing translations, while a multi-perspective generate-and-rank architecture selects among them. Beyond system development, our primary contribution is an analysis of how retrieved affordances propagate through the translation process. We find that generators actively exploit retrieved opportunities, evaluators progressively concentrate around stronger sound-meaning bridges, and exact phonological collisions are selected at disproportionately high rates when available. At the same time, many puns still yield no usable affordances, suggesting that retrieval remains the central bottleneck in computational pun translation. The resulting picture is remarkably close to the process envisioned by Low. Successful pun translation emerges not from preserving source-language words, but from discovering new places in the target language where sound and meaning collide.