每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-07-24
日期2026-07-24
已评分—
均分—
最高—

Daily Papers — 2026-07-24

9 papers on audio, speech, music, and acoustics.

1. How Meta-Learning Shapes LoRA Adapter Geometry in Speech Deepfake Detection

Authors: Ivan Kukanov, Janne Laakkonen, Ville Hautamäki

Categories: eess.AS | 7 pages, 5 figures, 3 tables. Submitted to SLT 2026 IEEE Score: 7.16/10 (Obj:8 Id:8 Ind:8 Comp:8 Eff:6 Nov:8 Rep:6)

  • Strength: 在严格单变量设计（固定架构/rank/数据/5种子，仅变目标）下，用 empirical Fisher + RankME 定位 MLDG 相对 ERM 在 LoRA 适配器中留下的几何差异，q/k 集中、out 分散在 30/30 seed×dataset 单元全部成立且经 data-free stable-rank 复现，是一个可靠且可迁移的机制发现。
  • Weakness: 诊断仅做事后描述，未验证能否预测新目标的泛化能力（无 RankME→EER 单调性测试）；功能性后果（perturbation sensitivity）只在 In-The-Wild 单语料上验证，因果链接作者自承为假设。
  • Full review: Claude Code 全文七公理审稿

Meta-learning for domain generalization (MLDG) improves out-of-distribution speech deepfake detection over empirical risk minimization (ERM) when both objectives train low-rank adapters on the same frozen self-supervised speech model. Because the architecture and adapter capacity are held fixed, this gap points to differences in how the training objective shapes the adapter, yet the field characterizes objectives through error rates rather than through the geometry of the solution they reach. We introduce a descriptive diagnostic for this question: holding architecture, rank, data, and seeds fixed and varying only the objective, we use the empirical Fisher on the finished adapter to compare the geometry that ERM and MLDG leave behind. We characterize each adapter with effective-rank diagnostics that separate where the adapter changes from where those changes matter to the loss, resolved by projection and by depth. Applied to ERM and MLDG, the diagnostic shows that the objective does not reshape all adapter projections alike: the loss-relevant update concentrates in the query and key projections while becoming more distributed in the output projection, consistently across six corpora and most strongly in the upper layers. The same contrast appears in the merged update independently of the low-rank factorization, indicating that it reflects the geometry of the effective update rather than the parameterization. These results show that the gap between ERM and MLDG is not only a difference in error rate, but a difference in how loss-relevant capacity is organized inside the adapter, and that loss-aware adapter geometry is a way to see it.

2. IQ-JEPA: A Joint-Embedding Predictive Architecture with a Hermitian Vision Transformer for Sound Speed and Attenuation Estimation from Ultrasound IQ Data

Authors: Masashi Sode, Gianmarco Pinton

Categories: cs.LG, physics.med-ph Score: 7.05/10 (Obj:9 Id:8 Ind:6 Comp:7 Eff:6 Nov:8 Rep:5)

  • Strength: 2^5 因子消融实验与冻结探针实验干净地隔离了自监督作为主导因素（10K 标签下 14.97 m/s 的增益），且复值编码器的 U(1) 对称性设计具有明确的物理依据。
  • Weakness: 全部实验均在 in silico 环境下使用作者自有求解器完成，无真实数据验证，无代码/数据发布，且未与基于物理的迭代方法（CUTE/FWI）进行定量比较。
  • Full review: Claude Code 全文七公理审稿

The speed of sound in tissue is a prerequisite for well-focused imaging and has diagnostic value, but recovering it from raw pulse-echo channel data is fundamentally a nonlinear inverse problem. Learned solvers are fast yet label hungry. Simulated sound-speed labels are expensive, while abundant real channel data is unlabeled. We propose IQ-JEPA to exploit both data types. An encoder is pretrained without labels to predict the latent representation of masked in-phase and quadrature (IQ) regions from visible context, then fine-tuned on simulated maps. Sound speed appears in the IQ signal as a phase difference, invariant to the constant phase offset. The encoder is a Hermitian vision transformer that operates on the complex signal directly. Its attention is equivariant to that phase and its conjugate-product feed-forward is invariant to it, so the encoder reads a quantity analogous to the one classical coherence methods use. On 79,293 Fullwave 2.5 simulations at 2.5 MHz, pretraining on the 63,435 unlabeled acquisitions reaches 15.60 m/s at 10,000 labels. This is a roughly threefold gain in label efficiency over supervised training, growing to over fourfold at 1,000 labels. It is about 2.2x below an InversionNet baseline, and 8.71 m/s at full labels. The gain still grows with more unlabeled pretraining data. Our comparisons point to self-supervision as the dominant factor. The same encoder transfers. Its frozen features expose sound speed and attenuation, and cross-distribution pretraining between layered and abdominal phantoms costs little accuracy. We see this as a first step toward a foundation model for quantitative ultrasound.

3. CODA: Cascaded Online Discontinuity-Aware Alignment for Real-Time Image-Based Score Following

Authors: Yining Yang, Ruogu Chen, Jie Han

Categories: cs.SD | 8 pages, 3 figures, accepted by the International Society for Music Information Retrieval Score: 6.95/10 (Obj:8 Id:6 Ind:9 Comp:6 Eff:8 Nov:6 Rep:9)

  • Strength: CODA 在 MSMD Setting I 将 ≤0.10s 跟踪准确率从 CYOLO-SB 的 .837 提升至 .914（bar accuracy .890→.975），实时 12.8 ms/frame，并在首个 repeat-aware jump benchmark 上将 repeat recovery@1s 从 .12 提升至 .78。
  • Weakness: Ablation 缺 break mode 对标准跟踪影响的报告，且未与”CYOLO-SB + 几何约束后处理”最简 baseline 对比，cascade 必要性论证不足；各模块（cross-attn/beam/priors）单点边际收益均 <3pp，synergy 未分析。
  • Full review: Claude Code 全文七公理审稿

Real-time score following from sheet images remains chal- lenging because the model must process streaming au- dio while resolving highly repetitive visual patterns un- der strict latency constraints. Recent image-based meth- ods have attempted to use multi-resolution prediction by simultaneously predicting the positions of the active sys- tem, bar, and note. However, their predictions across these different levels of notation are independent, which makes the predictions unstable and introduces unnecessary ex- tra search space for bar- and note-level predictions. Most existing methods also lack mechanisms to recover from score discontinuities, such as repeats, da capo (D.C.), or coda jumps. This paper proposes CODA, to the best of our knowledge, the first real-time score following system that addresses both gaps. CODA explicitly exploits the cascaded structure of music scores: it first selects the ac- tive system, then the active bar within it, and finally the active note within the selected bar. This enforces pre- diction consistency across resolutions. A silence-driven break mode enables recovery from arbitrary score discon- tinuities without requiring knowledge of the repeat struc- ture. Evaluated on the Multimodal Sheet Music Dataset (MSMD) piano benchmarks, CODA achieves state-of-the- art tracking accuracy and discontinuity-recovery perfor- mance under real-time throughput. Code is available at https://github.com/ValleyC/CODA.

4. Listen, Do Not Copy: Internalizing Audio-Grounded Scaffold Context for Robust Omni-Model Speech Understanding

Authors: Pengfei Zhang, Biao Tian, Tianxin Xie, Minghao Yang, Xiangang Li et al.

Categories: cs.SD | 9 pages, 4 figures Score: 5.84/10 (Obj:8 Id:5 Ind:7 Comp:6 Eff:5 Nov:5 Rep:5)

  • Strength: 在三个异构 omni 模型上，AGSC 训练后 no-clue capped mpWER 从 25%-71% 降至 9%-15%，silent-audio 控制为 context 捷径提供了干净的行为检测。
  • Weakness: 缺少”普通目标域 SFT 无 AGSC”对照和经典 SAT baseline 比较，无法区分 AGSC 线索设计 vs 单纯域微调的因果贡献；”perception bypass” 核心观察与 “All That Glitters Is Not Audio”（arXiv:2604.24401, 2026-04）高度重叠。
  • Full review: Claude Code 全文七公理审稿

Omni models transcribe clean, single-speaker speech well, but their accuracy drops sharply when speakers overlap and the scene is noisy, exactly where knowing who said what matters most. A natural fix is a short scene description. We show why this is risky: answer-bearing text lets the model copy instead of listen, so the score rises although nothing has been heard; a silent test exposes this shortcut at once. We call this failure mode perception bypass and address it with Audio-Grounded Scaffold Context (AGSC). AGSC links three steps: first, we build clues from audio to guide listening without giving the answer; second, answer-overlap and silence tests probe them for leakage and audio dependence; finally, those clues scaffold training but vanish at test time, yielding no-clue capability. Across three heterogeneous Omni models, training on AGSC lowers no-clue capped mean permutation word error rate (mpWER) on overlapping, noisy speech from 25%-71% to 9%-15%. For streaming control, we formulate a joint GDPO task in which the model learns when to use a clue and how to produce a speaker-attributed transcript from separately normalized format, gate, and transcript rewards. After internalization, AGSC adds almost no inference overhead.

5. Transforming Keystroke Noise to Text: Self-Supervised Acoustic Eavesdropping Attacks on Keyboards

Authors: Atsunori Okada, Akira Ito, Rei Ueno, Yuichi Hayashi, Naofumi Homma

Categories: cs.CR, cs.SD Score: 5.80/10 (Obj:9 Id:5 Ind:5 Comp:5 Eff:8 Nov:5 Rep:5)

  • Strength: 在无标注、便携设备、100–150 击键的低数据条件下实现 99% 文本重建准确率，显著优于 HMM 和 dictionary 基线，并在隔墙和在线会议等真实场景中保持 >90% 准确率。
  • Weakness: 方法是 7 模块工程拼装且缺少组件级 ablation，核心 LLM 纠正组件来自 Ayati 2025，Park 2502.09782（Transformer+LLM for ASCA）未被引用，仅 3 篇自撰短文本评测且无代码开源、依赖闭源 Gemini API。
  • Full review: Claude Code 全文七公理审稿

We present a self-supervised acoustic eavesdropping attack that reconstructs typed text solely from keystroke sounds, without requiring labeled data for the target device. The proposed attack enables stealthy eavesdropping in two real-world scenarios-physical spaces (public and semi-public) and online meetings. Our method combines unsupervised acoustic clustering with Transformer-based language model inference and iterative self-training, enabling stable character inference under highly uncertain acoustic-to-character mappings. We demonstrate that the proposed method achieves over 99% reconstruction accuracy with only 100-150 observed keystrokes under a close-proximity recording setup using a smartphone placed near the target device, significantly outperforming prior unsupervised baselines in low-data regimes. We further evaluate robustness across multiple laptop platforms and in realistic acquisition channels, including distance recording from approximately 3 meters away on the same desk, through-the-wall eavesdropping with a contact microphone, and background keyboard noise in online conferencing systems. Across these scenarios, the proposed method achieves high reconstruction accuracy (often exceeding 90%) with approximately 150-250 observed keystrokes. These results indicate that accurate text reconstruction from keystroke sounds is feasible in practice under an audio-only setting, even with limited observed keystrokes and without requiring device-specific labeled data, highlighting a realistic and previously underestimated privacy risk.

6. Music-JEPA: Learning a World Model of Sound from Action

Authors: Ziyu Wang, Kun Fang, Yann LeCun

Categories: cs.SD Score: 5.79/10 (Obj:8 Id:6 Ind:5 Comp:6 Eff:5 Nov:7 Rep:6)

  • Strength: 首次将 JEPA 重构为 action-conditioned world model 用于钢琴音乐，perturbation win rate 从 0.576 提升至 0.991（temporal target），beat tracking F1@100 达 0.6599 胜 MERT，6M 参数即 comparable 95M MERT。
  • Weakness: Transcription Note F1 仅 0.8381 远低于监督 SOTA 0.9375（不可用），key recognition 伪标签源自 pianoroll 与 action 输入同源存在循环论证，代码未开源，action predictor g 和 inverse planner 模块缺必要性 ablation。
  • Full review: Claude Code 全文七公理审稿

Joint Embedding Predictive Architectures (JEPA) have recently emerged as a paradigm for learning world models by predicting latent representations, offering a promising direction for self-supervised learning. While initial attempts have applied JEPA to the music domain, it remains unclear how such frameworks can naturally support the formation of a world model for music. In this work, we propose to learn a world model of piano sound using JEPA by framing music as an action-conditioned system: the audio is treated as the state, and the pianoroll as the instrument action. Given a current audio state and an action, the model predicts the resulting future audio state, mirroring how humans learn musical sound through interaction. The model is trained in a fully offline setting using paired audio-pianoroll data, without environment interaction. Experiments show that the learned model captures the relationships between musical actions and their resulting sound. The resulting representations support downstream tasks, including beat tracking, composer identification, and key estimation, and enable piano transcription via planning, by searching for actions that best explain a target sound.

7. MEUSLI: a Multilingual Projector for LLM-based ASR and Beyond

Authors: Lorenzo Concina, Seraphina Fong, Marco Matassoni, Alessio Brutti

Categories: cs.CL, cs.AI, eess.AS Score: 5.74/10 (Obj:8 Id:5 Ind:5 Comp:5 Eff:6 Nov:4 Rep:8)

  • Strength: 完全开源的 28 欧洲语言 Whisper→LLM 线性投影器家族（仅 17.31M 可训练参数），多语言预训练初始化使低资源语言 fine-tuning 和新语言 bootstrap 显著优于从零训练（Breton WER 84.3%→79.7%，Albanian 46min 数据即可收敛到 75.6% 而 from-scratch 完全失败 389%），且跨任务迁移到 ST/TID 在 <5h/语言 数据下有效（Finnish ST。
  • Weakness: “first open-science multilingual projector” 的 novelty claim 被 BLOOMZMMS (2024, 102 语言, 开源) 削弱；核心方法为已有组件组合无新机制；低资源语言绝对 WER 仍不可用（Irish 88.55%, Breton 73.40%）；与 Whisper 在 out-of-domain MLC 数据上差距显著且未与 BLOOMZMMS/MLC-SLM Chall。
  • Full review: Claude Code 全文七公理审稿

Lightweight projectors are an established way to connect pre-trained speech encoders with large language models (LLMs), mapping acoustic features into token-level embeddings for tasks like ASR and spoken question answering. Existing systems, however, typically only support a few languages and are often limited to English. We introduce MEUSLI, the first open-science multilingual projector family that links a Whisper encoder with open-source multilingual LLMs, enabling fully open-source end-to-end ASR in 28 European languages. MEUSLI extends prior monolingual pipelines, delivering strong results across high- and low-resource languages. Using proper continual leaning techniques, MEUSLI can be easily extended to other languages not seen in training. We further demonstrate that the MEUSLI projector can be leveraged beyond ASR, enabling multilingual speech translation and topic identification with only a few hours of task specific supervision per language. Overall, MEUSLI provides a solid foundation for multilingual speech understanding tasks, supporting scalable and inclu- sive open-source SpeechLLM

8. MemNMF: Memory-Augmented NMF on LPC Spectra for Anomalous Sound Detection

Authors: Phurich Saengthong, Takahiro Shinozaki

Categories: cs.SD, cs.LG Score: 5.70/10 (Obj:8 Id:5 Ind:8 Comp:8 Eff:5 Nov:5 Rep:5)

  • Strength: 在非平稳机器类型（Valve/Slider）和高噪（-6dB）条件下，MemNMF 相比 AE 基线有大幅 AUC 提升（MIMII Valve: 50.3→92.0，Slider: 73.4→94.1），NMF 初始化贡献在 -6dB 达 16.2 AUC（Table IV）。
  • Weakness: 在 DCASE2020T2 上输给 AudDSR [11]（74.9 vs 78.2 sec-dev Amean），未对比 DCASE 挑战赛 top 系统，且 AE 基线超参数不匹配（batch size/lr 不同），LPC 输入表示的增益无法干净归因。
  • Full review: Claude Code 全文七公理审稿

Autoencoder-based anomalous sound detection is attractive for machine condition monitoring because it can be trained using only normal recordings and yields an interpretable anomaly score from reconstruction error. Most prior work uses spectrogram autoencoders, but reconstructing detailed time–frequency patterns is sensitive to noise and transients, and models can reconstruct some anomalous inputs well, weakening normal–anomaly separation. We propose MemNMF, a constrained reconstruction method that operates on the Linear Predictive Coding spectrum, a compact estimate of the spectral envelope. MemNMF initializes a memory module from an NMF dictionary learned on normal LPC spectra and reconstructs each input as an attention-weighted combination of prototypical normal spectral patterns. Experiments on MIMII and DCASE 2020 Task 2 across multiple machine types and operating conditions show that LPC-spectrum inputs improve a standard autoencoder baseline and that MemNMF yields further gains, with especially strong robustness under noisy, non-stationary settings.

9. Reflector: Arrangement-Aware Harmonic Retrieval for Sample-Based Composition

Authors: Austin Rockman

Categories: cs.SD, cs.IR, cs.LG | 15 pages, 7 figures, 1 table, code, application, and other resources at https://github.com/austinrockman/reflector Score: 5.45/10 (Obj:8 Id:5 Ind:5 Comp:8 Eff:5 Nov:7 Rep:8)

  • Strength: 发现 interval-class kernel 直接用作检索规则时退化为 universal-donor 现象，并给出 L2 normalization + InfoNCE uniformity 逃脱退化的几何解释，配合 Eq. 6 四个数量级的代数加速实现交互式系统。
  • Weakness: 全部效用证据为 oracle 自身上的 intrinsic 重建（NDCG=0.85 对 oracle ground truth），训练与评测共享同一手工 kernel 构成闭环，无外部 perceptual 校验、无用户研究、无与 Stem-JEPA/SampleMatch 等已发表 baseline 的实验对比。
  • Full review: Claude Code 全文七公理审稿

Sample retrieval tools can help composers find harmonically compatible material, but querying from a fixed reference sample becomes less informative as arrangements evolve and the harmonic context shifts with each musical decision. We present Reflector, an interactive audio workstation that tracks harmonic combinations as they accumulate on the composer’s timeline and adapts retrieval as the arrangement develops. The system is organized around a fixed interval-class oracle: a hand-designed table of weights that scores how pitch-class content combines between sources. An encoder trained entirely on synthetic audio learns to approximate the oracle in a 128-dimensional embedding space, where dot products stand in for compatibility scores at interactive speed. As the composer arranges material on a multi-track timeline, a sweep-line analysis discovers co-sounding regions, computes oracle-weighted centroids, and retrieves against the composite harmonic identity of the session as it evolves. Session centroids projected into a navigable 3-D space reveal structural harmonic relations across the composer’s body of work. This paper is a systems account: we give the design rationale for each architectural decision, characterize Reflector’s behavior through intrinsic measurements on a working sample library, and describe the implementation. The characterization yields a central finding: the learned embedding preserves the kernel’s pairwise judgments while covering the whole library, something the kernel cannot do when used directly as a retrieval rule, because the embedding’s normalized geometry cannot express the degenerate solutions that direct scoring favors. The entire pipeline runs locally with no copyrighted training data. Reflector is free, and the training pipeline is open source.