Daily Papers — 2026-08-04

21 papers on audio, speech, music, and acoustics.

1. Agogic: Performance-Timed Music Tokens for LLM-Native Text-to-Symbolic-Music Generation

Authors: Junhao Chen, Mingjin Chen, Jingjia Mao, Lin Chen, Saining Zhang et al.

Categories: cs.SD, cs.CL | Project Page: https://yisuanwang.github.io/Agogic Score: 7.60/10 (Obj:9 Id:9 Ind:6 Comp:8 Eff:7 Nov:8 Rep:9)

Strength: 受控跨家族表示交换实验证明表示 Pareto-支配模型规模——0.8B PMT (FMD 159) 超过 27B beat grid (FMD 272)，在 26M from-scratch + 第二 tokenizer 上复现，因果识别严密（8 controls），释放 harness/25+ checkpoints/6.25M captioned corpus。
Weakness: audibility 未验证（人类实验未完成，auto listener p=0.07），caption adherence 弱（CK .10 vs MIDI-LLM .32），captioner-backbone-listener 同为 Qwen 家族构成潜在评测闭环。
Full review: Claude Code 全文七公理审稿

Text-to-music language models begin with a choice usually made by default: how to tokenize music. Normally entangled with backbone, data, and recipe, its effect has never been measured in isolation. We fix pretrained Qwen3.5 (0.8B-27B), data, budget, and decoding, and swap only the representation across seven tokenizations, anchoring texture metrics to each representation’s model-free ceiling. The ordering is clean and surprising: representation, not model size, is the binding variable for distributional fidelity. Scaling the backbone 34x barely moves Frechet Music Distance (FMD), whereas switching representation halves it. PMT, a performance-resolution stream we release (10 ms timing, per-note velocity, multi-track texture; 609 symbols), reaches FMD 159 at 0.8B against 272-286 for beat grids (1.7-1.8x lower, up to 2.8x elsewhere; non-overlapping bootstrap CIs), so a 0.8B performance-resolution model beats a 27B beat grid. It reappears on a 26M from-scratch backbone and a second performance-resolution tokenizer: a property of the class, not one lucky vocabulary. Nor is it a finer-lattice artifact: snapping PMT’s onsets to the beat grids’ resolution still leaves it 67-129 FMD ahead of both (n=500). The effect is distributional; whether it is audible is a separate question, left open by our probe, with a human study pre-registered. Native caption adherence is weak but separable: a lightweight decode-time constraint doubles instrument-F1 (.28 to .60) and Correct-Key (.16 to .35) at no distributional cost. We release the harness, 25+ checkpoints, two corpora (86.6k aligned across caption/MIDI/ABC/audio; 6.25M captioned, the largest for music), and an imprinting diagnostic: published text-to-MIDI systems reproduce their training distribution near-invariant to the caption (72% vs. 71% chord-time on disjoint domains). The field’s next representation claim can now be measured, not asserted.

2. Should We Type or Talk to LLM Agents? A Comprehensive Study of Voice and Keyboard Input Perturbations

Authors: Zizhao Hu, Nathan Elijah Segura, Mohammad Rostami, Jesse Thomason

Categories: cs.AI Score: 7.20/10 (Obj:8 Id:9 Ind:6 Comp:8 Eff:6 Nov:8 Rep:8)

Strength: 通过 550k 评分生成的系统实验，识别出 token survival（原始 token 存活率 r=+0.79）作为跨语音/键盘两信道的统一因果机制，并用前向/后向控制、去污染数据集和 thinking budget 实验多重验证。
Weakness: 仅覆盖 7-14B 开源英文模型且无真实音频/agent 场景，Finding 7（thinking budget）仅基于 1 模型 × 1 benchmark × 164 项目，证据基础过薄；语音算子生成模型（Qwen2.5-7B）与被评测模型部分重叠，存在独立性缺口。
Full review: Claude Code 全文七公理审稿

Human input reaches language models by typing or speaking, and each channel leaves a distinct signature: orthographic noise for keyboards; for voice, disfluency from conventional transcription and restructuring from AI-backed dictation tools. How do they impact an LLM’s performance? In this paper we present HIVE (Human Input-Variation Engine), a suite of voice transcription perturbations and QWERTY keyboard perturbations. We use HIVE to evaluate how robust models are to these perturbations. We present seven findings. (i) Voice transcription perturbations lower accuracy across every instruction-tuned model we test, and it is the structure of the transcription rather than its fillers that carries the cost. (ii) QWERTY keyboard perturbations cost less, and a model absorbs a lot of them before accuracy falls away. (iii) Both trace back to one cause, how many of the question’s tokens survive the perturbation: destroying a token is what hurts, while adding new ones alongside it costs little. (iv) The gap between the two channels appears only where the answer must be constructed or deduced; on multiple choice there is none. (v) The harm does not solely come from test-set contamination. (vi) It cannot be trained away with lightweight adaptation. (vii) A thinking budget recovers the keyboard channel almost entirely but leaves the spoken registers untouched, and compressed speech is worse with it.

3. Equivariant Music Transformer

Authors: Zixun Guo, Simon Dixon

Categories: cs.SD, cs.AI Score: 7.10/10 (Obj:8 Id:8 Ind:6 Comp:8 Eff:6 Nov:8 Rep:9)

Strength: 通过简洁的自蒸馏等变正则化损失（仅 λ=0.001 权重），在 96M 参数模型上同时改善生成能力（LNTP 降低 4.7%）和等变性（Lequiv 从 0.286 降至 0.049），并首次实证揭示”等变性不随 transformer scaling 涌现”的反直觉规律。
Weakness: 客观评测指标 Lequiv 同时用作训练损失和评测指标存在部分循环论证风险；外部 baseline 比较不够公平（架构、规模、特征工程混合），且缺少 LNTP 直接对比；听力测试样本量小（12 人）。
Full review: Claude Code 全文七公理审稿

Humans recognize a musical passage even when it is shifted in time or transposed in pitch, indicating a notion of equivariance in the representation space. Our analysis, however, shows that standard music transformers map such time-shifted or pitch-transposed inputs onto uncorrelated representations: these models become progressively less equivariant as they scale in size or train longer. This suggests that in standard music transformers, additional model capacity is allocated to memorizing absolute patterns rather than capturing shared musical structures. In this paper, we propose the Equivariant Music Transformer (EMT), which enforces equivariance through self-distillation by jointly optimizing a next-token-prediction and an auxiliary equivariance regularization loss. We find that the additional equivariance loss acts as a beneficial regularizer, simultaneously improving next-token prediction and producing equivariant latent representations. Through both objective and subjective evaluations, EMT demonstrates superior equivariance and generative capability compared to data augmentation, feature engineering, and state-of-the-art (SOTA) baselines. More broadly, our findings reveal that standard language modeling methods alone do not capture music’s translational symmetries, and dedicated inductive biases are required to produce better music representations. The code, weights and demos are available online.

4. CLASVS: Continuous-Latent Autoregression for Melody-Preserving Lyric Editing in Singing Voice Synthesis

Authors: Yizhong Geng, Tian-Hao Zhang, Chunfeng Wang, Wenxin Fu, Yingming Gao et al.

Categories: cs.SD Score: 6.95/10 (Obj:8 Id:8 Ind:6 Comp:6 Eff:8 Nov:6 Rep:5)

Strength: CLASVS 通过 SCT 路由将全局歌词规划与局部声学连续性分离，在 matched ablation 下相对 unified-context AR 将 macro-PER 从 .0447 降至 .0376，并在两个普通话 benchmark 上全面超越离散 AR 基线 Vevo2（macro-PER 降低 46.2%），同时保持旋律（FPC .9410）、歌手相似度（SIM .914）和感知质量（N-MOS 4.13）。
Weakness: 方法的组件复杂度较高（6 个前端/模块组合），训练依赖 8000h 不可再分发的私有数据，代码和 checkpoint 尚未释放，且仅在普通话 2-6 音节短编辑上验证，跨语言和长编辑泛化能力未经验证。
Full review: Claude Code 全文七公理审稿

Reference-conditioned melody-preserving lyric editing replaces words while retaining a performance’s timing, singer identity, and naturalness. Continuous-latent autoregression avoids finite codebooks and offers stepwise generation with learned stopping. Editing creates a conflict absent from ordinary reconstruction: training pairs reference cues with original lyrics, whereas inference asks revised lyrics to override source-lyric-correlated cues; one source-following patch can propagate through AR history. We introduce CLASVS. Its State-Control-Transition (SCT) routing keeps target-lyric and reference-melody controls persistent, returns semantic feedback on phonetic progress to the causal planner, and confines the previous latent patch to the local Transition. Progressive State-Control Grounding (PSCG) learns this contract through paired-edit-free, content-consistent Mandarin reconstruction. On two Mandarin benchmarks, CLASVS improves all four operations over discrete-AR Vevo2 and reduces macro-PER by 46.2%, while maintaining melody, singer similarity, and perceptual quality. Together, these results establish a strong continuous-AR operating point for score-annotation-free lyric edits and a basis for broader stepwise control. Audio demonstrations are available on our project page: https://piedpiperg.github.io/Liyric-SVS/.

5. Multi-Task Multi-Frame Visual Piano Transcription

Authors: Yonghyun Kim, Hoyeol Sohn, Juhan Nam, Alexander Lerch

Categories: cs.SD, cs.AI, cs.CV, cs.MM, eess.IV | Accepted to the 27th International Society for Music Information Retrieval (ISMIR) Conference, 2026 Score: 6.95/10 (Obj:9 Id:8 Ind:8 Comp:6 Eff:8 Nov:5 Rep:9)

Strength: V2N 是首个报告 note-level velocity F1 的 video-only VPT 系统，在 PianoVAM 上 +Off F1 从 PPAN 的 45.9% 跃升至 89.5%（+43.6 %p），R3 上 onset/offset 全面超越全部 baseline，代码和 checkpoint 已开源。
Weakness: 核心方法是将 Onsets-and-Frames 多任务范式迁移到视觉模态并组合已有组件（ResNet-18、Conformer ConvModule、BiLSTM），各组件均为已有技术，缺乏深层机制创新；跨数据集迁移几乎完全失败。
Full review: Claude Code 全文七公理审稿

Audio-based piano transcription performs well on onset, pitch, and velocity, but the sustain pedal lets sound persist long after key release, so audio systems predict pedal-extended offsets rather than physical key release. Yet existing Visual Piano Transcription (VPT) systems focus on onset detection from short video windows, offset accuracy lags onset by a wide margin, and note-level velocity has not been reported. To address these gaps, we present V2N (Video to Notes), the first complete VPT system: a shared temporal backbone feeds task-specific heads for onset, offset, key hold, and velocity, jointly trained with per-frame supervision rather than only at the window center. Ablations show that multi-task supervision enables offset and velocity prediction while improving onset accuracy; longer temporal context yields further improvements. V2N sets new state-of-the-art results on PianoVAM and R3.

6. On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs

Authors: Hendrik Vincent Koops, Hao Hao Tan, Elio Quinton

Categories: cs.SD | 8 pages, 4 figures, 4 tables. Accepted at the 27th International Society for Music Information Retrieval Conference (ISMIR 2026). Companion website: https://ismir26latentgeo.github.io/ Score: 6.89/10 (Obj:8 Id:5 Ind:8 Comp:9 Eff:6 Nov:6 Rep:9)

Strength: 发现 BWE 在多个神经编解码器潜在空间中表现为近似线性的全局方向（跨数据集余弦相似度达 0.98，仅需 8 样本即可估计），零参数向量加法在部分设置下 LSD/ViSQOL 优于 280M-1B 参数的生成模型，提出了一个有价值的社区基线。
Weakness: 核心比较依赖引用自其他论文的基线结果而非相同条件下重跑，AudioSR 在 mtd/ccmixter 上的指标与退化输入完全相同（疑似基线失效），使”competitive”声明的可信度受限；SiSpec 指标全面弱于强基线 A2SB；未引用 5 个月前发表的 CodecFlow (arXiv:2603.02022) 等密切相关工作。
Full review: Claude Code 全文七公理审稿

Recent audio restoration increasingly relies on large-scale conditional latent generative modeling, including diffusion, Schrodinger Bridges, and Flow Matching variants, to invert degradations such as bandwidth limitation or noise. We present an analysis of the performance of various state-of-the-art methods compared to simple arithmetic transformations in the latent spaces of multiple neural codecs for musical bandwidth extension. We show that estimating a single transport vector between the clean and degraded latent centroids on a reference set, and adding it to degraded latents, can yield restoration performance competitive with large diffusion models. This suggests, first, that some neural codec latent spaces exhibit structure aligned with audio bandwidth; and second, that in such cases complex conditional models may offer only limited gains over a simple vector addition. We argue that these findings reveal an interesting avenue for future research whereby models could take advantage of the latent space structure in order to offer greater training and parameter efficiency, and overall better performance. Additionally, we propose to consider this simple arithmetic transformation as a baseline for music bandwidth extension research, as it allows an assessment of the contribution of learnable parameters towards restoration performance.

7. GROW: Group-Relative Advantage-Weighted On-Policy Reinforcement Learning of Autoregressive-Diffusion Text-to-Speech model

Authors: Guanrou Yang, Tian Tan, Qian Chen, Ziyang Ma, Yakun Song et al.

Categories: eess.AS, cs.AI, cs.CL Score: 6.42/10 (Obj:8 Id:6 Ind:5 Comp:8 Eff:7 Nov:6 Rep:8)

Strength: GROW 通过 signed group-normalized advantage 替换正指数权重，在三个 multilingual zero-shot benchmark 上将 WER 从 2.016 降至 1.558、SIM 从 0.676 提升至 0.715，同时保持 UTMOS，且训练比 32-NFE DiTAR-GRPO 快 2.9×。
Weakness: 训练 reward 与评测使用相同的 Whisper/WavLM 模型（循环论证风险），且仅与 DiTAR-GRPO 一个 RL baseline 实验对比，缺少与 ARDM-DPO、VGPO、F5R-TTS 等同期方法的实验比较。
Full review: Claude Code 全文七公理审稿

Reinforcement learning for flow-matching text-to-speech is complicated by deterministic ODE sampling: trajectory-level policy-gradient methods typically convert the ODE into an SDE and track per-step likelihood ratios, introducing stochastic perturbations and substantial overhead. We propose GROW, a group-relative advantage-weighted on-policy RL method that acts directly on the standard flow-matching objective. For each prompt, GROW samples a group of on-policy utterances, separately standardizes intelligibility and speaker-similarity rewards within the group, and combines them to reweight flow-matching regression. A Wasserstein-2 velocity penalty anchors the updated model to a frozen pretrained reference. A group-mean reward baseline is introduced to convert reward weighting into advantage weighting. For strong pretrained TTS models with concentrated rewards, positive exponential weighting is dominated by reward-agnostic self-imitation, whereas a zero-mean signed advantage preserves effective within-group credit assignment. Instantiated on DiTAR and evaluated on LibriSpeech and Seed-TTS EN/ZH, GROW reduces average WER from 2.016 to 1.558 and raises speaker similarity from 0.676 to 0.715 while keeping UTMOS. With 10-NFE training rollouts and 32-NFE evaluation, GROW retains comparable performance while training 2.9x faster than 32-NFE DiTAR-GRPO. We will open-source complete GROW codes, faithful DiTAR reproduction, and all model checkpoints.

8. Hear to See: Discerning Stateful Listening for Audio-Visual Instance Segmentation

Authors: Leiye Liu, Miao Zhang, Jiahong Jiang, Jingjing Li, Jialong Zhong et al.

Categories: cs.MM, cs.CV, cs.SD | Accepted by ACM MM 2026 Score: 5.95/10 (Obj:8 Id:6 Ind:8 Comp:5 Eff:6 Nov:6 Rep:5)

Strength: ADM 通过音频调制 Mamba 的 Δ 参数实现异步状态感知跟踪，在异步子集上获得 46.75 mAP（较 AVISM +14.09 mAP），是一个真实的机制创新。
Weakness: ASP 核心依赖外部 MixIT 模型且 k-means+sparse attention 为标准技术组合；论文声称的 +7.8% 提升基于 AVISM 而非同期最强 ACVIS（实际差距仅 +1.86 mAP），且仅单一数据集验证、代码尚未公开。
Full review: Claude Code 全文七公理审稿

Audio-visual instance segmentation (AVIS) requires accurately identifying and tracking individual sounding objects with pixel-level masks. Existing methods struggle to match overlapping acoustic events with visual instances and handle asynchronous audio-visual dynamics. Therefore, two critical questions arise: how can a model establish precise correspondence between overlapping sound sources and visual instances, and how can a model maintain robust tracking when audio and visual signals are temporally misaligned?This paper proposes Hear to See (H2S), addressing these challenges through two mechanisms. The Acoustic-Semantic Projector (ASP) disentangles mixed audio and establishes hierarchical correspondence from semantic to spatial domains. The Asynchronous Dynamics Modulator (ADM) adaptively adjusts state transitions via audio-modulated Mamba, prioritizing current information during dynamic variations and maintaining continuity in stable periods.Experiments on AVISeg show H2S achieves SOTA performance, attaining 48.54 mAP with a COCO pretrained ResNet50 and surpassing the previous by 7.8\%. The code will be open-sourced once the paper is accepted. The source code will be publicly available at https://github.com/leiyeliu/H2S.

9. Echo-Aware Modulation for Compact-Latent Frequency-Time Modeling in Lightweight Acoustic Echo Cancellation

Authors: Ye Ni, Ruiyu Liang, Qingyun Wang, Kai Xie, Cairong Zou et al.

Categories: eess.AS Score: 5.95/10 (Obj:8 Id:6 Ind:8 Comp:5 Eff:7 Nov:5 Rep:5)

Strength: 在 0.2M params / 100M FLOPs/s 的严格预算下，EAM 模块以 26.1% 额外 FLOPs 使 Bark-domain AEC 在 SDR 上超越近 2× FLOPs 的 frequency-domain 对应模型，并在 AEC Challenge blind test 上取得最优 EchoMOS（DT 4.28, FE 4.26）。
Weakness: EAM 模块是已有技术（concat/subtract/multiply 交互 + 多尺度 DWC + 双门控注意力）的组合，缺少组件级消融证明各部分必要性，且未与 CAGCRN (Interspeech 2025) 和 Seidel et al. (ICASSP 2024) 两个最直接的 Bark/ERB-domain 同类工作做实验对比。
Full review: Claude Code 全文七公理审稿

Existing lightweight acoustic echo cancellation (AEC) systems often combine linear AEC with Bark-domain DNN-based suppression to lower the computational footprint. In such systems, downsampling layers further compress the input features into a compact bottleneck representation, but this compression weakens frequency-time modeling capacity and degrades performance. To mitigate this limitation, we propose MSA-EchoLite, a lightweight Bark-domain AEC framework with an asymmetric dual-branch encoder and an echo-aware frequency-time modulation (EAM) module. The EAM module enriches the compressed bottleneck representation by modeling discrepancy and correlation cues between the dual-branch microphone and echo-related latent features. Experimental results show that the Bark-domain variant of MSA-EchoLite offers a better performance-complexity trade-off than its frequency-domain counterpart but is more sensitive to feature compression. With only 26.1% additional FLOPs over its non-EAM Bark-domain variant, its EAM-enhanced version achieves 99.1% of the PESQ of the frequency-domain counterpart, which requires nearly twice the FLOPs, and even surpasses it in SDR. Overall, MSA-EchoLite outperforms state-of-the-art lightweight AEC models while using only 0.2 M parameters and 100 M FLOPs/s.

10. Towards Real-world Environment-aware Zero-shot Text-to-speech Synthesis via Disentangled Audio Infilling

Authors: Ye-Xin Lu, Xin Wang, Yang Ai, Hui-Peng Du, Zhen-Hua Ling et al.

Categories: eess.AS, eess.SP | Submitted to IEEE Transactions on Audio, Speech, and Language Processing Score: 5.84/10 (Obj:7 Id:5 Ind:8 Comp:5 Eff:6 Nov:5 Rep:6)

Strength: 在模拟测试集上环境保真度接近上界（FAD 2.11 vs vocoder 0.13，ESMOS 3.55 vs GT 3.59），TCFG 机制实现了噪声/混响的独立可控调节（Table V），两阶段训练有效弥合模拟-真实域差距（DAIEN-TTS-FT 在真实数据上 FAD 3.62 为最优）。
Weakness: 缺少与直接前作 [20][21][22] 的实验比较，DAIEN-TTS-N→NR 的提升未隔离数据增强效应与架构改进，五个新增贡献均为已有技术的增量组合而非机制创新，代码未开源。
Full review: Claude Code 全文七公理审稿

Recent zero-shot text-to-speech (TTS) systems achieve remarkable naturalness and speaker similarity but typically require high-quality speaker prompts and either strip away or entangle the acoustic environment with speaker characteristics, limiting their real-world applicability. We present an extended DAIEN-TTS, an environment-aware zero-shot TTS framework that disentangles and jointly models speech, background noise, and reverberation, enabling independent control over timbre and acoustic environment through separate speaker and environment prompts. Built upon the flow-matching-based F5-TTS, it uses a speech-environment separation module to decompose environmental speech into speech, noise, and reverberation components, which are injected into the Diffusion Transformer for environment-aware generation. Training uses simulated data constructed by mixing clean speech with noise and room impulse responses, together with a cross-speaker conditioning strategy that suppresses speaker information leakage from the environment branch. When real-world data are available, the system can be further fine-tuned to bridge the simulated-to-real domain gap.At inference, a triple classifier-free guidance mechanism enables fine-grained control over speech, noise, and reverberation, and a signal-to-noise-ratio adaptation strategy aligns the synthesized speech with the environment prompt. Experiments on simulated and real-world test sets show that DAIEN-TTS generates environmental personalized speech with high naturalness, strong speaker similarity, and faithful noise and reverberation reproduction, while offering controllability beyond prior environment-aware TTS systems.

11. Towards More Expressive Spoken LLMs: Fine-Grained Intent Benchmarking and Acoustic-Lexical Decoupled Policy Optimization

Authors: Xiang Lin, Tian-Hao Zhang, Chunfeng Wang, Zhou Pan, Kun Zhan et al.

Categories: cs.SD Score: 5.70/10 (Obj:8 Id:6 Ind:5 Comp:8 Eff:6 Nov:5 Rep:5)

Strength: ALPO 通过 token-specific advantage routing 使 GRPO 在 ES 上提升 +2.55（synthetic）/ +2.52（human-recorded），gradient norm ratio 从 0.69 平衡至 1.12，ablation 证明 routing 是关键组件。
Weakness: GLM-5 同时用于数据生成、训练 reward（R^txt）和 IF 评测形成闭环，ES 与 acoustic reward 直接对齐，且未与 ParaS2S/WavAlign 原始实现直接对比，核心自动指标独立性不足。
Full review: Claude Code 全文七公理审稿

Spoken emotional dialogue requires a model to understand a user’s spoken input and generate a response that is both semantically appropriate and emotionally expressive. This is challenging because communicative intent may be stated explicitly in lexical content or conveyed more implicitly through paralinguistic cues, which can complement or diverge from the words themselves. However, two limitations constrain progress in this area: the scarcity of benchmarks that distinguish these intent expressions, and the lack of reinforcement learning objectives that jointly account for response quality and emotional expression. To address the lack of suitable benchmarks, we introduce ParaIntent, a Chinese benchmark comprising 14 intent categories with balanced explicit and implicit samples, together with a multidimensional evaluation protocol covering intent fulfillment, response quality, and emotional expression. For policy optimization, existing approaches either use a shared objective for text and speech or apply reinforcement learning to only one modality, leaving modality-specific learning signals entangled within policy optimization. Motivated by this, we propose Acoustic-Lexical Decoupled Policy Optimization (ALPO), which computes independent textual and acoustic advantages and routes them to the corresponding text and speech tokens within a unified rollout. Under identical reward functions and training budgets, ALPO improves over standard GRPO on most automatic metrics and achieves the best subjective results among the fine-tuned variants, with particularly clear gains in emotional expressiveness on both the synthetic and human-recorded test sets.

12. Transfer Learning for Avian Bioacoustics under Sparse Positive Labels

Authors: Dhyey Patel, Yunting Yin

Categories: cs.SD | Accepted to IEEE MLSP 2026 Score: 5.60/10 (Obj:8 Id:6 Ind:8 Comp:5 Eff:5 Nov:4 Rep:8)

Strength: 在 BirdCLEF+ 2026 公共验证集上诚实报告了负迁移现象，并进行了严格的文件级交叉验证和排列测试，揭示 PAM 数据源比焦点录音迁移更有效这一有价值的经验发现。
Weakness: 核心方法贡献（L2 逻辑回归元分类器 + 59 维手工特征）本质是 stacking+特征工程，且最佳模型与多个替代方案在统计上不可区分（p>0.05），除 WABAD 外所有迁移增益均未达常规显著性阈值。
Full review: Claude Code 全文七公理审稿

Passive acoustic monitoring is an important tool for biodiversity assessment and wildlife conservation because it supports continuous and non-invasive monitoring of species across large spatial and temporal scales. Robust monitoring remains challenging because many datasets contain sparse positive labels, where species presences may be confirmed while unannotated species cannot be assumed absent. In this work, we study transfer learning under sparse positive labels using BirdCLEF+ 2026 as a target benchmark and BirdCLEF 2021, iNatSounds, WABAD, and BirdSet as external bioacoustic sources. We introduce a multi-source reliability framework that models heterogeneous bioacoustic datasets as distinct supervision sources with differing reliability. Our approach achieves 0.584 macro average precision and 0.860 macro AUC on public BirdCLEF+ 2026 validation labels while outperforming naive source pooling strategies. The strongest gains arise from passive acoustic monitoring datasets and biologically informed source selection. Our findings suggest that transfer learning in bioacoustics is fundamentally a weak supervision and negative transfer problem.

13. Language-Specialized Multi-Teacher On-Policy Distillation for Multilingual LLM-Based ASR

Authors: Yuan Xie, Jiaqi Song, Xianliang Wang, Ming Lei, Jie Gao et al.

Categories: cs.CL, cs.SD, eess.AS Score: 5.50/10 (Obj:6 Id:5 Ind:8 Comp:5 Eff:6 Nov:5 Rep:4)

Strength: LS-MOPD 在仅 50k 训练池上使 2.3B 模型达到 4.45% 平均错误率，超越 8B-30B 开源模型和 best-teacher oracle，并揭示 static acoustic prefix 的蒸馏优势与多教师互补性机制。
Weakness: 仅覆盖 4 种语言变体（中文+英语）却声称 “multilingual”，基座模型 NIM4-ASR 不开源导致不可复现，且 OPD for ASR 已有先驱工作削弱了 “first” 声称。
Full review: Claude Code 全文七公理审稿

Modern LLM-based ASR systems have established multilingual capability as a standard feature, leveraging large-scale multilingual corpora and LLMs’ cross-lingual knowledge to achieve competitive performance across multilingual benchmarks. However, joint modeling of languages with heterogeneous acoustic, phonological, and lexical characteristics inevitably introduces optimization conflicts, undermining language-wise specialization. To address this challenge, we propose Language-Specialized Multi-Teacher On-Policy Distillation (LS-MOPD), which decouples language-specific knowledge acquisition from multilingual capability integration: language-specialized teachers are independently optimized via reinforcement learning (RL), after which their expertise is integrated into a generalist multilingual student through language routing and token-level multi-teacher distillation, thereby reducing direct cross-lingual optimization conflicts. We further explore two acoustic-prefix configurations, static and dynamic, to examine how teacher–student prefix consistency influences the efficacy of on-policy distillation. Experiments on benchmarks covering Mandarin, Mandarin subdialects, Cantonese, and English demonstrate that LS-MOPD substantially outperforms RL baselines and consistently surpasses the empirical performance envelope defined by best-performing RL teachers, revealing its potential to generalize beyond all teachers in multilingual ASR.

14. Identity-Faithful Audio-Visual Target Speaker Extraction with QIANGDA and VOXBLINK2-AVSE

Authors: Peijun Yang, Zhan Jin, Juan Liu, Ming Li

Categories: eess.AS Score: 5.50/10 (Obj:8 Id:5 Ind:8 Comp:5 Eff:6 Nov:5 Rep:5)

Strength: QIANGDA 是首个 AV-TSE 真实同期录音 benchmark，paired identity test 有效暴露 visual bypass 失败模式（ResNet no-FiLM baseline 0% speaker margin, 84.28% residual overlap），评测链使用独立模型（Qwen3-ASR CER + WeSpeaker similarity + WavLM OSD）设计合理。
Weakness: 方法为已有组件标准组合（TF-GridNet + 冻结 AV-HuBERT + FiLM + ECAPA-TDNN margin loss），最佳 checkpoint 的训练 CLI 未保留导致 speaker-margin loss 因果效应无法确认，且未与外部 SOTA（AV-HuMAR, C²AV-TSE, AV-GridNet）进行直接性能比较。
Full review: Claude Code 全文七公理审稿

Audio-visual target speaker extraction should return the speaker indicated by the video, yet a separator can ignore the visual cue and repeatedly output the acoustically dominant voice. We introduce QIANGDA, a Mandarin AV-TSE benchmark of jointly recorded real two-speaker mixtures with synchronized multi-view video. Each scene also contains preceding A-only and B-only stages that provide in-scene speaker references. It contains 77 scenes and 7,598 clips (11.84 hours), including 6,042 dual-annotated mixtures. After processing there leave 6,038 evaluable mixtures and 12,076 target-speaker rows. We additionally curate VOXBLINK2-AVSE from VoxBlink2, comprising 250,828 synchronized audio–lip-ROI pairs from 28,421 identities and 766.17 hours of speech. Our extractor uses frozen, 1,280-dimensional projected AV-HuBERT features, target-conditioned training, and layer-wise feature modulation. We jointly evaluate content with Qwen3-ASR-1.7B CER and target identity with WeSpeaker ResNet34 plus Overlapped Speech Detection (OSD). On the complete manifest, the best archived checkpoint obtains 0.2261 CER, 82.22% strict output correctness, and 69.53% both-output strict success.

Authors: Jingwei Zhao, Gus Xia, Ziyu Wang, Ye Wang

Categories: cs.SD, cs.AI, cs.MM, eess.AS | Accepted by ISMIR 2026 Score: 5.47/10 (Obj:8 Id:5 Ind:6 Comp:5 Eff:6 Nov:5 Rep:4)

Strength: 首次将 BLIP-2 Q-Former 范式迁移到 audio-symbolic 音乐风格学习，实现跨模态钢琴编曲，在风格连贯性指标（GPC 79.2%, VCC 76.6%, TA 83.6%）和 OOD 泛化上超越 PCG2 和 A2M baseline，并首次定义跨模态 audio-to-symbolic 风格迁移任务。
Weakness: 内容保真度显著低于 SOTA（MCA 32.0% vs PCG2 39.3%），缺少最简 baseline 和关键模块消融（Q-Former vs 简单 cross-attention、query 数量 K、数据配对策略），代码和评测脚本未公开，架构本质是 BLIP-2 的跨领域迁移而非新方法。
Full review: Claude Code 全文七公理审稿

What is music style? Though often described using text labels such as “swing,” “classical,” or “emotional,” the real style remains implicit and hidden in concrete music examples. In this paper, we introduce a cross-modal framework that learns implicit music styles from raw audio and applies them to symbolic music generation. Inspired by BLIP-2, our model leverages a Querying Transformer (Q-Former) to extract style representations from a large, pre-trained audio language model (LM), and further applies them to condition a symbolic LM for generating piano arrangements. We adopt a two-stage training strategy: contrastive learning to align auditory style with symbolic expression, followed by generative modeling for music arrangement. Our model generates piano performances jointly conditioned on a lead sheet (content) and a reference audio example (style), enabling controllable and stylistically faithful arrangement. Experiments demonstrate the effectiveness of our approach in piano cover generation, style transfer, and audio-to-MIDI retrieval, achieving substantial improvements in style-aware alignment and music quality.

16. DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learning

Authors: Tristan Wu, Daniel Chin, Junan Zhang, Junyan Jiang, Yansen Jing et al.

Categories: cs.SD | Accepted to the 27th International Society for Music Information Retrieval Conference (ISMIR 2026). 8 pages, 3 figures, 1 table Score: 5.30/10 (Obj:6 Id:5 Ind:4 Comp:5 Eff:5 Nov:5 Rep:8)

Strength: 将 masked discrete diffusion 首次应用于 synthesizer inversion 任务，在 OOD NSynth 上的 GRPO 微调显著降低了音频距离指标（wMFCC 从 10.04 降至 5.96），且代码和数据已开源。
Weakness: GRPO 奖励与评测指标严重重叠（wMFCC/CLAP/MSS/SOT/RMS），构成循环验证；AR baseline 在域内大多数指标上优于 DD；最直接的竞争方法 SynthRL 未被作为 baseline 实现。
Full review: Claude Code 全文七公理审稿

Synthesizer inversion is challenging for two main reasons: 1) Distinct parameter configurations can produce perceptually similar sounds. 2) Parameter-space losses often fail to reflect rendered audio similarity, while the synthesizer being a non-differentiable black box prevents simple audio-domain supervision. To address the one-to-many mapping induced by the first challenge, we formulate synthesizer inversion as conditional generation over discrete synthesizer parameters and use masked discrete diffusion as the generator. This treatment additionally avoids the fixed-order assumption of autoregressive models and the continuous-relaxation mismatch of flow matching when modeling categorical synthesizer controls. To address the second challenge, we further fine-tune the model with GRPO-style audio-domain rewards computed from rendered outputs. Experiments on Dexed show that, after supervised training, the discrete diffusion model is competitive with autoregressive and flow-matching baselines, and reward-based fine-tuning further improves out-of-domain audio matching performance. Code and demos are available at: https://github.com/DDSynth-RL/DDSynthRL.

17. MeloCodec: Harnessing Melodic Priors for High-Fidelity Singing Voice Representation

Authors: Yizhong Geng, Wenxin Fu, Kecan Mao, Qifei Li, Yingming Gao et al.

Categories: cs.SD | 7 pages, 3 figures, 3 tables. Accepted at IEEE ICME 2026 Score: 5.26/10 (Obj:8 Id:5 Ind:6 Comp:5 Eff:5 Nov:5 Rep:4)

Strength: 在低比特率 (1.5 kbps) 下，通过 Tokenize-then-Fuse 范式，F0-RMSE 从 2.68 (DAC) 降至 1.64，并诊断出 codebook collapse (<5% utilization) 为 Direct Fusion 的根本原因。
Weakness: 没有代码发布，依赖不可公开获取的 5500 小时内部训练数据，缺失 FACodec 基线，且损失权重和 chromagram 超参数未报告——阻碍了独立复现和公平比较。
Full review: Claude Code 全文七公理审稿

Neural audio codecs serve as fundamental tokenizers for LLM-based audio generation. While semantic priors are widely exploited to enhance linguistic intelligibility, the integration of explicit acoustic priors remains underexplored, limiting synthesis fidelity in frequency-sensitive domains. To address this gap, we introduce MeloCodec, a novel framework designed to effectively incorporate melodic priors, a critical form of acoustic information for singing. To address the optimization instability typically caused by the direct fusion of such explicit priors, we propose a Tokenize-then-Fuse paradigm that pre-trains a discrete melodic branch to lock in structures before feature fusion. To robustly realize this paradigm, we further propose a two-stage training strategy that prevents codebook collapse and ensures stable convergence. Experiments show that MeloCodec outperforms baselines in singing voice representation, improving pitch consistency and enabling controllable pitch manipulation with minimal timbre degradation.

18. Cross-cultural evaluation of taste-sound correspondences in AI-generated music

Authors: Matteo Spanio, Massimiliano Zampini, Luisa Torri, Riccardo Migliavada, Bruno Mesz et al.

Categories: cs.HC, cs.SD | Submitted to PLOS ONE Score: 4.84/10 (Obj:6 Id:5 Ind:9 Comp:5 Eff:4 Nov:5 Rep:9)

Strength: 首次跨文化（阿根廷/意大利/日本，N=361）验证 AI 介导味觉–声音对应，发现同一粗糙/不协和声学区域在意大利和日本被映射为 sour 而在阿根廷被映射为 bitter，且 within-subject z-scoring 消除了国家主效应但保留了语义组织差异（three-way interaction partial η²=.018, p<.001），数据与代码完全开源。
Weakness: 效应量整体偏小（preference r≤.30, ANOVA partial η²≤.03），被试间一致性极低（Krippendorff α=0.10–0.19），仅 39/100 片段被听为预期味觉，salty prompt 在所有三国均失败且未尝试替代方案来隔离原因混淆。
Full review: Claude Code 全文七公理审稿

Sonic seasoning research has shown that listeners attribute systematic gustatory and emotional meaning to sound, and text-to-music generative artificial intelligence has recently been used to render gustatory prompts as musical stimuli. Whether the taste-sound correspondences acquired by such models hold beyond the cultural context in which they were validated remains untested. We extended a single-country study to a three-country online experiment conducted in Argentina, Italy, and Japan (N = 361). Participants first indicated their preference between base and fine-tuned MusicGen excerpts generated from four taste prompts (sweet, sour, bitter, salty), and then rated fine-tuned excerpts on twelve taste, emotion, and thermal descriptors. Preference for the fine-tuned model was confirmed in Argentina and Italy but not in Japan, and the salty prompt yielded the weakest correspondence in all three cohorts. Ratings differed substantially between countries, yet the main effect of country was no longer detectable once ratings had been standardized within participant, whereas the interactions characterizing the mapping of prompts onto descriptors remained essentially unchanged. Much of the apparent cross-cultural divergence is therefore attributable to differences in scale use; a structural component nevertheless persists. In addition an exploratory factor analysis indicated that the twelve descriptors were organized along different latent dimensions in each cohort. These results indicate that cross-cultural variation in AI-mediated sonic seasoning operates at two levels: the overall level at which taste is attributed to a given stimulus, and the relational structure of those attributions. Evaluations of generative music systems across populations should accordingly distinguish response-style bias from genuine perceptual reorganization.

19. Speaker Verification Under Real Classroom Conditions for English Speech

Authors: Saba Tabatabaee, Jing Liu, Megh Krishnaswamy, Carol Espy-Wilson

Categories: eess.AS Score: 3.95/10 (Obj:6 Id:5 Ind:5 Comp:3 Eff:5 Nov:2 Rep:2)

Strength: 在真实教室环境（18个英语课堂，218小时，含儿童+成人+babble噪声）上验证 WavLM-TDNN + two-stage 训练，相比 ECAPA-TDNN 基线相对 EER 降低 23.99%，且 classroom-level 5-fold 交叉验证保证了 speaker 独立性。
Weakness: 方法为已有组件（WavLM、TDNN、MoCo、two-stage fine-tune）的工程拼装无机制创新，绝对 EER 15.40% 远未实用，与同团队前期工作（EER 5.35%）存在未解释的 3 倍差距，私有数据集+无代码发布导致不可复现。
Full review: Claude Code 全文七公理审稿

Developing speaker verification (SV) models that are robust to classroom noise and effective across both children and adult speakers is critical for AI tools supporting educational environments. In this study, we use a real-world English-speaking classrooms dataset containing partial speaker identity annotations, with most recordings remaining unlabeled. We adapt the WavLM-TDNN model for classroom SV, achieving average relative reductions in Equal Error Rate (EER) of 23.99% and 6.32% compared to the ECAPA-TDNN baseline and the ECAPA-TDNN model trained on classroom data, respectively. Additionally, we investigate two training strategies for SV in classroom settings: self-supervised learning (SSL) and a two-stage approach that first pre-trains with SSL and then fine-tunes with limited annotated data. Five-fold cross-validation demonstrates that the two-stage strategy consistently outperforms the SSL-only approach, achieving an average relative EER reduction of 13.39%.

20. AI-Based Sound Effect Generation: A Narrative Review of Generative Models Across Input Modalities

Authors: Sandy Abdo, Bill Kapralos, Priyamvada Tripathi, KC Collins, Adam Dubrowski

Categories: cs.SD, cs.AI | 29 pages, 5 figures, to appear in G. A. Tsihrintzis, M. Virvou, N. Bourbakis, and L. C. Jain (Eds.), Advances in Global Applied Artificial Intelligence: Springer, Learning and Analytics in Intelligent Systems Book Series Score: 3.68/10 (Obj:6 Id:2 Ind:5 Comp:5 Eff:4 Nov:2 Rep:5)

Strength: 系统覆盖 30 篇近五年 AI 音效生成工作，按 input modality（text/visual/audio/multimodal）组织并提供 PRISMA flow diagram，Table 1 的模型摘要对新手有入门参考价值。
Weakness: 未引用直接竞争综述 Camara et al. (IEEE TASLP 2024)，分析深度不足（逐篇复述为主，无 meta-comparison、无变量隔离、无可迁移经验规律提炼），检索截止 2025-03 但 2026-08 才发表导致内容过时，排除 preprint 造成系统性覆盖缺口。
Full review: Claude Code 全文七公理审稿

Sound effects play a crucial role in conveying actions, events, and environmental cues across digital applications, often requiring a high degree of variation and contextual adaptability. Artificial intelligence (AI)-driven audio generative models are rapidly growing in popularity and have the potential to transform the way sound is synthesized and used across various applications. In response to this growing momentum, this chapter reviews and analyzes recent AI-based generative models for sound effect synthesis, with a focus on how different input modalities (text, visual, audio, and multimodal) affect the quality, controllability, and contextual relevance of the generated audio. It examines 30 peer-reviewed articles sourced from Google Scholar, IEEE Xplore, and the ACM Digital Library, exploring the evolution of AI generative models over the past five years. The results show that multiple models achieved state-of-the-art performance, producing high-fidelity, semantically aligned, and increasingly temporally coherent sound effects across tasks. However, despite these advances, the review identifies persistent challenges, including limitations in temporal synchronization for complex multi-event scenarios, gaps between objective metrics and human perception, and trade-offs between controllability and generative diversity. Overall, the chapter highlights that AI-driven sound effect generation is progressing toward more adaptive, scalable, and context-aware systems, offering significant implications for future sound design workflows and interactive media applications.

21. Calliphony: A Calligraphy-Driven Interface for Real-Time Generative Music Performance

Authors: Tristan Wu, Ruiji Yu, Gus Xia

Categories: cs.SD | 6 pages, 4 figures. Published in the Proceedings of the International Conference on New Interfaces for Musical Expression (NIME 2026). The first two authors contributed equally Score: 3.16/10 (Obj:5 Id:2 Ind:4 Comp:3 Eff:2 Nov:2 Rep:5)

Strength: 将书法动作通过陀螺仪传感器映射为对公开实时 MIDI 生成模型（Notochord）的外部控制信号，构建了一个可现场演示的低延迟跨模态表演系统。
Weakness: 完全缺乏定量评估（无延迟测量、无音乐质量评分、无用户研究、无 baseline 对比），且核心贡献为已有模块的工程拼装，方法新颖性不足。
Full review: Claude Code 全文七公理审稿

While music generative models have recently gained significant attention, how they can be effectively integrated into live music performances still requires further exploration. This paper presents Calliphony, a calligraphy-driven interface for real-time generative music performance. Specifically, we build a low-latency pipeline that captures brush motion with an attachable sensor and maps it to control signals for real-time symbolic music generation. Using a generative model, the system produces multi-track MIDI in performance settings, while brush-derived control signals constrain event timing and activate additional musical layers. The generated melody is then extended with real-time harmony and additional voices, and finally rendered through a DAW for live staging. Calliphony contributes: (1) a performance-oriented prototype that uses calligraphic motion as an external control layer for a real-time symbolic music generation model, controlling note density, pitch constraints, and accompaniment-layer activation; and (2) a cross-modal performance scenario that extends calligraphy beyond a primarily visual practice into an audiovisual, AI-assisted setting.