每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-09-28
日期2026-09-28
已评分—
均分—
最高—

Daily Papers — 2026-09-28

4 papers on audio, speech, music, and acoustics.

1. SPEAR-Gen: Generation-Aware Pre-training for Unified Speech Representations

Authors: Xiaoyu Yang, Arthur Hinsvark, Antonios Alexos, Osama Hanna, Philip C. Woodland et al.

Categories: eess.AS, cs.SD | In Submission Score: 5.05/10 (Obj:6 Id:4 Ind:6 Comp:6 Eff:5 Nov:5 Rep:4)

  • Strength: 在同尺寸、同数据的受控对照下，生成感知预训练让最后一层 SID 从 $55.47$ 升到 $70.23$，同时重合成 SIM 从 $0.467$ 升到 $0.892$。
  • Weakness: 「生成」只用 test-clean 上的声码器重合成来验证，对 WavCube 的差距在 $0.01$–$0.04$ 且无方差，UniWav、OLIVE 被引用却未比较。
  • Full review: Claude Code 全文七公理审稿

Speech understanding and generation place different demands on speech representations, and existing models are typically optimised towards one capability or the other. To reduce this gap, we introduce SPEAR-Gen, a speech representation model that learns a single representation for both capabilities. Task-aligned feature aggregation consolidates complementary linguistic and paralinguistic information across a frozen encoder into discrete targets for masked prediction, while a coarse-to-fine objective combines log-Mel reconstruction with residual flow matching to preserve spectral structure and fine-grained acoustic variation. Experiments on SUPERB and speech resynthesis show that SPEAR-Gen maintains strong understanding performance while substantially improving resynthesis quality and speaker preservation. These results demonstrate that a single speech representation can effectively support both understanding and generation.

2. InstCharVoice: Grounding Natural-Language Instructions for Character-Level Control in Text-to-Speech

Authors: Sihang Nie, Xueru Li, Xiaofen Xing, Deyi Tuo, Cheng-Bin Jin et al.

Categories: eess.AS, cs.SD | 5 pages, 3 figures, 5 tables; Submitted to ICASSP 2027 Score: 4.95/10 (Obj:7 Id:4 Ind:4 Comp:6 Eff:5 Nov:5 Rep:4)

  • Strength: 在 WordVoice 上只加一个 grounding head 和损失重加权，就把自建测试集上的关键词边界错误率从最强基线的 24.27% 压到 14.04%，同时保留了显式字级控制。
  • Weakness: 缺少「LLM 解析指令 + WordVoice」级联基线，且不看指令的 w/o Instruction 变体已胜过全部外部基线，增益无法与训练域及评测器同源的优势分开。
  • Full review: Claude Code 全文七公理审稿

Instruction-based text-to-speech (ITTS) systems enable natural-language control of expressive speech generation, but often offer limited transparency and fine-grained control over individual text units. Character-level controllable TTS systems provide explicit acoustic control, yet typically rely on user-specified acoustic attributes. To bridge this gap, we propose InstCharVoice, a unified framework that grounds natural-language instructions in character-level acoustic control. We first construct grounded instruction annotations on the WordVoice-5A-zh corpus using Qwen3-Omni. With this supervision, we train an autoregressive model to identify instruction-relevant characters and predict their acoustic attributes before generating the corresponding speech tokens. Keyword prediction and grounding-aware loss weighting help the model focus on instruction-relevant characters and attributes. Experiments show improved instruction following and keyword-level acoustic control over representative ITTS systems, with competitive speech naturalness and explicit character-level controllability. Audio samples are available at https://xxh333.github.io/instcharvoice-demo/.

3. Audio Tokens as a Budgeted Resource: Marginal-Utility Allocation for Scalable Audio Representations

Authors: Mingyu Zhao, Jinchao Zhang, Zhiyong Wu

Categories: eess.AS | 32 pages Score: 4.47/10 (Obj:7 Id:3 Ind:6 Comp:5 Eff:4 Nov:4 Rep:4)

  • Strength: 精确序列化前缀约束在 800 次因果评测中零预算违例，离线分配在语音上使 PESQ 显著提升 $0.130$（$1.913\to2.043$，bootstrap 区间排除零）。
  • Weakness: 因果分支只有约 $\pm1\%$ 的 Log-STFT 变化且无区间、D=3 变差，方法对照只有自家固定深度，VRVQ、FlexiCodec 与自家前作 BAMU 均未比较。
  • Full review: Claude Code 全文七公理审稿

Discrete audio tokens are widely used as a representation interface, yet fixed-depth RVQ tokenizers allocate equal capacity to every frame despite varying refinement value. We introduce UniAdapt, which learns marginal utility of RVQ refinements on a frozen codec and allocates them under exact serialized-bit budgets. A rate-independent causal controller predicts acoustic utility, while an optional semantic head supports speech-only utterance-level allocation; measured acoustic and semantic marginal gains on speech have a correlation of 0.42. For causal allocation, a primal-dual allocator selects prefix-valid depths, while an exact guard constrains each sequence prefix to its matched fixed-depth serialized budget. Under utterance-level allocation, UniAdapt reduces Log-STFT distortion by 1.07-4.39 percent across speech, music, and environmental audio without larger budgets. Causally, it improves three of four speech rates with zero violations across 800 utterance-rate evaluations and runs faster than real time. A 20-listener utterance-level MUSHRA study shows a significant 3.52-point speech improvement, with no significant differences on music or environmental audio. These results support separating utility prediction from budget enforcement for scalable, budget-conditioned audio representations.

4. Beyond Textual Chain-of-Thought: JEPA-Conditioned Latent Reasoning for Large Audio Language Models

Authors: Donghang Wu, Haoyang Zhang, Yizhou Peng, Shreyas Gopal, Yi-Wen Chao et al.

Categories: eess.AS Score: 4.37/10 (Obj:6 Id:3 Ind:6 Comp:6 Eff:4 Nov:4 Rep:3)

  • Strength: 在论文受控设置下，把文本 CoT 换成 WavJEPA 条件的 20 步 latent 推理，使 MMAR 从 $43.70$ 升到 $52.80$（$+9.10$，七个子类全升）。
  • Weakness: 唯一受控基线是重训的 Audio-Reasoner，对官方权重 MMAU-mini 仅领先 $0.69$，且未与机制几乎相同、MMAR 已达 $52.60$ 的 CoAT（2606.18273）比较。
  • Full review: Claude Code 全文七公理审稿

Explicit textual Chain-of-Thought (CoT) has improved the reasoning ability of large audio language models (LALMs). However, textual CoTs are often constructed from text captions of audio and provide limited access to the acoustic evidence, which can introduce problems like hallucination. To address this modality-gap issue, we introduce JELAR, a Joint-Embedding Predictive Architecture (JEPA)-based latent reasoning framework that conditions latent reasoning supervision on acoustic representations learned from raw waveforms. During training, a frozen WavJEPA model provides representations learned directly from raw waveforms. A non-causal expert first constructs answer-aware queries, which cross-attend to WavJEPA embeddings to produce latent reasoning targets. The LALM is trained to predict these targets before generating its response. Experimental results show that JELAR improves the Audio-Reasoner baseline by 2.70 and 9.10 absolute percentage points on MMAU-mini and MMAR, respectively, demonstrating the effectiveness of JEPA-conditioned latent reasoning as an alternative to explicit textual CoT supervision.