每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-08-18
日期2026-08-18
已评分—
均分—
最高—

Daily Papers — 2026-08-18

13 papers on audio, speech, music, and acoustics.

1. Fair ASR: Re-Evaluating Black-Box Jailbreaks under Shared Target-Call Budgets

Authors: Zhida He, Xiaoyu Wen, Han Qi, Ziyuan Zhou, Peng Yu et al.

Categories: cs.CR, cs.AI | 29 pages, 8 figures, 13 tables. Code: https://github.com/xsddys/Fair-ASR Score: 7.79/10 (Obj:9 Id:8 Ind:7 Comp:8 Eff:8 Nov:7 Rep:8)

  • Strength: Fair-ASR 以共享目标调用预算重评测 11 个黑盒越狱攻击，发现排名随预算大幅翻转（如 PAIR 从 B=1 第 8 升至 B=5 第 3），ReCode 在 B=20 下于 GPT-5 达到 85% ASR 且仅用 7.19 次攻击者调用。
  • Weakness: ReCode 头条结果与主重评测使用不同判题模型（GPT-4o+X-Teaming vs LlamaGuard4），跨表判定基准不一致，且目标调用轴未量化 token 用量与 API 定价等现实成本。
  • Full review: Claude Code 全文七公理审稿

Reliable jailbreak evaluation is essential for assessing LLM safety, but most existing studies rely solely on attack success rate (ASR) without accounting for its dependence on attack budgets, resulting in unfair comparisons across methods. Existing compute-aware evaluations reduce heterogeneous resources into FLOPs, which is difficult to estimate for black-box models and fails to capture resource-specific constraints. To provide a comparable evaluation basis, we introduce Fair-ASR, an evaluation protocol for black-box jailbreak attacks under shared target-call budgets B, using target calls as a directly observable and method-agnostic comparison axis while tracking attacker calls separately for efficiency analysis. We re-evaluate 11 representative attacks under the Fair-ASR protocol and find that attack rankings change substantially across target-call budgets, simple stochastic perturbations and hand-crafted templates remain highly competitive under equal target access, and no evaluated LLM-driven method is efficient in both target and attacker calls. Motivated by this efficiency gap, we introduce ReCode, a compositional budget-efficient attack that combines desensitization rewriting with two effective low-cost primitives identified by Fair-ASR. Under a budget of 20 target calls, ReCode achieves 85% ASR on GPT-5 while requiring only 7.19 attacker calls per request on average, showing strong efficiency in both target and attacker calls.

2. FM Synthesizer Audio-Parameter Shared Embeddings

Authors: David Braun, Adam Finkelstein

Categories: cs.SD | Accepted to DAFx 2026 Score: 7.53/10 (Obj:9 Id:7 Ind:6 Comp:7 Eff:7 Nov:8 Rep:9)

  • Strength: DX7-GNN 在 held-out 算法检索达 52.2% R@1 / 88.5% R@10，以 3.38M 参数超越 Transformer（34.6%）与 Highway（24.8%）约 17–27 个百分点，且音频编码器在 timbremetrics 上 69% 逼近 LAION-CLAP 的 72%。
  • Weakness: 检索查询为被检索 preset 的自身渲染音频，属封闭 identity 任务，缺少用户研究验证真实声音设计工作流中的效用，且评测仅覆盖 DX7 单一音高单次渲染。
  • Full review: Claude Code 全文七公理审稿

Given a target sound, finding the synthesizer preset that best reproduces it remains a core problem in sound design. Existing methods treat synthesis parameters as flat vectors, discarding the signal routing and parameter interactions that produce audio. We make two contributions. First, to learn a representation of parameters including their signal routing, we design a graph neural network whose message passing structure imitates FM signal processing. Second, we adapt the multimodal objective from SLAP to learn joint embeddings of audio and FM synthesizer parameters, enabling preset retrieval from a gallery. We focus on the Yamaha DX7, where six identical sinusoid operators interact according to one of 32 routing topologies. Our graph encoder’s message passing weights are shared across all nodes and layers, enabling processing of arbitrary topologies of any size. When every topology is seen during training, the DX7-GNN and two baselines achieve strong audio-to-preset retrieval. When some topologies are held out for testing, the DX7-GNN substantially outperforms both baselines despite having the fewest parameters. Our ablations further support the claim that imitating FM signal flow in a parameter encoder improves generalization to unseen topologies.

3. Why GPT-Style Models Do Not Directly Transfer to Symbolic Music: Compression in the Wrong Coordinate System

Authors: Yi Wang

Categories: cs.LG, cs.AI, cs.SD Score: 7.50/10 (Obj:8 Id:9 Ind:9 Comp:8 Eff:7 Nov:6 Rep:7)

  • Strength: 11 个受控表示臂在共享模型包络下显示时间坐标去嵌套使预测码长相对最小 baseline 降低 42.73%（A→D，1.813576 bits/event），而 BPE 压缩 71.07% 序列长度反而增加 14.00% 码长，直接证明坐标构造优于载体压缩。
  • Weakness: 核心「更高阶音乐组织涌现」结论仅由两例人工挑选的生成样例支撑（与源 Event Jaccard 25.8%/50.8%），无人类评测与下游任务验证，且源码与模型权重未公开。
  • Full review: Claude Code 全文七公理审稿

GPT-style models achieve strong performance by representing language with finite vocabularies of reusable discrete tokens. This success has motivated symbolic music tokenizations to treat recurring musical structures, such as chords, motifs, and phrases, as reusable units analogous to linguistic tokens. However, tokenization derives its advantage not from reusable combinations alone, but from compression: effective compression requires coordinates in which recurring regularities form stable and predictable conditional distributions. The key problem is therefore not to find larger musical combinations, but to discover the coordinate system in which musical facts become predictively compressible. We formulate the Effectiveness–Losslessness Framework and define tokenization as the construction of a predictively effective and relationally lossless coordinate system. The Predictive Effectiveness Principle defines the Fact–Token Boundary: decoupling and denesting construct coordinate interfaces that expose predictive regularities. The Relational Losslessness Principle defines the Token–State Boundary: tokenization stops before context-dependent relations are fixed, leaving their computation to model states. Controlled symbolic-music experiments validate these boundaries. Effective coordinate construction improves predictive compressibility, while fixed relational projections constrain contextual modeling. Sequence compaction alone does not guarantee predictive compression, while preserving contextual freedom allows higher-order musical organization to emerge without explicit structural labels. These results reveal why GPT-style models do not transfer directly across modalities: architectures transfer, but tokenization interfaces do not. Tokenization must discover effective representations while preserving the relational freedom from which contextual structure can emerge.

4. FireRedTTS3: Unified Speech Generation and Editing with Semantically Enriched Speech Representations

Authors: Feiyu Shen, Kun Xie, Yichen Wu, Ziqi Dai, Yichen Han et al.

Categories: cs.SD Score: 7.26/10 (Obj:8 Id:8 Ind:6 Comp:6 Eff:8 Nov:6 Rep:9)

  • Strength: FireRedTTS3 在 Seed-TTS-Eval（平均 WER/CER 3.04%、SIM 78.8%）与 MiniMax-MLS-Test（平均 3.75%、SIM 84.8%）上取得双最佳，并在 InstructTTSEval 六个子任务上全部最优，代码与模型权重均已开源。
  • Weakness: 缺少消融实验量化”冻结语义教师正则化”对缓解误差累积的独立贡献，且多处榜单优势仅 0.02-0.10 个百分点，语义编辑插入子任务指标落后于对比系统。
  • Full review: Claude Code 全文七公理审稿

Recent continuous autoregressive TTS models operate directly on continuous speech representations, preserving rich acoustic details while leveraging the instruction-following capabilities of text LLMs. This paradigm opens new possibilities for voice cloning, instruction-controlled voice design, and speech editing, but remains susceptible to error accumulation during autoregressive generation. Existing solutions often require additional semantic modules, multi-stage tokenizer training pipelines, or complex autoregressive architectures. In this work, we propose FireRedTTS3, a simple yet effective speech generation and editing framework that mitigates error accumulation at the representation level. Specifically, we leverage a frozen Audio Encoder trained on diverse speech understanding tasks as a semantic teacher to regularize the audio feature space. This improves text-speech alignment and stabilizes autoregressive generation while keeping the overall system simple. FireRedTTS3 provides two variants: FireRedTTS3-Base for multilingual and multi-dialect zero-shot voice cloning, and FireRedTTS3-Instruct for unified voice cloning, instruction-controlled voice design, and speech editing. Experiments show that FireRedTTS3-Base achieves the best average speech intelligibility and speaker similarity among compared systems on Seed-TTS-Eval and MiniMax-MLS-Test, while FireRedTTS3-Instruct outperforms competing systems on InstructTTSEval and Ming-Freeform-Audio-Edit. These results demonstrate that semantically enriched continuous speech representations, combined with a simple architecture, enable stable, controllable, and high-fidelity speech generation and editing. Code and models are available at https://github.com/FireRedTeam/FireRedTTS3.

5. UniVerse: Benchmarking and Enhancing LALMs on Culturally Inclusive Low-Resource Music Understanding

Authors: Ziya Zhou, Shangda Wu, Shenyang Xu, Yutong Zheng, Dafang Liang et al.

Categories: cs.SD, cs.MM | 21 pages, 7 figures, 8 tables Score: 7.00/10 (Obj:9 Id:7 Ind:7 Comp:6 Eff:7 Nov:6 Rep:8)

  • Strength: 公开的 UniVerseBench（5,042 QA 对/38+ 语言）与 UniVerseSet（510,078 QA 对）配合训练脚本与严格解析评测代码已上线 GitHub/HF，thinking 格式 SFT 使 Qwen2.5-Omni 提升 14.9 个百分点（33.9%→48.8%），且证明 25 语言训练量与实际准确率无显著相关。
  • Weakness: 多数失衡学习策略增益有限（语言重加权仅 +1.2 点、Text DPO 仅 +2.0 点，Qwen3 上默认 SFT 53.4% 反超全部高级策略），且未报告随机/人类上限基线、未控制 Gemini 建基准与评测的同族污染，训练后权重亦未发布。
  • Full review: Claude Code 全文七公理审稿

Recent advances in large audio-language models (LALMs) have significantly improved performance in tasks such as music captioning, genre classification, and sound event detection. However, limited attention has been paid to improving their adaptability across diverse musical traditions, particularly folk music rooted in distinct cultural contexts. Folk-music traditions are typically resource-scarce, unevenly represented across regions, and poorly documented. Even when such samples appear in large-scale pre-training, LALMs often fail to capture their structural and stylistic characteristics, partly due to the absence of dedicated evaluation protocols and training solutions. To address these limitations, we introduce UniVerse, a reproducible solution for low-resource music understanding. Specifically, we propose UniVerseBench, a benchmark of 5,042 Q&A pairs across more than 38 cultural and linguistic entities, constructed via an expert-guided yet highly automated pipeline. In parallel, we construct a fully automated, model-generated multi-turn dialogue training dataset UniVerseSet. By training LALMs on UniVerseSet, we systematically adapt and investigate representative multimodal imbalance learning strategies across both dense and Mixture-of-Experts (MoE) architectures. Experimental results indicate that fully automated data curation combined with imbalance-aware training yields non-trivial improvements, but models still struggle to capture fine-grained acoustic features, indicating a gap between surface-level alignment and deep musical comprehension.

6. Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges

Authors: Syeda Faiza Ahmed, Zien Sheikh Ali, Hunzalah Hassan Bhatti, Firoj Alam, Shammur Absar Chowdhury

Categories: cs.CL, cs.AI, cs.SD | Multi-turn Conversational AI; Multimodal Dialogue; AudioLLMs; Conversational Memory; Tool-Augmented Agents; Dialogue Evaluation Score: 6.60/10 (Obj:9 Id:6 Ind:6 Comp:7 Eff:7 Nov:6 Rep:6)

  • Strength: 以 PRISMA-ScR 流程从约 4K 篇文献筛出 200 篇详审，给出覆盖文本/语音/多模态/视频/omni-modal/文化六维的会话级统一框架（表 2-11），并配套公开的按数据集/模型/策略/评测分组的 GitHub 资源库。
  • Weakness: 核心缺口统计（43/52 数据集纯英语、32/53 基准纯文本）依赖作者自建标注且无筛选记录/一致性协议发布，相关综述对比（表 1/11）系自我评定，缺少独立验证。
  • Full review: Claude Code 全文七公理审稿

Conversational AI is moving beyond isolated text prompts toward sustained, multimodal interaction. In real conversations, users clarify goals, revise requests, interrupt responses, switch topics, and introduce new evidence while expecting systems to preserve context across turns. This makes multi-turn dialogue a distinct challenge requiring systems to maintain and update memory, ground responses across modalities, tools, and external knowledge, and adapt across languages and cultures. This study reviews multi-turn conversational AI across text-only dialogue, AudioLLMs and speech-native systems, multimodal and omni-modal systems, and tool-augmented agents. We organize the literature around datasets and benchmarks, modeling paradigms, training strategies, evaluation setups, and cross-cutting challenges. Our analysis shows that support for multiple modalities has advanced faster than the ability to sustain coherent interaction across a session. Despite stronger capabilities to perceive, speak, and act across modalities, current systems still struggle with persistent memory, cross-turn grounding, full-duplex interaction, robust evaluation, and cultural alignment. We conclude with a research agenda for systems that can remember, revise, ground, speak, listen, act, and adapt across turns, modalities, and cultures. (https://github.com/faiza-sfa/multiturn-conversational-ai-survey)

7. ChiroEcho: extending automated bat vocalisation classification beyond the learned taxonomy

Authors: Burooj Ghani, Welmoed Eversteijn, Milan van Hirtum, Juan Sebastián Cañas, Vincent J. Kalkman et al.

Categories: cs.LG, cs.SD, eess.AS | 24 pages, 3 figures. Accepted at the CV4E workshop, ECCV 2026 Score: 6.37/10 (Obj:8 Id:5 Ind:7 Comp:6 Eff:7 Nov:7 Rep:4)

  • Strength: 受控 held-out 实验表明属预测 + 地理规则可在物种头完全失效时恢复训练外物种（P. kuhlii 69.5%、P. maderensis 95.4%），把欧洲蝙蝠自动分类的操作覆盖从 35/48 提升到 41/48 种（73%→85%）。
  • Weakness: 缺少与 BSG-BATS、BatSpot 等既有欧洲蝙蝠分类器的物种覆盖与准确率量化对比，且代码库为占位页、训练/推理代码与预训练权重均未发布。
  • Full review: Claude Code 全文七公理审稿

Bats are key indicators of ecosystem health and are protected throughout Europe, making reliable population monitoring a conservation priority. Their cryptic nocturnal lifestyle makes passive acoustic monitoring essential, yet automated identification remains difficult as echolocation calls vary with behaviour and environment and overlap among species. We present a deep learning framework that jointly predicts species and genus and combines genus predictions with geographic species distributions at inference. When only one species of a predicted genus occurs in a region, the framework can resolve species absent from the learned taxonomy. This reframes geographic information as a means of extending, rather than constraining, a classifier’s effective taxonomy. Using recordings spanning 35 European bat species, we evaluate closed-set classification, examine the instability of performance estimates for sparsely represented species, and conduct a controlled held-out proof-of-principle experiment. The rare-species analysis shows how limited evaluation data can obscure species-level performance, while the held-out experiment shows that genus predictions and location can recover labels unavailable to the species head. Geographic resolution extends operational coverage from 35 to 41 of the 48 native European bat species, increasing coverage from 73% to 85%. To our knowledge, this is the broadest operational coverage reported for automated European bat classification. More broadly, the bat framework provides proof of principle for resolving unseen fine-grained classes by combining coarse predictions with transparent external constraints.

8. On computational approaches to Pop music culture

Authors: Arthur Flexer

Categories: cs.MM | 18 pages, 1 figure Score: 6.32/10 (Obj:8 Id:8 Ind:6 Comp:7 Eff:6 Nov:6 Rep:3)

  • Strength: 综述系统梳理了约 112 篇 Pop 音乐多模态计算文献（含 464,411 首、145,912 首、350,000 首歌词等大尺度研究），归纳出「真正多模态稀缺」与「采样导致外部效度存疑」两个开放问题。
  • Weakness: 全文为议程设定性质，未提供任何自身量化结果、试点实验或可复现制品，三条未来路径与作者已发表工作高度同源。
  • Full review: Claude Code 全文七公理审稿

This overview article presents arguments why the computational study of Pop music culture needs to be conducted in a multi-modal way beyond mere audio analysis, gives a survey of already published quantitative work on analyzing Pop music at scale, and discusses challenges and promising research avenues for future work. We argue that Pop music culture is a rich tapestry of audio, visual, textual and cultural connotations and relations which needs to be studied in an integrative way as a multi-modal socio-cultural phenomenon. What is needed is an approach which is reminiscent of “distant reading”, i.e. algorithmic analysis of thousands of books as a research tool in digital humanities. In addition to listening to audio, algorithms need to view album artwork and music videos, to read meta-information, lyrics, music magazines and books. Our review of already available work on distant reading/listening/viewing and multi-modal combinations thereof reveals two major open issues: a scarcity of truly multi-modal approaches and questionable external validity rooted in sampling practices when building music corpora. In trying to overcome these shortcomings we sketch three exemplary avenues for future research on Pop music culture: charting the topic universe of music lyrics, providing an iconography of album cover art, tracking retro cycles in music’s timeline.

9. The Last Mile of Deepfake Speech Detection: An Industry-Academia Experience Report

Authors: Anton Firc, Kamil Malinka, Vojtěch Staněk, Miroslav Hlaváček, Marek Bartoň

Categories: cs.SD, cs.CR | Accepted at the 6th Symposium on Security and Privacy in Speech Communication (SPSC 2026) Score: 6.21/10 (Obj:8 Id:6 Ind:8 Comp:3 Eff:8 Nov:6 Rep:3)

  • Strength: 首次系统披露商用深度伪造语音检测器产品化的内部障碍，给出 codec 增广缺失使 miss-rate 从约 4% 升至约 60%、电话场景 false-alarm 从约 5% 升至约 70% 的量化警示，并映射为 9 项可执行社区行动。
  • Weakness: 全部关键数字无样本量、置信区间与阈值并标注为 proprietary（Table 1 自述不可复现），无代码/数据/权重产出，量化结论无法被第三方检验。
  • Full review: Claude Code 全文七公理审稿

Synthetic speech detection benchmarks now report sub-1% error rates on some in-domain evaluations, yet performance degrades under unseen attacks, channel mismatch, and distribution shift. Based on a three-year effort with Phonexia, a commercial speaker-recognition vendor, we report barriers encountered while building and deploying a detector. Many public benchmarks are not licensed for commercial model development. Real inputs are not four-second clean clips but long, codec-degraded, sometimes partially synthetic recordings. And when a calibrated system returns a log-likelihood ratio of 2.5, no one can tell the customer what it means for their decision. Rather than proposing a new model, we connect these barriers to concrete research and coordination proposals: shared standards for commercially usable datasets, realistic deployment benchmarks, and scores that non-experts can act on. These observations come from one project and should be tested in other settings.

10. SpeechSense: A Paralinguistic-Focused Dataset for Fine-Grained Speech Sentiment Analysis

Authors: Shicheng Ma, Wenqian Cui, Irwin King

Categories: cs.CL, cs.MM, cs.SD | 7 pages, 2 figures, 5 tables. Accepted to ACM Multimedia 2026 (Dataset Track). Dataset and code: https://github.com/Sher13cked/SpeechSense Score: 6.11/10 (Obj:8 Id:6 Ind:6 Comp:5 Eff:6 Nov:6 Rep:8)

  • Strength: 语义-韵律解耦的 8 类立场数据集 SpeechSense（669 条人工验证测试集，Kappa 0.4437）在统一评测中使带声学访问模型达到 42-57% Macro F1，显著高于文本模型的 4.60-22.27%。
  • Weakness: 全合成数据缺乏真实语音的迁移/泛化量化，且未与最接近的合成基准 EmoNet-Voice 做模型级对比，效用与识别证据不足。
  • Full review: Claude Code 全文七公理审稿

Recent advances in AI have revolutionized speech processing, yet effective speech understanding requires discerning not just what is said, but how it is said. Speech Sentiment Analysis plays a critical role in decoding these paralinguistic cues for diverse real-world applications such as recruitment and customer service. However, existing Speech Sentiment Analysis research faces two primary limitations. First, dominant approaches rely on text-centric pipelines that cascade Automatic Speech Recognition with text analysis. This process inevitably discards essential acoustic features like prosody and tone, failing to capture attitudinal meanings in acoustically ambiguous utterances. Second, current benchmarks suffer from a mismatch in label granularity, prioritizing basic emotions (e.g., happy, sad) over the nuanced interpersonal stances (e.g., confident, impatient) necessary for social sensitivity. To address these limitations, we propose a novel dataset, SpeechSense, for fine-grained speech sentiment analysis. Specifically, we define a specialized 8-class taxonomy of interpersonal stances detectable primarily through prosodic cues beyond lexical content alone. We then construct a curated dataset based on this taxonomy, built from high-fidelity speech synthesis and rigorous human validation. Comprehensive experiments across multi-modal LLMs, text-only LLMs, and speech encoders demonstrate that models with acoustic access consistently outperform text-only baselines. These results empirically validate the primacy of acoustic cues in detecting subtle speaker attitudes, highlighting the necessity of SpeechSense. Dataset and supplementary materials are available at https://github.com/Sher13cked/SpeechSense.

11. Low-Power, Neuromorphic, Acoustic Anomaly Detection for Persistent Machine Monitoring

Authors: Steven C. Nesbit, Victor M. Vergara, Michael A. Felix, Evan T. Kain, Luis R. García Carrillo et al.

Categories: cs.NE, cs.AI, cs.ET, cs.LG, cs.SD, eess.AS | 5 pages, 2 figures, 2 tables Score: 6.05/10 (Obj:8 Id:5 Ind:7 Comp:6 Eff:8 Nov:5 Rep:3)

  • Strength: 在 Loihi 2 上部署的自编码器声学异常检测在 DCASE 2026 ToyCar 带噪基准上全部超越官方 baseline（target AUC 0.5317→0.6466），且动态能量 0.0406 mJ/样本，较 CPU 低约 496 倍、较 GPU 低约 132 倍。
  • Weakness: 未发布任何代码、模型权重或部署配置，且 DCASE 结果来自受开发集标签影响的开发集评测、能量优势仅对比 2014 年 CPU 与旧 GPU 并排除静态功耗，单芯片部署功耗仅为外推投影。
  • Full review: Claude Code 全文七公理审稿

Persistent acoustic monitoring can detect machine faults without physical contact, but always-on inference is constrained by power, latency, and deployment complexity. We demonstrate autoencoder-based acoustic anomaly detection on an Intel Loihi 2 neuromorphic processor under clean and noisy conditions. Log-mel features are computed off chip; normalization, autoencoder inference, L1 reconstruction scoring, and thresholding run on chip. In a clean, microphone-position-invariant ToyADMOS ToyCar benchmark, the on-chip model achieves 0.9959 AUC and 0.9785 standardized pAUC at maximum false-positive rate 0.1. In the DCASE 2026 Task 2 ToyCar noisy benchmark, the model achieves source AUC 0.7990, target AUC 0.6466, and pAUC 0.6426, exceeding reported baseline metrics. Power profiling on a 16-chip Loihi 2 VPX system shows real-time throughput with 0.0406$\unicode{x2013}$0.0426 mJ dynamic energy per sample, two orders of magnitude lower than both a CPU and GPU. These results support neuromorphic acoustic anomaly detection as a practical candidate for low-power, persistent machine monitoring.

12. DNN-Based Frequency-Dependent Estimation of Speech, Music, and Noise Power in Acoustic Mixtures for Hearing-Aid Scene Analysis

Authors: Mats Lang, Thomas Haubner, Nina Kiessling, Christoph Hoog Antink, Henning Puder

Categories: eess.AS | Accepted to International Workshop on Acoustic Signal Enhancement (IWAENC) 2026 Score: 6.05/10 (Obj:8 Id:5 Ind:7 Comp:7 Eff:6 Nov:6 Rep:4)

  • Strength: 因果低复杂度 CRNN（181.9k 参数 / 30.9 MFLOPs）把混合谱分解为语音/音乐/噪声功率比例，unseen 域（HEAR-DS+TIMIT）LE 1.71 dB、PCC 0.875，且仅靠语音比例阈值即可在下游 VAD 达到 0.845 平衡准确率，与 Silero VAD（0.848）持平。
  • Weakness: 功率估计任务未与任何替代基线对比（LE/PCC 仅对自身目标），三个声称支持的下游任务只验证了 VAD 一个，且未提供任何代码或权重释放。
  • Full review: Claude Code 全文七公理审稿

Acoustic scene analysis is essential for adapting hearing-aid signal processing algorithms to the current listening environment. However, state-of-the-art (SOTA) systems typically rely on multiple independent estimators for tasks such as scene classification, Voice Activity Detection (VAD), or Signal-to-Noise Ratio estimation, which increases computational complexity and fails to exploit dependencies between related tasks. To address this problem, we propose a unified and interpretable acoustic scene representation by decomposing the observed mixture spectrum into speech, music, and noise power components. This is motivated by the typical listening targets of hearing-aid users. In particular, we estimate time- and frequency-dependent power proportions by a causal low-complexity Deep Neural Network, from which multiple downstream acoustic scene analysis measures can in principle be derived by simple post-processing. In this work, we validate the proposed representation using VAD as a representative downstream task and show performance comparable to a SOTA estimator while providing a substantially richer scene description.

13. Target Speaker Identification: A Low-Latency Streaming Pipeline

Authors: Patrick S. Burke, Satyam Raj, Sean Kinahan

Categories: cs.SD | 8 pages, 4 figures Score: 5.21/10 (Obj:8 Id:5 Ind:5 Comp:6 Eff:5 Nov:4 Rep:5)

  • Strength: 基于 Diart 流式日志 + Pyannote 验证的解耦控制信号流水线在 17 集 This American Life 播客上达到中位准确率 0.93、特异性 0.99（余弦距离阈值 0.7）、1 秒识别延迟，验证了助听器选择性放大的可行性。
  • Weakness: 表 1 部署口径（57s 注册）验证 AUC 仅 0.449，与表 2 及摘要宣称的 >0.90 准确率内部矛盾，且整定仅在单集上进行、评测为单次运行、系统代码未公开发布。
  • Full review: Claude Code 全文七公理审稿

We present a real-time pipeline of open source, pretrained models for streaming identification of a target speaker, motivated by hearing-aid applications where latency as low as 10 ms can be perceptible. We formulate a two-step approach in which incoming audio is first segmented by speaker using low-latency streaming diarization, followed by speaker verification against a registered target speaker. To emulate conversational speech while minimizing overlap, we use the This American Life Podcast Transcripts dataset and select the host as a consistent target speaker. We benchmark offline diarization with Pyannote and LIUM using diarization error rate (DER) and select Pyannote based on baseline performance and compatibility with streaming. We then evaluate speaker verification using Pyannote and TitaNet-Large and generate ROC curves to select an operating region. We integrate Diart and tune clustering parameters to reduce DER while maintaining real-time operation. We pair Diart with Pyannote verification and evaluate system-level performance by converting predicted and ground-truth speech regions into 100 ms binary masks. Across 17 evaluation episodes, the system achieves greater than 0.90 median accuracy with high specificity (0.95-0.98) at cosine distance thresholds of 0.7-0.75, demonstrating a practical proof of concept for downstream low-latency selective amplification.