Daily Papers — 2026-08-24

16 papers on audio, speech, music, and acoustics.

1. AT-ADD: A Benchmark and Challenge for Robust and All-Type Audio Deepfake Detection

Authors: Yuankun Xie, Haonan Cheng, Jiayi Zhou, Xiaoxuan Guo, Tao Wang et al.

Categories: cs.SD Score: 7.90/10 (Obj:9 Id:8 Ind:8 Comp:6 Eff:9 Nov:7 Rep:8)

Strength: 全类型音频深伪检测基准，规模领先（Track 2 四类音频、68 生成器、46.7 万段），最强基线 76.73%/79.47% Macro-F1，获胜系统 90.71%/96.10%；bootstrap 与生成器级分析质量高。
Weakness: 评测含私有/inhouse 数据源，第三方无法完全复现；与 ESDD 2、Xie et al. 全类型基准的差异边界论证不足，生成器与真实源覆盖偏保守。
Full review: Claude Code 全文七公理审稿

Recent audio generation models can synthesize high-fidelity speech, environmental sound, singing voice, and music, creating new risks for multimedia trust. Existing audio deepfake detection (ADD) benchmarks remain predominantly speech-centric and often underrepresent realistic channel variation and diverse audio types. This paper presents AT-ADD, a large-scale benchmark and challenge designed to evaluate both robust speech deepfake detection and all-type audio deepfake detection. Track 1 evaluates binary speech detection under unseen generators, diverse recording conditions, signal perturbations, and replay effects. Track 2 evaluates type-agnostic real/fake detection over speech, sound, singing, and music when the audio type is unknown at test time. We detail the dataset construction, evaluation protocol, and reproducible baselines, and analyze the final systems submitted to the ACM Multimedia 2026 Grand Challenge. The strongest official baseline obtains 76.73% and 79.47% Macro-F1 on the Track 1 and Track 2 evaluation sets, respectively, whereas the winning challenge systems reach 90.71% and 96.10%. Beyond aggregate rankings, sample-level analysis of the top five submissions examines generator- and type-level difficulty, cross-system error complementarity, and ranking stability. The results show that large-scale self-supervised representations, condition-aware augmentation, multi-crop inference, and structured fusion or routing are central to generalization, while generator-specific robustness and consistent performance across diverse audio types remain unresolved.

2. Toward Sub-1 kB Identity-Preserving Face Compression: A Benchmark of Codecs, a Custom Learned Codec, and Studies of Resolution, Demographic Fairness, Recompression, and Adversarial Robustness

Authors: Petr Hurtik, Jakub Sochor

Categories: cs.CV Score: 7.50/10 (Obj:9 Id:8 Ind:9 Comp:5 Eff:8 Nov:7 Rep:6)

Strength: 512 B 预算下的「场重排」发现——AVIF/HEIF/JPEG XL/JPEG 在 512 B/112 px 坍缩到 28–98% FNMR@1e-4，而自研 Ours-ACCURATE 在 KK 达 6.9%（WebP 24.3%），且经独立 CVLFace-IR101 复核，评测协议与统计严谨度为同类最高档。
Weakness: 自研编解码器的身份侧流在独立 matcher 上增益趋零（id-cos 差 -0.0002）、字节重分配反而 PSNR +0.82 dB，方法复杂度与增量收益错配，且编解码器权重与源码未发布、核心被测对象不可复现。
Full review: Claude Code 全文七公理审稿

Storing face images under a hard sub-kilobyte budget, as required for identity documents, smart-card biometrics and bandwidth-constrained verification, forces a codec to discard most of the signal while keeping what a face matcher actually reads: identity. Generic codecs optimize pixel fidelity, not the embedding distances that drive verification, so which codec, resolution and setting best preserve identity at 1024 bytes or less, and how that degrades at 512, is unclear. We benchmark ten general and face-specific codecs across resolutions, byte budgets, two datasets (controlled Color FERET, in-the-wild AI-Solutions-KK) and four anchor face matchers, with a fourteen-model ViT and CNN roster confirming the ranking is backbone-invariant. We then train a custom identity-preserving codec that hits the byte budget exactly via binary search over a frozen gain table, and run four studies: resolution, demographic fairness, recompression, and no-box adversarial robustness. Sub-kilobyte identity preservation is feasible, but which codec to deploy depends entirely on the budget. At 1024 bytes and the 112 px working resolution the problem is close to solved: modern codecs hold Color FERET equal-error rate under 0.35 percent on the ArcFace anchor. At 512 bytes the field re-sorts: AVIF, HEIF, JPEG XL and legacy JPEG collapse to 28 to 98 percent false-non-match rate at FMR 1e-4, while WebP, JPEG-AI and our byte-budgeted learned codecs stay out of that band, with 24.3 percent for WebP against 6.9 percent for our accurate variant in the wild. That re-sort, not the 1024-byte ranking, is the operational result: a codec chosen at 1 kB is not the codec to deploy at half that.

3. WnW: Waxing-and-Waning KV Cache for Long-Form Speech LLMs

Authors: Yiming Yao, Chenyang Lyu, Xuanfan Ni, Longyue Wang, Weihua Luo et al.

Categories: cs.CL, cs.SD | Accepted at EMNLP 2026 Main Conference. 8 pages, 5 figures Score: 7.40/10 (Obj:1 Id:2 Ind:1 Comp:1 Eff:2 Nov:2 Rep:1)

Strength: WnW 在 GPU 上仅保留 20% 音频 KV 时，两个 3B 骨干上 WER 距 Full Cache 仅 1.6 点以内（Voxtral 6.23% vs 6.79%），而 prefill-only 基线同预算下全部无法终止（WER>190%），24B 骨干同样保持领先。
Weakness: 代码仓库（github.com/XMUDeepLIT/WnW）review 时仍 404 未发布，关键基线 AudioKV 官方实现不可得，且核心对比使用了截断 WER 口径，独立复现与归因验证尚缺。
Full review: Claude Code 全文七公理审稿

Long-form audio inputs make the KV cache the dominant memory cost of speech LLMs. Prefill-only KV compression methods permanently discard audio KV positions once evicted, with no pathway to recover them during decoding. We show this is fragile on long-form audio: prefill attention concentrates near the audio start (an attention-sink effect), while decode-time attention distributes broadly, and the two rankings overlap weakly. We propose WnW (Waxing-and-Waning KV cache), which classifies KV-heads into anchor, tidal, and fixed roles via offline calibration. Anchor heads remain on GPU and serve as a decode-time importance observer; tidal heads keep a CPU-resident complement that is recalled chunk-by-chunk based on aggregated anchor-head scores; fixed heads keep only an on-GPU subset, with the rest permanently discarded. On LibriSpeech-Long with two 3B backbones (Voxtral-mini-3b and Qwen2.5-Omni-3B), WnW preserves near-Full-Cache accuracy while keeping only 20% of audio tokens on GPU, where prefill-only baselines fail to terminate. Results generalize across language, task, and domain shifts, and CPU-GPU recall adds little decode-time overhead in our measurements.

4. The ISCSLP 2026 Real-World Audio-Visual Speech Enhancement Challenge

Authors: Challenge Organizers

Categories: eess.AS, cs.SD | The First Real-World Audio-Visual Speech Enhancement (AVSE) Challenge Score: 7.40/10 (Obj:1 Id:2 Ind:1 Comp:1 Eff:2 Nov:2 Rep:1)

Strength: 发布 ISCSLP 2026 真实世界 AVSE 挑战的可复现协议（mix/remix 双场景 + 五类视觉退化 + 说话人不相交划分），附完整开源基线：Track 1 remix 开发集 SI-SDR −4.069 dB/STOI 0.388，AV-ConvTasNet checkpoints 与离线评测器全部公开。
Weakness: Track 2 同时改变退化视频、独立 checkpoint 与远场子集三重变量，与 Track 1 无法构成成对视觉消融，视觉退化增益始终未被自身量化；基线全指标极弱（测试 CER 高达 102%），且缺音频-only/无处理对照与人类听感校准。
Full review: Claude Code 全文七公理审稿

Audio-visual speech enhancement (AVSE) uses visual-speech cues from a target speaker to recover that speaker’s speech from noisy or overlapping speech. Many widely used protocols construct mixed signals from separately recorded audio sources and assume reliable video, leaving their performance under natural overlap and visual failure insufficiently characterized. The Real-World AVSE Challenge evaluates two related settings. Track~1 comprises two scenarios: real-world mixtures recorded with two speakers speaking simultaneously, without a corresponding clean reference signal, and synthetic remixes obtained by manually mixing the separately recorded speech of two speakers, with a clean reference signal available; Track~2 reuses audio but pairs it with a degraded target video and contains additional 3-m far-field recordings. The speakers in the development and test sets are disjoint. Evaluation metrics include clean-waveform fidelity, learned quality estimates, transcription accuracy, and speaker identification. In the remix task on the development set, the baseline model achieved an SI-SDR of $-4.069$~dB and an STOI of $0.388$ on Track~1, and an SI-SDR of $-2.851$~dB and an STOI of $0.470$ on Track~2. We release the AV-ConvTasNet checkpoints, the offline evaluator, and the official baseline results on the development and test sets.

5. PolyChirp: Multi-Species Birdsong Classification Using TinyML on Low-Power Acoustic Sensors

Authors: Nathan Duboisset, Zhaolan Huang, Felix Bießmann, Roudy Dagher, Antoine Lavandier et al.

Categories: cs.LG, cs.AI Score: 7.40/10 (Obj:8 Id:8 Ind:8 Comp:7 Eff:7 Nov:6 Rep:8)

Strength: 系统级评测完整：INT8 下多物种分类在 k=1→10 时宏 F2 从 0.98 缓降到 0.97，NPU 把 mel 模型推理时延降到 2–39ms（约降 2 个数量级），代码双仓库公开可重跑。
Weakness: 核心主张「整季单电池」无季节能耗核算支撑，预测真值全来自 BirdNET 打标且无实地部署录音验证，SincNet/LEAF/Transformer-Time 等模型在 INT8/算子层面即失效徒增复杂度。
Full review: Claude Code 全文七公理审稿

Recent progress in the field of TinyML has demonstrated that low-power hardware based on microcontrollers can achieve bird species monitoring in real time based on acoustic sensor data for an entire breeding period on a single battery charge. However, the state of the art on low-power microcontrollers was so far limited to binary classification of a single species. In contrast, real fauna monitoring deployments often target multiple species simultaneously. To address this challenge we develop PolyChirp, an approach combining biological domain expertise, automated dataset curation, neural architecture optimization and novel hardware to achieve multiclass bird species detection in the wild. PolyChirp is based on newly designed tiny multiclass models that leverage recent microcontrollers and hardware acceleration with a neural processing unit (NPU). We evaluate the predictive performance of these models, and we measure their computational performance – memory footprint, latency, energy consumption – on common microcontroller hardware. Our results demonstrate that PolyChirp not only outperforms state-of-the-art on single species binary classification, but also achieves robust classification of up to 10 species simultaneously, while still fitting with the resource envelope of a sensor that must remain operational in the field for a full season on a single battery charge.

6. EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis

Authors: Tianchi Liu, Zeyang Song, Tianrui Wang, Zhipeng Li, Chenglin Xu et al.

Categories: eess.AS, cs.AI | Accepted to EMNLP 2026 (Main Conference) Score: 7.30/10 (Obj:8 Id:7 Ind:7 Comp:7 Eff:8 Nov:7 Rep:7)

Strength: 冻结 639M decoder 仅用 +0.43% 参数零延迟开销实现句内平滑情绪转换，MOS-Tra 3.63 领先最强基线 2.80，对 GPT-4o mini TTS / ElevenLabs v3 偏好胜率 72.0% / 64.4%（95% CI 下限超 60%），并以 7 种注入架构的系统失败分析佐证解耦设计。
Weakness: 帧级时变情绪条件与 training-free 句内控制已有前身工作、TED-TTS 引用标题失真；合成训练数据与评测共享同一 VAD 插值范式及预测器存在同源性；每说话人单模型、零样本泛化与真实录音语料表现均未验证，EMNLP 2026 接收声明与专利通知无法独立核实。
Full review: Claude Code 全文七公理审稿

Psychological research on emotion dynamics has established that human affect is a continuous, evolving process: emotions rise, decay, and transition within seconds. Current emotional text-to-speech (TTS) systems, however, condition on a single discrete label or static embedding per utterance, fundamentally misaligning with the temporal nature of affect. While recent LLM-based TTS systems may implicitly vary prosody through text understanding, such variation is neither explicitly controllable nor precise enough for targeted intra-utterance transitions. We address three challenges: (1) a multi-pass flow blending pipeline synthesizes frame-aligned transition audio, circumventing the scarcity of natural intra-utterance transitions; (2) dual-stage Valence-Arousal-Dominance (VAD) conditioning guides prosodic planning in the LLM and acoustic realization in the flow decoder via frame-level VAD embeddings; (3) direction-magnitude decoupled injection structurally separates emotion direction from injection magnitude, preventing content degradation. EmoTra-TTS adds only +0.43% parameters with no latency overhead, achieves 30%-87% relative improvement on emotion transition quality, corroborated by 64.4%-79.5% overall win rates in pairwise preference tests against four SOTA baselines and two commercial systems.

7. Better Retrieval, Worse Robustness: How Multi-hop RAG Amplifies Upstream ASR Errors

Authors: Zhenghua Bao

Categories: cs.CL, cs.IR, eess.AS | Accepted to EMNLP 2026 (Main Conference) Score: 7.20/10 (Obj:9 Id:8 Ind:8 Comp:7 Eff:7 Nov:5 Rep:8)

Strength: 用 2×2 消融与跨基准/跨 ASR 系统/真实语音三重验证证明多跳 RAG 结构扩展会放大上游 ASR 错误——IRCoT+HippoRAG2 相对 Naive RAG 的 oracle–NG F1 差距在各基准扩大 36–67%，而实体损坏占 2Wiki 降级案例的 87–96%。
Weakness: 全部结论基于每口音单一 TTS 声线的合成语音，真实语音验证未覆盖端到端 RAG 行为，且依赖闭源 OpenAI 组件与规则化实体标注，外推与复现均留有余地。
Full review: Claude Code 全文七公理审稿

Speech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline as a fixed upstream constraint. We empirically test whether two extensions to standard retrieval-augmented generation (RAG), entity-graph linking and iterative reformulation, absorb or amplify these errors. Using four English accents synthesized through neural TTS, we evaluate four RAG configurations on three multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA and MuSiQue) against a clean-text oracle. Although the structurally richer configurations generally retain higher absolute F1 under ASR input, both extensions amplify the error: the F1 gap from clean text to the highest-WER accent is 36-67% larger under their combination than under naive dense retrieval, on all three benchmarks. The dominant failure mode is corruption of one or more query entities, accounting for 87-96% of degradation cases on 2WikiMultiHopQA across all four methods. Two lightweight surface-form mitigations leave most of the gap intact, indicating that downstream retrieval structure amplifies remaining entity errors. We release code and data at https://github.com/Continuum-AI-Corp/spoken-multihop-rag .

8. Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds

Authors: Nan Duan, Haoyang Huang, Weiyang Jin, Haoran Li, Yaowei Li et al.

Categories: cs.CV | Project page: https://echo-team-joy-future-academy-jd.github.io/Echo-1.5-Page/ Score: 7.20/10 (Obj:1 Id:2 Ind:1 Comp:1 Eff:2 Nov:2 Rep:1)

Strength: 双变体系统在长视频 7 指标中 6 项第一（ViCLIP 0.8264、Voice 0.8524），世界模型 WBench 81.7 居首，四步蒸馏因果变体仅 0.7 分差距且 Interaction 87.9 最高，代码与权重均已开源。
Weakness: 长视频评测沿用自家 v1.0 协议且记忆取自真值视频、缺乏逐机制消融与蒸馏延迟量化，训练超参与数据规模未披露，核心方法多为既有技术（DMD/SGF/UCPE/MeanFlow）的组合。
Full review: Claude Code 全文七公理审稿

Video generation is progressing beyond isolated clips toward long-form narratives and interactive worlds, requiring models to preserve identities, follow user controls, and remain stable over extended rollouts. We present JoyAI-Echo-1.5, a unified audio-visual generation system with two purpose-built variants. The long-video variant introduces composable cross-shot memory that aggregates visual evidence across multiple prior shots and speaker cues derived from speech-filtered full-shot audio, enabling persistent character appearance and voice identity across flexible combinations of text, image, and memory conditioning. The world-model variant converts heterogeneous navigation inputs into calibrated metric 6-DoF camera trajectories and injects them through a geometry-aware conditioning pathway, enabling controller-agnostic interaction across flexible viewpoints. To support efficient long-horizon generation, we transform a bidirectional audio-visual backbone into a causal few-step generator using progressive teacher forcing and short- and long-horizon Self-Gradient Forcing on self-generated rollouts. Experiments demonstrate strong performance in both settings. JoyAI-Echo-1.5 achieves improvements over existing long-video baselines in cross-shot consistency, visual quality, text alignment, and speech fidelity. Its world-model variant ranks first on WBench, with an average score of 81.7, and achieves leading visual quality and long-horizon persistence on SANA-WM-Bench. Together, these results indicate that memory, geometric control, and rollout-aware training provide a practical foundation for generating coherent stories and continuously evolving interactive worlds. Project page: https://echo-team-joy-future-academy-jd.github.io/Echo-1.5-Page/.

9. Do Spoken Language Models Hear Speech as They Read Text? Bridging Structural Gaps Between Speech and Text

Authors: Hyeonyu Kim, Hwayeon Kim, Youngwon Choi, Myeongkyun Cho, Huu-Kim Nguyen

Categories: cs.CL, cs.AI | Accepted to EMNLP 2026 Findings Score: 7.10/10 (Obj:1 Id:2 Ind:1 Comp:1 Eff:2 Nov:2 Rep:1)

Strength: 以 CKA（0.3570→0.6399）与 token 级相似度图系统诊断 SLM 语音/文本弱对齐，动态 query 分配解耦长度与语义后 SpeechR 52.91、Speech-IFEval CEQ 96.14、forgetting rate -12.18，均超越 Qwen2-Audio-Instruct 与 DeSTA2。
Weakness: 缺 ASR+LLM 级联与纯文本 LLM 上界两个对照基线，闭源对比全面落后 Gemini-1.5-Pro（SpeechR 52.91 vs 67.68），且 CKA 最高的 InfoNCE 变体性能反而最差，对齐强度权衡缺乏先验准则。
Full review: Claude Code 全文七公理审稿

Spoken Language Models (SLMs) generate textual responses directly from speech, offering an alternative to cascaded systems. Despite recent advances, existing SLMs still exhibit weaker instruction-following behavior and limited generalization across diverse tasks compared to text-based language models. Our analysis shows that speech and text representations in current SLMs remain weakly aligned despite strong downstream performance, indicating that structural differences between continuous, temporally varying speech and discrete text remain insufficiently addressed. To address this, we propose a simple framework that decouples length mismatch from semantic alignment and encourages closer correspondence between speech and text representations. Experiments across multiple benchmarks demonstrate competitive performance against strong baselines, underscoring the importance of explicitly addressing structural differences between speech and text in SLM training. Our code is publicly available at https://github.com/jaykim9870/Do_SLMs_Hear_Speech_as_They_Read_Text.

10. Unsupervised Speech Recognition at the Syllable Level

Authors: Liming Wang, Kai-Wei Chang, Kunio Kashino, David Harwath, Mark Hasegawa-Johnson et al.

Categories: eess.AS Score: 6.70/10 (Obj:1 Id:2 Ind:1 Comp:1 Eff:2 Nov:2 Rep:1)

Strength: 提出音节级 UASR 框架 SylCipher，在 G2P-free 设置下 LibriSpeech CER 21.8%（matched）/35.9%（unmatched），较 wav2vec-U 字符级降 40%/17%，普通话带声调 PER 12.2%，并首次给出多语种（MLS 三语）与 DRC/NSC 噪声信道下的可识别性理论；六数据集、五语种、matched/unmatched 双评测矩阵完整。
Weakness: 新颖组件（MLM+JE2E，无 PUSM 阶段）LibriSpeech CER 39.2-43.5% 反而劣于既有 PUSM 的 35.5%，头部增益主要来自追加既有 PUSM 目标，独立增量证据弱；代码仓库 github.com/cactuswiththoughts/SylCipher 评测时 404、权重未发布；绝对 CER 21.8% 仍为 G2P 基 REBORN（8.3%）的 2.6 倍。
Full review: Claude Code 全文七公理审稿

Training speech recognizers with unpaired speech and text – known as unsupervised speech recognition (UASR) – is a crucial step toward extending ASR to low-resource languages in the long-tail distribution and enabling multimodal learning from non-parallel data. However, existing approaches based on phones often rely on costly resources such as grapheme-to-phoneme converters (G2Ps) and struggle to generalize to languages with ambiguous phoneme boundaries due to training instability. In this paper, we address both challenges by introducing a syllable-level UASR framework based on masked language modeling, which avoids the need for G2P and the instability of GAN-based methods. Our approach achieves up to a 40\% relative reduction in character error rate (CER) on LibriSpeech and generalizes effectively to low-resource languages that have remained particularly difficult for prior methods. Code is publicly available\footnote{https://github.com/cactuswiththoughts/SylCipher}.

11. Reasoning-Oriented Post-Training and Inference-Time LoRA Rescaling for Audio-Dependent Question Answering

Authors: Weiteng Hu, Yin Cao, Jun Yang

Categories: cs.SD Score: 5.84/10 (Obj:8 Id:7 Ind:5 Comp:4 Eff:6 Nov:4 Rep:8)

Strength: 推理期 LoRA 重缩放以零训练成本把 Qwen-CoT 的 top-1 从 58.93% 提至 61.05%，且全流程代码、衍生数据与超参数公开可复现，骨干依赖性的实证观察（Qwen 微调提升、MOSS 微调掉 7.3–7.8 点）真实清晰。
Weakness: 结构化 CoT 多奖励管线与简单答案级 GRPO 最终成绩持平（58.93% vs 58.93%），最强 67.70% 恰恰来自零样本而非后训练，缩放思想又缺乏与 α-LoRA/SALMONN 的已有研究对照。
Full review: Claude Code 全文七公理审稿

Audio-Dependent Question Answering (ADQA) requires Large Audio-Language Models (LALMs) to answer questions whose correct answers depend on the given audio content. Successful ADQA requires accurate audio perception, identification of question-relevant evidence, and cross-modal reasoning. Using the official ADQA dataset of DCASE 2026 Task 5, we investigate reasoning-oriented post-training with Low-Rank Adaptation (LoRA) and inference-time LoRA rescaling for both Qwen2.5-Omni and MOSS-Audio-8B-Thinking. We introduce a structured Chain-of-Thought (CoT) framework that decomposes the reasoning process into question analysis, question type, audio evidence, and reasoning. We then analyze how task-specific LoRA adaptation affects the two backbones and further explore inference-time rescaling of trained LoRA adapters. Experiments on the development set reveal markedly backbone-dependent behavior: post-training improves the Qwen-based systems but substantially degrades MOSS-Audio under our supervised fine-tuning configuration. Moderate LoRA rescaling further improves the best Qwen system’s top-1 accuracy from 58.93% to 61.05% and partially restores the performance of the fine-tuned MOSS-Audio models, while the best MOSS-Audio system achieves 67.70% top-1 accuracy. Our submitted systems ranked third overall and second among lightweight systems under 10B parameters in the challenge.

12. LipsAM: Lipschitz-continuous Neural Networks for Convergent Plug-and-Play Audio Signal Recovery

Authors: Kazuki Matsumoto, Ren Uchida, Natsuki Yoshino, Kohei Yatabe

Categories: cs.SD Score: 5.80/10 (Obj:1 Id:2 Ind:1 Comp:1 Eff:2 Nov:2 Rep:1)

Strength: 理论完备——给出幅度-相位分离网络 Lipschitz 连续性的充要条件（零保持刻画），并把 Lipschitz 常数评估降维到 N=2 的七变量优化，PnP 收敛性在去混响实验中得到 Δx≈10^-12 的实证支持。
Weakness: 效用证据不足——提出方法在 SI-SNR 上系统性劣于无保证的 ReM-AM-M，仅「有限」优于 soft-thresholding 基线，且无数值表、无代码开源、仅 10 个测试对单任务验证。
Full review: Claude Code 全文七公理审稿

The Lipschitz continuity of deep neural networks (DNNs) is essential for establishing theoretical guarantees regarding their behavior. From both theoretical and practical perspectives, various methods have been proposed to construct Lipschitz-continuous architectures and control their Lipschitz constants. However, several DNN architectures common in audio signal processing fall outside the scope of existing theoretical frameworks, hindering the development of Lipschitz-continuous models in acoustic applications. In particular, despite their widespread adoption, DNNs that separately process the magnitude and phase of complex-valued signals cannot be Lipschitz continuous under existing frameworks. In this paper, to address this limitation, we establish a theoretical foundation for constructing amplitude modifiers (AMs), a class of DNN architectures that operate solely on the magnitude of a complex-valued input, with provable Lipschitz continuity. Specifically, we derive a necessary and sufficient condition for an AM to be Lipschitz continuous and propose LipsAMs (Lipschitz-continuous AMs) corresponding to common architectures for audio signals, including time-frequency masking. Furthermore, we develop an efficient framework for evaluating their Lipschitz constants and analytically derive these constants for some of the proposed architectures. As an application, we propose CoReM-LipsAM (Controlled Residual Maps via LipsAM) for plug-and-play (PnP) audio signal recovery, integrating a DNN as a data-driven prior within a model-based signal processing algorithm. The convergence of the obtained PnP algorithm is structurally guaranteed by the CoReM-LipsAM architecture and empirically validated through speech dereverberation experiments.

13. EXAM$^2$: $\underline{Ex}tending$ $\underline{A}udio$ $Understanding$ $in$ $\underline{M}ultilingual$ $and$ $\underline{M}ultimodal$ $Analysis$

Authors: Jiawen Wang, Xiaoxue Gao, Zi Haur Pang, Nancy F. Chen

Categories: cs.SD, cs.AI | 8 pages, 2 figures Score: 5.20/10 (Obj:8 Id:6 Ind:4 Comp:4 Eff:5 Nov:5 Rep:4)

Strength: 首个覆盖 6 语言 × 4 音频域 × 22,614 视觉候选的多语言多模态音频 MCQ 基准，系统评测 10+ 模型并揭示闭源 73.83% > 开源 61.24% 等结构化差距。
Weakness: 论文承诺的全开源仓库 werywjw/EXAM-2 实测不可访问（GitHub API 返回 404），可复现性存疑；Gemma3n-EXAM$^2$ 的 +21.65% 提升源于”在 EXAM$^2$-train LoTA 后在 EXAM$^2$-test 评测”的循环对比，且闭源 GPT-4o-audio 在所有六语言上仍领先 10+ 点。
Full review: Claude Code 全文七公理审稿

Recent large audio language models (LALMs) have achieved impressive progress in audio understanding. However, existing evaluations remain largely constrained to English and narrow audio domains. Prior benchmarks typically focus on a single audio modality, i.e., speech, sound, or music, limiting the systematic investigation into how these models generalize across diverse visual scenarios. In this paper, we introduce EXAM$^2$, a benchmark for multilingual and multimodal audio understanding spanning six languages and multiple modalities, including speech, sound, music, mixed-audio settings, and visual images. By incorporating visual information alongside heterogeneous audio inputs, EXAM$^2$ enables more realistic evaluation of scene-aware audio reasoning and cross-modal comprehension. EXAM$^2$ comprises $5,667$ multiple-choice questions, $22,614$ image instances, and $135,684$ multilingual translations. We evaluate state-of-the-art open-source and proprietary LALMs as well as multimodal LLMs, revealing substantial performance gaps in multilingual and cross-modal understanding. Furthermore, we propose Gemma3n-EXAM$^2$, a lightweight fusion-model fine-tuned on EXAM$^2$-train, achieves up to $12.4\%$ improvement in multilingual settings and $21.7\%$ gains in multimodal evaluation over a strong baseline. Empirical results establish EXAM$^2$ as a challenging benchmark and pioneer future multilingual and multimodal audio intelligence research.

14. DiaScriber: A Speech LLM for Joint Diarization and Transcription in Multi-Speaker Scenarios

Authors: Bingshen Mu, Xian Shi, Xiong Wang, Zhifang Guo, Ting He et al.

Categories: eess.AS, cs.SD Score: 5.10/10 (Obj:8 Id:5 Ind:6 Comp:5 Eff:5 Nov:5 Rep:2)

Strength: 三阶段训练（CPT/SFT/RL）加三条数据管线的端到端 MSASR 系统在 AliMeeting 上取得 DER 3.6/cpWER 12.0/tcpWER 12.4 全面最优，消融实验证实阶段递进与三奖励组合的机制性增益。
Weakness: 仅对两个自认「产生幻觉」的端到端基线对比、缺级联基线，内部测试集占比过半且构造与 RL 数据同源性未披露，代码/权重/数据全部未发布，第三方无法复现。
Full review: Claude Code 全文七公理审稿

Multi-speaker automatic speech recognition (MSASR) aims to jointly predict content transcriptions, speaker identities, and timestamps, thereby addressing the key question of “who spoke what and when” and holds substantial practical value in real-world multi-speaker scenarios. However, MSASR still encounters considerable challenges in the presence of fast turn transitions, overlapping speech, and complex, diverse multi-speaker scenarios. In this work, we propose DiaScriber, an end-to-end multi-speaker diarization and transcription model built on a speech large language model. We first construct diverse data pipelines to cover a wide variety of multi-speaker scenarios and their complexities, including validation and refinement, turn-transition and overlapping-speech simulation, and multimodal annotation. Furthermore, DiaScriber is developed based on the pretrained version of Qwen3.5-Omni through a three-stage training strategy involving continual pretraining, supervised fine-tuning, and reinforcement learning. Experiments show that DiaScriber achieves superior performance over comparison methods across extensive multi-speaker scenario test sets and demonstrates outstanding generalization ability in unseen multi-speaker scenarios.

Authors: Naman Garg, Sarika Jain, George Fazekas

Categories: cs.AI Score: 4.90/10 (Obj:8 Id:6 Ind:6 Comp:5 Eff:2 Nov:3 Rep:2)

Strength: 透明的竞赛失败报告，系统提交 Blind B 复合 0.321（第 37/40 名），累计消融与 9 信号权重全公开，诚实记录 LLM 注入激进度的边界（9 会话 +0.8%、54 会话 −18.9%），并把 Blind A/B 差距分解为可归因的 −0.272 组件项。
Weakness: 未发布任何代码/权重/种子且依赖付费 API 与私有 embedding 不可复现；关键观察（album continuation 58.1%、保守注入 +0.8%）全部建立在开发期或训练 ground-truth 统计上而”核心组件未部署”，效用与多模态新意均无独立证据，作者在赛前已拥有更优配置（Blind A 0.6012）却未部署。
Full review: Claude Code 全文七公理审稿

We present Team Semiintelligencn’s solution for the ACM RecSys 2026 TalkPlayData Challenge, addressing conversational music recommendation through a multi-modal and personalized conversational recommender system. Our submitted system employs a three-stage pipeline: (1) multi-modal retrieval constructing decay-weighted centroids across seven dense embedding spaces - track- and user-level CF-BPR, Qwen3 (metadata, lyrics, attributes), CLAP audio, and SigLIP visual - supplemented by BM25 lexical retrieval and an artist substring-match signal, all fused via weighted Reciprocal Rank Fusion (RRF) with optimized signal weights; (2) lightweight reranking (history filtering, popularity smoothing, and catalog diversity penalization); and (3) persona-diversified response generation using GPT-4o-mini. Beyond this submitted configuration, we report development-time experiments with additional components - constrained LLM-guided artist injection, album continuation signals, XGBoost LambdaMART, and a superior GPT-4.1 response prompt - that were not deployed to Blind B due to cost and complexity constraints. We optimize RRF weights on a 500-session development split via differential evolution, improving MRR by +19.5%. On Blind A, we observe that unconstrained LLM-guided injection across 54 sessions causes catastrophic nDCG regression (-18.9%), while conservative injection on only 9 sessions yields the best observed Blind A nDCG - a finding we present as a Blind A observation warranting further validation. The submitted system achieves a Blind B composite score of 0.3213.

16. MLLM-Assisted Audio VOS: A 3rd Place Report for the MeViS-Audio Track, 8th LSVOS Challenge

Authors: Liangtao Shi, Jinxia Xie, Xiantao Hu, Ting Liu

Categories: cs.CV | 5 pages Score: 4.90/10 (Obj:1 Id:2 Ind:1 Comp:1 Eff:2 Nov:2 Rep:1)

Strength: 零训练的 MLLM+ASR+SAM 四阶段流水线在 MeViS-Audio 官方测试集取得 Final Score 66.80（季军），消融显示 DAM4SAM 把 J 从 49.72 提升到 53.81、校验模块把 N-acc 从 41.38 大幅提升到 96.55。
Weakness: 无任何与第 1、2 名或既有 AVOS 方法的对比，关键提示工程与校验细节未披露，代码未发布，且依赖 Gemini-3 闭源 API 使整套流水线无法复现。
Full review: Claude Code 全文七公理审稿

In this technical report, we present a training-free framework for audio-guided video object segmentation, which integrates Multimodal Large Language Models (MLLMs) with SAM-based segmentation models. We decompose the task into several stages and identify suitable foundation models for each stage. Without introducing additional model training or task-specific fine-tuning, our approach leverages the strong multimodal reasoning capabilities of MLLMs to model text-visual correspondence and employs SAM-based models for accurate object mask generation. The proposed framework demonstrates the effectiveness of leveraging foundation models for audio-guided video segmentation and achieves competitive performance in the MeViS-Audio Track of the 8th LSVOS Challenge.