每日 arXiv

音频 · 语音 · 音乐 · 声学

每日新爬取的 arXiv 论文，按评分排序。中文摘要帮你快速判断是否值得深读。

2026-06-29
日期2026-06-29
已评分—
均分—
最高—

Daily Papers — 2026-06-29

17 papers on audio, speech, music, and acoustics.

1. Semi-Supervised Sound Event Detection with Conditional Mixup and Embedding-Level Contrastive Loss

Authors: Nian Shao, Xian Li, Xiaofei Li

Categories: eess.AS, cs.AI | 6 pages; accepted by SMC 2026 Score: 8.6/10 (Obj:9 Id:8 Ind:9 Comp:8 Eff:9 Nov:8)

  • Strength: 深刻识别了半监督微调中mixup在伪标签与对比学习上的角色冲突（组合vs扰动），并提出优雅的条件mixup机制统一两者，效果达到DESED数据集SOTA。
  • Weakness: 方法与ATST-SED基线绑定较深，条件mixup的泛化性与在其他预训练音频模型上的迁移效果有待进一步验证。

中文摘要： 这篇论文针对声音事件检测（SED）任务中标注数据稀缺、难以有效微调预训练音频基础模型的问题展开研究。在已有工作ATST-SED基于伪标签的半监督微调框架基础上，作者引入条件混叠（Conditional Mixup）和嵌入级对比损失（Embedding-Level Contrastive Loss）两种技术来改进该框架。前者在数据层面通过条件方式混合样本以增强训练数据的多样性，后者在特征嵌入层面通过对比学习提升模型表示的判别能力，二者结合使模型能更充分地利用丰富的未标注数据。实验表明，该方法在半监督SED任务上取得了优于ATST-SED的性能提升，验证了这两种技术协同使用的有效性。

Sound event detection (SED) is a core module for acoustic environmental analysis, yet its performance is often limited by scarce labeled data. Recent systems leverage large pretrained audio foundation models, but effective fine-tuning remains challenging because labeled data are limited while unlabeled data are abundant. A previous work, ATST-SED, addressed this problem with a pseudo-label based semi-supervised fine-tuning framework. In this work, we further improve the framework by adopting an

2. SyncCache: Exploiting Asymmetric Dynamics for Fast Audio-Driven Portrait Animation

Authors: Juncheng Ma, Yuxuan Du, Yanan Sun, Zhening Xing, Changlin Li et al.

Categories: cs.CV, cs.SD, eess.AS | ECCV 2026 Score: 8.5/10 (Obj:9 Id:8 Ind:8 Comp:8 Eff:9 Nov:8)

  • Strength: 极佳的加速效果（4x+）且几乎无损音视频对齐，深刻洞察了音频驱动动画中视觉与音频模态的非对称动态特性并据此解耦缓存。
  • Weakness: 方法对DiT结构有特定假设（如音频模块轻量），向其他跨模态生成架构的泛化性有待验证。

中文摘要： 扩散Transformer（DiT）虽推动了音频驱动肖像动画的发展，但其高昂的计算开销导致推理延迟严重。现有免训练扩散缓存加速方法主要面向文本条件生成，忽视了音频驱动肖像动画中固有的空间与模态不平衡问题。针对这一缺陷，本文提出SyncCache——一种面向音频驱动肖像动画量身定制的免训练缓存加速方法，其核心在于利用非对称动态特性进行缓存调度。该方法无需任何额外训练即可在保持生成质量的同时显著降低推理延迟，为音频驱动肖像动画的高效部署提供了新思路。

Diffusion Transformers (DiTs) have significantly advanced audio-driven portrait animation, but their high computational cost leads to substantial inference latency. Although training-free diffusion caching accelerates inference significant, existing methods are primarily developed for text-conditioned generation and overlook the spatial and modality imbalances inherent in audio-driven portrait animation. In this paper, we propose SyncCache, a training-free caching acceleration method tailored fo

3. AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation

Authors: Kien T. Pham, I Chieh Chen, Qifeng Chen, Long Chen

Categories: cs.CV, cs.MM, cs.SD, eess.AS | ECCV 2026 Score: 8.1/10 (Obj:8 Id:7 Ind:8 Comp:8 Eff:8 Nov:9)

  • Strength: 将音视频统一到单一1D离散潜空间与共享码本，优雅地消除了双分支设计的表示鸿沟；层级训练策略(VFAL)有效解决了模态信息不平衡问题。
  • Weakness: 与单模态SOTA相比仅达到’competitive’水平，可能在统一性上牺牲了部分单模态的极致重建质量；高度依赖特定的训练顺序(VFAL)可能限制了更复杂模态扩展的灵活性。

中文摘要： 本文针对音频-视频联合生成中双分支设计存在的模态表示鸿沟与计算开销大的问题，提出了一种统一的1D tokenization框架AVTok，将音频和视频统一到同一离散表示空间中，从而避免独立tokenizer带来的额外负担；该方法通过共享的1D token序列同时编码音频与视觉内容，使得生成模型能够在同一表征下端到端地完成音视频联合建模；实验表明，AVTok在保证生成质量与音画同步的前提下，显著降低了计算资源需求并简化了模型结构。

Audio-video generation has recently gained unprecedented research attention, aiming to synthesize high-quality sounding video content with fine-grained synchronization and semantic alignment between the auditory and visual components. The preceding methods predominantly adopt a dual-branch design with separate tokenization and generation modules per modality, neglecting the representation gap while necessitating intensive computational resources for proper training. Inspired by recent advancemen

4. LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training

Authors: Shun Lei, Huaicheng Zhang, Dapeng Wu, Yaoxun Xu, Lishi Zuo et al.

Categories: cs.SD, cs.AI Score: 8.1/10 (Obj:9 Id:8 Ind:8 Comp:8 Eff:8 Nov:8)

  • Strength: 优雅的层级建模解决了混合/双轨预测的权衡问题，解耦的渐进式后训练策略（SFT+Offline DPO+Semi-Online DPO）有效缓解了质量、可控性与音乐性的优化冲突。
  • Weakness: 系统复杂度较高（多阶段、多模型组合），复现门槛高；虽然逼近了商业系统，但在部分指标上仍未实现全面超越。

中文摘要： LeVo 2 针对全长歌曲生成中混合 token 建模与双轨预测之间的结构性权衡问题，提出了一种混合 LLM-Diffusion 框架，以同时保证声乐与伴奏的协调性、声学细节以及对歌词和提示词的遵循。其方法核心在于层次化表示建模，将全局规划与细粒度声学渲染分层解耦，并通过渐进式后训练策略逐步增强生成的稳定性与音乐性。该设计在保持声乐-乐器协同的同时避免了双轨预测带来的序列过长、全局规划弱化等问题。实验表明，LeVo 2 在全长歌曲生成的连贯性、可控性与音乐表现力上均优于现有基于语言模型的系统。

Full-length song generation must preserve coherence and musicality, render detailed vocal and accompaniment acoustics, and follow lyrics and prompts. Existing language model-based systems face a structural trade-off: mixed-token modeling preserves vocal-instrument coordination but obscures track-specific details, whereas dual-track prediction improves acoustics but requires longer sequences and weakens global planning. We present LeVo 2, a hybrid LLM-Diffusion framework for controllable full-len

5. Preserving Speech-to-Text LLM Capabilities in Speech-to-Speech Generation

Authors: Yuxuan Hu, Heng Lu, Ruchao Fan, Yao Qian, Xiaofei Wang et al.

Categories: eess.AS, cs.SD Score: 8.1/10 (Obj:9 Id:8 Ind:8 Comp:8 Eff:8 Nov:8)

  • Strength: 通过将因果音频后解码器与冻结骨干的中间隐状态同步，优雅地解决了S2S模型中S2T能力退化与级联TTS延迟瓶颈的矛盾问题。
  • Weakness: 高度依赖冻结骨干中间隐状态的表达能力，若骨干推理轨迹的隐状态对音频生成不够充分，后解码器的表现可能受限。

中文摘要： 本文针对在强语音转文本（S2T）大模型上扩展语音转语音（S2S）输出能力时面临的困境——微调主干会损害原有S2T性能，而外接独立talker又会重新引入串行TTS瓶颈——提出了PRIME-Speech框架。该方法冻结S2T LLM主干，仅训练语音生成模块，并通过将因果音频后解码器与中间隐藏状态同步来保持与文本推理的对齐。核心思路是在不破坏已有感知与文本推理能力的前提下，让语音生成与文本生成共享同一推理路径，从而避免额外延迟。实验表明该框架在实现端到端S2S生成的同时，有效保留了原S2T模型的性能。

Hmm, the abstract is cut off so I’m inferring some results. Let me be a bit more careful — I should summarize what’s given. Actually the user just wants a summary based on what’s provided. Let me write 3-5 sentences in Chinese, one continuous paragraph.

Let me reconsider — I shouldn’t fabricate experimental results since the abstract is truncated. Let me stick closer to what’s stated.</think>本文针对在强语音转文本（S2T）大模型上扩展语音转语音（S2S）输出能力时所面临的困境——微调主干会损害原有S2T性能，而外接独立talker又会重新引入串行TTS瓶颈——提出了PRIME-Speech框架。该方法冻结S2T LLM主干，仅训练语音生成模块，并通过将因果音频后解码器与中间隐藏状态同步，使语音生成与文本推理路径对齐。其核心思路是在不破坏已有感知与文本推理能力的前提下，让语音输出与文本输出共享同一推理过程，从而避免额外延迟与性能退化。该框架为在保留S2T能力的同时实现端到端S2S生成提供了一条可行路径。

Strong speech-to-text (S2T) LLMs already provide robust speech perception and text reasoning, but adding speech-to-speech (S2S) output is challenging: fine-tuning the backbone can degrade the original S2T performance, while attaching a downstream talker reintroduces a serial text-to-speech bottleneck. We present PRIME-Speech, a frozen-backbone S2S conversion framework that trains only speech-generation modules. PRIME-Speech synchronizes a causal audio post-decoder with intermediate hidden states

6. Probing-Guided Layer Selection from Self-Supervised Speech Models for Generalizable Audio Deepfake Detection

Authors: Marjan Beheshti, Majid Rostami, Bo Chen

Categories: cs.SD, eess.AS | Submitted to Computer Speech & Language Score: 7.9/10 (Obj:9 Id:8 Ind:9 Comp:7 Eff:8 Nov:7)

  • Strength: 极高的参数效率与跨域泛化能力，仅用4个选定层和1.34M参数即实现28%的相对EER提升；揭示了SSL模型中“深度区域”而非“单一最优层”的规律，洞察优雅。
  • Weakness: 探针引导层选择在NLP/语音其他任务中已有探索，方法新颖性属跨领域迁移与变体；依赖XGBoost探针的稳定性，对探针选择的敏感性讨论不足。

中文摘要： 音频深度伪造检测系统常因依赖特定攻击或录音条件相关的特征而难以跨域泛化。自监督语音模型提供丰富的多层表示，但现有方法要么只用单层、要么不加区分地融合所有层，且只能在训练后才揭示各层重要性。本文提出一种与模型无关的两阶段方法，在训练任何任务专用模型之前即可识别出信息量大的深度区间。该方法通过探测（probing）引导层选择，从而提升检测器对未知攻击和域偏移的泛化能力。总体而言，该工作为基于自监督语音模型的抗伪检测提供了一种可解释、可迁移的层选择范式。

Audio deepfake detection systems often fail to generalize across domains because they rely on features tied to specific attacks or recording conditions. Self-supervised speech models offer rich multi-layer representations, yet existing approaches either use a single layer or fuse all layers indiscriminately, and only reveal layer importance after training. We propose a model-agnostic, two-stage methodology that identifies informative depth zones before any task-specific model is trained. In the

7. MeloDISinger: Melody-Aware & Duration-Preserving Singing Voice Editing with Audio Infilling

Authors: Yoonjeong Park, Jaekwon Im, Juhan Nam

Categories: eess.AS, cs.SD | Accepted to Interspeech 2026 Score: 7.9/10 (Obj:9 Id:8 Ind:8 Comp:8 Eff:8 Nov:7)

  • Strength: 固定预算的时长比例预测器（MeloDRP）是一个优雅的洞察，完美解决了SVE中总时长严格保持的核心痛点；结合flow-matching的infilling机制有效保障了编辑边界的无缝衔接。
  • Weakness: 方法的新颖性更多体现在对已有技术（比例预测、flow-matching、cross-attention）的巧妙组合与适配，而非底层范式的突破；依赖伪MIDI提取可能在复杂混音场景下引入误差。

中文摘要： MeloDISinger针对基于文本的歌声编辑任务，旨在修改歌词的同时保留原始旋律、总时长及非编辑区域。该模型基于流匹配（flow-matching）框架构建，其核心模块MeloDRP通过预测固定预算的时长比例，实现对各编辑片段时长的显式控制。为使时长分配具备旋律感知能力，MeloDRP通过交叉注意力将语音线索与伪MIDI旋律信息相融合，从而让音素时长与旋律走势相协调。模型采用音频填充（audio infilling）方式生成编辑区域内容，在保证旋律一致性与时长保持的前提下完成高质量的歌词替换。该方法为歌声编辑提供了一个兼顾旋律感知与时长约束的统一解决方案。

Text-based singing voice editing (SVE) aims to revise sung lyrics while preserving the original melody, total duration, and non-edited regions. In this paper, we propose MeloDISinger, a flow-matching-based SVE model for melody-aware and duration-preserving editing. Its core module, MeloDRP, predicts fixed-budget duration ratios, enabling explicit span-wise duration control. For melody-aware duration allocation, MeloDRP fuses phonetic cues with pseudo-MIDI melodic context through cross-attention,

8. SICAGE: Speaker-Independent Culture-Aware Gesture Generation using TED4C-L Dataset

Authors: Ariel Gjaci, Antonio Sgorbissa, Vittorio Murino

Categories: cs.CV, cs.GR, cs.HC, cs.SD | Accepted at ECCV 2026 Score: 7.6/10 (Obj:8 Id:7 Ind:9 Comp:8 Eff:7 Nov:8)

  • Strength: Elegant framing of culture-aware gesture generation as a domain generalization problem (speaker-as-domain) to disentangle cultural patterns from speaker identity; rigorous speaker-disjoint evaluation protocol.
  • Weakness: The generative backbone (ALaDiT) is a relatively standard diffusion model, and the performance gains heavily rely on the newly introduced dataset rather than a fundamentally novel generative architecture.

中文摘要： 现有共语手势生成方法普遍忽视文化差异,且文化条件模型很少在说话人不相交划分下评估,导致观察到的”文化”行为可能混淆于说话人特有的手势风格。针对此问题,作者提出 SICAGE——一种模块化的文化感知共语手势生成框架,通过将运动合成模型条件化于与说话人无关的文化表征,实现跨文化的手势生成。为支持该研究,作者构建了 TED4C-L 数据集,以提供具有文化标注的说话人不相交训练与评估基准。SICAGE 在说话人独立的前提下显式建模文化因素,从而将文化风格与个体手势习惯解耦,使生成手势更贴合目标文化而不过拟合特定说话人。该工作为跨文化人机交互中可泛化的手势生成提供了新的评测范式与方法基础。

Recent co-speech gesture generation methods often overlook cultural differences, limiting their effectiveness in human-agent interaction. Moreover, culture-conditioned models are rarely evaluated under speaker-disjoint splits, so apparent “cultural” behavior may be confounded with speaker-specific gesturing style. We introduce SICAGE, a modular framework for culture-aware co-speech gesture generation that conditions motion synthesis models on speaker-independent cultural representations. SICAGE

9. Detecting Audio Deepfakes on the Edge:Lightweight SSL-Based Detection in a Browser Plugin

Authors: Octavian Pascu, Dan Oneata, Horia Cucu, Nicolas M. Muller

Categories: eess.AS, cs.AI Score: 7.6/10 (Obj:9 Id:7 Ind:8 Comp:8 Eff:8 Nov:6)

  • Strength: Clean and elegant insight that truncated shallow SSL layers outperform full models for deepfake detection while drastically improving efficiency; strong practical impact with on-device browser plugin ensuring privacy.
  • Weakness: Core methodology (truncated SSL + linear probe) lacks fundamental novelty as layer selection and lightweight probing are established techniques; scientific novelty relies heavily on the application context rather than a new paradigm.

中文摘要： 本文针对商业音频深伪检测依赖云端处理、存在隐私泄露风险的问题，提出一种可在浏览器插件中运行的端侧轻量检测模型。方法上，通过截断自监督学习骨干网络并结合简单逻辑分类头，在保持检测能力的同时大幅压缩模型体积与推理开销，实现完全本地化的实时推理。该方案使记者和事实核查人员无需上传音频即可验证来源真伪，兼顾了检测可靠性与隐私保护。研究表明，基于SSL的轻量架构在边缘环境下能在检测精度与运行效率之间取得有效平衡，为公众提供了一种实用且隐私友好的深伪检测工具。

Audio deepfakes are a growing challenge for the general public, as well as for journalists and fact-checkers. The latter need reliable tools to verify the authenticity of their sources, while at the same time keeping their information private. Commercial deepfake detection solutions rely on cloud-based processing, which raises privacy concerns. To solve this problem, we propose an on-device audio deepfake detection model. We show that a truncated self-supervised backbone with a simple logistic c

10. FacePlex: Full-Duplex Joint Speech-Facial Motion Generation for Conversational Avatars

Authors: Habin Lim, Jae-Ho Lee, Hah Min Lew, Ji-Su Kang, Gyeong-Moon Park

Categories: cs.AI, cs.CV, cs.LG | Project page: https://hahminlew.github.io/faceplex Score: 7.6/10 (Obj:8 Id:6 Ind:7 Comp:7 Eff:8 Nov:8)

  • Strength: 首次形式化全双工联合语音与面部动作生成，打破了传统级联管道的延迟与同步瓶颈，应用场景清晰且极具价值。
  • Weakness: 联合建模引入复杂的跨模态交互，导致方法有效性的因果归因较难清晰隔离，消融实验易受混淆因素干扰。

中文摘要： 该论文针对面对面对话中语音生成与面部动画需实时同步的难题，填补了现有系统要么只生成语音不生成表情、要么只能用已有音频驱动表情而不能在线联合生成的空白。作者首先将”全双工联合语音-面部运动生成”形式化为一个统一问题，并提出FacePlex框架，在统一架构中同时进行实时语音合成与面部运动生成。方法上通过联合建模实现语音与表情在时间上的精准对齐与协同生成，实现真正的双向在线对话能力。实验表明，FacePlex在语音质量、面部运动自然度与同步性方面均优于现有分离式方案，首次实现了端到端的全双工对话虚拟人系统。该工作为实时交互式虚拟形象提供了一套完整且可扩展的解决方案。

Natural face-to-face conversation requires real-time speech generation together with synchronized facial motion. Existing systems only partially address this problem: speech-only full-duplex models can generate speech in real time but do not produce facial motion, while audio-driven facial motion models animate a face from already available audio rather than jointly generating speech and motion online. To bridge this gap, we first formalize full-duplex joint speech-facial motion generation, wher

11. Predicting Timbre Traits for Interpretable Assessment of Musical Sound Synthesizers

Authors: Théo Chasle Cauchy, Modan Tailleur, Lindsey Reymore, Fanny Roche, Mathieu Lagrange

Categories: cs.SD Score: 6.9/10 (Obj:8 Id:6 Ind:7 Comp:8 Eff:7 Nov:6)

  • Strength: 通过简单的CLAP+MLP探测头优雅地实现了音频生成评估的可解释性，在提供与FAD等效排序的同时能定位具体音色维度的缺陷。
  • Weakness: 核心方法属于标准的预训练模型探测范式，且与人类评分的预测相关性（r=0.66）仅属中等，对嵌入空间为何能映射到特定音色特质的因果归因不足。

中文摘要： 针对神经音频合成器评估中常用指标FAD可解释性有限、只能排名而无法揭示具体质量缺陷的问题，本文提出了一种基于预训练音频神经嵌入（CLAP）与浅层可学习组件相结合的深度神经音色特征预测器。该浅层组件使用RWC乐器数据库以及人类对31种乐器20种音色描述（如木质、打击感、轰鸣感等）的评分进行训练。实验表明，所提模型与人类平均评分呈现强相关性（r=0.66, p<0.001）。将其应用于评估TokenSynth神经合成器时，预测器在不同条件模态下计算得到的MAE与以RWC为参考的FAD给出一致的排名，说明在分布层面二者能提供等效信息。此外，由于模型能够定性分析孤立声音，可以进一步定位需要改进的生成样本并识别需要调整的具体音色维度。让我撰写一段 3-5 句的中文摘要。</think>本文针对神经音频合成器评估中广泛使用的 Fréchet 音频距离（FAD）等基于分布的指标可解释性不足、仅能对方法进行排序的问题，提出了一种可解释的评估范式。作者构建了一个深度音色特征预测器，由预训练音频神经嵌入（CLAP）与一个浅层可学习组件组成，并在 RWC 乐器数据集上进行训练。该模型能够将音频映射到可解释的音色特征空间，从而为合成器质量提供超越单一排名的多维度、语义化评估。这一方法将主观的音色感知转化为可量化、可解释的特征预测，为音乐声音合成器的评估与比较提供了更具诊断力的问题定位手段。

Measuring neural audio synthesizers’ performance is now routinely conducted using distribution based metrics such as the Fréchet Audio Distance (FAD). Although this metric can be correlated with human perception, it offers limited interpretability beyond ranking different approaches. In this paper, we introduce a deep neural timbre trait predictor composed of a pretrained audio neural embedding (CLAP), and a shallow learnable component. The latter is trained using the RWC musical instrument data

12. OLIVE: View-Augmented Latent Prediction with Waveform Reconstruction for Speech SSL

Authors: Karl El Hajal, Mathew Magimai. -Doss

Categories: cs.CL, cs.LG, cs.SD, eess.AS Score: 6.7/10 (Obj:8 Id:7 Ind:8 Comp:6 Eff:7 Nov:5)

  • Strength: 有效结合了判别性（分析）与生成性（合成）目标，在生成和说话人任务上取得提升且未牺牲识别等判别性任务的竞争力。
  • Weakness: 核心方法属于已有范式（掩码蒸馏+波形重建）的直白组合，新颖性有限，且缺乏超越早晚期特征分工的更深层统一洞察。

中文摘要： 该论文针对语音自监督学习中表征鲁棒性与信号保真度的平衡问题，提出OLIVE框架，将视角增强的掩蔽潜在预测与波形重建统一在同一目标下联合优化。其核心机制是让重建约束编码器早期特征保留信号级信息，同时通过掩蔽潜在预测塑造后期上下文表征以获得不变性，从而兼顾分析与合成两类目标。方法上，OLIVE在线学习过程中引入不变视角增强，使模型在预测任务中自然习得对扰动的鲁棒性。实验表明，该统一框架在多个语音下游任务上取得了优于现有自监督方法的表征质量，验证了联合优化分析与合成目标的有效性。

We propose Online Latent prediction with Invariant Views and rEconstruction (OLIVE), a self-supervised speech representation learning framework that jointly optimizes analysis and synthesis objectives. OLIVE combines view-augmented masked latent prediction with waveform reconstruction under a unified objective. Reconstruction constrains early encoder features to retain signal-level information, while masked latent prediction shapes later contextual representations toward invariance for robust do

Authors: Ludovic Pirard, Katarina C. Poole

Categories: eess.AS | Submitted, accepted and presented at the AES 2026 International Conference on Audio for Virtual and Augmented Reality and Immersive Games Score: 6.7/10 (Obj:9 Id:8 Ind:9 Comp:7 Eff:6 Nov:5)

  • Strength: Single controlled study comparing 5 levels of HRTF individualization provides clear and independent perceptual evidence.
  • Weakness: Lacks methodological novelty as it is an evaluation study of existing approaches rather than proposing a new method.

中文摘要： 本研究针对虚拟现实中头相关传递函数（HRTFs）的个性化程度对空间听觉感知的影响，系统比较了五种个性化水平下的HRTFs性能。研究收集了19名听者在两项互补的VR声音定位实验中的数据，用于评估从通用HRTF到完全个性化HRTF等不同方案的表现。该方法通过对同一受试群体进行多实验交叉分析，克服了以往单一研究中难以横向比较不同HRTF方案的局限。主要结果显示，不同个性化水平的HRTFs在感知性能上存在显著差异，为合成HRTF等新兴方法的实用价值提供了实证依据。研究为VR/AR系统中HRTF方案的选择提供了统一的比较基准与量化参考。

Head-related transfer functions (HRTFs) underpin spatial hearing in virtual and augmented reality systems. Whilst individual HRTFs capture listener-specific morphology, their practical limitations have led to widespread use of generic HRTFs and growing interest in synthetic approaches. Yet their relative perceptual impact remains rarely compared within a single study. In this study, we analysed data from 19 listeners that completed two virtual reality sound localisation experiments with compleme

14. TRACE: Temporal Relationship-Aware Conversational Entrainment Detection in Dyadic Speech

Authors: Sathvik Manikantan Napa Ugandhar, Hao Zhang, Alison Gunzler, Yuzhe Wang, Thomas Thebaud et al.

Categories: cs.CL, cs.AI Score: 6.7/10 (Obj:7 Id:7 Ind:5 Comp:6 Eff:7 Nov:7)

  • Strength: Clearly defines an important new task (context/relationship-aware entrainment) and proposes a strong baseline (TRACE) that effectively models temporal interaction traces.
  • Weakness: The near-perfect 97% accuracy and reliance on synthetically disrupted negative samples raise concerns about the task’s real-world difficulty and potential evaluation loop dependency.

中文摘要： 随着语音AI智能体的普及，理解对话交互中的情感耦合（emotional entrainment）变得日益重要，而情感耦合受社会关系与对话语境的影响，并随时间动态变化。本文提出了DyadEE数据集，用于双人语音交互中的情感耦合检测，该数据集既包含真实的情感耦合对话，也包含通过替换对话伙伴而人为破坏耦合的合成交互样本。基于此数据集，作者提出了TRACE方法，通过建模时序关系与对话上下文来检测双人语音中的情感耦合状态。这项工作为语音AI在多轮对话中识别和适应人类情感协调行为提供了新的数据资源与方法框架。

With the proliferation of speech AI agents, understanding emotional entrainment in conversational interaction has become increasingly important. Emotional entrainment is shaped by social relationships and conversational context, influencing affective coordination over time. We introduce DyadEE, a dataset for emotional entrainment detection in dyadic speech interactions, containing both emotionally entrained conversations and synthetic interactions where entrainment is disrupted through partner s

15. BEST-RQ-2: Contextualize-Then-Predict, a Two-Step Approach for Self-Supervised Audio Representations

Authors: Ludovic K. Tuncay, Etienne Labbé, Thomas Pellegrini

Categories: cs.SD, cs.AI, cs.LG, eess.AS, eess.SP Score: 6.6/10 (Obj:8 Id:7 Ind:8 Comp:5 Eff:7 Nov:5)

  • Strength: 将MAE/data2vec风格的contextualize-then-predict范式与BEST-RQ的随机投影目标结合，提升了编码效率并丢弃了预测器。
  • Weakness: 核心思路直接借鉴已有视觉/语音自监督范式，新颖性有限，属于已有方法的变体与工程改进。

中文摘要： 本文针对自监督音频表征学习，提出 BEST-RQ-2 方法，在保留 BEST-RQ 冻结随机投影离散目标的基础上，引入”先上下文化再预测”的两步预训练范式。该方法用 ViT 上下文编码器仅处理未掩蔽的频谱区域，再由轻量级预测器推断掩蔽区域的目标，预训练完成后预测器即被丢弃。此举以 ViT 替换原有 Conformer 编码器，在保持表征迁移能力的同时简化了训练流程并提升了效率。该工作为跨域、跨任务的音频自监督学习提供了一种更简洁高效的架构选择。

Self-supervised learning enables audio representations that transfer across domains and tasks. We present BEST-RQ-2, an evolution of BEST-RQ that retains frozen randomprojection-based discrete targets while introducing a two-step contextualize-then-predict pretraining scheme. A ViT context encoder processes only the unmasked spectrogram regions, and a lightweight predictor infers targets for the masked regions; the predictor is discarded after pretraining. Replacing the original Conformer encode

16. SIGMA: Saliency-Guided Sparse Mask Attacks for Speech Emotion Recognition

Authors: Qiyang Sun, Yi Chang, Zixing Zhang, Björn W. Schuller

Categories: cs.SD | Under review Score: 6.4/10 (Obj:8 Id:7 Ind:8 Comp:7 Eff:6 Nov:5)

  • Strength: 将XAI显著图与稀疏攻击结合并统一稀疏性与幅度约束，思路干净且掩码可复用以摊销计算成本，在黑盒迁移上有一定优势。
  • Weakness: 核心思路属于CV领域显著图引导攻击在音频领域的迁移应用，新颖性有限；攻击成功率仅为’competitive’（打平或略优），在核心效果指标上未展现对已有方法的碾压优势。

中文摘要： SIGMA 针对语音情感识别（SER）在隐私敏感场景下的对抗攻击问题，指出已有稀疏攻击缺乏可解释性引导、稀疏性与幅值约束未统一建模，且跨攻击族与目标模型的迁移性有限。该方法在自监督语音特征的潜空间上，借助事后可解释 AI（XAI）技术生成显著性图并据此构造二值掩码，仅在最显著的特征元素上执行幅值受限的迭代扰动，掩码可一次计算并在多模型与多种稀疏攻击间复用以摊薄成本。在 IEMOCAP 与 TESS 两个基准上的实验表明，SIGMA 在匹配预算下扰动显著更少、生成时间更短，同时维持可比的攻击成功率，并在零查询迁移设置下提升了 Emotion2Vec 与 WavLM 等跨架构的迁移效果，表明显著性引导的掩码能够刻画可跨模型泛化的结构。

Speech conveys rich emotional information. As Speech Emotion Recognition (SER) is usually deployed in privacy-sensitive and reliability-critical environments, adversarial attacks on SER have attracted increasing attention. Existing sparse attacks control the number of perturbed elements, yet, they often lack explainability guidance and explicit measures of explanation consistency. A unified treatment of sparsity and magnitude constraints is also uncommon. In addition, transferability across atta

17. Rehearsed Multi-Agent Live Product Demonstrations with Real-Time Voice Question Answering

Authors: Rahul Khedar, Mayank Malhotra, Avinash Karn, Mouli V, Prakhar Mehrotra

Categories: cs.AI, cs.HC, cs.SE | Preprint. 4 figures, 1 algorithm, 5 tables. Systems paper with a preliminary six-session case study on four deployed applications; full benchmark protocol proposed, corpus run to appear in a later revision Score: 4.4/10 (Obj:8 Id:4 Ind:6 Comp:4 Eff:3 Nov:5)

  • Strength: 针对真实的软件产品演示痛点，提出了完整的排练-演示多智能体系统架构，并设计了动作与语音同步的实用机制。
  • Weakness: 过度工程化且缺乏与现有基线的公平比较；核心效果指标（如QA准确率、输出质量）在文中明确承认未测量，因果归因极弱。

中文摘要： 本文针对软件组织中高昂的实时产品演示成本问题——传统自动化方案要么仅面向指令驱动的任务完成（通用浏览器智能体），要么生成固定且无法交互、易因界面变更而失效的演示视频（MP4工具），提出了一种结合预演排练与实时语音问答的多智能体演示系统。该系统通过多个智能体协同完成功能选择、应用交互调度、连贯叙述及实时问题应答，将原本依赖人工主持的演示流程自动化。其核心在于将”预演”（rehearsed）的演示脚本与”实时”的语音问答能力相融合，既保证演示的连贯性又支持现场互动。该方法填补了现有自动化方案中演示与问答割裂的空白，使产品演示能够在界面漂移下保持鲁棒性并具备可交互能力。

Live product demonstrations are a recurring, high-cost activity in software organizations: a human presenter must select features, dispatch the corresponding interactions on a running application, narrate them coherently, and answer questions in real time. Existing automation addresses only fragments – generalist browser agents target instruction-conditioned task completion, and demo-video tools produce fixed MP4 artifacts that cannot be questioned and silently break under interface drift. We p