Voxtral Realtime
本条目由 AI 按收录标准筛选与撰写，经独立的 AI 核查在原文中逐条找到依据，未经人工审核；数字与结论以原文为准。
相关报告
收录理由
该报告为 arXiv 论文，阐述基于 Delayed Streams Modeling 的原生流式 ASR 架构、因果音频编码器与 Ada RMS-Norm 延迟条件、多语大规模预训练及多基准评测，并开源权重与 vLLM 流式推理集成。
关键要点
- Unlike approaches that adapt offline models through chunking or sliding windows, Voxtral Realtime is trained end-to-end for streaming, with explicit alignment between audio and text streams.
- 该模型为 4B 参数原生流式 ASR，支持 13 种语言；贡献包括从零训练的因果音频编码器与 Ada RMS-Norm，使单模型可在 80 ms 整数倍的目标延迟下工作。
- 预训练使用 spanning 13 languages 的大规模数据集。
- At a delay of 480 ms, Voxtral Realtime achieves performance competitive with Whisper and ElevenLabs Scribe v2 Realtime.
- Increasing the delay to 960 ms further closes the gap, with Voxtral Realtime surpassing past both Scribe v2 Realtime and Whisper.
- We release the resulting model as open weights under the Apache 2.0 license.