Abstract
In this paper, we present Qwen-Audio-3.1-TTS, a production-oriented speech synthesis system that jointly advances content consistency, speaker similarity, prosodic naturalness, audio quality, controllability, multilingual coverage, efficiency, and robustness. It combines a 12.5 Hz low-frame-rate speech tokenizer for reduced inference latency with a five-stage progressive training paradigm for coordinated LM and FM optimization. The model provides production-level control through free-style natural-language instructions and fine-grained inline tags, while supporting 16 languages, 20 Chinese dialect regions, one-pass long-form synthesis up to 3 minutes, and robust generation from noisy, reverberant, or unclear reference speech. Across SEED-TTS-Eval, CV3-Eval, instruction-following, long-form, and acoustic-robustness evaluations, Qwen-Audio-3.1-TTS achieves state-of-the-art performance on many reported dimensions or the strongest aggregate results. It also ranks first on the independent Artificial Analysis Text-to-Speech Leaderboard. These results establish Qwen-Audio-3.1-TTS as a strong foundation for production-level speech synthesis.
Key Contributions
- Low-frame-rate speech tokenizer: A 12.5 Hz supervised speech tokenizer reduces autoregressive decoding cost while retaining content and speaker information.
- Progressive training paradigm: The training pipeline combines independent LM and FM pretraining, joint training with high-quality data annealing, LM reinforcement learning, FM robustness training, and FM reinforcement learning to improve content consistency, prosodic naturalness, voice fidelity, perceptual quality, and robustness.
- Production-grade controllability: The model interprets free-style natural-language instructions describing role, emotion, speaking style, rate, timbre, and accent. In parallel, 86 newly added fine-grained inline tags enable localized control at phrase and word level, including expressive transitions and non-verbal events such as laughter, breathing, coughing, and sighing.
- Broad and robust deployment coverage: The model supports 16 languages, seven of them newly added, and 20 Chinese dialect regions; it handles hard text-normalization cases, one-pass synthesis up to 3 minutes, and degraded prompts without an explicit denoising mode. A reproducible speaker fine-tuning protocol and vocoder super-resolution further support target-voice adaptation and 48 kHz output.
- Comprehensive evaluation: We evaluate zero-shot voice cloning, multilingual and cross-lingual synthesis, free-style instruction following, fine-grained control, text normalization, long-form generation, adverse-prompt robustness, and dialect synthesis through objective benchmarks and arena-style human evaluation.
Listen, compare, and explore.
Select any sample to play. Audio loads on demand.