Verified project record

FunAudioLLM/CosyVoice

CosyVoice is a multilingual zero-shot text-to-speech system covering 9 languages and 18+ Chinese dialects. It provides voice cloning, instruction-based control, and bi-streaming with latency as low as 150ms.

Project overview

Project type: Model Runtime · Audio & Speech
Use cases: Audio & Speech
Deployment: Refer to project documentation
License: Apache-2.0

Best for

AI engineers and developers needing multilingual zero-shot TTS with dialect coverage and instruction control.
Advanced users requiring training and inference scripts for speech synthesis.

Key capabilities

Provides zero-shot speech synthesis covering 9 common languages and 18+ Chinese dialects/accents.
Supports both multi-lingual and cross-lingual zero-shot voice cloning.
Supports pronunciation inpainting of Chinese Pinyin and English CMU phonemes for more controllability.
Supports both text-in streaming and audio-out streaming with latency as low as 150ms.
Supports various instructions such as languages, dialects, emotions, speed, and volume.
Provides advanced users with training and inference scripts.
Supports reading of numbers, special symbols and various text formats without a traditional frontend module.

Limitations and risks

vLLM versions between 0.10.x are not tested, and versions older than 0.9.0 do not support CosyVoice inference.
Content provided is for academic purposes only.
Requires Repetition Aware Sampling (RAS) inference to address LLM stability issues.

Getting started

Clone the repository, install the Conda environment, download pretrained models, and run example.py or webui.py to start.

Evidence and sources

GitHub project description: Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.
README: **Fun-CosyVoice 3.0** is an advanced text-to-speech (TTS) system based on large language models (LLM), surpassing its predecessor (CosyVoice 2.0) in content consistency, speaker s…
README: Language Coverage**: Covers 9 common languages (Chinese, English, Japanese, Korean, German, Spanish, French, Italian, Russian), 18+ Chinese dialects/accents (Guangdong, Minnan, Si…
README: Pronunciation Inpainting**: Supports pronunciation inpainting of Chinese Pinyin and English CMU phonemes, providing more controllability and thus suitable for production use.
README: Bi-Streaming**: Support both text-in streaming and audio-out streaming, and achieves latency as low as 150ms while maintaining high-quality audio output.

Reviewed at: 2026-07-17T11:32:24Z