Verified project record
QwenAudio/CosyVoice
CosyVoice is a self-hosted multilingual text-to-speech model that provides zero-shot voice cloning, cross-lingual generation, and bi-streaming inference. It requires local GPU configuration and manual dependency management, supporting developers who need customizable speech synthesis.
Project overview
The project supports bi-streaming text-in and audio-out inference with latency documented as low as 150ms across 9 common languages and 18+ Chinese dialects.
- Project type
- Model Runtime · Audio & Speech
- Use cases
- Audio & Speech
- Deployment
- Refer to project documentation
- License
- Apache-2.0
Best for
- Developers, AI engineers, and researchers who require self-hosted deployment and have the capacity to manage Python environments and GPU configurations.
Key capabilities
- Supports zero-shot voice cloning across multiple languages and dialects without needing a traditional frontend module.
- Covers 9 common languages (such as Chinese, English, and Japanese) and 18+ Chinese dialects/accents.
- Supports instructions for languages, dialects, emotions, speed, and volume.
- Supports pronunciation inpainting of Chinese Pinyin and English CMU phonemes for more controllability.
- Supports text-in streaming and audio-out streaming with latency as low as 150ms.
- Supports reading of numbers, special symbols, and various text formats without a traditional frontend module.
- Provides deployment scripts for gRPC and FastAPI servers via Docker.
- Supports vLLM 0.11.x+ and 0.9.0 for accelerated inference.
Limitations and risks
- Specific vLLM versions have strict requirements and may corrupt existing environments if hardware is unsupported.
- GPU and minimum hardware requirements are not documented, requiring adopters to independently verify hardware compatibility.
Getting started
- Setup is rated hard because it requires Python environment configuration, GPU setup, manual model downloads, and navigation across deployment platforms.
- Users clone the repository, create and activate a Conda environment, install system dependencies (sox, libsox-dev), install Python requirements, download pretrained models using ModelScope or HuggingFace SDK, and run python example.py or python3 webui.py.
Alternatives and comparisons
- Provides a deep learning toolkit for text-to-speech generation with voice cloning capabilities and support for over 1100 languages.
- Provides a text-to-speech model optimized for dialogue scenarios with fine-grained prosodic control and multi-speaker support.
- A tokenizer-free text-to-speech model supporting 30 languages, 48kHz audio output, and highly accurate voice cloning.
Project comparisons
Evidence and sources
- GitHub project description: Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.
- GitHub repository metadata: Python
- README: **Fun-CosyVoice 3.0** is an advanced text-to-speech (TTS) system based on large language models (LLM), surpassing its predecessor (CosyVoice 2.0) in content consistency, speaker s…
- README: - **Language Coverage**: Covers 9 common languages (Chinese, English, Japanese, Korean, German, Spanish, French, Italian, Russian), 18+ Chinese dialects/accents (Guangdong, Minnan…
- README: - [x] 2025/12 - [x] release Fun-CosyVoice3-0.5B-2512 base model, rl model and its training/inference script
AI Search
Find projects, verify facts, compare options, or turn a complex need into an actionable plan
Try a searchA click only fills the search box; you stay in control
Project Details
0