Verified project record

babysor/MockingBird

A voice cloning and speech synthesis toolkit supporting Mandarin Chinese and English, capable of generating speech from a 5-second audio sample. It offers self-hosted execution and training workflows but requires advanced technical knowledge to set up.

Project overview

Enables real-time multilingual voice cloning with Mandarin Chinese support, a capability differentiated from the original project fork.

Project type: Model Runtime · Audio & Speech
Use cases: Audio & Speech
Deployment: Refer to project documentation
License: License pending

Best for

Developers and AI engineers who need to clone voices and synthesize speech locally or via a web server, particularly requiring Mandarin Chinese support.

Key capabilities

Clones a voice from a 5-second sample to generate arbitrary speech in real-time, supporting Mandarin Chinese and English.
Allows users to use or easily fine-tune pre-trained encoder and vocoder models for immediate results.
Includes scripts to preprocess datasets and train encoder, synthesizer, and vocoder components.
Provides a built-in web server to serve synthesis results via remote calling.
Includes a desktop GUI toolbox (demo_toolbox.py) for testing and interacting with the models.
Offers a command-line script to generate voice from text and wav files directly.

Limitations and risks

demo_cli is not working because the original pretrained synthesizer model is incompatible with Chinese symbols.
PyQt5 in demo_toolbox.py is not compatible with M1 chips without workarounds.
Training requires significant VRAM and may require manual batch size adjustment.
Training may require changing virtual memory to 100G.

Getting started

Setup is rated as hard. Users must manually install Python dependencies, PyTorch, and ffmpeg, run pip install -r requirements.txt, prepare or download synthesizer models, and then run python web.py or python demo_toolbox.py.

Alternatives and comparisons

Provides a toolbox and CLI implementation of the SV2TTS deep learning framework with a real-time vocoder for synthesizing cloned speech from arbitrary text.
A tokenizer-free text-to-speech model supporting 30 languages, 48kHz audio output, voice design from text descriptions, and highly accurate voice cloning.
A complete voice I/O stack running locally that bridges voice cloning, speech generation, and global dictation with agent voice output.

Project comparisons

Evidence and sources

GitHub project description: 🚀Clone a voice in 5 seconds to generate arbitrary speech in real-time
README: 🌍 **Chinese** supported mandarin and tested with multiple datasets: aidatatang_200zh, magicdata, aishell3, data_aishell, and etc. 🤩 **PyTorch** worked for pytorch, tested in versi…
README: 🚧 While I no longer actively update this repo, you can find me continuously pushing this tech forward to good side and open-source. I'm also building an optimized and cloud hosted…
README: Note that we are using the pretrained encoder/vocoder but not synthesizer, since the original model is incompatible with the Chinese symbols. It means the demo_cli is not working…
README: Train the synthesizer: `python train.py --type=synth mandarin <datasets_root>/SV2TTS/synthesizer`

Source updated at: 2026-07-16T18:06:14Z

Open original GitHub project