VideoLingo is an automated pipeline for translating and dubbing video content, producing single-line subtitles and voiced audio tracks. It requires configuring external or local models for transcription, LLM processing, and text-to-speech.
Project overview
The project combines a 3-step translation process (Translate-Reflect-Adaptation), terminology management, and single-line subtitle segmentation to address stiff machine translation and multi-line subtitle issues.
Project type
Workflow · Video · Audio & Speech
Use cases
Translation & Subtitles · Video Creation · Audio & Speech
Deployment
Refer to project documentation
License
Apache-2.0
Best for
Creators and general users needing automated single-line subtitle generation and dubbing for videos.
Key capabilities
Downloads video and provides NLP and AI-powered subtitle segmentation with a 3-step Translate-Reflect-Adaptation process.
Generates and applies custom terminology to help ensure translation coherence.
Limitations and risks
WhisperX transcription performance may be affected by video background noise.
Using weaker models can lead to errors during processes due to strict JSON format requirements.
The dubbing feature may not be completely accurate due to differences in speech rates and intonation between languages.
Multilingual video transcription recognition will only retain the main language.
Cannot dub multiple characters separately as WhisperX's speaker distinction capability is not sufficiently reliable.
There is a risk of repeated execution errors if the output folder is not deleted after an error occurs with weaker models.
Getting started
Clone the repository, run python setup_env.py, configure APIs or local models in settings, and run streamlit run st.py.
Alternatives and comparisons
Provides a one-click workflow for video translation and audio transcription, supporting local offline deployment and speaker diarization.
Offers an easy one-stop tool for video subtitling, including speech recognition, subtitle optimization, translation, and video synthesis.
Framework for automating video and short content creation, simplifying footage sourcing, voiceover synthesis, and editing tasks.
README: VideoLingo is an all-in-one video translation, localization, and dubbing tool aimed at generating Netflix-quality subtitles. It eliminates stiff machine translations and multi-lin…
README: - 🎥 YouTube video download via yt-dlp
README: FFmpeg is required.
README: 3. Start the application ```bash .venv\Scripts\streamlit run st.py # Windows .venv/bin/streamlit run st.py # macOS / Linux ``` Or double-click `OneKeyStart.bat` on Windows.