This project provides a workflow for translating videos and transcribing audio, supporting both local offline models and online APIs. It handles speech recognition, subtitle translation, speech synthesis, and video synthesis in a multi-stage pipeline.
Project overview
It supports local orchestration using local ASR models alongside online API options, and provides multiple interface modes including a Windows desktop GUI, CLI, and WebUI.
Creators, general users, or developers who need a multi-stage video and audio translation pipeline with options for local models or external APIs.
Users who require interactive proofreading at the recognition, translation, or dubbing stages of the workflow.
Key capabilities
Executes a one-click workflow: Speech Recognition (ASR), followed by Subtitle Translation, Speech Synthesis (TTS), and Video Synthesis.
Batch converts audio or video files to SRT subtitles and supports Speaker Diarization to distinguish between different roles.
Assigns different AI dubbing voices to different identified speakers.
Integrates models such as F5-TTS, CosyVoice, and GPT-SoVITS for zero-shot voice cloning.
Supports pausing and manual proofreading at the recognition, translation, and dubbing stages.
Includes auxiliary tools such as vocal separation, video and subtitle merging, audio-video alignment, and transcript matching.
Supports headless operation for server deployment or batch processing tasks.
Provides a browser-based interface for remote access or internal network deployment.
Limitations and risks
The pre-packaged version must be extracted to a path that contains no Chinese characters or spaces.
Getting started
Download the latest pre-packaged version, extract the compressed file to a path without Chinese characters or spaces, and double-click sp.exe. This path requires no Python environment configuration.
Alternatives and comparisons
Provides an automated pipeline to generate single-line subtitles and dubbing for videos, focusing on cutting, translating, aligning, and voicing content.
Provides a one-stop tool for video subtitling, including speech recognition, subtitle optimization, translation, and video synthesis.
Framework for automating content creation that simplifies video creation, footage sourcing, voiceover synthesis, and editing tasks.
GitHub project description: Translate the video from one language to another and embed dubbing & subtitles.
Release: win-pyvideotrans-v4.05
README: pyVideoTrans is dedicated to seamlessly converting videos from one language to another, offering a complete workflow that includes speech recognition, subtitle translation, multi-…
README: - **🎙️ Audio Transcription / Subtitle Generation**: Batch convert audio/video to SRT subtitles, supporting **Speaker Diarization** to distinguish between different roles.
README: We provide a pre-packaged `.exe` version for Windows 10/11 users, requiring no Python environment configuration.