Audio & Speech Open-Source Projects

Showing 35 indexable projects from the current snapshot.

• Zackriya-Solutions/meetily

  Meetily is a desktop application that captures, transcribes, and summarizes meeting audio entirely on your local machine. It provides AI-generated meeting notes without sending sensitive audio data to third-party cloud servers.

• openai/whisper

  Whisper is a general-purpose speech recognition model that performs multilingual transcription, speech-to-English translation, and language identification. It replaces many stages of a traditional speech-processing pipeline using a single…

• Comfy-Org/ComfyUI

  A modular node graph interface and backend for designing generative AI workflows across images, videos, audio, and 3D models without writing code. It operates fully offline with smart memory management that supports GPUs with as low as 1GB…

• huggingface/diffusers

  Diffusers provides modular inference pipelines, interchangeable schedulers, and pretrained building-block models for diffusion-based image, audio, and video generation in PyTorch. It is intended for developers and researchers who need to r…

• RVC-Boss/GPT-SoVITS

  GPT-SoVITS is a self-hosted application for zero-shot and few-shot voice cloning and text-to-speech. It produces trained models using as little as one minute of voice data and supports cross-lingual inference in Chinese, English, Japanese,…

• 2noise/ChatTTS

  ChatTTS is a text-to-speech model optimized for dialogue scenarios, providing fine-grained prosodic control and multi-speaker support. It is released for academic purposes only under a CC BY-NC 4.0 license and supports English and Chinese.

• CorentinJ/Real-Time-Voice-Cloning

  A local SV2TTS toolbox and CLI for cloning a voice from a short audio sample and synthesizing arbitrary text in real time. Adopt it for local experiments or workflows that accept an older voice-cloning approach; do not select it when curre…

• ggml-org/whisper.cpp

  ggml-org/whisper.cpp is a plain C/C++ inference library for running OpenAI Whisper automatic speech recognition across supported platforms. It provides a dependency-free implementation with zero runtime memory allocations, voice activity d…

• jamiepine/voicebox

  VoiceBox is a local-first, open-source voice I/O application that combines voice cloning, multi-engine text-to-speech, global dictation, and AI agent output. It runs entirely on your machine, ensuring voice data and captures are never tran…

• coqui-ai/TTS

  A deep learning toolkit for Text-to-Speech generation that provides pretrained models in over 1100 languages, voice cloning, and model training tools. It is intended for developers, AI engineers, and researchers building speech synthesis a…

• myshell-ai/OpenVoice

  OpenVoice clones a reference voice's tone color and generates speech with controllable emotion, accent, and rhythm across multiple languages. It processes reference speech inputs to produce cross-lingual generated audio.

• OpenBMB/VoxCPM

  VoxCPM is a tokenizer-free text-to-speech model that outputs 48kHz audio, supports 30 languages, and creates or clones voices from text descriptions or short reference clips. It is suitable for developers and creators who need self-hosted,…

• fishaudio/fish-speech

  Fish Speech is a multilingual text-to-speech and voice cloning model that supports over 80 languages and multi-speaker generation. It accepts text and short reference audio to generate speech audio without requiring phoneme preprocessing.

• SYSTRAN/faster-whisper

  Faster Whisper re-implements OpenAI Whisper inference on CTranslate2 to deliver up to 4x faster transcription with lower memory usage on CPU and CUDA GPUs. It supports 8-bit quantization, batched inference, voice activity detection, and wo…

• m-bain/whisperX

  WhisperX adds batched inference, word-level timestamps via wav2vec2 alignment, and pyannote-audio speaker diarization to Whisper large-v2, reaching up to 70x realtime transcription. It is designed for developers and researchers who need pr…

• QwenAudio/CosyVoice

  A multilingual text-to-speech model providing zero-shot voice cloning, cross-lingual generation, and bi-streaming inference with latency as low as 150ms. It covers 9 languages and 18+ Chinese dialects, and requires manual environment confi…

• index-tts/index-tts

  A controllable zero-shot text-to-speech system that separates speaker identity from emotional expression. It requires local execution with an NVIDIA GPU and CUDA Toolkit 12.8 or newer.

• DrewThomasson/ebook2audiobook

  This tool converts e-book files into audiobooks using text-to-speech technology across 1158 languages and dialects. It supports multiple TTS engines, optional voice cloning, and can run on low-resource hardware with a minimum of 2 GB RAM.

• modelscope/FunASR

  FunASR is an industrial-grade speech recognition toolkit for offline, streaming, and edge deployment of ASR, VAD, punctuation, speaker diarization, and emotion recognition tasks. It provides transcription across multiple languages and is d…

• nari-labs/dia

  Dia is a self-hosted text-to-audio dialogue model designed for research and educational use, generating multi-speaker conversations with non-verbal sounds and voice cloning from audio prompts. A CUDA-capable GPU is required, as CPU inferen…

• jianchang512/pyvideotrans

  A one-click video translation and audio transcription tool that handles speech recognition, subtitle translation, multi-role AI dubbing, and video synthesis. It supports both local offline deployment and various online APIs.

• NVIDIA-NeMo/Speech

  A PyTorch-based generative AI framework for researchers and developers to build, customize, and deploy Automatic Speech Recognition, Text-to-Speech, and Speech Large Language Models. The project requires a pre-configured Python environment…

• WEIFENG2333/VideoCaptioner

  A desktop GUI and CLI application that consolidates video subtitling tasks—transcription, optimization, translation, synthesis, and dubbing—into a single pipeline. It includes no-configuration free options alongside LLM-based processing fo…

• kaldi-asr/kaldi

  An open-source toolkit for building speech recognition systems, written in C++ with example builds for developers and researchers. It runs on UNIX-like platforms with cross-compilation support for Android and WebAssembly, though it require…

• alphacep/vosk-api

  alphacep/vosk-api is a self-hosted, offline speech recognition library designed for developers who need continuous large vocabulary transcription across 20+ languages without relying on cloud services. It provides compact 50 MB models, zer…

• waooAI/waoowaoo

  An AI video-production application that automatically converts novel text into storyboards, character and scene images, voices, and complete videos. It is self-hosted via Docker Compose and requires an external AI service API key.

• Open-LLM-VTuber/Open-LLM-VTuber

  A cross-platform, voice-interactive AI companion application featuring Live2D avatars, visual perception, and real-time voice conversations. It supports extensive LLM, ASR, and TTS backends and can run completely offline using local models.

• PaddlePaddle/PaddleSpeech

  PaddleSpeech is an open-source speech and audio toolkit covering ASR, TTS, voice cloning, and audio classification. It runs locally via Python CLI and server components, and requires the PaddlePaddle framework as an external dependency.

• speechbrain/speechbrain

  SpeechBrain is an open-source PyTorch framework designed to accelerate conversational AI development. It provides over 200 training recipes and 100+ pretrained models for building and evaluating speech, text, and EEG processing systems.

• rany2/edge-tts

  rany2/edge-tts is a Python library and command-line tool that converts text to speech using Microsoft Edge's online TTS service without requiring an API key, the Edge browser, or Windows. It outputs audio files and subtitle files from text…

• livekit/agents

  livekit/agents is a framework for creating conversational, multi-modal voice agents that run as realtime programmable participants on servers. It provides integrated job scheduling and a selection of STT, LLM, TTS, and Realtime API integra…

• ostris/ai-toolkit

  ai-toolkit is a free, open-source training suite for fine-tuning diffusion models on image, video, and audio generation tasks. It is designed to run on consumer-grade hardware via a web interface or command line.

• abus-aikorea/voice-pro

  A Gradio web application that combines speech recognition, multilingual translation, dubbing, text-to-speech, voice cloning, audio processing, video download, and vocal isolation for creators and developers.

• TEN-framework/ten-framework

  ten-framework is an open-source framework for developing and deploying real-time multimodal conversational AI agents. It provides extensive examples and deployment options while addressing the need to integrate various models and services…

• Baiyuetribe/paper2gui

  Paper2GUI packages more than 50 AI models into install-free desktop GUI tools for image, video, and audio tasks. It targets non-technical users, but operating-system support varies by tool and some TTS functions may rely on third-party ser…