Verified project record

Open-LLM-VTuber/Open-LLM-VTuber

A voice-interactive AI companion application that combines real-time speech, visual perception, and Live2D avatars. It supports local execution and extensive LLM, ASR, and TTS backend integrations, with data boundary and external inference dependent on the configured providers.

Project overview

It provides a customizable AI companion capable of voice interaction and Live2D avatar display, featuring a transparent desktop pet mode and the ability to orchestrate local model backends.

Project type: Model Runtime · Audio & Speech
Use cases: Chat Assistants
Deployment: Refer to project documentation
License: License pending

Best for

General users and developers seeking a customizable voice-interactive AI companion with visual perception that can be configured for local execution or remote backends.

Key capabilities

Features a Live2D avatar with expressions, emotion mapping, and touch feedback.
Supports an offline mode using local models, though users may also configure external or remote inference APIs.
Transparent background, global top-most, and mouse click-through features for the desktop client.
Integrates a rich variety of LLM inference, text-to-speech, and speech recognition solutions including Ollama, OpenAI, sherpa-onnx, Azure, etc.

Limitations and risks

Live2D sample models are not covered by the MIT license and may require additional licensing for commercial use.
Accessing the server remotely requires configuring HTTPS due to microphone secure context restrictions.
v1 is only receiving bug fixes while development focuses on v2.0.
v1.0.0 introduced breaking changes requiring re-deployment and configuration migration.

Getting started

Install dependencies via uv or Docker. Configure conf.yaml. Run the application server. Access via web client or desktop client. Setup may require separate installation of extra tools like uv, ffmpeg, or deeplx.

Alternatives and comparisons

An SDK to run LLMs, vision, speech-to-text, and text-to-speech locally across iOS, Android, Flutter, React Native, and Web.
A speech processing library providing fast and efficient text-to-speech, speech-to-text, and speech-to-speech capabilities optimized for Apple Silicon.
A distilled version of Whisper that is 6x faster, 49% smaller, and performs within 1% word error rate (WER) for English speech recognition.

Project comparisons

Open-LLM-VTuber/Open-LLM-VTuber vs Distil-Whisper

Evidence and sources

README: :warning: This project is in its early stages and is currently under **active development**.
README: **Open-LLM-VTuber** is a unique **voice-interactive AI companion** that not only supports **real-time voice conversations** and **visual perception** but also features a lively **…
README: offers two usage modes: web version and desktop client (with special support for **transparent background desktop pet mode**, allowing the AI companion to accompany you anywhere o…
README: 🖥️ **Cross-platform support**: Perfect compatibility with macOS, Linux, and Windows. We support NVIDIA and non-NVIDIA GPUs, with options to run on CPU or use cloud APIs for resour…
README: 🔒 **Offline mode support**: Run completely offline using local models - no internet required. Your conversations stay on your device, ensuring privacy and security.

Source updated at: 2026-07-17T18:16:28Z

Open original GitHub project