A desktop application and local API server for running large language models privately on your own hardware. It supports text generation, vision input, tool calling, image generation, and exposes OpenAI/Anthropic-compatible API endpoints without telemetry.
Project overview
It provides local execution with zero telemetry and serves as a drop-in local replacement for OpenAI and Anthropic APIs.
Project type
Model Runtime
Deployment
Refer to project documentation
License
AGPL-3.0
Best for
Developers and general users who want to run large language models on their own hardware without relying on external APIs or cloud services.
Users who require local execution with zero telemetry during operation.
Key capabilities
Run text generation models locally with instruct and chat modes, supporting branching and notebook interfaces.
Attach images to messages for visual understanding during chat.
Models can call custom functions during chat, including web search, page fetching, and math, defined in single .py files. Supports MCP servers.
Serves Chat, Completions, and Messages endpoints locally as a drop-in replacement for OpenAI/Anthropic APIs.
Fine-tune LoRAs on multi-turn chat or raw text datasets.
A dedicated tab for diffusers models like Z-Image-Turbo with quantization and gallery features.
Supports built-in and community extensions including TTS, voice input, and translation.
Limitations and risks
Some model types require the full installation instead of the portable build.
Full installation requires approximately 10GB of disk space and downloads PyTorch.
Getting started
Download, unzip, and double-click the textgen executable to open the application window.