LocalAI is an open-source AI engine that runs text, audio, image, and video models behind a single API without requiring a GPU. It is designed to keep data within the user's own infrastructure.
Project overview
The project uses a composable core that downloads backend images only when needed, enabling multimodal model serving on hardware without GPUs.
Developers and AI engineers needing to run multimodal AI models on their own infrastructure without a dedicated GPU.
Key capabilities
Executes LLM, vision, voice, image, and video models through a single API.
Offers drop-in compatibility with OpenAI, Anthropic, and ElevenLabs APIs across backends.
Provides autonomous agents with tool use, RAG, skills, and SSE streaming.
Includes an integrated WebUI for operating the application.
Installs and removes backends on the fly through OCI images.
Provides horizontal scaling with PostgreSQL and NATS.
Limitations and risks
The macOS DMG is not signed by Apple.
macOS users may need to manually remove the application quarantine attribute after installing the unsigned DMG.
Getting started
Run the documented Docker container command, load a model using the local-ai run command, and start an interactive chat with the local-ai chat command.
Evidence and sources
GitHub project description: LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
README: - **Any model, any modality**: LLMs, vision, voice, image, and video behind one API
README: - **Privacy-first**: your data never leaves your infrastructure
README: - **Drop-in API compatibility**: OpenAI, Anthropic, and ElevenLabs APIs across every backend
README: - [Built-in Agents](https://localai.io/features/agents/) — Autonomous AI agents with tool use, RAG, skills, SSE streaming, and [Agent Hub](https://agenthub.localai.io)