zylon-ai/private-gpt provides an API-first backend layer that follows the Claude API model to build applications on local models without rebuilding backend primitives or relying on cloud APIs. It requires a separately hosted OpenAI-compatible inference server and is designed for developers rather than as a finished chat or enterprise search application.
Project overview
zylon-ai/private-gpt provides an API-first backend layer that follows the Claude API model to build applications on local models without rebuilding backend primitives or relying on cloud APIs. It requires a separately hosted OpenAI-compatible inference server and is designed for developers rather than as a finished chat or enterprise search application.
Project type
MCP · AI Agent · RAG
Use cases
Knowledge Q&A
Deployment
Refer to project documentation
License
Apache-2.0
Best for
Developers and AI engineers who need a local backend API layer that follows the Claude API model to power internal applications, private delivery systems, and enterprise RAG solutions.
Key capabilities
Supports the ingestion of files and artifacts with retrieval augmented generation, including the provision of citations.
Provides support for Model Context Protocol (MCP) within the API and enables connections to remote MCP servers.
Features built-in structured querying for databases and CSV files without requiring exclusive dependence on external tools or code execution.
Allows users to configure and utilize custom tools within the system.
Limitations and risks
The platform does not run models itself; it requires a connection to a separate OpenAI-compatible inference server.
The provided user interface is a demonstrator, not the core product offering.
A separate running LLM server is required for the application to function.
Getting started
Setup involves installing PrivateGPT alongside a separate OpenAI-compatible LLM server such as Ollama. After installation, running 'private-gpt serve' and opening http://localhost:8080/ui allows initial interaction.
Evidence and sources
README: PrivateGPT is the open-source API layer that turns local models into production AI applications.
README: PrivateGPT ships a built-in workbench UI for testing and demos, available at `/ui`. The API is the actual product.
README: Production-tested: [PrivateGPT powers Zylon](#privategpt-vs-zylon), the on-premise AI platform providing Private AI to enterprises across the globe.
README: - Standard messages API (streaming, async, token counting) - File and artifact ingestion - Retrieval with citations and agentic RAG - Built-in tools mirroring the Claude API (web…
README: PrivateGPT follows the Claude API as the reference for modern AI application APIs.