An LLM gateway and AI asset management system that unifies access, authentication, and billing across multiple large language model APIs. It supports cross-conversion between OpenAI, Claude, and Gemini formats and requires authorized upstream API keys and endpoints for operation.
Project overview
Provides a unified gateway for cross-converting multiple LLM API formats while managing internal quota allocation, user permissions, and usage accounting.
Project type
Infrastructure
Deployment
Refer to project documentation
License
AGPL-3.0
Best for
Enterprise teams and developers who need to manage access, format cross-conversion, and billing across disparate large language model APIs from a private deployment.
Teams requiring user-level model rate limiting, token grouping, and a visual statistical analysis console for model usage.
Key capabilities
Converts requests between OpenAI Compatible, Claude Messages, and Google Gemini formats.
Supports channel weighted random distribution, automatic retry on failure, and user-level model rate limiting.
Provides internal top-up, quota allocation, per-request usage, and cache-hit cost accounting with flexible billing policies.
Allows configuration of token grouping, model restrictions, and user management.
Provides a visual console and statistical analysis tools for model usage.
Limitations and risks
32-bit systems are not supported. A 64-bit system architecture (amd64 or arm64) is required.
If using external databases, MySQL must be version 5.7.8 or higher, or PostgreSQL must be version 9.6 or higher.
Google Gemini to OpenAI Compatible conversion currently supports text only; function calling is not supported.
Operating as a public generative AI service or API resale service requires users to fulfill all regulatory, licensing, and tax obligations.
Getting started
Setup is rated as easy. The recommended path is to clone the project, edit the docker-compose.yml configuration, start the service, and visit http://localhost:3000 to access the web GUI.
Alternatives and comparisons
Unifies access to multiple LLM providers under a standard OpenAI API format for key management and secondary redistribution. This system is fully compatible with the original One API database.
Provides an open source AI gateway for 100+ LLMs with a unified interface in OpenAI format, cost tracking, load balancing, and an admin dashboard.
Serves as a high-throughput and memory-efficient inference and serving engine for large language models.
GitHub project description: A unified AI model hub for aggregation & distribution. It supports cross-converting various LLMs into OpenAI-compatible, Claude-compatible, or Gemini-compatible formats. A central…
Release: v1.0.0-rc.21
README: This project is intended solely for lawful and authorized AI API gateway, organization-level authentication, multi-model management, usage analytics, cost accounting, and private…
README: Next-Generation LLM Gateway and AI Asset Management System