A self-hostable AI gateway that aggregates 250+ external AI providers into a single OpenAI-compatible endpoint with automatic failover and token compression. It routes requests to remote third-party models and depends on their availability, terms of service, and API keys.
Project overview
It consolidates over 250 AI providers with 18 routing strategies and stacked token compression claiming 15-95% savings, differentiating it from typical routers that offer fewer providers, strategies, or compression features.
Project type
AI Agent · AI Coding · Infrastructure
Use cases
Coding & Development
Deployment
Refer to project documentation
License
MIT
Best for
Developers and AI engineers who need to manage multiple external AI providers and API keys for coding or prompt engineering tasks.
Teams relying on coding agents that require an OpenAI-compatible endpoint with built-in routing and token compression.
Key capabilities
Routes requests to over 250 AI providers using 18 different strategies (e.g., priority, cost-optimized, round-robin) to optimize for cost, latency, or headroom.
Automatically falls back across providers in milliseconds when quota limits are hit or providers fail.
Compresses tool outputs and context payloads by 15-95% to save tokens using stacked RTK and Caveman engines.
Includes a built-in MCP server with 94 tools, 3 transports, and 30 scopes.
Supports A2A (Agent-to-Agent) protocol with 6 skills using JSON-RPC 2.0.
Distributes a provider's time-based quota fairly across multiple keys in a pool, with options for weights, dimensions, policies, and caps.
Uses JA3/JA4 TLS fingerprint spoofing via wreq-js to access AI services from anywhere and bypass blocks.
Translates OpenAI, Claude, Gemini, and Responses API formats so any tool can be pointed at a single endpoint.
Limitations and risks
Reliance on free tiers means susceptibility to provider rate limits and quota exhaustion, necessitating fallback mechanisms.
Depends entirely on the availability and terms of service of third-party AI providers, which can change.
Getting started
Install globally via npm, run the application, and open the local dashboard at http://localhost:20128 to begin.
Alternatives and comparisons
Spins up a complete local LLM stack with backends and frontends pre-wired to work together using a single command.
Deploys, runs, and manages MCP servers locally or in a Kubernetes cluster within isolated containers with security guardrails.