agentgateway is a proxy built on AI-native protocols (MCP and A2A) that provides security, observability, and governance for agent-to-LLM, agent-to-tool, and agent-to-agent communication. It routes traffic to major LLM providers through a unified API and can be self-hosted locally or deployed on Kubernetes.
Project overview
It provides a structured proxy layer for governing multi-agent communication, routing, and policy enforcement across any framework.
Project type
MCP · Infrastructure
Deployment
Refer to project documentation
License
Apache-2.0
Best for
Developers who need to secure, route, and monitor agent-to-LLM, agent-to-tool, or agent-to-agent communication.
Operations teams managing agent infrastructure on Kubernetes with custom Gateway API configurations.
Enterprise teams requiring fine-grained RBAC, telemetry, and guardrails for agentic workflows.
Key capabilities
Route traffic to major LLM providers through a unified OpenAI-compatible API with budget and spend controls, prompt enrichment, load balancing, and failover.
Connect LLMs to tools and external data sources via MCP with tool federation, stdio/HTTP/SSE/Streamable HTTP transports, OpenAPI integration, and OAuth authentication.
Enable secure agent-to-agent communication using A2A, with capability discovery, modality negotiation, and task collaboration.
Intelligent routing to self-hosted models using Kubernetes Inference Gateway extensions based on GPU utilization, KV cache, LoRA adapters, and queue depth.
Multi-layered content filtering with regex, OpenAI moderation, AWS Bedrock Guardrails, Google Model Armor, and custom webhooks.
Auth, fine-grained RBAC with CEL policy engine, rate limiting, TLS, and OpenTelemetry metrics/logs/tracing.
A built-in UI to explore agentgateway connecting agent-to-agent or agent-to-tool.
Deploy on Kubernetes using the built-in controller and Gateway API support.
Limitations and risks
The project is currently in active development.
The repository lacks explicit setup steps and provides links to external quickstart documentation.
GPU requirements, cost dependencies, data boundary guarantees, and coding requirements are not documented.
Getting started
Setup requires consulting external quickstart documentation, as explicit steps are not provided in the repository.
Use the built-in UI to explore the proxy connecting agents to other agents or tools after installation.
Alternatives and comparisons
Provides a lightweight, unified Python library with a standardized Chat Completions API for multiple LLM providers rather than a standalone proxy server.
An open-source AI engine running multimodal models behind one API, keeping data in the user's infrastructure instead of acting as an agent proxy.
Turns agent skills declared in standard formats into a POST /run HTTP endpoint that works with any LLM and is self-hostable.