Architectures
Overview
The AI Gateway system is designed with a two-plane architecture consisting of a Management Plane and a Traffic Plane. The two planes are intentionally decoupled and connected only through native Kubernetes configuration mechanisms, enabling centralized governance with reliable runtime enforcement.
Management Plane
Operators manage the system through a Management Panel, covering the following functional areas:
- Gateway: gateway lifecycle and related components
- Backend: definition and management of destinations (LLMs, MCP, APIs)
- Route/API: routing rules, endpoints, and API mappings
- Policy: access control and content governance policies
- Secret: credentials and keys management
- Gateway Config: operational gateway settings
- Topology/Monitoring: system visibility and monitoring views
All UI actions are materialized as Kubernetes CRDs. The agentgateway continuously watches these CRDs and reconciles runtime behavior accordingly. This is the single, controlled integration point between management and production traffic.
Traffic Plane (Runtime Flow)
Requests flow left → right:
Input Traffic (AI Clients)The gateway accepts requests from common AI clients such as Claude Desktop, Cursor, GitHub Copilot, as well as custom applications/agents.
agentgateway (Rust Proxy)All requests pass through agentgateway, which enforces and optimizes traffic before forwarding:
- Auth / AuthZ
- Rate Limiting
- Prompt Guard
- Routing
- Caching
- Observability
Processed requests are forwarded to one of three backend categories:
- AI Models: OpenAI, Anthropic, Bedrock, Gemini, Vertex, etc.
- MCP Servers: tool/data servers (e.g., GitHub, Slack, databases, …)
- REST APIs: internal Kubernetes services or external HTTP endpoints
Components Summary
Management Plane (this repository)
| Component | Technology | Role |
|---|---|---|
| Vue UI | Vue 3.4 + TypeScript 5.4 + Vite 5.3 | Operator console, standalone or MFE (ai-gateway) on port 8230 |
| CORE service | REST /core/v1 | Profile, menu, project (cross-cutting) |
| AI_GATEWAY service | REST /ai-gateway/v1 | CRUD for all CRDs, realm + project scoped |
Runtime Plane (agentgateway upstream project)
| Component | Technology | Role |
|---|---|---|
| Control plane | Go | gatewayReconciler (Helm render) + agentgatewaysyncer (XDS converter) |
| Data plane | Rust | TLS, AuthN/Z, rate limit, prompt guard, routing — all in one pod |
| Backend dispatchers (built-in) | 4 types — ai, mcp, static, dynamicForwardProxy |
Kubernetes Resources
| CRD | API Group | Purpose |
|---|---|---|
| Gateway | gateway.networking.k8s.io | Entry point with listeners (host / port / TLS); GatewayClass = agentgateway |
| HTTPRoute | gateway.networking.k8s.io | Match request → backendRef; supports filters (rewrite / redirect / mirror) |
| AgentgatewayBackend | agentgateway.dev/v1alpha1 | Upstream definition (ai / mcp / static / dfp) |
| AgentgatewayPolicy | agentgateway.dev/v1alpha1 | Frontend / traffic / backend policies (incl. AI sub-policies) |
| AgentgatewayParameters | agentgateway.dev/v1alpha1 | Data-plane provisioning (image, replicas, HPA, etc.) |
| Secret | core/v1 | TLS certificates and backend API keys |