Architectures

Overview

The AI Gateway system is designed with a two-plane architecture consisting of a Management Plane and a Traffic Plane. The two planes are intentionally decoupled and connected only through native Kubernetes configuration mechanisms, enabling centralized governance with reliable runtime enforcement.

Management Plane

Operators manage the system through a Management Panel, covering the following functional areas:

• Gateway: gateway lifecycle and related components
• Backend: definition and management of destinations (LLMs, MCP, APIs)
• Route/API: routing rules, endpoints, and API mappings
• Policy: access control and content governance policies
• Secret: credentials and keys management
• Gateway Config: operational gateway settings
• Topology/Monitoring: system visibility and monitoring views

All UI actions are materialized as Kubernetes CRDs. The agentgateway continuously watches these CRDs and reconciles runtime behavior accordingly. This is the single, controlled integration point between management and production traffic.

Traffic Plane (Runtime Flow)

Requests flow left → right:

Input Traffic (AI Clients)

The gateway accepts requests from common AI clients such as Claude Desktop, Cursor, GitHub Copilot, as well as custom applications/agents.

agentgateway (Rust Proxy)

All requests pass through agentgateway, which enforces and optimizes traffic before forwarding:

• Auth / AuthZ
• Rate Limiting
• Prompt Guard
• Routing
• Caching
• Observability 

Backends

Processed requests are forwarded to one of three backend categories:

• AI Models: OpenAI, Anthropic, Bedrock, Gemini, Vertex, etc.
• MCP Servers: tool/data servers (e.g., GitHub, Slack, databases, …)
• REST APIs: internal Kubernetes services or external HTTP endpoints

Components Summary

Management Plane (this repository)

Component	Technology	Role
Vue UI	Vue 3.4 + TypeScript 5.4 + Vite 5.3	Operator console, standalone or MFE (ai-gateway) on port 8230
CORE service	REST /core/v1	Profile, menu, project (cross-cutting)
AI_GATEWAY service	REST /ai-gateway/v1	CRUD for all CRDs, realm + project scoped

Runtime Plane (agentgateway upstream project)

Component	Technology	Role
Control plane	Go	gatewayReconciler (Helm render) + agentgatewaysyncer (XDS converter)
Data plane	Rust	TLS, AuthN/Z, rate limit, prompt guard, routing — all in one pod
Backend dispatchers (built-in)		4 types — ai, mcp, static, dynamicForwardProxy

Kubernetes Resources

CRD	API Group	Purpose
Gateway	gateway.networking.k8s.io	Entry point with listeners (host / port / TLS); GatewayClass = agentgateway
HTTPRoute	gateway.networking.k8s.io	Match request → backendRef; supports filters (rewrite / redirect / mirror)
AgentgatewayBackend	agentgateway.dev/v1alpha1	Upstream definition (ai / mcp / static / dfp)
AgentgatewayPolicy	agentgateway.dev/v1alpha1	Frontend / traffic / backend policies (incl. AI sub-policies)
AgentgatewayParameters	agentgateway.dev/v1alpha1	Data-plane provisioning (image, replicas, HPA, etc.)
Secret	core/v1	TLS certificates and backend API keys