Kong AI Gateway extends Kong's API platform with LLM routing, semantic caching, and MCP governance. But its per-service pricing punishes AI-native teams experimenting with models.
Kong Gateway has over 43,900 GitHub stars and more than 11,267 commits. That history is entirely in API management. The Kong AI Gateway is a newer layer on top of that foundation, an AI-enabled extension of Kong Gateway that adds LLM routing, semantic caching, prompt guards, and observability to a platform originally designed for REST microservices.
For enterprises already running Kong for API traffic, this extension is a logical add. For teams adopting a gateway purely for LLM orchestration, the calculus is different. Kong's pricing model charges approximately $105 per month per Gateway Service, and each LLM provider or model you route to counts as a separate service. That per-service structure, combined with request-based billing, creates real friction for AI teams that swap models weekly and run agentic workflows generating dozens of internal calls per request.
Kong Gateway is built on top of Nginx and the OpenResty framework, designed to sit in front of RESTful microservices and handle traffic flow before requests reach the backend. The AI Gateway extends this with a set of specialized plugins, using the same model you use for any other Kong Gateway plugin. You can assemble a sophisticated AI management platform without custom code or deploying new and unfamiliar tools.
The core value proposition: a provider-agnostic API that routes AI requests to various providers. Client applications stay decoupled from provider-specific APIs. Credentials are managed centrally. Request routing can shift dynamically to optimize for cost, latency, or availability.
On top of this sits the Konnect control plane, which provides routing and load balancing, streaming responses, access control, usage analytics, and visual traffic maps showing how requests flow between clients and models in real time.
You can launch a demo instance with a single command: curl -Ls https://get.konghq.com/ai/v1 | bash. That gets you from zero to a running gateway quickly. Production deployments are more involved, but the on-ramp is genuinely low-friction.
The routing layer centers on the AI Proxy plugin, which standardizes API signatures across LLM providers. Your application sends a single request format. The proxy translates it to whichever provider backs the route. Swap providers without touching application code.
Beyond basic proxying, Kong AI Gateway supports three distinct routing capabilities:
For teams managing multiple providers, this collapses what would otherwise be per-provider integration work into a single configuration layer. The question is whether the routing alone justifies the cost structure underneath it. If you want a broader look at how AI gateways compare to firewalls and control platforms, the breakdown in AI firewall vs. AI gateway vs. control platform covers the taxonomy.
Kong AI Gateway advertises three security mechanisms: PII sanitization, semantic prompt guards, and access control.
The PII sanitization claim deserves scrutiny. Kong positions it as a way to "stop data leakage and avoid compliance nightmares." But the implementation runs through Lua plugins on the Nginx/OpenResty stack. For teams operating under GDPR, KVKK, or sector-specific data residency rules, the depth of that sanitization matters: does it redact before the request leaves your network, or after? Does it handle structured PII in tool-call arguments, not just free-text prompts? Kong's documentation describes the capability at a feature level without detailing the enforcement boundary.
This is one area where purpose-built AI gateways diverge from extended API gateways. Our own platform, SHIM, treats PII redaction as a core pipeline stage rather than an optional plugin, stripping identifiers before any request reaches a provider. That architectural difference matters when compliance requires proof that sensitive data never left your boundary, not just that a filter was enabled.
On the cost governance side, Kong supports token quota management with user, model, and time-bound quotas, plus showback and chargeback for LLM, agent, and MCP usage across the enterprise. The observability layer tracks AI consumption, tool usage, and token spend, with predictive consumption models for cost tuning and debugging via logging and tracing.
Kong has extended beyond LLM call proxying into the agentic protocol layer. The gateway now claims to secure, govern, and observe AI-native systems end to end by supporting both Model Context Protocol (MCP) and Agent2Agent (A2A) traffic.
For MCP, the gateway can automatically generate MCP tools and servers on top of Kong-managed APIs, enforce authentication for MCP server access, and optimize token spend through MCP context optimization.
For A2A, Kong captures detailed telemetry on every A2A call, including payloads, latency, token usage, errors, caller identity, and capabilities invoked. It also enforces centralized authentication and authorization for all A2A traffic.
These are the right capabilities to build. Whether Kong's implementation is mature enough for production agentic workflows is a separate question, and one that early adopters will answer over the next 12 months. We have covered other gateway approaches to this problem in our writeups on Envoy AI Gateway and Databricks AI Gateway.
This is where Kong's API management heritage creates genuine friction for AI use cases.
Kong Konnect SaaS charges approximately $105 per month per Gateway Service and roughly $34.25 per 1 million API requests, plus about $720 per month for dedicated cloud gateway instances. Each LLM provider counts as a separate billable service. Route to OpenAI, Azure, Anthropic, and a local Llama instance, and that counts as 4 distinct services.
Two dynamics make this pricing model particularly painful for AI teams:
The experimentation tax. AI teams test new open-source models weekly. Adding a new model endpoint in Kong Gateway Enterprise can trigger a license upgrade event if you exceed your service quota. Enterprise features like AI Rate Limiting Advanced or specialized analytics often require higher-tier licenses, pushing contracts frequently above $50,000 annually even for mid-sized deployments.
The agentic multiplier. A single agent workflow might trigger a chain of 20+ internal API calls, spanning RAG lookups, vector DB searches, prompting, and re-prompting. In traditional API management, 1 million requests is substantial volume. In agentic workflows, that volume accumulates rapidly, accelerating consumption of usage-based tiers far faster than standard REST traffic.
The per-service model was designed for a world where backend services change infrequently. In AI, the model roster is the variable you iterate on most.
Kong is a comprehensive enterprise platform designed to handle the complexity of banking and telco legacy systems. It includes extensive support for SOAP, gRPC transcoding, XML validation, and monolithic architectures. For a generative AI application processing JSON-based REST payloads, you do not need XML-to-JSON transformation or SOAP services. But because Kong is a bundled platform, a portion of your licensing fee supports capabilities you will never invoke.
The plugin architecture compounds this. Kong's extensibility runs on Lua, layered on Nginx/OpenResty. AI teams overwhelmingly work in Python. Building custom AI middleware, writing prompt transformation logic, integrating with evaluation frameworks: all of this is more natural in Python than in Lua. If your team does not already have Lua expertise (and most AI teams do not), extending Kong's AI capabilities means learning a language ecosystem that has minimal overlap with your existing toolchain.
Kong AI Gateway makes clear sense for one profile: enterprises already running Kong Gateway for REST and gRPC API management who want to extend that infrastructure to cover LLM traffic without deploying a separate system. The plugin model means no new tools to learn. The Konnect control plane unifies API and AI observability. The institutional knowledge your platform team has built around Kong transfers directly.
It is a poor fit for teams building AI-native applications from scratch. The per-service licensing penalizes model experimentation. The request-based billing amplifies costs in agentic architectures. The Lua plugin system is orthogonal to the Python tooling AI engineers already use. And the bundled legacy protocol support adds cost without adding value.
The routing capabilities are real. The semantic caching and prompt guards solve genuine problems. The MCP and A2A support targets the right emerging protocols. But the billing model was designed for API management at scale, not for AI teams iterating on model selection, testing new providers weekly, and running agent workflows that generate 20x the request volume of traditional API calls.
If you are evaluating gateways specifically for LLM traffic, understanding what an AI gateway is and how the category differs from traditional API management is worth the detour before committing to a platform whose economics were built for a different workload.