The GPT-5 API has shipped six model versions since August 2025. Pricing tiers, rate limits, sub-variants, and mid-stream safeguards make a gateway layer essential.
Six model versions since August 2025. GPT-5 launched on August 7, 2025 and is already described as OpenAI's "previous model", superseded by GPT-5.6. Between those two releases: GPT-5.1, 5.2, 5.4, 5.5, and now 5.6, each with different pricing, different rate limit bands, and new features that change how requests behave mid-stream.
If your application talks directly to the gpt 5 api, you are building against a specification that OpenAI rewrites on its own schedule. A gateway layer is how you stop that from being your problem.
Yes. GPT-5 is accessible through the OpenAI API and has been since launch. But OpenAI's own model guidance now recommends GPT-5.6, and GPT-5.6 ships with three sub-variants: gpt-5.6-sol (flagship capability), gpt-5.6-terra (balanced performance at lower cost), and gpt-5.6-luna (efficient, high-volume workloads).
So the answer to "which GPT-5 model should I use" is already a routing decision. The gpt-5.6 alias defaults to gpt-5.6-sol, but teams optimizing for cost or throughput need to route different workloads to different sub-variants. That is middleware work, not application logic.
The price spread across the GPT-5 family is 25x from bottom to top.
| Model | Input (per 1M tokens) | Cached Input | Output (per 1M tokens) |
|---|---|---|---|
| GPT-5 | $1.25 | $0.125 | $10.00 |
| GPT-5 mini | $0.25 | - | - |
| GPT-5 nano | $0.05 | - | - |
A 25x gap between nano and full GPT-5 means routing decisions have direct budget impact. Sending a classification task to the full model when nano handles it fine costs 25 times more per token. Multiply that across thousands of daily requests and the difference is material.
Two common questions. Is the GPT-5 API free? No. Free-tier accounts cannot use GPT-5 at all. API access requires Tier 1 or above. Is GPT-5 mini free? Also no, though Vercel's AI gateway offers GPT-5 mini with $5 of free credits every 30 days for users who have not yet made a payment.
The gpt 5 api enforces rate limits that vary by 30x depending on your spending tier.
| Tier | RPM | TPM | Batch Queue |
|---|---|---|---|
| Free | Not supported | - | - |
| Tier 1 | 500 | 500,000 | 1,500,000 |
| Tier 2 | 5,000 | 1,000,000 | 3,000,000 |
| Tier 3 | 5,000 | 2,000,000 | 100,000,000 |
| Tier 4 | 10,000 | 4,000,000 | 200,000,000 |
| Tier 5 | 15,000 | 40,000,000 | 15,000,000,000 |
500 RPM at Tier 1 is a hard ceiling for production workloads. Tier limits auto-increase with spend, but a new integration starts constrained. Without a gateway handling queuing, retry logic, and overflow routing, those 429 errors land directly in your application.
The pace of releases is the real integration risk. Six versions since GPT-5's August 2025 launch: 5.0, 5.1, 5.2, 5.4, 5.5, and 5.6. Each version changes recommended configurations, deprecates previous defaults, and introduces features that affect cost and latency.
GPT-5.5 illustrates how unpredictable the rollout pattern can be. OpenAI withheld API access at launch, saying API deployments required "different safeguards." API access followed a day later, on April 24, 2026. If your system was waiting on that model, you had no programmatic way to know when it would become available.
Direct integrations absorb this churn as maintenance work. Every version bump is a potential breaking change: new model IDs, new parameter defaults, new behaviors. A gateway abstracts the model version behind your own routing rules. When OpenAI ships the next version, you update one configuration instead of every service that calls the API.
GPT-5.6 brings a significantly larger API surface. Four new capabilities each introduce operational complexity:
Programmatic tool calling. GPT-5.6 can write JavaScript to call eligible tools, pass results between calls, and process intermediate outputs in a hosted runtime. Each tool call adds tokens and latency.
Multi-agent coordination. A GPT-5.6 instance can coordinate multiple subagents in parallel and synthesize their results. This multiplies token consumption in ways that are hard to predict from the initial prompt.
Persisted reasoning. The model can reuse reasoning items across turns to improve multi-turn quality and cache efficiency. Useful, but it changes the cost profile of conversations over time.
Explicit prompt caching. GPT-5.6 lets you mark which reusable prompt prefixes OpenAI caches. The catch: cache writes cost 1.25x the uncached input rate, while cache reads remain discounted. Misconfigure your cache boundaries and you pay more than you would without caching at all.
Each feature is a knob that affects cost and latency. Without centralized controls, each team configures them independently, and the bill reflects the inconsistency.
GPT-5.6 deploys real-time classifiers that pause generation for several seconds mid-stream while they review outputs. These safeguards may intervene on legitimate work, particularly in dual-use areas where defensive and offensive activity look similar at first.
From a systems perspective, this creates unpredictable latency spikes that your application cannot anticipate or prevent. A request that normally completes in two seconds might take seven. Without a gateway providing timeout management, retry logic, or fallback to an alternative model, those pauses propagate directly to your users.
GPT-5 introduced a 400,000-token context window with up to 128,000 max output tokens. For comparison, GPT-4 maxed out at 32,000 tokens, making this more than a 12x increase. Entire codebases, full contract sets, and months of financial data fit in a single request.
The operational reality: a max-context request at GPT-5's $1.25/M input rate costs roughly $0.50 in input tokens alone. Output tokens at $10/M add up faster. Cache write costs at 1.25x the uncached rate mean that caching strategy requires careful management.
Fine-tuning is not supported for GPT-5. Developers are directed to use system instructions, custom tools, or model distillation (training smaller models on GPT-5 outputs). That makes prompt engineering and context management the primary optimization levers, both of which benefit from centralized control.
GPT-5 also exposes multiple specialized API endpoints: Chat Completions for conversational turns, the Responses API for multi-step agentic workflows, Realtime for streaming and voice sessions, and Assistants for persistent AI agents with structured logic and toolkits. Each endpoint has different cost and latency characteristics.
The gpt 5 api is powerful. It is also a moving target with a pricing structure that punishes careless usage. A gateway layer, sitting between your application and OpenAI, addresses the specific problems outlined above.
Model routing across the family. Route classification tasks to nano ($0.05/M), general work to mini ($0.25/M), and complex reasoning to the full model ($1.25/M) or its GPT-5.6 sub-variants (sol, terra, luna). The application requests "complete this task." The gateway decides which model handles it.
Rate limit management. Queue requests when you approach tier limits instead of returning 429 errors. Distribute load across provider accounts. Fall back to alternative models when one provider throttles.
Cost budgeting. Set per-team, per-project, or per-endpoint spend limits. Track token consumption in real time rather than discovering overages in the monthly invoice. This is AI FinOps applied at the request level.
Cache optimization. Manage prompt caching centrally so cache boundaries are consistent and cache write costs (1.25x uncached rate) produce net savings instead of net losses.
Safeguard retry logic. When mid-stream classifiers pause or block a request, the gateway retries with adjusted parameters or routes to a fallback model. Your application sees a completed response.
Version insulation. When OpenAI ships the next model version, update the gateway configuration. Your application code stays the same. Given six versions since August 2025, this is a maintenance reduction you will use repeatedly.
The GPT-5 family is the most capable set of models OpenAI has released. It is also the most operationally complex, with a 25x pricing spread, 30x rate limit range, mid-stream safeguards, cache write surcharges, and a release cadence that has already produced six versions. The gap between "works in development" and "runs reliably in production" is exactly the gap a gateway layer fills.
shim sits in front of OpenAI, Anthropic, Gemini, and every other provider, routing by model and enforcing per-route budget caps. It makes one billable attempt per request, so retries stay in your code.