OpenAI rate limits use RPM, TPM, and tier-based caps to throttle API access. Learn how the system works, what triggers 429 errors, and how to handle them.
You shipped a feature that calls GPT-4o. It works in dev. It works in staging. Then three users hit it at the same time in production, and every request comes back 429: Too Many Requests.
The fix is not "get a higher tier." The fix is understanding how openai rate limits actually work, because the system is more nuanced than a single number, and most teams misconfigure their applications around it.
Rate limits are restrictions that the API imposes on the number of times a user or client can access services within a specified period of time. They exist for three reasons: to protect against abuse or misuse, to ensure fair access across all users, and to manage aggregate infrastructure load.
The limits are measured across multiple metrics: RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), TPD (tokens per day), IPM (images per minute), and audio minutes per minute for some streaming audio models. Most developers only think about RPM. That is the first mistake.
The enforcement logic is whichever-hits-first. You might send 20 requests with only 100 tokens to the ChatCompletions endpoint and that would fill your limit if your RPM is 20, even if you have 150k tokens of TPM headroom left. Conversely, a single massive prompt can exhaust your TPM allocation in one call.
Three details that trip teams up:
Limits are scoped to your organization and project, not to individual users. Rate limits are defined at the organization level and at the project level, not user level. Every API call from every service sharing a project key counts against the same pool. This means your internal chatbot and your customer-facing summarizer can starve each other without either one doing anything unusual.
Model families share pools. Some model families have shared rate limits. Any models listed under a "shared limit" in your organization's limit page share a rate limit between them. If the listed shared TPM is 3.5M, all calls to any model in that group count towards it. Switching from GPT-4o to GPT-4o-mini within the same family does not give you a separate allocation.
Sub-minute bursts trigger errors. Even if your total request volume fits within the per-minute cap, rate limits can be applied over shorter periods, for example, 1 request per second for a 60 RPM limit, meaning short high-volume request bursts can also lead to rate limit errors. A batch of 30 requests fired in the first second of a minute can fail even though you only planned to send 50 that minute.
OpenAI gates access through five usage tiers that automatically upgrade as your cumulative spend increases:
| Tier | Qualification | Monthly usage limit |
|---|---|---|
| Free | Allowed geography | $100/month |
| Tier 1 | $5 paid | $100/month |
| Tier 2 | $50 paid | $500/month |
| Tier 3 | $100 paid | $1,000/month |
| Tier 4 | $250 paid | $5,000/month |
| Tier 5 | $1,000 paid | $200,000/month |
Each tier also sets different RPM, TPM, and RPD caps per model. Tier upgrades happen automatically as your spend crosses the threshold. You do not need to apply.
Every API response includes headers that expose your current limit status in real time. Most applications ignore them entirely.
| Header | Example value | What it tells you |
|---|---|---|
| x-ratelimit-limit-requests | 60 | Maximum requests permitted |
| x-ratelimit-limit-tokens | 150000 | Maximum tokens permitted |
| x-ratelimit-remaining-requests | 59 | Requests left before limit |
| x-ratelimit-remaining-tokens | 149984 | Tokens left before limit |
| x-ratelimit-reset-requests | 1s | Time until request limit resets |
| x-ratelimit-reset-tokens | 6m0s | Time until token limit resets |
The remaining and reset fields are the ones that matter for runtime decisions. If x-ratelimit-remaining-requests is 2 and you have a queue of 10 pending calls, you know exactly what is about to happen. Building against these headers instead of guessing is the difference between graceful degradation and a cascade of 429s.
Two specialist limits that catch teams off guard:
Batch API queues are token-based. Batch API queue limits are calculated based on the total number of input tokens queued for a given model. Tokens from pending batch jobs count against your queue limit. Once a batch job completes, its tokens are no longer counted. This means you cannot simply fire off unlimited batch jobs and wait; large queued batches block subsequent submissions until they finish.
Vector store ingestion caps at 300 RPM per store. The file upload and file batch endpoints share a limit of 300 requests per minute for each vector store. If you are bulk-loading documents into a vector store, you will hit this wall quickly and need to throttle your ingestion pipeline accordingly.
The natural instinct is to retry immediately. This makes things worse.
Continuously resending a request without backoff doesn't work because unsuccessful requests still count against your per-minute limit. Every failed retry eats into the same quota that caused the failure. A tight retry loop can hold your application at the rate limit ceiling indefinitely, burning through your RPM on requests that will never succeed.
The error appears as a 429 status code or RateLimitError in the OpenAI Python library. Both indicate the same thing: you have exceeded one of your rate limit metrics for the current window.
Exponential backoff. Perform a short sleep when a rate limit error is hit, then retry the unsuccessful request. If the request is still unsuccessful, the sleep length is increased and the process is repeated until the request succeeds or a maximum retry count is reached. In Python, the backoff library handles this with a decorator:
from openai import OpenAI, RateLimitError
import backoff
client = OpenAI()
@backoff.on_exception(backoff.expo, RateLimitError)
def completions_with_backoff(**kwargs):
return client.completions.create(**kwargs)Client-side throttling. Rather than reacting to 429s, prevent them. Implement client-side request throttling mechanisms to limit the rate of outgoing requests based on predefined rate limits and usage quotas.A token bucket or leaky bucket algorithm, calibrated to your tier's RPM and TPM, smooths bursts before they reach the API.
Caching. Cache API responses at strategic points in your application architecture to minimize redundant requests, mitigate the risk of rate limit errors, and reduce costs caused by excessive API usage. If 40% of your requests are semantically identical (common in customer support and FAQ workflows), caching alone can cut your effective RPM usage nearly in half.
Per-user usage caps. OpenAI recommends that you set a usage limit for individual users within a specified time frame (daily, weekly, or monthly) and consider implementing a hard cap or a manual review process for users who exceed the limit. Without per-user controls, a single power user or a misbehaving integration can consume your entire organization's allocation.
Separate API keys per service. Sharing API keys across multiple applications or users aggregates usage, increasing the likelihood of hitting rate limits unexpectedly. Isolate each service or application into its own project with its own key so that limits are scoped independently.
Each strategy above works. The problem is that every service in your stack needs its own implementation. Your chatbot needs backoff logic. Your summarizer needs its own throttle. Your RAG pipeline needs caching. Your batch processor needs queue management. Multiply that by the number of models and providers you use, and you are maintaining the same retry/throttle/cache code in a dozen places.
This is the architectural problem that an AI gateway solves. A gateway sits between your application code and the model APIs, centralizing the concerns that rate limits force you to deal with:
Admission controls reject requests that exceed configured rate, quota, or spend policy before a provider call starts.
Per-user and per-team controls are configured once and enforced at the shared boundary.
Usage accounting records provider attempts without adding a response cache.
Retries and fallback remain explicit application policy. SHIM does not automatically retry billable provider calls or fail over across providers.
The pattern is straightforward: instead of each service implementing its own rate limit handling, you solve it once at the infrastructure layer and every service behind the gateway inherits the solution.
If you are building against OpenAI's API directly and starting to feel the friction of managing rate limits across multiple services, Shim's documentation walks through how this works in practice. You can get started here.