A 429 is the least specific 4xx a model provider can send you. No other 4xx code means "I will not do this, but ask again — maybe — and there are three different reasons why." And the reason changes what you should do: back off, fail over, or stop permanently.
The confusion about what a provider's 429 actually means is a recurring topic in gateway issue trackers, and the shape of the discussion over on the community side mirrors the shape of the code. This is the playbook: how to decode the cause, then what to do about each one.
Three causes behind one status code
1. A rate-limit burst. You crossed a per-minute ceiling (RPM, TPM, or concurrent requests). Transient. The window resets. Retrying is correct — with backoff. Usually ships with a retry-after header or a cause string that names the meter you hit.
2. A key or project cap. This key (or the project it belongs to) hit a platform spend cap — daily or monthly. Not transient. Retrying against a condition that is deterministically reached just burns errors until the period rolls over and the next hour is wiser. The correct moves are to refuse, alert, or fail over — none of which involve asking the same key again.
3. A plan or team quota. Plan-tier limits, seat allocation, organization quota. The reset is measured in days, not seconds. A 429 that lands at the same wall-clock time, month after month — period start, a specific calendar boundary — is a quota, not bad luck.
There is a fourth one that is not a cap at all: a 429 your tooling invents. If you have routed consumer subscription session keys through a proxy or toolchain, requests that go through clean directly to the provider come back with rate-limit errors that direct requests do not — a sign that the traffic pattern on that key is wrong, not that the provider is busy. Do not put infrastructure traffic on session keys; it is a terms violation and an architecture bug at the same time. Similar patterns at the edges of local runtimes and multi-provider orchestrators are tracked on Olla's issue list and Cascadr's; the common root cause is the key path, not the provider.
Decode before you decide
The cause is on the wire, and the three fields to read are the same across providers:
- The cause word in the body. Names differ, families do not: a rate-limit noun, a cap noun, a quota noun.
retry-after— present or absent. Its presence says the condition resets.- The shape over time: a burst that comes and goes, a flat wall, or a wall at the same time every month.
Concrete shapes (the strings vary by provider; the combination is the signal):
--- burst (rate limit) ---
HTTP/1.1 429 Too Many Requests
retry-after: 7
{ "error": { "type": "rate_limit", "message": "Rate limited on requests per minute." } }
--- cap ---
HTTP/1.1 429 Too Many Requests
(no retry-after header)
{ "error": { "type": "cap_reached", "message": "Key spend cap for this period has been reached." } }The burst has the header; the cap does not, or if it does it points at the next period, not the next second. retry-after: 7 is a backoff signal. "Cap reached" is a decision point — nothing you do in the next few seconds changes it.
The playbook, per cause
- Bursts: exponential backoff with jitter, honoring
retry-afterwhen it is present, and a sane ceiling (45 seconds covers most per-minute windows). Keep in-flight requests under the provider's TPM, not just the RPM — output-heavy work exhausts token ceilings long before request ceilings. - Cap reached: do not retry. This is a deterministically-failing call. Automatic retries convert a clean stop into noisy logs — and in several harnesses, into retries that bill for the attempt even though the work was refused. The moves: fail over to a second key or provider if you have one; otherwise queue the work in something durable that drains when the period rolls; and alert a human, because a cap is a design decision being reached, not a transient condition.
- Quotas: same shape as caps (do not retry against them) on a longer timescale. The queue pattern is the right answer here, and if the workload can absorb it, a 24-hour batch API is the cheapest "not yet" you can buy.
A production account of exactly this failure shapes an LLM tooling thread from the community, and the one-line summary from it is worth repeating: the retry policy was the incident. That is where a cap stops being an error and starts being a spend — not in the request, in the policy that retries it.
Circuit breakers keyed on cause, not status
A breaker that trips on any 429 over-corrects. A single burst 429 is a valuable signal — "this key is near its ceiling" — and breaking on it makes your next request more expensive by landing it on a less-preferred provider. Key the breaker on provider + model + cause, and trip on sustained patterns — N consecutive burst 429s with rising retry-after — not on a single event. A cap-reached trips a different, longer-lived breaker, because it will not clear until a human or a calendar does.
The architectural lever: your real limit is a sum
Per-key limits are per-key. Two keys on one provider roughly double your effective RPM and TPM, within what the provider's accounting allows. A second provider doubles it again — and, the part that gets missed, decorrelates your failure modes, because a cap on one platform is not a cap on the other. This is why the effective ceiling of a well-built system is the sum across providers and keys, not the listing for any single key. The cost side of that decision — what failover does to the bill, including context replay and cache invalidation — is in the failover post.
On an endpoint with multiple upstreams, a single provider's cap becomes a routing decision the platform makes instead of you. What stays yours is the shape of the 429 you see: the cause, retry presence, and whether the condition is transient. Exact error semantics are in the docs; per-model rates and ceilings are in the model catalog.