In the first 30 seconds I want to stop turning a provider limit into a fleet wide retry storm. A 429 can mean different things: a short rate window, exhausted token allocation, a specific model or organization limit, or a billing condition that will not clear in seconds. Read the provider error and headers. OpenAI's rate limit guide says a temporary 429 may include Retry-After, which is a minimum delay for that API. It does not mean every 429 is safe to retry, and a provider SDK may already be retrying under the gateway.

Immediately cap attempts at one controlled layer. Respect the signal and the user's deadline, add jitter if a retry is genuinely useful, and reserve a small retry budget rather than allowing every app to try twice. Track accepted, rejected, and retried requests separately so the dashboard does not show a fake recovery from an increased number of attempts. Protect existing streams from a blanket kill switch unless continuing them makes the incident worse or the provider itself ends them. A request that has already emitted a partial answer cannot be replayed as a transparent fresh answer.

I would also split traffic by contract in those first seconds. Some customers pin the provider. Some workflows have a proven fallback. Others have no acceptable alternate. For a pinned or high consequence path, return an honest temporary failure or queue only within a bounded deadline. Do not let a router silently trade quality or a data residency rule for HTTP success. If a read only low risk query has a tested alternate, move new eligible requests there within its spare capacity, not all 429 traffic at once.

At five minutes, the question is whether this is our demand exceeding the allocation or a change in effective provider capacity. Check rate limit headers and per model quota, our prompt and output mix, recent rollout, retry volume, region, and the provider status. Compare planned admission with actual upstream attempts. A token heavy tenant can exhaust a quota while QPS looks normal. Throttle or shed lower priority traffic at the platform edge and keep queue time visible. If a delayed answer would be useless, reject it before we buy input tokens for a late attempt. Google's overload guidance is relevant here: early shedding and controlled degradation preserve useful work when a dependency is saturated.

Now the fallback fails an important quality slice. I would name the slice and make that routing exclusion explicit. Perhaps a smaller model drops a contract exception or emits invalid tool arguments. Existing evals may allow it for general documentation answers but not this workflow. A canary on the safe slices can expand cautiously while the unsafe slice gets a clear “try later” response, a human path, or a delayed job if that product contract permits it. A degraded answer must be labeled and useful, not an unsupported confident answer from the wrong model.

At an hour, we need a sustainable operating mode rather than a clever retry timer. Decide how much traffic we can actually serve under the current allocation, revise admissions, and report the affected customers and tasks. Ask the provider about the limit and timeline, but do not plan the next hour on an unconfirmed capacity increase. If the incident is an enduring quota mismatch, buy or reserve more capacity, change product tiers, or move eligible workloads after qualification. If our own rollout inflated tokens, revert it. Keep the backlog bounded and expire tasks whose deadline passed.

What if provider health briefly returns? Ramp admissions up under a shared budget. Do not reopen every client circuit at once and make the next 429 wave ourselves. Track success under live load, queue age, retry attempts, quality by fallback slice, and spent tokens per completed task. A stable recovery means the new offered load fits available capacity and the quality contract, not that the last 429 disappeared from a graph.