I would draw the call stack and count the attempts before proposing a circuit breaker.

The API tries the retriever three times. Each retriever attempt tries the reranker three times. Each reranker attempt tries the model three times. In the all failing case, one user request can cause 3 × 3 × 3 = 27 model calls. If the model client's own SDK also makes three attempts, the bound becomes 81. These are maxima for that retry structure, not a prediction that every failure reaches the bound. Timeouts, cancellations, and partial successes change the actual count.

Now imagine 30 user requests per second hit that path during the outage. At the bound, the model receives 810 attempts per second from those requests alone, before other traffic. The service is already overloaded, so the retries make recovery less likely. AWS's Builders' Library gives the same multiplication problem for a deeper service stack and recommends avoiding independent retries at every layer.

“The retriever only retries once, so surely it is harmless.” One retry at each of three layers still makes up to 2 × 2 × 2 = 8 leaf calls. The multiplication is the issue. I would ask for a measured improvement in successful requests under realistic faults, including the extra load on a recovering provider, before keeping any layer's retry.

I would put one owner in charge of retrying the whole operation, usually the layer that knows the user deadline and acceptable degraded behavior. Lower clients should still handle transport errors, but their automatic retries must be disabled, limited, or counted against the same budget. A three second deadline is propagated to every call. A retriever that spent 2.7 seconds waiting has no business starting a model retry that normally takes another second. The remaining time, not a local timeout copied into each layer, decides whether another attempt can help.

The retry decision also depends on the error. A short connection reset before any work may be retryable. A 429 under capacity pressure is a signal to reduce admitted work and respect the provider's retry guidance, not for every caller to send two immediate copies. A bad schema or invalid request will not improve on repetition. An ambiguous write can have happened despite a timeout and needs idempotency or reconciliation before another attempt. A model response that is merely low quality is not a transport failure. Blindly asking again can increase cost while changing the answer unpredictably.

For transient failures, I would use bounded attempts with backoff and jitter, plus a local retry budget or token bucket so a burst of failed requests cannot dominate capacity. Admission control comes before the overloaded dependency. If a reranker is unavailable and the application has measured that lexical plus vector ranking is acceptable for a low-risk class, it may degrade there. If the model is unavailable, a fallback provider is only valid for tasks whose quality, data handling, and tool behavior have been evaluated. Otherwise return a clear unavailable result instead of pretending the fallback is equivalent.

I would instrument attempts per logical request, remaining deadline at each attempt, time spent in each dependency, retry reason, 429 rate, queue length, provider health, and cost per successful task. A trace should make it obvious whether one user request created one model call or 27. Incident response can then cut retry budgets and shed low-priority work quickly, rather than guessing from aggregate QPS.

Then they suggest a circuit breaker. It can protect a struggling dependency, but its open and half-open states need careful recovery behavior. If every instance probes at once, the service sees another burst. A shared or bounded probe rate, jitter, admission, and a limited retry budget are still needed. A circuit breaker is one control, not the policy for the whole stack.

The AWS Builders' Library treatment of timeouts, retries, and jitter is the primary reference for the multiplicative load argument. The three second deadline and 30 requests per second are illustrative.