Distributed Reliability · Staff
A zone fails during a model rollout. Why do both fleets miss their deadlines?
The question
Interview question
A three-zone GPU fleet runs a model near its target utilization. A new model version is warming in each zone while old replicas still serve. One zone fails. The rollout controller reports that the old version still has enough replicas and the new version is healthy in two zones. Users see long queues and missed first-token deadlines on both versions. Why is replica count misleading, and what should the rollout budget have reserved?
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
The fleet lost a fault domain while it was already paying for two model versions. A replica count does not state how many tokens per second the remaining replicas can serve at the current prompt and output mix, how much memory the new version needs, or whether they are ready for the requests being routed to them. The surviving zones now take diverted traffic, and the new model's weight load and warmup can compete for their GPU and storage bandwidth. Queues grow before a pod necessarily fails a liveness probe.
I would model capacity in the worst allowed combination: peak contracted traffic, one zone unavailable, and the permitted rollout surge. The useful unit is goodput at the latency SLO for traffic classes, not average GPU utilization. Compute capacity separately for old and new versions while both receive requests. If the new version has lower tokens per second or larger KV use, one new replica is not equivalent to one old replica. Keep enough ready old-version capacity in the surviving zones for rollback and choose a maximum new-version exposure that still leaves headroom under the failure condition.
Kubernetes rolling update controls describe maxSurge and maxUnavailable, while topology spread constraints help place pods across zones. Neither control by itself guarantees GPU availability, weight locality or token-level SLO capacity. A rollout can satisfy pod availability and still overcommit the expensive part of the service. The capacity plan must include placement, GPU type, local caches, interconnect dependencies and load balancer routing.
When the zone fails, freeze further rollout and drain work from failed replicas with a bounded retry policy. Route new requests only to a version and zone that can meet their deadline and data boundary. Shed or degrade lower-priority work under an explicit policy before queue debt poisons every request. Do not immediately roll back if the old fleet also lacks capacity to absorb all traffic. A rollback is a workload transition with its own model loading and cache costs. First stabilize admission and preserve truthful user responses, then choose the safest serving version.
What if there is no budget for a full spare zone of GPUs? We can contract for a lower availability or performance tier during a zone failure, keep a smaller fallback model for eligible tasks, reduce rollout speed at high traffic, or reserve capacity through another mechanism. But we cannot claim a one-zone-loss SLO while running with no headroom in a three-zone fleet. Test an actual zone evacuation during a canary, including a slow new-model load, and measure queue age, first-token p99, successful tasks, rejected requests and rollback time.
A rolling model deploy runs out of GPU capacity covers the capacity consumed by rolling out a model. This scenario combines rollout with a fault-domain failure and asks whether the fleet can still serve at the promised deadline. The release gate is a capacity calculation and a failure drill, not a deployment controller saying its replica count is healthy.
Continue reading
Related questions
Read beyond the question
Explore more distributed reliability
Follow another question in this area, or return to the full Interview Prep index.
Browse this area →