I would start by asking what a failure costs. If this is a drafting assistant, a weaker answer may be acceptable with disclosure. If it is an action that can move money or an answer that requires a particular evidence standard, a weaker model is not an automatic substitute. The router's choice set includes a guarded downgrade, queue within the user's deadline, reject early, or pause the unsafe part of the workflow. “Always give something” is not a quality policy.

The offline number assumed the premium model was available at the measured latency and price. In production, routing changes arrival rates at each backend, and queueing changes latency nonlinearly as utilization approaches capacity. A model that is best for the hardest 15 percent of requests can still be the wrong route when that lane is saturated. Use current quota, observed service rate, reservation headroom, remaining deadline, estimated token load, cost budget, tenant entitlement and the request's minimum quality requirement. A threshold can be one policy input, but it is not the objective.

I would admit requests against bounded per-provider budgets for requests, input tokens, output tokens and in-flight work, because providers and models can impose different limits. OpenAI rate-limit guidance describes request and token limits. Each reservation needs a conservative estimate and a correction when actual usage arrives. Underestimation must not allow unlimited queued work. Include retry traffic in the budget, or a spike can become a retry storm. A model provider returns 429s at peak. What happens over the first hour? covers provider 429 behavior in the first hour. The decision here is how the router trades quality against finite capacity on each new request.

I would separate requests into quality and safety bands before optimizing cost. A band with a proven lower-capability fallback may use it if the estimate of task success at the remaining deadline beats waiting. A band with no acceptable fallback gets a short bounded queue or an explicit unavailable response. Protect capacity for tenants and critical use cases with reservations or weighted fair admission. Otherwise a few long, expensive prompts can consume the premium lane and make small urgent requests miss deadlines.

Now the interesting pushback: the difficulty classifier can be wrong. A request that looks easy may involve a hidden exception, and a cheap model's confident output may hide it. Evaluate routing on outcomes, not just the classifier's accuracy. Compare successful task rate, groundedness and severity-weighted failures by route and by cohort, at real utilization. Shadow a sample through both models only where policy, privacy and capacity permit. Log the decision features and fallback reason without retaining sensitive prompts by default. Do not blindly train on only accepted routed requests, since the rejected and downgraded cases are the ones we most need to understand.

Another pushback is that a fallback provider has spare capacity but different tools, residency or context behavior. Recheck those constraints before routing. “Spare tokens” does not mean an equivalent execution environment. In a drill, reduce premium capacity, then look at completed task quality, deadline misses, queue age, retries, tenant fairness and cost. A router that merely keeps its own p99 low by dropping difficult requests has not solved the user's problem.