Checkout is overloaded. Should premium customers get every available slot?

Interview question. An API fleet can safely complete 12,000 units of work per second, but a promotion drives demand to 20,000. Requests come from premium subscribers, free users, and background internal jobs. Build an admission policy that protects revenue-critical work without silently starving everyone else or allowing one tenant to monopolize capacity.

There is a deceptively simple answer: let premium users through first, drop the rest. It sounds defensible until the free tier includes prospective customers, the internal job settles already-paid invoices, and one premium organization consumes the entire fleet's available capacity. Priority is about protecting business outcomes under shortage. It is not permission to forget fairness, safety, or the distinction between work that can wait and work that cannot.

I would first translate the workload into cost rather than count requests. A cached profile read might cost one arbitrary work unit, while exporting a year's transactions can cost two hundred. If the fleet's effective capacity is 12,000 work units per second, accepting 12,000 requests per second is safe only if each has cost one. During an overload, the request mix often changes too. Background recalculation and report generation may compete for the same database pool as checkout even when their HTTP arrival rates look small.

We need classes defined by outcomes. For example: Class A might be an authenticated payment confirmation or already-authorized refund, where delaying too long risks correctness or customer harm. Class B could be normal interactive checkout and account changes. Class C covers recommendations and discretionary dashboard queries. Class D contains offline exports and cache warming. This is not simply a pricing hierarchy. Some free-user operations have higher criticality than paid-user analytics. A failed fraud verification might require stronger protection than a paid marketing search.

One workable starting policy reserves 3,000 units per second for essential transaction completion, 7,000 for ordinary interactive traffic, and 2,000 for deferrable work. Those numbers are illustrative capacities, not automatically sensible ratios for every business. A work-conserving scheduler can lend unused reservations to another class, so capacity does not sit idle during quiet periods. The lending must be bounded so critical work can reclaim its share promptly when demand returns. If a background job starts a thirty-second uninterruptible query after borrowing protected capacity, the reservation exists only on paper.

That leads to admission versus preemption. Dropping an expensive request before it starts saves capacity. Stopping it after it has mutated inventory or captured a payment can create a correctness problem. Protect irreversible operations with explicit state transitions and allow controlled completion or compensation. For purely read-only analytics, cancellation and short execution deadlines make work stealing much safer. A scheduler cannot guarantee strict priority over a resource it cannot preempt or meter.

Within each class, add fairness by tenant or principal. Weighted fair queuing and deficit round robin are examples of schedulers that give multiple flows opportunities to progress despite unequal request sizes. With cost-aware scheduling, a tenant's large export consumes more credit than a cheap read. A practical implementation uses per-tenant bounded queues, weight or minimum-share policy, and admission counters near the scarce resource. A queue per tenant alone does not isolate shared workers or database pools. Shared asynchronous queues require their own isolation and fairness policy. Here the learning decision is who may consume scarce synchronous API capacity right now.

Consider a numerical spike. We have 20,000 requested work units per second and capacity for 12,000. Suppose all 3,000 essential units must be served, 9,000 units of other work remain eligible, and 17,000 nonessential units arrive. The nonessential classes together cannot receive more than 9,000. A priority scheduler should reject or defer 8,000 units early rather than admit all 17,000 and let them time out. The exact assignment depends on the contract, fairness weights, starvation floors and the cost of each task. Do not call those rejected units successful completions in throughput graphs.

Is starvation ever acceptable? Some best-effort workloads can be delayed for minutes, but a permanently starved free tier can become a broken product. Define a guaranteed minimum share or maximum waiting bound if the business promises one. Low-priority queues need caps and deadline checks because yesterday's recommendation refresh is not useful tomorrow. Where work is durable and reschedulable, enqueue it rather than forcing an HTTP request to wait. But queue acceptance is a different contract from completed work, and the acknowledgment must be honest.

The HTTP response should reveal the decision without disclosing private policy details. For a client-specific quota violation, 429 with a meaningful retry hint may fit. If the entire service is unable to accept work because of temporary capacity shortage, 503 can better describe system overload. Both require clients to respect deadlines and avoid multiplying traffic with synchronized retries. A paid tier may earn a higher weight or explicit reservation, but that does not justify indefinite retries or letting authorized requests bypass security checks.

Operationally, I want admitted and completed useful work by class and tenant, error rates, shed demand, queue age, wasted work, deadline expiry, and starvation duration. Simulate a major tenant surge, partial-region outage, sudden arrival of critical refunds, misclassified priority tags and an internal batch job holding too many database connections. A caller must not be allowed to label its own request “critical” merely by sending a header. The server assigns class from verified identity and operation semantics.

The engineering answer is not “premium wins.” It is a capacity allocation contract that preserves essential correctness, delivers useful work under shortage, and explains exactly which customers lose service first and for how long. That answer can be debated openly with product, security, reliability and customer teams before the next real overload makes the decision for them.