Who Gets In When the API Is Full?

Follow the decisions in order

1. Part 1 of 6

   Senior

   Fifty services sit behind one gateway. Why is every launch waiting on its team?

   Separate cross-cutting edge responsibilities from domain authorization and client-specific BFF composition without creating a shared business monolith.

2. Part 2 of 6

   Staff

   The contract says 100 requests per second. Why did 200 requests pass at midnight?

   Explain fixed window burst boundaries and choose among exact or approximate sliding counts, token buckets and paced queues based on workload semantics.

3. Part 3 of 6

   Principal · AI Engineering reading

   A global rate limit cannot be guessed independently in each region

   Design global per-identity rate decisions across 100 gateways using atomic shared buckets or leased allocations, with explicit overshoot and partition costs.

4. Part 4 of 6

   Staff

   We admit only 500 requests per second. Why does latency still collapse?

   Differentiate request arrival rate from concurrent scarce work using Little’s Law and build a feedback limiter with bounded queueing and oscillation controls.

5. Part 5 of 6

   Principal

   Checkout is overloaded. Should premium customers get every available slot?

   Allocate scarce API work by criticality and work cost, preserve tenant fairness, and shed early rather than treat paid-user priority as unlimited access.

6. Part 6 of 6

   Staff

   The policy service is down. Should the gateway let everyone through?

   Specify different safe degraded behaviors for authentication, rate limiting, fraud scoring and feature flags with bounded freshness and auditable recovery.