The interview question. A 32-shard cache handles product lookups. Thirty-one shards are comfortable, but one is pinned at its throughput limit. Traces show that the same product key receives about 300,000 reads per second. The team doubles the cluster to 64 shards and redistributes the key space. The problem remains.
Explain why extra shards did not solve it. Propose a safe read design, then change the requirement: the hot key now represents inventory decrements, not a product description. What must change?
Think first
A celebrity product page has one cache key receiving 300,000 reads per second. Scaling the cluster from 32 to 64 shards does not improve it. Design a plan if the description can lag by five seconds. Then make that key a scarce-inventory counter and explain what you can no longer replicate freely.
Take a few minutes to form an approach, then compare it with the worked explanation.
Check the exercise numbers
For the follow-up, 40 decimal MB × 1,000 full-value reads/s = 40 GB/s, or 320 Gbit/s of payload before protocol overhead. That exposes a bandwidth demand, not proof that networking always saturates before CPU or serialization. For inventory, four disjoint buckets of fifty tickets authorize at most 200 sales before coordinated rights transfers.
Reveal the worked explanation
First find out what is hot
A busy node is evidence of load, not an explanation of its source. Three problems can look almost identical from a host-level dashboard.
One hot key receives an extraordinary number of requests. One hot range receives many requests for different adjacent keys, perhaps because new records have increasing timestamps. Or the placement policy simply put too many ordinary keys on one node. The remedy depends on which pattern exists.
For this interview, traces already identify a single hot product key. That matters. Changing which server owns the key merely relocates the problem. Whether the cluster has 32 or 64 shards, a scheme with one authoritative cache owner for that key still sends those requests to one place.
A single hot key remains pinned to one cache owner after rebalancing, while a bounded read-fanout layer can distribute acceptable stale reads across copies
If 300,000 reads per second exceed one server's useful capacity, better hash distribution for unrelated keys is not the missing fix. We need either to reduce origin requests, allow several copies to answer, or limit demand. These choices require a freshness contract.
A product description is a reasonable candidate for read copies
Suppose the product description can be a few seconds stale. We can add bounded in-process caches, regional cache copies, or a CDN where the access pattern permits it. Requests hit these copies instead of hammering one shared cache shard. Cache entries need versioned invalidation or a clear TTL. For a five-second end-to-end freshness bound, budget the source version’s age, delayed fills, invalidation propagation and serving lifetime together. Starting a five-second TTL after an arbitrarily stale fill does not establish that bound. A single-flight mechanism can collapse many simultaneous misses into one load per cache node.
I would not choose an exact TTL from the air. Product should say whether an edit must appear instantly, within five seconds, or on the user's next session. The answer determines whether stale copies are acceptable and what an invalidation path must deliver. It also determines how much cost we can move away from the hot owner.
Think about the dangerous moment when a version expires. Ten thousand application workers might miss in the same second. Local caches can protect the shared owner during normal traffic and then collectively overload it during a synchronized cold start. Add jittered expiration, bounded refresh concurrency, stale-while-revalidate where permitted, and request coalescing. Keep enough capacity at the backing source for a controlled refill, not an unlimited stampede.
Redis notes that frequently requested keys can load their owning shards heavily and that replicas may be used to serve reads where staleness is acceptable. This is not permission to present replica reads as fresh under every failure mode. Redis Cluster specification
If the product description must become visible to every reader immediately after an edit, a casual stack of TTL caches no longer satisfies the requirement. Use a stronger version or validation protocol, route those operations to a current authority, or renegotiate the freshness guarantee. An invalidation message can be delayed or missed. “We publish invalidations” is not a complete correctness argument.
Change one noun and the architecture changes
Now imagine the same hot key is the inventory count for the last 200 concert tickets. Each request attempts to decrement it, and overselling is forbidden. Reading a stale copy is not equivalent to authorizing a sale.
Replicating the number onto ten writable cache nodes and letting all ten subtract independently can sell more tickets than exist. Two nodes may each observe 200 and independently authorize 200 successful purchases. The invariant is no longer a freshness preference. It is a shared limit that must be enforced across every successful decrement.
For the simple design, keep one authoritative, conditionally updated inventory record. It can check available stock and decrement atomically for a purchase intent. Put waiting-room admission and fair rate limiting ahead of it so a flash sale does not turn every failed attempt into expensive contention. At 300,000 attempted writes per second, the service may need to reject or queue many attempts even if the database is healthy.
Can we spread the writes? Possibly, but the meaning changes. We could allocate disjoint ticket quotas to several independent buckets, for example fifty tickets per bucket across four buckets. Each bucket sells only its assigned quota. Transfers between buckets require a separate protocol, and some buckets may run out while others still have capacity. The system must not grant overlapping rights during a partition. This escrow-like model trades flexibility for safe parallelism.
DynamoDB's adaptive capacity can split hot item collections in some conditions, but it does not mean any single item can absorb arbitrary writes by magically acquiring multiple independent authorities. Its own guidance distinguishes partition limits, item-collection splits, and monotonically increasing sort keys. AWS guidance
The interview decision is therefore not “hot key means replicate.” It is which operations can be answered from independent copies, and which require controlled authority?
The next system has a different problem. Its traffic is not concentrated on one value. It needs ordered customer history, and the partition key that spreads writes makes its reads expensive.
Go deeper
Go deeper: distinguish read skew from ownership pressure
Before changing the application, inspect the server's useful throughput, CPU, bandwidth, serialization cost, key size, connection count, and latency. A large value retrieved a few times per second can saturate bandwidth while a small frequently read value can saturate CPU. Hot keys and big keys are not synonyms.
Read replicas can reduce read pressure when acceptable, but each replication scheme has an acknowledgment, lag, and failover story. If users must see their own edits, session-based routing or version tokens may allow a narrower guarantee than global instant freshness.
For writes, conditional updates protect the inventory predicate, not user fairness. A waiting room or token bucket may reduce load without deciding who is entitled to stock. That policy and its abuse resistance belong to the service contract.
An escrow design only works if the sum of allocated rights never exceeds the real inventory and transfers are coordinated without duplication. If four buckets each possess fifty tickets, the total authorization is at most 200 until a valid transfer or return changes those rights. Its extra complexity is usually unjustified for ordinary low-volume inventory.
Check your reasoning
Differentiate single hot key from hot range, oversized value, or unbalanced placement with evidence.
Explain why changing shard count or hash function merely moves a single-key bottleneck.
Use bounded read copies, miss coalescing and cold-start origin protection under explicit freshness requirements.
Explain why independent writable copies can oversell and name the authority enforcing conditional decrements.
Discuss admission and optionally nonoverlapping reservation rights with their reallocation cost.
Common mistake: Assuming a more uniform distribution of keys guarantees uniform distribution of requests, or that caching stale inventory safely authorizes sales.
Read the answer explanation
Preallocate disjoint quantities of tickets to independent authoritative buckets, with controlled rights transfers. — Disjoint rights bound total successful decrements, whereas independently copied full counts do not.
Apply it elsewhere
A global feature flag is read millions of times per second but can be revoked urgently. Which parts of the hot-read design remain valid?
Compare your approach
Define whether “urgent” means a bounded propagation delay or an immediate authorization barrier. That requirement governs cache validation and fail-closed behavior.
One more constraint
One key is both 40 MB and read 1,000 times per second. Which resource saturates first?
Quick revision
Adding shards can spread many keys but not the traffic to one key if one owner still handles that key. Confirm whether the hotspot is a key, range, or placement imbalance. For a read-heavy description with bounded staleness, use controlled caches, read copies, and miss coalescing. Define freshness and protect cold starts. For inventory decrements, stale writable copies cannot enforce an aggregate limit. Use one conditional authority, admission control, or carefully allocated nonoverlapping reservation rights.
