Learning series · 5 episodes · Senior / Staff / Principal
The shard that moved
Five interview lessons on placement stability, hot keys, range-locality, live shard ownership, and repair cascades.
Start series →Follow the decisions in order
- Part 1 of 5
Senior
We added one cache server. Why did nearly every key move?Compute actual key movement under modulo 4 to 5, compare stable rings and logical slots, and separate owner mapping from physical migration.
- Part 2 of 5
Staff
We doubled the cache shards. Why did one shard get no relief?Diagnose one hot key versus a hot range or placement imbalance, then distinguish bounded-staleness read distribution from inventory-write authority.
- Part 3 of 5
Staff
The writes are spread evenly. Why does one customer's history need 80 queries?Choose physical placement from point-read and per-tenant ordered scan patterns, then manage whale tenants and the ordering cost of sub-bucketing.
- Part 4 of 5
Principal
The new shard is caught up. Can we send it writes now?Specify online shard migration with snapshot and CDC boundaries, tenant-scoped write fencing, stale-router rejection, and rollback after target writes.
- Part 5 of 5
Staff
One storage node died. Why did the rest of the cluster follow it?Diagnose feedback between lost capacity, replica repair I/O, health checks and rebalancing, then size and prioritize recovery under a durability deadline.