Distributed Systems & System Design
Reason through the system.
Then through its failures.
Prepare for Senior, Staff and Principal interviews with production scenarios and worked engineering explanations. Follow connected decisions in a learning series, or find the question behind a specific failure.
Start with a connected problem
Each episode changes a constraint. Carry your reasoning forward as the failure boundary moves.
5 episodes · Senior / Staff / Principal
One order, five failure boundaries
Follow one order through checkout retries, event publication, redelivery, historical replay and regional failover.
Follow this series →4 episodes · Staff / Senior
Where the time goes
Five engineering questions about network latency, load distribution, constrained concurrency, queueing, and retry overload.
Follow this series →5 episodes · Senior / Staff
Inside the database
Five distinct storage-engine questions from append-only files to durable WAL, B-tree splits, LSM runs, and compaction stalls.
Follow this series →Browse by engineering subject
A few questions to think through
Reliability & Failure Recovery · Senior
The customer clicked once. Why are there two orders?An HTTP timeout leaves the outcome unknown. Stable operation identity and a database constraint turn retries into one business decision.
Database Internals & Transactions · Staff
COMMIT succeeded. What exactly survived the crash?Defining a local durable-commit boundary and the role of WAL ordering, fsync, checkpoints, torn pages, and recovery under process versus system failure.
Replication, Consistency & Consensus · Staff
The lease expired. Why can its previous owner still write?Distinguish lease expiration from the physical ability to write, and enforce non-regression with destination-side fencing.
Explore the learning series
5 episodes · Senior / Staff / Principal
One order, five failure boundaries
Follow one order through checkout retries, event publication, redelivery, historical replay and regional failover.
Follow this series →4 episodes · Staff / Senior
Where the time goes
Five engineering questions about network latency, load distribution, constrained concurrency, queueing, and retry overload.
Follow this series →5 episodes · Senior / Staff
Inside the database
Five distinct storage-engine questions from append-only files to durable WAL, B-tree splits, LSM runs, and compaction stalls.
Follow this series →5 episodes · Senior / Staff
The database has a plan
Five original interview lessons on access paths, composite sort order, parameter-sensitive plans, hash join spills and MVCC visibility.
Follow this series →5 episodes · Senior / Staff
When transactions disagree
Five original interview lessons on lost updates, MVCC history, write skew, deadlocks, and serializable retry pressure.
Follow this series →5 episodes · Senior / Staff / Principal
The shard that moved
Five interview lessons on placement stability, hot keys, range-locality, live shard ownership, and repair cascades.
Follow this series →4 episodes · Staff / Senior
The copies disagree
Five connected lessons on WAL replay visibility, session freshness, quorum boundaries, tombstone repair, and replica fanout.
Follow this series →5 episodes · Staff / Principal
Who gets to decide?
Five lessons on Raft elections, current-term commitment, fencing, safe reads and changing voters.
Follow this series →6 episodes · Senior / Staff / Principal
The cache was fast until it wasn't
Six lessons about stale fills, expired hot keys, miss amplification, negative caches, privacy and cold recovery.
Follow this series →5 episodes · Senior / Staff / Principal
The stream keeps moving
Six lessons about producer buffering, partition expansion, hot consumers, poison records, retention gaps and broker fairness.
Follow this series →6 episodes · Senior / Staff / Principal
The Batch Finished. What Did We Actually Produce?
Six technical episodes on shuffle, skew, joins, data layout, atomic snapshots and trustworthy analytics.
Follow this series →5 episodes · Senior / Staff
The Query Changed the Data Model
Six connected lessons on key-value access, alternate identities, document modeling, change streams and Cassandra query tables.
Follow this series →6 episodes · Staff
The Numbers Were There. The Answers Weren't.
Six connected lessons about telemetry signals, series identities, compression, rollups, percentiles and query overload.
Follow this series →6 episodes · Senior / Staff / Principal
Search Returned an Answer. Was It the Right One?
Six connected lessons on inverted indexes, relevance, refresh semantics, distributed top-K, faceted counts and live reindexing.
Follow this series →6 episodes · Staff / Principal
The Network Was Up. The Requests Were Not.
Six networking incidents connecting TCP windows, packet loss, listen queues, NAT exhaustion, tunnel MTUs and gray failures.
Follow this series →6 episodes · Senior / Staff
The Connection Is More Than an IP
Six networking investigations covering DNS, safe address cutovers, HTTPS identity, TLS setup, HTTP transport blocking and QUIC mobility.
Follow this series →6 episodes · Senior / Principal / Staff
The Route Exists. Does the Request Arrive?
Six investigations into private network paths, multi-region topology, BGP outages, anycast session ownership, IPv6 rollout and the WAN latency tax.
Follow this series →6 episodes · Staff / Senior / Principal
The Machine Had More Resources. Why Was It Slower?
Six investigations into CPU quotas, JVM pauses, lock contention, NUMA, memory pressure and zero-copy I/O.
Follow this series →5 episodes · Senior / Staff / Principal
The Object Exists. Is It Safe?
Six investigations into storage interfaces, multipart publication, object consistency, erasure coding, silent corruption and safe reclamation.
Follow this series →6 episodes · Senior / Staff / Principal
The File Left the Origin. Was It Safe to Deliver?
Six connected investigations into direct uploads, image transformations, CDN tiers, stale HTTP content, edge policies and global updates.
Follow this series →6 episodes · Staff / Senior / Principal
The Backup Succeeded. Can We Recover?
Six connected investigations into recovery targets, cross-system consistency, point-in-time recovery, compromised backups, restore bottlenecks and DR proof.
Follow this series →6 episodes · Staff / Senior / Principal
The Dashboard Was Green. The System Was Not.
Six connected production investigations into async traces, sampling bias, silent data loss, SLO burn rates, telemetry outages, and incident experiments.
Follow this series →5 episodes · Senior / Staff / Principal
Who Gets In When the API Is Full?
Six connected design decisions about gateways, temporal rate limits, global quotas, adaptive concurrency, fair work and degraded security policy.
Follow this series →5 episodes · Senior / Staff / Principal
The Failure Should Have Stopped There
Six connected engineering decisions about circuit breakers, bulkheads, cascading failures, cells, shuffle sharding and metastable recovery.
Follow this series →6 episodes · Senior / Staff / Principal
The Config Changed. Who Told the Fleet?
Six connected engineering decisions about coordination on the request path, stale discovery, config rollouts, feature flags, routing safety and controller convergence.
Follow this series →