Calculate storage overhead, then interrogate placement
A 60 MB object under three-way replication occupies approximately three 60 MB full copies, or 180 MB. With a 6+3 code, split the data into six 10 MB data fragments and create three additional 10 MB coding fragments. That totals 90 MB. The exact encoding includes stripe, padding, metadata, and implementation overhead, especially for small objects, so this is a clean large-object approximation.
Now suppose all nine fragments are accidentally placed across just two physical racks. A single rack failure could erase more than three fragments and make the object unavailable despite the attractive 'three failures tolerated' description. Coding durability only follows when fragment placement matches the promised failure model. Independent hosts, racks, power domains, zones, and regions have different economics and failure correlations. It may be impossible or very expensive to place all nine fragments in independent availability zones, so do not assume the code automatically survives three zone failures.
Replication can be placed across three zones with one whole copy in each. It offers a simple way to serve reads locally from one surviving complete replica. An erasure-coded stripe may be distributed across hosts or racks within a zone, with separate cross-zone replication or other safeguards. Those are architecture choices, not properties of Reed-Solomon mathematics.
Read and repair paths are not equivalent
For a healthy read, an erasure-coded store might serve the six data fragments and concatenate their contents, possibly using coding reconstruction or caches depending on placement and implementation. It may need coordinated access to multiple fragment servers. For a lost fragment, a basic 6+3 reconstruction commonly reads six surviving fragments to rebuild one missing fragment. In the 60 MB example that can mean transferring 60 MB of surviving fragments to reconstruct a 10 MB replacement fragment. By comparison, replacing a lost full replica usually transfers one 60 MB complete object from one surviving replica, but the source and network shape are simpler.
That arithmetic is not the whole performance story. Different codes offer different repair locality. Some systems can avoid unnecessary network traffic for local failures using special code layouts. Reads may also be served from caches or reconstructed in parallel. The honest comparison uses measured tail latency, fanout, CPU encoding cost, and network bottlenecks under normal and degraded conditions.
Consider the failure rate of disks and the time required to repair each object. A design that saves half the raw bytes but causes a tenfold surge in cross-rack traffic during repair could make the cluster less safe during large incidents. Repair has to coexist with foreground reads and writes. The storage system needs bandwidth budgets, capacity headroom, and degraded-state admission rules. Otherwise a failed node can trigger repair traffic that saturates the next group of nodes, creating a correlated outage that the code theoretic tolerance did not predict.
Workload matters as much as durability
For large, mostly immutable backups and media, coding is attractive because it amortizes encoding, large stripes, and remote-fragment fetch costs. For tiny frequently updated metadata records, updates may require read-modify-write of stripes and parity, depending on implementation. Replicated storage can be more economical in latency and operational simplicity even when its raw-capacity cost is higher. A practical design often uses replication for small mutable metadata and erasure coding for large immutable data.
A second complication is the difference between availability and durability. An object may remain recoverable from six remaining fragments but have unacceptable read latency when several fragment servers are slow. The system may choose to serve degraded reads or declare the object temporarily unavailable. This is a user-visible service-level decision rather than a coding-theory footnote.
Before rolling out the change, I'd model failure domains, expected correlated incidents, latent corruption, repair-time distributions, and storage growth. Then I'd run controlled experiments: one disk out, one rack unavailable, an additional slow fragment, concurrent rebuilds, and large customer reads during repair. Measure usable capacity, p99 reads, time spent below redundancy target, and operator effort. Keep a migration and rollback strategy because transitioning 12 PB of live data is itself a risk.
My recommendation would be to pilot 6+3 for a carefully chosen immutable object tier, not rewrite the entire service overnight. The space saving is credible. The safety claim requires demonstrating that placement, repair, and degraded-read behavior satisfy the actual failure and latency objectives. The most experienced interview answer knows which part is algebra and which part still needs an experiment.
