Skip to content

We upgraded the inter-region link. Why is checkout still taking a second?

The Route Exists. Does the Request Arrive? · Part 6 of 6 · 6 min read

Read the worked explanation ↓

Interview question. A checkout service in region A calls a database-backed inventory service in region B. The private link has plenty of available bandwidth and transfers large files quickly. Yet an individual checkout spends more than a second waiting on the network. Its trace shows twelve dependent request-and-response exchanges over a path with roughly 90 ms round-trip time. Explain where the delay comes from and redesign the interaction without pretending a proxy can change the speed of light.

The first thing to ask is whether those exchanges depend on each other. A bandwidth graph tells us how many bytes can move over time, but a dependent request can't proceed until its predecessor returns. If the application waits on twelve sequential round trips, the wire time alone is at least 12 × 90 ms = 1,080 ms under the stated approximation, before remote processing, TLS setup, queuing or retries. Doubling bandwidth doesn't divide 1,080 by two.

This is a useful interview trap because teams often spend money on faster private connectivity when the delay is caused by protocol shape. A request carrying a few hundred bytes doesn't need much bandwidth. It needs the next result before it can send the next instruction. That difference explains why a database or remote filesystem can feel fast in bulk benchmarks and painfully slow under one chatty application workflow.

A chatty twelve-exchange checkout path costs twelve wide-area round trips, while one coarse request performs local inventory work near the data.
A chatty twelve-exchange checkout path costs twelve wide-area round trips, while one coarse request performs local inventory work near the data.
Think first

A checkout operation uses twelve synchronous inventory RPCs from region A to B. The RTT is 90 ms and all remote CPU spans total 30 ms. Propose a latency budget and redesign, then explain how the business should behave when B is unreachable for ten minutes.

Take a few minutes to form an approach, then compare it with the worked explanation.

Check the exercise numbers

Twelve dependent round trips plus 30 ms of total remote CPU give an approximate lower bound of 12 × 90 + 30 = 1,110 ms. One coarse remote operation with the same 30 ms of CPU would take about 90 + 30 = 120 ms before queueing, serialization, setup and retries: about 990 ms less WAN waiting. This assumes the original CPU work remains necessary and stays on the critical path; measure the actual trace after redesign.

Reveal the worked explanation

Reconstruct the actual dependency chain

Suppose checkout performs a lookup for inventory, then fetches an item row, then checks a reservation, then requests a discount adjustment and so on. It's tempting to say the code makes twelve queries, but the important detail is where those queries run. If a backend in region A opens a database connection directly to region B and makes twelve serial calls, each response must cross the inter-region path before the next call begins.

A single prepared statement or stored procedure executed on the database side might combine some operations. A domain API placed beside the data can run a local transaction that does its work within region B and return one coherent outcome to region A. The remote client still makes a request and receives a response, so one WAN round trip remains. We have changed the number of dependent WAN interactions, not made the remote region physically closer.

Now check correctness. If several inventory decisions must be atomic, moving the logic behind a coarse-grained API can preserve one authoritative transactional boundary. Blindly issuing twelve calls in parallel would be wrong when later operations depend on earlier results or when ordering matters to stock reservation. If calls are truly independent, controlled parallelism can reduce the critical path to approximately the slowest branch plus overhead, but that is an application property to prove, not an optimization the network can guess.

A proxy only helps when it changes the boundary

Someone suggests adding a proxy. Where should it run? A proxy next to the caller that relays the same twelve serial commands still pays the wide-area latency twelve times. It may provide connection pooling, authentication and retries, but it doesn't eliminate the underlying dependency chain.

A proxy or execution service co-located with the database can accept one coarse operation over the WAN, execute the twelve short exchanges locally, and return a compact result. That improves the architecture if the protocol supports batching and the execution semantics are safe. But if the proxy merely tunnels one request at a time and waits for another instruction from region A, nothing fundamental changes. The location and responsibility of the proxy matter more than its product name.

Reuse TLS and TCP connections to avoid unnecessary handshakes. Avoid repeated DNS resolution when connection pooling is safe. Compression may help very large payloads. Those optimizations are valuable, but they remove setup costs or bytes, not sequential WAN round trips. An answer that confuses them should be challenged in the interview.

What if the business wants regional independence?

The product team then adds a requirement: checkout in region A should continue if region B becomes unreachable for ten minutes. That changes the problem from pure latency to ownership and consistency. If region B alone owns authoritative inventory reservations, a network partition removes A's ability to guarantee new reservations without either an alternate authority or a predefined allocation of stock.

One option is to keep some inventory authority local through explicit per-region allocation. Another is to accept a pending order and complete reservation asynchronously when connectivity returns. Those choices change user experience and business invariants. We cannot simply cache the remote inventory number and continue decrementing it independently while claiming global no-oversell guarantees. Replicated snapshots can answer product browsing faster, but reservation correctness needs a separate protocol.

This lesson therefore ends with a design choice rather than a latency tweak. Which operations require strong, authoritative answers right now, and which can tolerate a queued or stale outcome? Read-only catalog details could use region-local materialized views. A stock reservation might remain a synchronous call to its owner, or the business may prefer pending confirmation. Group remote operations at the service boundary that understands the invariants.

Measure the new design honestly

I would annotate a distributed trace with number of cross-region request legs, observed path RTT, connection reuse, remote CPU time and queueing. Compare the old twelve-exchange design with one coarse reservation call, using the same workload and error budget. Capture the p95 and p99, not only a fast median. Then cut the inter-region path to see whether the failure contract matches what the product promised.

A successful redesign reduces the number of dependent round trips without weakening ordering, authorization or data correctness. More bandwidth cannot rescue a conversational protocol that asks a distant server twelve dependent questions. Fewer WAN conversations, clear ownership and explicit degraded behavior can.

Go deeper

A useful lower-bound model is T >= N_sequential × RTT + processing_critical_path, while real traces include transport setup, serialization, queuing, bandwidth delay and possibly retries. With 12 sequential RTTs at 90 ms, the network wait is at least roughly 1.08 seconds. Parallel operations reduce depth only when their dependencies allow it. Moving computation beside the authoritative data can collapse many local exchanges into one WAN request but does not remove the WAN from the synchronous operation. Offline availability requires different ownership or a deferred business contract, not merely an extra proxy.

Check your reasoning

  • Compute the 12 × 90 ms = 1,080 ms sequential RTT lower-bound example.
  • Identify which calls are dependent and which can be parallelized safely.
  • Explain why available bandwidth and connection pooling do not remove sequential RTTs.
  • Move coarse operation execution beside authoritative inventory state.
  • Choose an explicit region-isolation failure contract and validate it with traces.

Common mistake: Increasing private bandwidth tenfold makes twelve WAN round trips take one tenth the time.

Twelve dependent inventory RPCs cross a 90 ms WAN path. Which redesign reduces the round-trip floor while preserving inventory correctness?

Read the answer explanation

Convert to one authoritative remote operation with local data-side execution — One coarse request can reduce dependent WAN exchanges while preserving one authoritative inventory decision.

Apply it elsewhere

An analytics coordinator remotely opens, scans and closes thousands of small objects one at a time across regions. Explain whether prefetching, batching, parallelism or relocating the coordinator has the strongest effect, and where memory or ordering limits change the answer.

Compare your approach

Quantify the dependent RTT critical path, separate bandwidth and connection setup, move coarse work to data ownership, preserve inventory invariants, consider offline contracts and measure the actual cross-region call count.

One more constraint

If twelve remote calls are sequential and each requires a round trip, will upgrading available bandwidth from 1 Gbps to 10 Gbps remove their 1,080 ms round-trip floor?

Quick revision

Throughput is not the same as request latency. Twelve dependent exchanges over a 90 ms RTT path cost about 1.08 seconds of network waiting before processing. More bandwidth doesn't reduce those sequential round trips. Move related work beside the authoritative data, expose a coarse-grained operation, parallelize only independent requests and reuse connections for setup efficiency. Region outages require explicit ownership or pending-operation semantics, not a proxy that silently weakens inventory correctness.

References