DNS TTL is not a connection drain
Even after a client obtains the new IP, an established HTTP connection to the old IP can remain active. HTTP connection pooling, long-lived streams and application-specific DNS caches can retain earlier destination choices. A JVM, a browser, a sidecar proxy and a corporate resolver may have different cache policies. Some resolvers clamp unusually short TTLs, and some clients do not respect TTL behavior exactly. We should investigate those cases, not design for an impossible global cache flush.
There is another subtlety. During trouble reaching authoritative nameservers, a recursive resolver implementing serve-stale behavior may deliberately return expired records for resilience. RFC 8767 standardizes this pattern under specific conditions. That is a good availability feature in many incidents, but it weakens the idea that TTL expiry itself is a strict operational deadline for retiring an address.
The observed 20 percent could also reflect something other than DNS. Maybe one address family was updated and the other wasn't. Maybe a CNAME chain retains an old provider target. Maybe a portion of traffic is pinned to old connections, or one resolver's cache is behaving differently. I would log the destination IP that actually receives each request, compare it with sampled resolver answers, and determine whether the clients are making new connections to that IP or continuing existing ones. Those are different fixes.
Plan the migration around overlap, not a stopwatch
I would keep the old load balancer functional while gradually introducing the new one. Both destinations should serve compatible application versions, correct certificates, routing rules and access policies. If the service writes to durable state, old and new entry points must reach the same authoritative backend or an explicitly designed migration path. Running two API front doors must not accidentally split state or create duplicate ownership.
Days or hours before the change, depending on the previous TTL and observed client behavior, lower the TTL. Verify the low TTL from multiple independent recursive resolvers, not only by querying the authoritative provider. Then bring the new endpoint online and probe it directly before changing public answers. An initial canary using a small traffic segment or distinct test hostname is safer than relying on DNS records as fine-grained load balancing.
At cutover, change the relevant record sets and monitor new versus old destination traffic, error rates, request latency and connection age. Drain old instances at the application or load-balancer layer where possible, but continue accepting traffic to the old address during the compatibility window. If the old entry point can safely proxy or redirect requests to the new system, that may help, provided hostnames, certificate validation, authentication and non-idempotent requests remain correct. A bare HTTP redirect cannot repair clients that fail before making an HTTP request.
The rollback path matters. If the new balancer has a route defect, we can restore the old DNS answer, but many clients will still have the new address cached. Therefore rollback also requires keeping the new endpoint capable of serving or forwarding until those clients can recover. DNS rollback is another gradual transition, not an atomic undo button.
What if old servers are already gone?
Now the problem is containment. Restore the old address or a compatible listener as quickly as the infrastructure permits, while keeping the new address healthy. If the previous public IP cannot be reclaimed, determine whether the old provider can keep forwarding requests. Communicate to affected customers, collect destination-IP diagnostics and avoid a sequence of frantic DNS changes that generates additional cache populations.
A useful release gate is based on observed traffic, not an arbitrary multiple of a low TTL. I would require sustained low or zero traffic to the old destination across a suitably long observation period, consideration of long-lived clients, and explicit exceptions for embedded or managed clients with unusual caching. Some clients may be permanently misconfigured, so perfect zero may be unachievable. The team must make that residual risk visible.
For Staff-level interviews, the key architectural insight is that the old endpoint is part of the migration plan. A DNS record can steer future discoveries, but a reliable cutover requires overlap, compatibility, observability and rollback on both sides. That mindset becomes equally important when the next stage, HTTPS identity, has to remain valid across both destinations.
