The 404 tells me one serving path no longer returns the page. It says very little about the rest of the AI system. I would take the quoted paragraph and its citation ID, find the source document version that produced it, and follow its descendants. We may have an extracted text blob, several chunks, lexical postings, vectors with metadata, a reranker candidate cache, assembled context, a final answer cache, a conversation summary, traces, and backups. Some contain the text. Others only contain a vector or an identifier, but they can still point to a copy that does.

One source version fans out into extracted text, indexes, answer caches, and retained records. A deletion tombstone blocks serving immediately while a tracked purge removes each derived copy. Backups follow their retention and restore procedure.
Block new disclosure first. Track the later purge separately.

I would ask what “deleted” means in this product. Is it an author removing a page, an administrator requesting erasure of the bytes, a tenant leaving, or an access change? Those lead to different treatment of history and backups. We should not invent a single universal retention promise. The source owner needs to define the event, scope, and deadline for stopping disclosure, then the storage owner needs to define what physical removal and backup handling can honestly guarantee.

For the serving path, the source ID and version need a durable tombstone or invalidation record. The answer and context builders check that record before using a candidate, even if search returns it from a stale index. A final answer cache that already contains the paragraph cannot be served merely because the query embedding is similar. Either invalidate entries by their source dependency or keep a reverse map from source version to derived cache keys. If that dependency was never recorded, disable the affected cache namespace while investigating. An expiry of several hours is not a safe answer to a shorter stop-serving requirement.

Then purge the derived copies. Use lineage from source version to extraction job, chunk IDs, index generation, object storage key, and cache entries. Delete from the lexical and vector indexes, wait for search visibility, and verify with exact text, source ID, and a semantic query that previously found the document. A delete operation returning success does not prove every shard and replica stopped serving it. Elastic's delete-by-query documentation, for example, describes snapshot and version-conflict behavior. If the document changed during the delete, an unexamined conflict can leave a version behind.

There is a race with ingestion. A delayed connector event can reinsert an older version after the purge. Tombstones therefore need monotonic source versions or another ordering rule at the index writer. An old upsert must lose to a newer deletion. I would test deletion while an old backfill, retry, and generation cutover are all running. Also check copied content in another corpus. If a second document is genuinely independent, the first source's tombstone does not automatically define its deletion policy, but a duplicated extract with the same lineage should be found.

The traces and backups are the awkward part. Do we log full prompts and model responses? Did a third party receive the text? What is retained in a diagnostic snapshot? These are inventory and contractual questions, not things I would guess in an interview. For an immediate incident, restrict access to the retained copies and stop new logging of this content. For longer term removal, apply the documented retention and restore process. If a backup still contains the old bytes, restoring it must replay the tombstone before that data becomes queryable. In a versioned object store, a plain delete may only create a delete marker while older versions remain. Amazon S3 documents this behavior. That is one example of why “the URL is gone” does not prove erasure.

The interviewer changes the fact: the document is live, but this user lost access. Now deleting everyone's chunks would damage other users' search. The right immediate action is current authorization at every disclosure path, including cached answers and citation clicks. A cache keyed only by query text or embedding is unsafe for private material. It needs an identity and permission scope that is valid at serve time, or it should contain source IDs and rerun authorization before fetching text. If a response has already streamed to this user before the revocation, we cannot take those tokens back. We can prevent new disclosure from the effective revocation point.

I would close the incident with two different proofs. For a true source deletion: a source-version inventory showing every derived serving copy removed or blocked, plus a restore test that does not resurrect it. For an access revocation: negative tests for the affected principal across search, answer cache, logs exposed through product features, and citation open, while another authorized principal still gets the live page. Mixing those cases is how a system either leaks content or deletes the wrong thing.

Sources for the implementation examples: Elastic delete by query and Amazon S3 delete markers. The deletion ledger and serving gate are proposed system design, not features guaranteed by those products.