Retrieval and RAG · Staff
The answer cites the right document, but the quoted paragraph moved. Which span is authoritative?
The question
Interview question
Yesterday the assistant quoted paragraph 18 of a policy PDF, revision 12. A reprocessing job finds a previously missed table, so that same paragraph becomes paragraph 27 in the new extraction. Later an editor changes the paragraph's meaning, but the public document URL stays the same. A reader clicks yesterday's citation and sees a different passage. What identity should a citation carry, and when can it be repaired automatically?
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
The public URL identifies a document location, not the exact evidence that supported yesterday's sentence. Paragraph number and byte offset can change when extraction changes. A quote can remain identical while its section heading or exception changes its meaning. I would store a citation as a relation among the answer claim, source document ID, immutable source revision, extracted span or quote, surrounding context, extraction version, and any location needed to render it. That lets us say exactly what the assistant saw then.
The W3C Web Annotation Data Model describes text position and text quote selectors. Position is brittle as content changes. A quote with surrounding context is a better reanchoring candidate, but it is not a proof of unchanged meaning. If the source PDF bytes are the same and only the parser's offsets shifted, compare the exact quote, nearby words, section and page image. A unique match can be reanchored to the new extraction while retaining the old source revision and recording the parser mapping. If there are two identical clauses under different sections, stop. Choosing the first match would quietly change the evidence.
If the PDF itself changed, do not overwrite yesterday's evidence pointer with the new paragraph. The historical answer should continue to cite revision 12 where retention and current access rules permit. A new answer can evaluate revision 13, including its effective date and changed meaning. If the product wants to show a correction beside an old answer, make it a new dated notice linked to the old one. The source corrected yesterday's data. Should the assistant rewrite yesterday's answer? covers corrected answers after source changes. Here the narrower failure is that a citation resolver points to a different span while the answer text appears unchanged.
There are privacy and retention edges. If revision 12 was deleted under a binding restriction, the service may not be allowed to display the archived bytes merely to keep a citation working. It can show that the historical citation is no longer available, and follow the applicable correction or takedown path. A content hash without retained bytes can detect that something differs, but cannot reconstruct the passage for a reader. A protected snapshot is useful only if retention is allowed.
I would test a table inserted before the quote, OCR whitespace changes, repeated boilerplate, a moved paragraph under a new heading, an actual wording change and an access revocation. Measure citation resolution to the intended span, not just “URL returns 200.” A user updates a document and search answers from the old revision asks why search is serving the wrong document revision. This question asks what a citation means when document identity stays fixed but the span representation moves. A relevant page is still an inadequate citation if it cannot lead the reader to the evidence that supported the claim.
Continue reading
Related questions
Read beyond the question
Explore more retrieval and rag
Follow another question in this area, or return to the full Interview Prep index.
Browse this area →