One vector per passage compresses the passage into one location in embedding space. That is efficient, but it can blur which words matched. A late-interaction model such as ColBERT stores contextualized vectors for many document tokens, then compares query-token vectors to document-token vectors at search time. A common scoring shape takes the best document-token match for each query token and sums those matches. The ColBERTv2 paper describes this multi-vector design and compression to reduce its footprint. It is a more expressive relevance signal than one passage vector, with a much larger index and more query-side work unless the implementation prunes aggressively.

I would ask which failures this fixes. If a query says “timeout 504 in client v3 after token refresh,” a single vector may retrieve generally relevant timeout pages while missing the exact combination. Token-level matches may rank the passage that covers more of those terms. But matching each term somewhere in a long document does not prove that the terms describe the same failure, that a negation applies, or that the version is current. Keep an exact lexical lane for error codes and API symbols. Treat version and permission as eligibility constraints. The final answer still needs claim-level evidence.

The cost estimate starts with number of indexed tokens, not number of documents. For 500 million documents, measure the chunk and token distribution, vectors per chunk, dimension and compression format, postings or centroid structures, replicas, rebuild time and network traffic. A published compression result on a research corpus is not this fleet's storage bill. Run a pilot on a representative shard with real ACL filters and update rates. Compare recall of answer-bearing spans, final grounded answers, p95 and p99 latency, memory, disk, ingestion cost and cost per useful completed question against the existing hybrid stack.

If the index is too large, there are intermediate choices. Use late interaction to rerank a lexical and single-vector shortlist, or enable it for a corpus where the slice gain is real. That gives up the ability to recover a candidate never found in the first stage, so measure first-stage recall separately. The PLAID paper explores pruning for efficient late-interaction retrieval, but its benchmark speedups are not a deployment guarantee here. If exact codes still drive the errors, a better lexical analyzer and field design may beat an expensive neural index. I would buy the representation only for a measured gain at the actual storage and latency budget.