I would stop treating the six slots as the first six rows of a ranking. The goal is a sufficient, permitted evidence set for this question. One passage might state the rule, another its exception, and a third the product version. Six similar paragraphs from the same manual can have a better average relevance score and still fail to answer.

Before packing, filter by tenant and current permission. Then preserve exact identifiers, source authority, version, effective date, and document relationships. Similarity and a reranker score help find candidates. They do not establish that a passage is allowed, current, or sufficient. A high-ranked summary might point to the right source while omitting the qualifying line. The selector should be able to expand to the parent paragraph or table and spend tokens on that context instead of keeping a sentence that cannot stand on its own.

Imagine the question is “Can version 8.4 roll back a data migration in the EU region?” The top hit is a current release note about the 8.4 migration feature, but it says nothing about rollback. A lower-ranked runbook describes rollback and another passage limits it for EU deployments. I want those two pieces and the relevant version scope. If the runbook points to a more recent exception, follow that relationship before answering. This may require another retrieval step, not a clever permutation of the first eighty results.

At selection time I would compare a candidate's marginal value to what is already chosen. Does it add a missing condition, an independent source, an exact error string, or a contradiction we must resolve? Or does it repeat a chosen chunk with slightly different wording? A per-document cap can prevent one long manual from filling all six slots, but it should not forbid two adjacent pieces from the same document when both are needed. Token budget matters more than the literal count of six because a table and its headers may cost more than a short paragraph.

The top passage is current but lacks the answer. I would keep it only if it establishes a needed premise. “Current” is not equivalent to “answers the question.” If the other candidates do not supply the missing rollback condition, the system should search using the version, region, and rollback terms, perhaps following a link from the release note. If that still fails, ask for a missing detail or say the available evidence is insufficient. The model should not fill the gap from a plausible older runbook.

There is a stopping decision too. A system that always spends all six slots can make a supported answer worse by injecting weak conflicting text. A system that stops after one good paragraph can miss an exception. I would test whether the selected set covers the answer's material claims and known qualifiers, using claim-level labels from human-reviewed cases. That test is imperfect, so high-impact queries with unresolved authority conflict need an abstain or review path. “Enough evidence” should be measured against the task, not the model's own confidence sentence.

For evaluation, record where the required supporting spans lived among the eighty, what the selector chose, and whether a later retrieval could find the missing piece. Score evidence-set recall under a fixed token budget, citation support, false abstention, unsupported claims, and latency. Break out versioned questions, two-document questions, and queries with many duplicates. If the oracle spans were never in the eighty, changing the context selector cannot repair first-stage retrieval. If they were present and discarded, increasing vector top-k is the wrong fix.