A score has meaning inside the system that produced it. BM25 values depend on term statistics and fields. A vector similarity may use a different range and orientation. A cross-encoder score has yet another scale. Sorting the raw numbers assumes they are measurements of the same thing, which we have not established. First make the candidate sets eligible: current tenant permissions, product version, document effective time and source scope. Fusion cannot repair a restricted or obsolete source that should not have entered the comparison.

For an initial robust merge, reciprocal rank fusion uses position within each source's ranked list and avoids treating raw scores as commensurate. Elasticsearch's RRF reference shows this rank-based approach. It is a useful baseline, not a proof that rank 1 from an optional incident source has the same authority as rank 1 from the policy source. Keep enough candidates from each required source before the final cutoff. Preserve source identity and coverage so the coordinator knows when a required source is absent or failed.

If we have representative relevance labels and want controlled weights, normalize or calibrate the component scores on an eligible candidate population, then evaluate the fused result by query type and source. Elastic's linear retriever documentation makes the normalization and weight choices explicit. Per-query min-max scaling can make the best hit in a weak list look strong relative to that list, and a single-item list needs defined handling when minimum equals maximum. The scale also changes when candidate windows change. So I would compare RRF, calibrated combination and source-aware reranking on answer-bearing recall after the context cutoff, unsupported claims and latency. Source authority and required coverage still need explicit rules.

Now add the new source. Its scores and rank distribution are unknown, and a single weak hit can enter a rank-fusion list at rank 1. Do not give it equal influence by default. Start it as a shadow or a bounded candidate lane, label where it actually supplies missing evidence, and canary by query slice. If the policy source times out, a well-merged list of code and incident results cannot justify “no policy exception exists.” The question about a timed-out policy source handles that missing-source case. Fusion is only meaningful after we know which eligible sources actually responded.