Retrieval and RAG · Principal
Search code and docs when exact names and fuzzy questions mix
The question
Interview question
Design search for code and technical docs where queries mix exact API names, error strings, and fuzzy conceptual requests. Half the queries specify a product version.
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
First I want to know what a result means. A code symbol definition, an example that calls it, an API reference page, a release note, and a migration warning are different evidence. If someone asks “Why does ERR_SCHEMA_42 happen in 8.4?”, a beautifully similar page about schema validation in 9.0 is the wrong result. If they ask “How should I handle transient schema failures?”, exact string matching alone may miss the answer.
I would index source objects with durable identities: repository and commit or release for code, document and revision for docs, symbol and path when known, product version range, language, section, tenant or visibility, and a resolvable source location. Parsing needs to preserve code blocks, headings, tables, signatures, and nearby explanation. A chunk that contains an error code without its condition or a method name without its class is not useful evidence. Keep links from a chunk to its parent section and adjacent pieces so retrieval can expand the right span without embedding an entire manual.
The query planner extracts literal anchors before a model rewrites anything: backticked symbols, error strings, paths, versions, language, and any named product. It can still form a conceptual query for the fuzzy part. Run a lexical path that respects exact tokens and a semantic path over the permitted, version-compatible corpus. For a specific version, use a version range or release mapping with an explicit policy for undocumented versions. A page tagged 8.x is not necessarily valid for every 8.x release. Some documentation is current-only, some describes historical behavior, and code may have different branches. Resolve those meanings at ingestion rather than pretending string tags define compatibility.
I would merge the candidate lists by ranks or calibrated scores, not add raw BM25 and vector scores as though they share units. Elastic's RRF documentation gives a concrete rank-fusion mechanism. Preserve a few high-confidence exact matches even if the semantic branch dominates. Then rerank under a bounded budget with the whole question, version constraints, source type, and enough surrounding text to judge whether it answers the question. A query for an exact error code might need the diagnostic reference and the versioned fix, not six copies of the same error string.
Authorization comes before content reaches the model. The index may narrow candidates with tenant and document filters, but current access must also be checked at the evidence boundary if revocation can race index updates. If filtering leaves fewer candidates, search deeper within the authorized slice. Never let the restricted hit become a summary, score explanation, or citation that the user can infer from.
The citation should name the actual code commit or doc revision and a stable line or section range. Render the excerpt from that revision, then verify that the final claim is supported by that excerpt. A search hit's URL is not a citation guarantee. Line numbers can move on a new commit, and a code example can be illustrative rather than normative. For versioned answers, I would show the version context beside the citation so a later search visitor can tell what it applies to.
Suppose half the queries specify a version, as the question says. I would make that a first-class evaluation slice, not just another filter in an average. Build cases for exact symbol in the wrong version, renamed API, alias, old migration guide outranking a current reference, and a conceptual question where both versions share words but differ in behavior. Measure candidate recall, answerable evidence recall, exact identifier recall, citation support, and latency per slice. If version-specific recall is poor, broadening the semantic search without fixing metadata makes the system confidently wrong more often.
There is a harder follow-up: what if the right answer spans a release note and the implementation at a commit? Search should preserve their relationship and retrieve both, with dates and authority visible. If they disagree, don't let the reranker silently settle the conflict. An API contract may take precedence for expected behavior, while code at a specific deployed commit may explain observed behavior. The answer needs to say which question it is answering.
Continue reading
Related questions
Read beyond the question
Explore more retrieval and rag
Follow another question in this area, or return to the full Interview Prep index.
Browse this area →