Retrieval and RAG · Staff
The customer typed the exact incident ID. Why did keyword search miss it?
The question
Interview question
The document contains `INC-42-B`, and the query contains exactly `INC-42-B`. Yet the keyword branch returns unrelated documents about `INC` and `B`, while `42` disappears from the analyzed terms and the correct incident is absent from the first page. How can exact text fail an exact-looking search?
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
Because a full-text query is not necessarily an exact-term lookup. The index analyzer may split at punctuation or discard numbers, lowercase terms, or use a different analyzer from the query path. For example, Elasticsearch's simple analyzer splits at non-letters and removes numbers. That is useful for natural words and destructive for an incident ID. A document containing the visible string is not proof that the index stored that string as one searchable term.
I would run the analyzer on the document field and on the query, print the tokens, then inspect the generated query against the mapping. If the identifier has its own structured field, index a canonical keyword representation and query that field exactly, with a carefully defined case and punctuation policy. Elasticsearch's keyword analyzer preserves the whole input as one token. Keep an analyzed text field as well so a user can still search the surrounding incident description. Make exact-ID matches eligible for a dedicated retrieval path before any semantic reranker or fusion step can bury them.
Canonicalization needs discipline. INC-42-B, inc-42-b and INC 42 B may or may not be the same ID in the source system. Normalize according to the identifier contract, not a generic lowercasing rule, and verify that two distinct IDs never collapse. Test the actual query API, not just the analyzer in isolation. A filter or permission check may still remove a valid hit after exact matching.
If an interviewer says BM25 should rank the exact match highest, ask what terms BM25 actually saw. The scoring algorithm cannot recover a full identifier that the analyzer discarded. This is a different failure from a hybrid merger demoting a correct keyword hit that it did retrieve.
Continue reading
Related questions
Read beyond the question
Explore more retrieval and rag
Follow another question in this area, or search the complete Question Library.
Browse this area →Browse Question Library →