Agent Architecture · Principal
Two investigation agents agree. What if both read the same stale document?
The question
Interview question
An incident agent asks two subagents to determine whether a new policy permits a refund. Both say yes and cite different paragraphs. The parent treats agreement as independent verification. Later you find both citations came from two chunks of one outdated PDF. How much confidence did the second agent really add?
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
Very little about the policy's current authority. Two reasoning processes can make independent mistakes, but here the decisive input was shared. Their agreement may show that the old document consistently says yes. It does not give two independent observations that the old document is the right policy. In evidence reasoning, dependence matters. A second vote based on the same source is not a second source. Research on correlated information in collective decisions examines how common observations change the value of apparent agreement. We do not need a numeric confidence formula for this incident. We need lineage.
I would inspect each answer's evidence graph: source system, document ID and revision, retrieval timestamp, chunk IDs, quotation spans, and any shared search result or parent-provided summary. “Different paragraphs” is not the same as “different authorities.” Did one agent read the other's conclusion? Did both receive the same cached context pack? Were they asked to verify the rule against the policy system of record, or simply answer from whatever search returned? The parent should compare evidence, not count votes. If both depend on the same stale PDF, it should say, “Both agents interpreted revision A as permitting the refund, but we have not established that A is current.”
For a consequential action, define an authoritative check. Fetch the currently effective policy revision and scope for the customer, product and time, validate the source version, and evaluate the specific exception. A second subagent can independently search for a conflicting amendment, or inspect the authoritative source through a different path. That may reveal missing evidence. Do not require two model calls when one well-specified lookup is enough. The design goal is a second failure mode, not a second personality.
There is a tricky case: two separate sites can copy the same erroneous policy text. Distinct URLs are not automatically independent either. Track upstream provenance where possible, including effective dates and document lineage. If the source of truth is unavailable, defer the refund or present a bounded uncertainty rather than turn agent agreement into permission. Two investigation agents disagree. What goes into the next context? handles what enters context when investigation agents disagree. This case is more deceptive because they agree for the same wrong reason. The real question is what observation would falsify their shared premise.
Continue reading
Related questions
Read beyond the question
Explore more agent architecture
Follow another question in this area, or search the complete Question Library.
Browse this area →Browse Question Library →