Agent Architecture · Staff
The multi-agent demo is better, but production pays for it
The question
Interview question
A multi-agent design improves demo quality but triples tool calls and makes incident reconstruction difficult. Would you keep it? A single agent reaches the same result on most tasks.
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
I would not keep the multi-agent path for all traffic on this evidence. “Better demo” is a selection process. The demo may favor long tasks with clean handoffs and no contested tool output. The production decision is whether the extra agents improve a meaningful outcome on a defined slice enough to pay for their cost, latency, external effects, and debugging burden.
First make the comparison fair. Same task distribution, same model options and tool permissions, same deadline, same source snapshot, and equivalent opportunity to retrieve evidence. Count successful outcomes with their constraints, not just an evaluator's preference for a polished final answer. For a coding task, did tests pass and was the patch safe? For support, did the answer cite the right policy and avoid duplicate actions? Track p95 and p99 completion time, model tokens, tool calls, retries, rate-limit events, and cost per successful task. If multi-agent succeeds on 94 of 100 tasks and single-agent on 92, those numbers alone tell us little without uncertainty, task mix, and the severity of the two different failures. They are hypothetical numbers, not a claim about a product.
The threefold tool traffic matters beyond a bill. Parallel branches may hammer the same index, read inconsistent source revisions, produce overlapping actions, and make the slowest branch the completion bottleneck. If two children both propose a credit, the coordinator must deduplicate effects using a business operation identity, not pick the prettier explanation. A parent saying “delegate” doesn't make branch permissions or result provenance automatic.
The reconstruction problem is a product defect. I need run and branch IDs, parent-child edges, accepted attempts, tool arguments and outcomes, source revisions, authorization decisions, handoff summaries, and the precise evidence set used in the final answer. OpenAI's Agents SDK tracing illustrates tool and handoff spans. A trace does not by itself prove that every cited claim used the right branch result, so the assembly contract still needs immutable references. If we cannot explain a bad payment or a wrong citation after the fact, I would not run that path for consequential tasks.
The probe says a single agent reaches the same result on most tasks. I would route the common slice to it and identify a hard slice where parallel specialization changes the outcome: independent verification of a complex source conflict, deep code investigation with separable components, or a genuinely broad research task under a deadline. Predefine a trigger from information available before the outcome, such as task size and source diversity. Don't spend three times as much on every request and justify it with a few memorable wins.
Could a second agent act as a reviewer instead of parallel solver? Possibly. Evaluate whether an independent review catches real failures that a cheaper deterministic check or one additional retrieval would miss. The reviewer should see enough evidence to challenge the answer, and its dissent should have a resolution path. An agent agreeing with a sibling that shares the same flawed source is not independent assurance.
I would ship a limited cohort only after a reproducible evaluation shows a material improvement on the hard slice, the cost fits, and incidents can be reconstructed. Then monitor slice routing and rollback to the single-agent baseline if the gain disappears. If the multi-agent design cannot earn its complexity with a measured outcome, the simpler system wins on this workload.
Continue reading
Related questions
Read beyond the question
Explore more agent architecture
Follow another question in this area, or return to the full Interview Prep index.
Browse this area →