An agent architecture is not a model call surrounded by a loop. It is a system that turns a request into bounded decisions, executes effects, records what happened, and can explain or recover the run.

The model can propose the next action. It cannot be the source of truth for which actions already happened, which identity is acting, or whether an uncertain external write is safe to repeat.

Architecture diagram: a request passes through context assembly to a model proposal. The execution runtime validates and authorizes it, records durable intent, and then invokes a bounded tool. Results return to the runtime. Memory informs context, permissions constrain runtime actions, evaluation assesses outputs, and observability records the run.
The model is one decision component. The execution runtime owns progress, policy checks and external effects.

Follow one request through the system#

Consider an engineering agent asked to investigate a rising error rate, inspect deployment and service evidence, and prepare a change if the evidence supports one.

The useful architecture is not “send the prompt, then call tools.” It is a sequence of separate responsibilities:

request and trigger
  -> context request
  -> authorized evidence selection
  -> model decision
  -> runtime validation
  -> tool operation
  -> durable outcome
  -> evaluation and operational record

Each boundary answers a different question. What task is being attempted? Which evidence is allowed and fresh enough? What did the model propose? Is that proposal valid and authorized? Did an external effect happen? What should a replacement worker do if the answer is unknown?

If those questions collapse into a transcript, recovery becomes guesswork.

A model is not the agent runtime#

A model invocation maps input to output. It may classify, extract, plan, or produce a tool proposal. Even when the output is structured, it is still a proposal until trusted code validates it.

For example, a model may return:

{
  "decision": "inspect_deployment",
  "arguments": { "service": "billing-api", "window": "15m" }
}

The runtime must still check that the decision is in the allowed action set, the service is in scope, the caller has permission, the arguments satisfy the tool schema, and the request budget permits another call. A schema-valid answer can still be stale, unauthorized, irrelevant, or too expensive.

The planner can ask for another decision when the system state changes. The runtime decides when to ask, what state to include, which actions are available, and when the task must stop or wait for a person.

That separation prevents a prompt from becoming an accidental policy engine. It also lets the system replace a model without rewriting ownership, retry, audit, and authorization semantics.

Context assembly is a controlled read path#

The context layer turns a task and its current state into evidence for one decision. It can combine a compact run summary, recent observations, retrieved records, constraints, and the available tool descriptions. It should also record why each item was admitted.

Do authorization before sensitive content enters the model context. Filtering a denied result after retrieval is too late if the text has already crossed the model boundary. Freshness and authority are separate checks: a current ticket can still be untrusted, and an authoritative policy record can still be stale.

Context is not the durable workflow. It is a derived view for a particular model call. Build it again from durable state and selected evidence when the next decision is needed. Do not infer that a tool succeeded because its result appears in a conversation, or infer that an action did not happen because the transcript ends before a response.

The deeper design is in Designing the Context Layer for Production AI Agents. For the more specific distinction between reusable information and run progress, see Agent Memory vs Execution State.

Tools are adapters around effects#

A tool should expose a narrow operation, not a general-purpose escape hatch. Its contract describes validated inputs, identity requirements, permission checks, timeouts, side-effect behavior, idempotency support, reconciliation, and the evidence returned to the runtime.

Separate read tools from writes. A query that reads deployment status can usually be repeated. A tool that creates a pull request or changes a cloud setting may produce an external effect that must be tracked with a stable operation ID.

The runtime should persist the intended operation before calling the provider:

validate proposal
authorize exact action
persist operation intent
commit local transaction
call tool with stable operation_id
persist provider result or mark outcome unknown

That order makes a crash visible. It does not create a distributed transaction with the provider. There is still a point where the provider can commit and the local worker can fail before saving the response. The system must represent that uncertainty instead of translating a timeout into “nothing happened.”

The recovery protocol is worked through in Designing Long-Running AI Agents That Survive Failures, including the uncertain outcome between a provider action and its local record.

Durable state owns progress#

Store the current run state separately from the messages used to prompt the model. At minimum, a long-running run usually needs a stable run ID, current step, attempt history, pending operations, approvals, ownership or lease data, and a terminal outcome.

The durable record answers operational questions:

  • Which step is legal next?
  • Which tool operations are prepared, committed, rejected, or unknown?
  • Which worker currently owns the run?
  • Is approval bound to the exact action being executed?
  • What evidence allows recovery to continue without repeating an effect?

Conversation history can help the planner. It is not a reliable ledger. Agent Memory vs Execution State shows why these stores should not be substituted for each other.

Permissions travel with the action#

Identity and policy should be checked at the point where the system commits to an external effect. A role attached only to the model request does not explain which human, agent version, workload, and delegation chain authorized a particular payment or production change.

Carry a bounded action request through authorization and execution. Bind approval to the tool, normalized arguments, resource version, and relevant policy version. Recheck the binding immediately before the effect; the model may have proposed a different action after approval.

See Designing Identity and Delegation for Production AI Agents for the identity chain and An Allow Is Not a Permit for the runtime enforcement boundary.

Memory is not a second workflow database#

Memory can preserve a useful preference or a verified fact across tasks. It should not be used to record that a refund is currently halfway through execution, that a tool call has already been sent, or which worker has the lease.

Memory needs its own write policy: what is useful later, how it was learned, who can access it, when it expires, and how a person can correct it. Execution state needs transactional transitions and recovery semantics. Those are different consistency needs.

For the architecture of shared, versioned agent procedures, see A Skill Is Not a File. It Is a Deploy. A skill may provide instructions and tools, but it does not replace the run ledger or permission checks.

Evaluation and observability answer different questions#

Evaluation asks whether the system made a good decision for a known case. It can test retrieval evidence, tool selection, argument quality, policy adherence, and final outcomes. An overall answer score hides which boundary failed, so retain intermediate evidence and evaluate the path as well as the result.

Observability helps explain a particular production run. Keep a logical run ID across model calls, tool calls, retries, approvals, and worker replacements. Record context manifests, model and tool versions, operation IDs, policy outcomes, state transitions, and reconciliation results. Do not use a trace ID as the only run identity if one logical task can span multiple traces.

Designing Observability for Production AI Agents works through the operational record. The article Your RAG Evals Are Measuring the Wrong Thing shows why the final answer alone is not enough evidence.

Do not split into multiple agents by default#

One runtime with explicit steps is easier to reason about when work is sequential and shares one permission boundary. Multiple agents may help when work can be partitioned independently, needs genuinely separate capabilities, or benefits from isolated context and ownership.

The split adds coordination state: task ownership, shared evidence versions, cancellation, partial completion, conflicting results, and failure propagation. If two agents can write to the same resource, the design still needs one authority that serializes or rejects conflicting effects. Delegation does not remove the need for durable execution state.

Start with a single logical run and explicit tools. Add a coordinator and delegated runs only when measurements show that parallelism, specialization, or isolation is worth the extra state and operational burden.

Keep the boundaries testable#

A useful first design review asks whether the system can demonstrate these properties:

  1. A model response cannot bypass runtime validation or permissions.
  2. Every external write has a durable operation identity.
  3. An unknown result is reconciled before a potentially duplicate action.
  4. A retry can continue from stored state without reconstructing truth from chat history.
  5. Evaluation can identify whether context, planning, tools, policy, or recovery caused a failure.
  6. An operator can follow one logical run across workers and traces.

The model may be nondeterministic. The boundaries around its decisions still need explicit owners, durable state, and observable outcomes.