Context Engineering · Principal
The user corrected the agent's memory. Why does the old fact return next week?
The question
Interview question
A support agent remembers that a customer wants invoices sent to the finance mailbox. The customer corrects it: “That mailbox was closed. Use the billing portal. Do not email invoices.” The agent acknowledges the correction. Next week another run emails the old mailbox. The memory store contains the new preference, but a summary, a vector index, and an older paused workflow still contain the old one. Design the correction path. Then the same old preference is found in a team-shared memory used by several customers.
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
The system has two active versions of a fact and no rule that makes the correction win at the point of use. Adding a new embedding beside the old one can make the problem worse. Similarity search may return either version, or both, depending on phrasing. A fluent summary can turn the older one back into an instruction.
I would start by being precise about what the user corrected. The invoice destination is an action constraint, not a harmless writing preference. The new instruction has a subject, scope, effective time, source, and authority: this customer, invoice delivery, from now on, stated by an authorized account user. It says both “use the portal” and “do not email.” The durable policy or account settings service should own the actual destination and permitted channel. The agent's memory can help find that setting, but it must not be the final authority for sending an invoice.
Represent a memory claim as a versioned record with a stable logical key, such as customer plus invoice-delivery preference. Store the old value's provenance and status, the correction event, the new value, and the relation supersedes. Make the write conditional on the version observed, so two sessions cannot silently overwrite each other's edits. A correction should first make the old claim ineligible for new context and actions. Updating the vector index, compacted summaries, and cached context can follow, but the read path must already filter by the authoritative current version. An index refresh is not a revocation mechanism. A stale retrieval result can still be surfaced for historical explanation if permitted, clearly marked as superseded, but it must not be promoted into current instructions.
The paused workflow needs special treatment. Its checkpoint is evidence of what it saw earlier, not permission to execute that old plan. On resume, the runtime rebuilds the working context from current account settings and checks the destination again just before sending. If email is now prohibited, the old pending email cannot be sent, even if the model wrote “finance mailbox” into its last plan. If a send was already submitted before the correction, the system may not be able to undo it. Report that outcome separately and investigate exposure. The earlier state, context and memory question covers the broad lifecycle. This question is about conflicting versions and a live correction winning across every projection.
How do we find all copies? Keep derivation links from source turns and memory claim IDs to extracted memories, summaries, search entries, cached contexts, and workflows that used them. W3C PROV-O provides terms for revision and derivation, while the concrete index and cache invalidation contract is ours to build. A background reconciler can scan projections and old checkpoints for still-eligible references to the superseded claim. The release test should create an old checkpoint and a delayed memory extraction job, apply the correction, and then resume both. Neither may reintroduce the old instruction. Microsoft's agent memory documentation describes item operations and scoped deletion in its preview product, but a successful store update alone cannot prove this end-to-end behavior for our system.
The team-shared memory changes the answer. Maybe “send invoices to finance” was an organization default, not a personal preference. The customer correction might override it only for that customer's account. It should not silently change other customers' destinations. Model the scope and priority explicitly: organization default, account setting, authorized user instruction, and policy prohibition. When two authorized people in one account disagree, the agent should not pick the newest sentence. It should ask the account owner or use the governed setting. For the corrected customer, a lower-priority shared memory cannot override the account-specific rule. For other customers, it may still be relevant if valid.
There is also a difference between correction and deletion. If the customer asks to forget the old mailbox entirely, stop future use immediately and run the documented deletion process over retained derivatives. A superseded historical record may otherwise be kept under the product's retention policy to explain why an invoice was sent last month. Do not call that record current memory. The real success condition is simple to state but takes system work: every new answer and every action uses the current governed value, while the audit can still explain the past within its permission and retention limits.
Continue reading
Related questions
Read beyond the question
Explore more context engineering
Follow another question in this area, or return to the full Interview Prep index.
Browse this area →