Stateless MCP Didn't Delete Your State: Designing Production MCP Infrastructure
Stateless MCP removes hidden protocol sessions, not application state. Production infrastructure still needs explicit interaction state, durable Tasks, identity, business effect tracking, routing, admission control, and failover.
On this page
An agent needs to call payments.refund for payment P9831. The refund amount is $480 and the agent is acting for a user in tenant T42.
In a small setup, the MCP client knows the payment server and calls the tool directly. In a larger environment there may be hundreds of MCP servers, thousands of tools, different tool versions, multiple regions, several identity providers, long running operations, and agents acting for users with different permissions.
Before this refund reaches the payment system, the platform has to find the tool, authenticate the caller, carry the delegated user identity, apply policy, check available capacity, select a compatible server version, authorize access to P9831, and make sure the same refund cannot be created twice if execution fails halfway through.
This design assumes remote MCP over HTTP using the 2026-07-28 revision. That revision removed the initialization handshake and protocol session from the stateless request path, exposed routing metadata over HTTP, formalized Multi Round Trip Requests, and moved durable long running execution into Tasks. The changes make MCP easier to distribute across ordinary HTTP infrastructure, but they also make the remaining state much more explicit. MCP July 2026 specification changes
Stateless MCP still has four kinds of state#
The old protocol session is no longer where the server keeps hidden state for the request path. That does not mean an MCP based system has no state.
I would separate four kinds.
Protocol state
No hidden connection session
required for normal requests
Interaction state
MRTR requestState
Durable asynchronous state
MCP Task
Business effect state
operation_id
plus external system state
These states belong to different systems and have different lifetimes.
requestState may exist only while the client obtains approval or another piece of information. An MCP Task may live for hours. A refund created in a payment system may exist for years. None of these should depend on the lifetime of the MCP server process that happened to receive the first request.
Stateless MCP removes hidden protocol session state, but interaction state, durable Task state, and external business effect state still have different owners, lifetimes, and recovery rules.
Open diagram
Treating all four as one generic agent session brings back the coupling that the stateless protocol removed.
This separation is the basis for the rest of the architecture.
Separate the control plane from live tool execution#
Live MCP traffic belongs in a data plane. Server registration, tool metadata, schemas, authorization policy, ownership, and rollout state belong in a control plane.
Production MCP infrastructure separates live tool execution from control-plane metadata. Gateways route from locally validated snapshots while registration, tool schemas, policy, rollout state, and emergency revocation are managed separately.
Open diagram
A server record can contain:
McpServer {
server_id
server_version
protocol_versions
endpoint
region
owner
auth_policy
health_status
}
A tool record needs execution metadata too:
McpTool {
tool_name
server_id
schema_version
input_schema_hash
effect_type
required_scope
timeout_policy
task_capable
status
}
I would keep the authoritative registry in a transactional store because registration, schema changes, ownership, and rollout state need predictable updates. Gateways should not query that database for every call. They consume a validated registry snapshot and keep routing data locally.
If the registry store becomes unavailable, existing traffic can continue from the last known valid snapshot for a bounded period while registration and rollout changes stop.
Tool visibility now needs an explicit enterprise layer#
There are two discovery problems.
The platform first needs to know that an MCP endpoint exists. That normally comes from service onboarding, deployment metadata, an internal service registry, or another infrastructure system.
Once the endpoint is known, MCP can describe the capabilities available from that server.
Platform onboarding
|
v
Known MCP endpoint
|
v
server/discover
|
v
tools/list
|
v
Enterprise Tool Catalog
Servers implementing the current revision support server/discover, while clients do not have to invoke discovery synchronously for every request. A catalog controller can use it during onboarding and refresh rather than putting discovery into the normal call path.
The stateless model also changes how tool visibility should be thought about.
Tool lists are no longer supposed to quietly depend on hidden per connection state. If two principals should see different capabilities, that difference needs to be represented explicitly through authorization scope, catalog policy, or another visible input.
That means the enterprise layer increasingly owns the effective view of available tools.
A cached catalog therefore needs more than tool_name.
CatalogCacheKey {
tenant
principal_scope
catalog_version
server_version
}
A catalog produced for an administrative agent cannot be reused for an agent with smaller permissions.
The current specification also allows list responses to expose cache hints such as ttlMs and cacheScope, which are useful inputs to this cache rather than a replacement for permission aware catalog design. MCP July 2026 changes
Keep the control plane out of the normal request path#
A live tool call should not query the registry database, ownership service, schema service, and policy store before it can be routed.
Compile the required control plane state into something the gateway can evaluate locally.
payments.refund
server_pool = payments_mcp_v12
region = us_east
effect_type = WRITE
auth_policy = payment_refund_v4
concurrency_class = PAYMENT_WRITE
schema_version = 12
A control plane update produces a new snapshot. Gateway instances validate it and switch versions atomically.
Normal propagation can tolerate some delay. Emergency revocation cannot.
If payments.refund is found to be unsafe, waiting several minutes for catalog caches and routing snapshots to expire is not acceptable. Keep a small revocation path that can disable a tool, server, or server version faster than the ordinary refresh cycle.
This also gives the platform a clear answer during control plane failure. Existing valid routes continue. Configuration changes pause.
Follow one refund through the system#
For payments.refund, the request path looks like this:
Agent Runtime
|
v
MCP Client
|
v
Gateway validates protocol metadata
|
v
Authenticate caller
|
v
Resolve payments.refund
from local catalog snapshot
|
v
Check tenant and agent policy
|
v
Check admission limits
|
v
Select compatible server pool
|
v
MCP Server validates request
|
v
Authorize P9831
for the delegated user
|
v
Create operation identity
|
v
Payment Provider
|
v
Record or reconcile result
|
v
Return MCP response
The gateway handles platform wide concerns such as routing, tool availability, tenant policy, and quotas.
The MCP server still validates the request and performs the authorization needed for the actual resource being changed. The payment system remains the final authority on whether P9831 may be refunded.
I would not make the gateway the only security boundary.
Validate MCP routing metadata against the request#
The new HTTP path exposes information such as:
MCP-Protocol-Version: 2026-07-28
Mcp-Method: tools/call
Mcp-Name: payments.refund
That allows load balancers and gateways to route and apply coarse policy without fully interpreting every JSON RPC body. MCP July 2026 changes
Those headers only become useful policy inputs if they agree with the actual request.
This must fail:
Mcp-Name: payments.lookup
body:
tools/call payments.refund
Otherwise the caller could try to obtain routing or authorization intended for one operation while invoking another.
The protocol metadata improves HTTP infrastructure integration. It does not replace normal request validation.
Carry user identity and agent identity separately#
One tool call can involve several principals.
Human User
|
v
Agent Principal
|
v
Agent Workload
|
v
MCP Gateway
|
v
MCP Server
|
v
Protected Resource
At minimum I would carry:
user_subject
agent_principal
workload_identity
tenant
delegated_scope
The agent may be permitted to invoke payments.refund, while the user may only be able to modify payments belonging to one account.
Both conditions need to hold.
can_execute(
user = U1841,
agent = reconciliation_agent_v12,
tool = payments.refund,
resource = P9831,
amount = 480
)
This is much narrower than assigning the agent a broad payments role.
The MCP roadmap published in August explicitly calls out agent identity, delegation, workload identity federation, token exchange, and enterprise security as active areas of work. MCP roadmap
Keep OAuth resource boundaries intact#
For an authenticated remote MCP server, the incoming access token is intended for that MCP resource.
If the MCP server then calls the payment API, it must not simply pass that token downstream.
User and Agent
|
| token
| audience = MCP Server
v
MCP Server
|
| validate audience
|
| obtain separate
| downstream credential
v
Payment API
MCP authorization guidance explicitly rejects token passthrough. The MCP server needs a credential intended for the downstream resource rather than turning possession of an MCP token into authority over everything behind that server. MCP authorization specification
This matters even more when one MCP server fronts several protected APIs.
A protected MCP call keeps user, agent, and workload identity separate while preserving OAuth resource boundaries. Platform policy is enforced at the gateway, resource authorization remains with the MCP server and protected system, and downstream APIs receive their own credentials.
Open diagram
Use MRTR for interaction inside the current request#
Suppose refunds above $250 require confirmation.
The server does not need to keep one server instance waiting while the user responds.
payments.refund
|
v
Policy requires confirmation
|
v
input_required
requestState
|
v
Client obtains approval
|
v
retry tools/call
inputResponses
requestState
|
v
Any compatible server instance
can continue
The client treats requestState as opaque and sends it back with the follow up request.
This creates an important security boundary because the state travelled through a client that cannot be trusted to preserve it correctly. If the state affects authorization or execution, protect its integrity and bind it to the principal, original request, and expiry.
A requestState issued for user U1841 should not be reusable by another principal. A state issued for refund P9831 should not authorize refund P7742.
Replay and cross user reuse need to fail.
The MRTR design calls out these risks directly. MCP MRTR SEP
MRTR is useful when the current logical operation needs another piece of input. Work that continues independently for minutes or hours belongs in Tasks.
MRTR carries short-lived interaction state when a request needs more input, while Tasks move long-running execution into durable state that survives the MCP server instance handling the initial request.
Open diagram
Use Tasks for durable asynchronous work#
A repository analysis, warehouse export, migration, or deployment should survive the MCP frontend that accepted the original request.
tools/call
|
v
MCP Server
|
v
Create durable Task
|
v
return task_id
|
v
Task Store
|
v
Worker
|
v
External System
There is an important ordering requirement here.
The server must not return the task handle until the task has been durably created and tasks/get for that identifier can resolve it.
If the task store is eventually consistent, the response has to wait until this visibility condition is true. Otherwise the client can receive a valid task ID and immediately get a not found response while polling it.
That behavior is part of the published Tasks extension rather than an implementation preference. MCP Tasks extension
Keep MCP Task status separate from internal execution state#
The protocol exposes a deliberately small task state machine:
working
input_required
completed
failed
cancelled
The implementation usually needs more detail:
QUEUED
RUNNING
WAITING_FOR_DEPENDENCY
WAITING_FOR_RETRY
EXTERNAL_STATUS_UNKNOWN
RECOVERING
Do not expose every worker condition as another protocol status.
A task record can keep both views.
TaskRecord {
task_id
tenant_id
principal
tool_name
operation_id
mcp_status
internal_state
created_at
updated_at
expires_at
poll_interval_ms
result_reference
worker_owner
fence_token
version
}
The distinction around failed matters too.
A tool can complete normally and return a result containing isError: true. That does not necessarily make the task status failed.
At the protocol level, failed represents failure of the JSON RPC execution. A completed tool execution can still return a tool level error result. MCP Tasks extension
That distinction prevents application errors and protocol execution failures from being collapsed into one state.
Task polling changes the storage design#
Suppose the system has:
100,000 active tasks
and clients poll every:
5 seconds
That produces:
20,000 tasks/get requests / second
before ordinary tool calls are included.
At that point the Task API is no longer a small metadata service.
tasks/get should be a cheap point lookup by task_id.
Do not reconstruct the current task state by replaying a workflow history on every poll.
Task API
|
+---------+---------+
| |
tasks/get tasks/update
| |
v v
Point read path State transition
| |
v v
Cache or replica Authoritative store
The authoritative store can be relational or key value based. The useful property is that current task state is directly addressable by task_id, while updates use a version or compare and set check.
A small cache can absorb repeated reads for tasks whose state is unchanged. Cache invalidation matters most for transitions such as input_required and terminal states because clients need those changes to become visible predictably.
Read replicas can help if their replication lag remains below the visibility the protocol and product need. Routing task creation to a primary and immediately polling a lagging replica can recreate the same read after write problem that the durability rule was intended to prevent.
The protocol exposes pollIntervalMs so the server can tell clients how often they should poll, and clients are expected to respect it. Servers may rate limit clients that poll more aggressively. MCP Tasks extension
At hundreds of thousands or millions of active tasks, subscriptions stop being a small optimization. Continuous polling produces read traffic even when almost every task is unchanged. Notification subscriptions can remove a large part of that amplification for clients that support them.
The point lookup still remains useful for reconnect, recovery, and clients that cannot maintain subscriptions.
Keep external business effects separate from protocol requests#
The JSON RPC request identifier correlates a protocol request and response. It is not enough to identify an external business action across recovery.
Suppose the payment provider commits the refund and the MCP server crashes before returning the response.
The next attempt should still represent the same logical refund.
operation_id = R9831_S14_REFUND
If the payment provider supports idempotency, the adapter uses that identity again. If it does not, the adapter needs a reconciliation path that can determine whether the original refund already committed.
An unknown result is reconciled before another external write is created.
The detailed design for business effect control goes deeper than this article needs. The important MCP infrastructure point is that the lifetime of the business action is independent of the JSON RPC attempt and independent of the server instance processing it.
Admission control should understand the tool#
Global MCP request rate is useful but too coarse for operations with different cost and risk.
customers.lookup
max_concurrency = 1000
payments.refund
max_concurrency = 50
warehouse.export
max_concurrency = 5
The gateway can enforce tenant limits, agent concurrency, server concurrency, tool concurrency, and downstream provider quotas.
Individual MCP servers still need local backpressure because their worker pools and dependencies can saturate before a global platform limit is reached.
The useful MCP specific input here is the tool metadata already present in the catalog. Admission control can use effect type, cost class, timeout policy, and task capability rather than treating every tools/call as equivalent.
Version tool schemas independently from deployment#
Tool schemas are part of the contract seen by the model facing client.
Suppose payments.refund originally accepts:
payment_id
amount
and the next version requires:
reason_code
An agent may still hold the old cached tool definition.
Keep explicit schema identity:
tool_name
schema_version
input_schema_hash
server_version
deprecated_at
minimum_client_version
A breaking server deployment should not replace every old server while clients may still send the earlier request shape.
publish new schema
|
v
run compatible server pools
|
v
allow discovery caches to refresh
|
v
move traffic
|
v
retire old version
For genuinely incompatible contracts, a separately versioned tool is often easier to operate than silently changing the meaning of an existing one.
The MCP protocol itself now has a defined deprecation policy. Enterprise tool contracts still need their own lifecycle. MCP July 2026 changes
Regional failover has to preserve state ownership#
Stateless gateways and MCP frontends can run regionally.
Durable Tasks and business writes need clearer ownership.
A task can have a home region:
task_id = TK842
home_region = us_east
or use a store whose consistency behavior supports updates from multiple regions.
External writes need more care.
If us_east sends the refund to the payment provider and becomes unavailable before recording the result, us_west cannot assume that no refund happened.
Failover must preserve the same operation_id, and the new region needs a reconciliation path before repeating the write.
Traffic failover and execution ownership failover are different operations. The first can happen at the gateway. The second depends on durable state.
Plan failure behavior by state owner#
Keeping the state types separate gives each failure a clear owner.
If the gateway fails, protocol requests can move to another stateless gateway.
If the registry store fails, validated routing snapshots can continue serving existing traffic while control plane changes pause.
If authorization infrastructure is unavailable, operations requiring a new privilege decision should fail closed.
If requestState cannot be validated, the MRTR continuation is rejected.
If the Task store is unavailable, a new durable task is not created in local process memory as a fallback.
If an MCP server disappears, another compatible instance can process the next stateless request.
If a business write has uncertain status, recovery uses the business operation identity and external reconciliation rather than assuming transport failure means business failure.
Protocol state, interaction state, task state, and business state fail differently because they belong to different systems.
Carry one audit identity through the complete call#
A production write should remain traceable across gateway, MCP server, Task worker, and downstream resource.
Useful identifiers include:
run_id
trace_id
user_subject
agent_principal
workload_identity
tool_call_id
operation_id
task_id
server_id
server_version
tool_name
authorization_decision_id
Tokens and credentials should not appear in telemetry.
For one refund, an investigation should answer:
Which user initiated the work?
Which agent acted for that user?
Which tool was invoked?
Which authorization allowed it?
Which MCP server version processed it?
Which external resource changed?
Which logical operation committed?
Keep these invariants through implementation changes#
-
Every protected tool call has an identifiable tenant, user context, agent identity, workload identity, and tool identity.
-
Tool discovery never exposes capabilities outside the authorization scope represented by the catalog view.
-
Protocol routing metadata is validated against the actual JSON RPC request before it is used for routing or policy.
-
Tokens presented to an MCP server are intended for that MCP resource and are not passed through to downstream APIs.
-
Platform policy at the gateway does not replace resource specific authorization at the system that owns the resource.
-
MRTR state that travels through a client is treated as untrusted and validated before execution resumes.
-
An MCP Task identifier is not returned until the durable task can be resolved through
tasks/get. -
MCP Task status remains separate from richer worker execution state.
-
tasks/getremains a cheap point read rather than rebuilding task history for every poll. -
Long running work survives the MCP server instance that accepted the first request.
-
Cached tool schemas remain compatible with the server versions receiving those requests.
-
A compromised tool can be disabled faster than ordinary catalog cache expiry.
-
An uncertain external write is reconciled before another business effect is created.
Keep reading
Continue with a guided sequence of free production engineering essays.
Find your next reading pathRelated reading
Continue this path
A Skill Is Not a File. It Is a Deploy: Designing Agent Skills Infrastructure
Production Agent Skills are deployments, not files. They need immutable versions, controlled rollout, revocation, trust boundaries, progressive resolution, and a runtime path designed for scale.
Read essayHow Coding Agents Actually Work: Inside the Agent Harness
A technical look inside coding agent harnesses, covering context management, repository search, tools, sandboxes, subagents, durable sessions, recovery, concurrency, completion and evaluation.
Read essayOpenAI Decisions API vs Jev: Designing Systems Around Decisions Instead of Generation
A technical comparison of OpenAI Decisions API and Jev, and how to design a decision layer for routing, tool selection, risk, abstention, calibration and AI agent control.
Read essay