A common envelope is useful. Application teams should not parse five response shapes to find generated text, usage, a tool proposal, or a refusal. But the gateway cannot claim two routes are interchangeable if their state, tool, or error semantics differ. A normalized field that says tool_call: true does not tell us who executes the tool, whether a partial streamed call can be acted on, whether a model response retry replays a proposal, or whether the provider itself ran an external action.

I would define a small portable core: request and trace IDs, provider and model version, prompt and tool-schema revision, ordered content blocks, tool-call IDs with structured arguments and validation status, finish reason, usage, latency, and a typed error with retry guidance. Then expose capabilities and route-specific semantics explicitly. Examples include context and output limits, parallel tool-call support, structured-output guarantees for the chosen model, streaming event types, data handling requirements, cache behavior, rate-limit scope, and whether tools execute in our runtime or provider-side. The route should be pinned for a step whose replay depends on it.

For client-side tools, the provider returns a proposal. Our runtime validates arguments, authorizes the operation, records its stable identity, executes it, and sends the result back with the matching call ID. OpenAI's function-calling guide describes application-executed functions. Claude's tool-use documentation describes tool_use and tool_result blocks, with an explicit parallel-result contract. The gateway can normalize these events, but it must preserve correlation and ordering. A provider's own server-side tool mode changes the effect boundary, so it requires a separately reviewed contract and permissions, not the same generic boolean.

Now the retry probe. Suppose provider A's model request fails before returning any tool proposal, while provider B returns a proposal and then the connection drops. A blind gateway retry could produce a second proposal with another call ID. That is generally fine if no tool was executed, but it can be dangerous if an adapter began executing a streamed partial call or the provider executed it server-side. We need an explicit step state: no complete proposal accepted, complete proposal accepted but no tool submitted, tool submitted with known or unknown outcome, and result returned. Retry policy follows that state and the tool's contract, not an HTTP status alone. Do not claim the providers have particular retry guarantees without their actual API contract. The gateway should make the uncertainty visible.

If provider B has no safe replay for this step, don't silently fail over to A after an unknown tool effect. The application owner needs a typed effect_outcome_unknown or equivalent, the original route and operation ID, and a reconciliation path. If it is only a generation failure before an effect, failover may be allowed if the new model meets the task's quality, data, and tool constraints. A fallback that omits a required tool or moves restricted content to an unapproved region is not a successful normalization.

The audit record should retain the provider-native request and response references or a protected representation sufficient to debug translation, alongside the normalized events. Redact secrets and bind access to the same principal and retention policy. During rollout, compare both representations on a corpus of tool proposals, malformed arguments, parallel calls, truncation, refusals, timeouts, and partially streamed outputs. A gateway earns trust when it makes provider differences manageable and observable, not when it erases the facts needed to decide whether an action was safe.