I would not turn six points of preference into permission to change every application. Preference measures which answer people like under the tested prompts. The agent workflow may depend on choosing the right tool, supplying valid arguments, waiting for the result, and declining a dangerous action. A nicer final response can coexist with a worse action path.

First I would look at the errors. Did the model select the wrong tool, emit malformed arguments, ask for an action outside policy, mishandle a timeout, or call a valid tool twice? Did a tool server or adapter change at the same time? Replay the same task inputs against the old and new model with the same tool schemas and environment. Use side-effect-free fixtures for writes. Attach the full trace and the exact model, prompt, schema, and policy versions so a reviewer can see the first diverging decision. OpenAI's agent evaluation guidance treats tool calls and workflow traces as evaluation material, not only final prose.

The small sample is a problem in both directions. Two failures in twenty cases might be a serious regression or noise, depending on what failed and how the cases were sampled. I would not announce a precise failure rate from that alone. Build a targeted set around the actual failure mechanism, then test independent cases from real workflow shapes. Include approvals, partial tool results, retries, permissions, and cases that should not call the tool. An exact policy violation can block rollout even if it appears once. A benign formatting regression may need more evidence and a compatibility fix rather than a global veto.

Who owns the decision depends on the blast radius. The platform team can qualify the new model, expose a versioned option, and prevent a known unsafe default. The application team owns the business meaning of that high-impact action and the acceptance criteria for its workflow. If the model is shared by hundreds of apps, no one team should silently change the default for all of them based on a general preference score. Keep the old snapshot pinned for the failing workflow while lower-risk apps opt into a measured canary. The platform can block a cross-tenant or security violation regardless of an individual app owner's enthusiasm.

For a canary, I want the high-impact workflow explicitly excluded until it passes its own gate. Shadow runs can compare proposals, but they must not issue real refunds or production writes. Live traffic needs stable assignment, version logging, and rollback ownership. Measure completed task success, valid tool calls, policy violations, human corrections, latency, and cost on the slices that matter. Compare the old and new paths under the same traffic mix. A rollback switch that changes the model but leaves a new tool adapter or prompt active may not restore the old behavior.

The workflow owner is another team, and they have not responded. That is not consent. I would show them the failing traces and give them a reproducible test and a decision deadline. The platform can make the new model available to other apps, but it cannot move this workflow by default until the owning team or an agreed risk authority accepts a concrete fix and gate. If the old model is being retired, this becomes a migration and business continuity problem. We may need a narrow adapter, a temporary human review path, or reduced functionality. We should not disguise it as a routine model upgrade.

Suppose the new model is excellent everywhere except that one tool call. Could we force that call through the old model? Maybe, if the workflow can make a clean handoff and the extra state, latency, and cost are measured. The handoff has to preserve tool authority and operation identity. A mixed-model run is a new system to evaluate, not an automatic rescue. I would also try a clear schema or prompt change, then rerun both the general preference and critical workflow slices, because the fix can trade quality elsewhere.

I would ship the new model as an opt-in capability after platform checks, and move this workflow only when its owner can point to passing action-level evidence and an operational rollback. A global six-point win does not pay for an unexamined error in a consequential tool path.