The green tests prove something about the code with the flag state and context used by CI. They do not prove the rollout branch was executed. With a remote flag service, “flag on” may also mean different things for different accounts, regions, environments and variant payloads. A default-off mock is especially misleading when production uses a targeted strategy. Unleash documents flag evaluation against environment and context. The exact resolver in this system is the thing to inspect.

I would start by capturing one failing production decision without customer secrets: release commit, flag key and configuration version, environment, evaluated context fields, selected variant, and the code path taken. Then reproduce it in a hermetic test with the resolver's inputs pinned. Run the old build and the patch under that same flag snapshot. The test should fail on the old build for the actual checkout bug and pass on the patch. Otherwise the agent has not shown it fixed the behavior users saw. An assertion around a helper function may miss the router, a variant payload or a second calculation path.

For this patch, the validation matrix is small and deliberate. Test flag off, on for an eligible account, off for an ineligible account, and any variant whose payload changes pricing. Include the boundary where region or account targeting differs, plus retries and rounding if the implementation touches them. We do not need a Cartesian product of every flag in the service. Record which relevant flag combinations and contexts were exercised, and fail the release gate if the target rollout path has zero coverage. When flags interact, use a scenario that reaches the interaction, not a checkbox claiming both flags were separately tested.

The agent should receive the intended behavior contract and representative flag fixtures, not an unrestricted ability to change the flag mock until tests pass. Its report should say which paths it tested and which were unavailable. CI can replay a signed or versioned configuration snapshot with test identities, then an integration or canary check should log the actual evaluated variant in the target production cohort. Keep the runtime kill switch ready. A canary that lands outside the 5 percent cohort is not evidence for the risky branch.

What if the remote flag changes after CI? Bind the release decision to a recorded configuration version or reevaluate the relevant scenarios before enabling a materially different rollout. If that is operationally expensive, define an allowed envelope and detect drift outside it. Do not freeze all feature management indefinitely. The coding agent made CI green by changing the test. Do you merge it? asks whether an agent weakened the tests. Here the tests can be honest and still exercise the wrong behavior because the flag context does not match the users receiving the change.