Zero observed does not mean zero possible. Under an independent, identically sampled Bernoulli model, if the true failure probability is p, the chance of seeing zero failures in n trials is (1-p)^n. For n = 200, the one-sided 95 percent upper confidence bound solves (1-p)^200 = 0.05, giving p = 1 - 0.05^(1/200) ≈ 0.0149. About 1.49 percent, far above 0.01 percent. To push that bound below 0.01 percent with zero failures would require roughly 30,000 independent representative trials. NIST's exact binomial documentation covers the one-sided method. The arithmetic is a useful sanity check, not a guarantee for an agent whose failures are clustered by task, model version, or tool.

Here half the cases are slight variants. Correlation means 200 lines in a spreadsheet do not necessarily carry 200 independent opportunities to fail.

There is another issue: did these tests even exercise the conditions that allow an unauthorized write? A corpus of safe read-only tasks can produce a perfect result while saying almost nothing about the write boundary. Specify the event first. Does “unauthorized” mean outside delegated scope, above a monetary cap, stale approval, wrong tenant, or an external action after cancellation? Record the attempted action and the actual effect, not just the model's final text.

I would make the release gate a layered argument. Deterministic broker checks enforce permissions for every effect, with adversarial tests around concurrency, revocation, and format changes. Then run scenario tests that deliberately reach each failure path. A separate representative sample estimates ordinary workload risk with intervals, including runs that error or never finish. Human review of the attempted actions checks whether the grader itself misses violations. Keep the statistical statement tied to the sampled population and version. Testing does not replace a hard authorization boundary for a rare, high-consequence effect.

Suppose the team offers ten thousand more synthetic variants of the same prompts. That may catch a narrow regression, but it cannot repair the sampling or oracle problem. Suppose instead they observe zero incidents in a million production requests. The rate might look much smaller, but only if unauthorized actions would have been detected, exposures were counted correctly, and the policy and traffic did not change. I would ask for detection coverage and the relevant high-risk denominator before turning a zero into a safety claim.