A coding agent runs repository tests that can read its secrets

The repository is an input to the task. Running npm test or make test executes code chosen by that repository, including dependency scripts and test fixtures. I would treat that execution like running an untrusted program even if the repository belongs to our organization. A malicious dependency, compromised branch, or accidental test script can read environment variables and contact a network endpoint. Telling the agent not to read secrets does nothing once the process has access to them.

The model proposes a command. A separate runtime decides which workspace and capabilities the command gets. Put the checkout and generated changes in an ephemeral sandbox with a restricted filesystem view, a writable scratch area, resource and time limits, and no host or production credentials. The host orchestrator and its identity material stay outside. A sandbox is only as strong as its actual implementation and configuration, so test escape and mount assumptions, use patched images, and record the image digest. Do not use a shared host directory as a shortcut if it contains other tenants' data. Anthropic's self hosted sandbox security guidance explicitly calls out network egress, filesystem, capabilities, and image hardening as operator responsibilities.

Default network egress to deny. The test runner may need to install packages, but opening the internet broadly makes secret exfiltration trivial and dependency resolution nondeterministic. Use a pinned dependency mirror or controlled proxy and record what was fetched. Also consider DNS, redirects, and a permitted host that can relay arbitrary payloads. An egress allowlist alone is not a guarantee if the allowed service accepts unconstrained writes.

The probe grants a staging API dependency. I would expose a narrow brokered interface for the specific test fixture, with a scoped, short lived test identity, a dedicated staging dataset, request limits, and an audit trail. The sandbox sees only the test endpoint or broker, never the credential used by the host. The broker checks the operation and data, not just the hostname. If the staging service is allowed to return private data, give the fixture synthetic or isolated accounts. An arbitrary shell process with a wide staging token is still an arbitrary client with that token.

This changes how we promote output. Tests inside an untrusted sandbox can report success while replacing a test script or forging a log. Their stdout can also contain instructions aimed at the agent and must remain tool output, not a new developer message. The orchestrator collects the patch and test artifacts, validates paths and size, and runs a trusted review or independent verification against a clean checkout before merging. Preserve the source commit, sandbox image, command, tool outputs, and artifact digest. A passing sandbox test is evidence about that environment, not an attestation that the change is safe.

Suppose a security scanner itself needs to inspect a customer secret or production-like data. That is a different task with a separately approved execution environment and narrower capability. I would not quietly mount a production vault into the same sandbox because the ordinary test suite now “needs it.” OpenAI's shell tool guidance also warns that allowing domains can create exfiltration paths. The strongest control is keeping the secret out of the arbitrary process and making any necessary access a brokered, inspectable action.