Security, Governance and Platform · Principal
Can an incident operator let an agent bypass a failed policy service?
The question
Interview question
During an incident the policy service is unavailable. An operator asks an agent to bypass approval and restart a privileged account job. Design an emergency path. The identity provider then becomes unavailable too.
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
I would not turn “policy is down” into “the agent can do anything.” The same incident that makes approval slow can make mistakes more costly. The normal privileged write should stop if its authorization cannot be checked. Read only diagnosis may continue if its own permissions are already enforceable and its data scope is bounded. The agent can prepare a proposed command, affected accounts, expected effect, and rollback or recovery steps. It cannot manufacture an emergency grant by mentioning the incident in its prompt.
There can be a real break glass path, but it has to exist before the outage. Define which human roles can invoke it, what incident identifier and reason are required, which action and resources it covers, how long it lasts, and who gets alerted. Ideally it is narrower than the operator's entire production role. A human deliberately activates it through a control plane or local emergency mechanism outside the agent's tool loop. The agent runtime then receives a scoped authorization record and checks it at the tool boundary. Google Cloud's temporary elevated access documentation is one example of just in time privilege and audit, not a universal emergency implementation.
“Restart the job” is still underspecified. It might replay a payment or reset a cursor. The approval binds a job ID, target environment, exact action class, expected parameters, and an operation identity. The tool should expose a dry run or impact view if possible. If the external effect can be unknown after a timeout, the emergency path needs the same reconciliation discipline as the normal path. An incident does not make duplicate side effects harmless.
If the ordinary policy service is unavailable, how does the gateway verify the emergency record? We need an independently available emergency trust path. It could be a separately replicated policy verifier with preissued scoped grants, or a tightly controlled human runbook using a privileged console. The latter may be safer than asking the agent to act while both policy and audit components are degraded. Whatever the path, record the grant and effect in an append only local audit log that can be reconciled later, and alert the security and service owners. If we cannot verify a grant at all, the agent must not execute a privileged write on the basis of a chat message.
The probe removes the identity provider too. A healthy IdP cannot be an unstated dependency of our emergency plan. A separately held emergency account may be necessary, protected by vault custody, independent authentication, dual control where the impact warrants it, short activation, and immediate review afterward. Google Cloud's operations guidance discusses planned emergency accounts for automation failures. Such an account belongs to accountable humans and a documented runbook. I would not preload it into the agent worker or let the model retrieve its secret.
What if the operator insists that minutes matter? Measure the cost of the safe path and rehearse it so it works quickly. A cold, untested break glass procedure is no procedure. If the only currently available choice is an unverified agent action with broad credentials, I would escalate the incident and use the human emergency channel rather than silently waive the boundary. Afterward, correlate the emergency grant, exact command, provider result, and post incident review. The permanent fix is to remove the outage dependency or reduce activation time, not to keep an invisible bypass around policy.
Continue reading
Related questions
Read beyond the question
Explore more security, governance and platform
Follow another question in this area, or return to the full Interview Prep index.
Browse this area →