During an outage, an engineer wants to bypass the pipeline and patch production directly. It would save 25 minutes. What is your position?

What the interviewer is testing

Whether you hold a principle under pressure, and whether you recognise that the request is evidence of a design problem rather than a discipline problem.

The position during the incident

Generally no, and the reason is specific: a manual patch produces an artifact nobody can identify, in a state nobody can reproduce, that the next deployment will silently overwrite — re-opening the incident, often hours later, in a way that is extremely confusing because "we fixed that".

It also means the fix has had no test, no scan, no review, and no rollback path, at exactly the moment when the person making it is tired and under pressure. The failure rate of changes made in that state is high, and a second failure on top of an active incident is materially worse than 25 minutes.

The exception where the answer is yes: when the pipeline itself is the thing that is broken, or when 25 minutes of continued outage causes irreversible harm — data loss, safety, regulatory breach — that exceeds the risk of an unverified change. That is a judgement call for the incident commander, made explicitly and recorded.

If it is taken, two things are mandatory: a second engineer observes and confirms the change, and the change is backfilled into the repository as the immediate first follow-up, before the incident is closed.

The real conclusion

A 25-minute emergency path is a design defect. The request is telling you something true: the pipeline is not fit for incident response.

The fix is an expedited path that is fast and controlled — a pipeline variant that runs the blocking safety checks (build, secret scan, policy, smoke test) and skips the slow non-safety stages (full regression, extended soak, staged waves), reaching production in a few minutes with a recorded artifact and a working rollback.

Target: under 10 minutes for an emergency deployment. Then rehearse it, because a path first used during an outage is not a path.

What a strong answer adds

Noting the cheaper mitigations that often make the hotfix unnecessary: a feature flag to disable the broken path, a rollback to the previous artifact, or shedding the failing traffic. Those are faster than any deployment and should be the first options considered.

Common weak answers

"Never bypass the pipeline" with no emergency path offered. Allowing the bypass without requiring backfill, which guarantees the fix is lost.