You deploy at 14:00. At 14:20 error rates spike. The release included a migration that dropped a column. What do you do, and what should have been different?
Show the full answer Hide the answer
What the interviewer is testing
Whether you recognise that rollback availability is a property the architecture either has or does not, and whether you can make a decision quickly under incident conditions rather than debating.
Why rollback is not available
The previous application version expects the dropped column. Deploying it means every code path touching that column fails, so rolling back converts a partial failure into a total one.
Restoring the database is worse: it is slow, and every transaction since 14:00 is lost. For twenty minutes of production traffic that is likely to be thousands of orders, payments or messages — an unrecoverable data loss chosen to fix a recoverable error rate.
So the answer is forward fix: identify the defect, ship a corrective change, and if the fix is not quick, mitigate first by disabling the affected feature with a flag or shedding the failing traffic.
What should have been different
The migration should have followed expand and contract:
| Step | Action | Rollback safe? |
|---|---|---|
| Expand | Add new column, nullable, unused | Yes |
| Migrate | Dual-write, backfill, move reads | Yes |
| Contract | Drop old column | Only after nothing reads it |
The contract step runs as its own deployment, days or weeks later, once telemetry confirms zero reads. Then a rollback of that release is safe because the column is genuinely unused.
The governing rule: schema changes and application changes ship in separate deployments, and every deployment must be independently reversible. A release that bundles a destructive migration with application logic has traded away rollback for a small amount of convenience.
What a strong answer adds
A pipeline check that fails any migration containing a destructive statement unless it carries an explicit approval marker, which turns the rule from a convention into a control. Also: the incident commander should decide within about five minutes and communicate the choice, since the worst outcome is the team debating rollback while the error rate continues.
Common weak answers
Choosing rollback reflexively without checking whether it is possible. Proposing a database restore without acknowledging the data loss.