practice

Forward Fix

Resolving an incident by shipping a correction rather than reverting, which is the right choice exactly when reverting is not actually available.

Rollback is the default advice and it is often unavailable. A schema migration has run; a message with a new shape is already in a partner's queue; a downstream consumer has cached the new format. In those cases "revert the deployment" restores old code against a world the new code changed, which is a second, less-understood failure.

The honest position is that the choice is decided at design time, not during the incident. If the change was made backwards-compatible — expand before contract, additive schema, tolerant readers — rollback stays available and is almost always right, because it is fast and rehearsed. If it was not, forward fix is the only option and its cost is that it must be written, reviewed and deployed under pressure.

What separates teams that recover quickly is that they know which of the two they are in before they need to decide. A deployment record that states whether the change is reversible, and until when, converts a panicked debate into a lookup.