Release Strategies
How new code reaches users — rolling, blue-green, canary, or flag-controlled — chosen by rollback speed and blast radius.
Definition
| Strategy | Mechanism | Rollback | Cost |
|---|---|---|---|
| Rolling | Replace instances in batches | Roll forward or back through batches | Low |
| Blue-green | Two full environments, switch traffic | Instant switch back | Double infrastructure during release |
| Canary | Small share of traffic to the new version, analysed | Stop and shift back | Requires good metrics and automation |
| Feature flag | Code deployed dark, enabled per cohort | Flag off, instant, no deployment | Flag lifecycle management |
How to choose
Two questions decide it:
How fast must rollback be? Flags are instant. Blue-green is a traffic switch. Rolling requires another deployment cycle. If a bad release costs money by the minute, rolling is too slow.
How confident are you before exposing everyone? Canary buys evidence: a small share of real traffic with automated comparison against the control, promoting or aborting on measured signal. This is the only strategy that provides information rather than merely limiting damage.
The part that matters more than the strategy
Rollback must be tested and fast. Every strategy assumes reversal is possible, and the assumption frequently fails for a reason nothing in the deployment mechanism covers:
- A schema migration that the previous version cannot read. This is the most common blocker, and the remedy is expand-and-contract so every intermediate state is compatible with both versions.
- Messages or events written in a new format that the old version cannot consume.
- State written by the new version that the old one misinterprets.
If the database has moved forward, rolling back the application does not roll back the system. That is the real constraint, and it is why schema compatibility is a release-strategy concern rather than a database one.
Canary analysis specifics
- Compare against a control, not against yesterday. Time-of-day effects will otherwise dominate.
- Measure the right things: error rate, latency percentiles, and at least one business metric.
- Automate the decision. A human watching a dashboard for ten minutes does not scale and is not reliable at 2am.
- Run long enough for slow-emerging problems — memory growth, cache pollution — to appear.
- Beware small samples. 1% of traffic may take an hour to produce a statistically meaningful signal.
Failure scenarios
- Rollback assumed and never tested.
- Canary too small or too short to detect the problem.
- Blue-green with a shared database, so the "instant rollback" does not cover the data.
- Flags accumulating forever, producing an exponential space of untested combinations.
Interview question
"You roll back a deployment and the system is still broken. What are the likely reasons?"