advanced 2 min answer

You need to ship a rewrite of the pricing engine. Same inputs, same expected outputs, completely new implementation. How do you release it?

releasecanarytestingrisk
Show the full answer Hide the answer

What the interviewer is testing

Whether you know that release strategy is part of architecture, and whether you reach for verification techniques beyond "test it well and deploy on a Tuesday".

The strategy

1. Ship it dark, behind a flag. Merge continuously to trunk with the new engine behind a flag defaulting to off. This avoids a long-lived branch and lets the work be integrated and tested against everyone else's changes throughout.

2. Shadow it first — this is the key move. Run the new engine on real production traffic, in parallel with the old one, with the old one's result still authoritative. Log both. Compare. Because the inputs and expected outputs are the same, discrepancies are directly measurable, and every one is either a bug in the new engine or a rule in the old one that nobody documented. The second category is what makes this worth doing: a pricing engine of any age contains rules that exist only in the code.

The new engine must be side-effect free during this period — no writes, no external calls.

3. Set an exit criterion before you start. For example: discrepancy rate below 0.01% sustained for two weeks including a month-end. Without a stated threshold, the shadow period ends when somebody loses patience.

4. Then canary by traffic percentage. 1%, then 5%, 25%, 100%, with automated rollback on error rate or latency regression. Pricing is high-consequence, so also alert on business metrics — average order value, discount rate — because a pricing bug can produce perfectly healthy technical metrics and wrong money.

5. Keep the old engine callable for one release cycle after 100%, so rollback is a flag flip rather than a redeploy.

6. Remove the flag and the old engine in a scheduled, ticketed piece of work. This is the step that never happens if it is not scheduled.

Why not blue-green

Blue-green is a single cut for everyone. For a change where correctness is uncertain and errors are financial, you want graduated exposure and real comparison, not an atomic switch — even one with a fast rollback, because the damage during the exposure window is already done.

What a strong answer adds

  • Canary caveats specific to this case: if pricing is called asynchronously from a queue, traffic-percentage splitting does not work as expected; and sticky sessions mean a user who lands on the canary stays there, so 1% of traffic may be 1% of users seeing 100% of the new behaviour.
  • Characterisation tests captured from the shadow comparison become the regression suite, permanently — the discrepancies you found are the edge cases nobody would have thought to write.