Your service handles 40 requests per minute. Product wants canary releases with automated analysis. What do you tell them?

What the interviewer is testing

Whether you understand that canary analysis is a statistical procedure with a minimum sample size, and whether you will say so rather than implementing something that looks like a canary.

The arithmetic

At 40 requests per minute, a 5% canary receives 2 requests per minute. Suppose the baseline error rate is 1% and the new version doubles it to 2%. In an hour the canary sees 120 requests, with an expected 1.2 errors instead of 2.4. That difference is indistinguishable from noise — you would need many hours, possibly days, to reach any confidence.

So an automated canary at this volume will do one of two things: promote everything regardless of quality, or fire false alarms on random variation. Both are worse than not having it, because they carry the authority of a measurement.

What to do instead

Blue-green with fast automated rollback. The exposure is total but brief, and detection comes from the same monitoring you already rely on. For a low-volume service the mean time to recovery, not the blast radius, is the lever worth pulling.

Raise the canary percentage substantially. At this volume a 50% canary is statistically far more useful than 5%, and the blast radius is still bounded. The instinct that smaller is safer is wrong when smaller means blind.

Synthetic traffic against the canary to exercise critical paths continuously, which gives a signal that does not depend on organic volume.

Longer bake with manual judgement, which is legitimate — an engineer watching for an hour is a better detector than an underpowered statistical test.

What a strong answer adds

Noting that the same problem applies to per-endpoint analysis on higher-volume services: the service may do 5,000 requests per minute while the endpoint that broke does 12. Canary analysis must be scoped to metrics with sufficient volume, and low-traffic critical paths need synthetic coverage.

Also worth raising: business metrics are usually much lower volume than technical ones, so a canary gated on conversion rate needs far more traffic than one gated on HTTP error rate.

Common weak answers

Implementing the canary anyway because it is best practice. Choosing a percentage without asking what signal it produces.