Search the practice set
275 questions, 991 terms and 600 topics in 30 areas.
33 results for “Rollback & Forward Fix”
Forward Fix
Resolving an incident by shipping a correction rather than reverting, which is the right choice exactly when reverting is not actually available.
Blue-Green Database Schema
The constraint that makes fast rollback actually work — both application versions must be able to run against one schema at the same time.
Blue-Green Deployment
Running two identical production environments and switching traffic from the old one to the new one in a single cut, with the old kept warm for rollback.
Compatibility Mode
The registry setting that defines which schema changes are permitted — backward, forward, full, or none — and therefore what upgrade order is safe.
Compensating Transaction
A business operation that semantically undoes a previously committed step — not a rollback, because the original effect was visible and may not be fully reversible.
Cutover Runbook
The step-by-step plan for a migration event, including verification points, decision criteria and the rollback path, rehearsed before it is used.
Pre-Mortem
An exercise in which a team imagines a project has already failed and explains why, surfacing risks that a forward-looking risk assessment misses.
Production is degraded after a release. The team tries to roll back and discovers a migration has already run. What do you do now, and what do you change afterwards?
Now: stop trying to roll back Reverting the deployment would run the previous code against a schema it does not know, which is a second and less understood fail
A codebase has 340 feature flags, most of them permanently on. What is the problem and how do you fix it?
The problem is combinatorial and it is not theoretical Every flag doubles the number of possible code paths. At 340, the number of configurations is unbounded a
A dashboard query that took 200ms now takes 40 seconds. The table has grown to 200 million rows. Walk me through diagnosis and fix, including what you would not do.
What the interviewer is testing Whether you diagnose with evidence before changing anything, and whether you know the costs of the fixes you propose. Diagnosis,
A list endpoint returning 2 million records times out on deep pages and returns 500 for validation failures. Fix both properly.
The pagination problem OFFSET 100000 requires the database to scan and discard 100,000 rows before returning the page, so each page is slower than the last. Off
A serverless API works in testing and fails under load with connection errors. The database is at 5% CPU. Explain and fix.
The mechanism Serverless functions scale by creating independent execution environments , each with its own process and its own connection pool. Two hundred con
A service writes to its database and then publishes an event to Kafka. Sometimes consumers see an event for a record that does not exist, and sometimes a record exists with no event. Why, and how do you fix it?
What the interviewer is testing Recognition of the dual write problem — one of the most common defects in event driven systems and one that testing rarely catch
After a content purge, your origin receives 400× normal traffic and falls over. The CDN is working as configured. What is missing and how do you fix it?
What is happening A global CDN has hundreds of points of presence, each with an independent cache. A purge invalidates the object everywhere at once , so the ne
After a five-minute network outage at your ingest tier, 200,000 devices reconnect simultaneously and the authentication service collapses. How do you fix this?
The mechanism Every device detected the disconnection at roughly the same moment and retried immediately. Reconnection is the most expensive operation in the fl
Every deployment produces a small spike of 502s that the team has learned to ignore. Fix it.
The cause In flight requests are being severed because the instance terminates before the load balancer has stopped sending to it, or before existing requests c
Finance and Sales report different revenue for the same quarter. Both are convinced they are right. You are asked to fix the data. Where do you start?
Assume neither number is wrong The instinct is to look for a bug. In practice the overwhelming majority of these cases are two defensible definitions with no me
Knight Capital lost roughly $460M in 45 minutes in 2012 after a deployment reached seven of eight servers. Which architectural failures made that possible, and which one would you fix first?
The case, as publicly reported Per the SEC's 2013 order, Knight Capital deployed new order routing code to eight production servers ahead of a NYSE programme la
Six teams share one integration test environment. Bookings are made a week ahead and releases slip when someone overruns. How do you fix it?
Name the cost first The queue is not an inconvenience; it is lead time. Measure it: for the last twenty changes, how many days elapsed between "ready to test" a
Three teams have independently chosen three different message brokers. Design the selection and governance process going forward.
Understand why it happened before correcting it Three independent choices usually mean there was no easy default, no forum for the decision, or a central proces
Uber reached roughly 2,200 microservices and found the estate incomprehensible. Their fix was not consolidation. What was it, and why does it generalise?
What they did Uber's 2020 publication on Domain Oriented Microservice Architecture describes three structural rules layered on top of the existing services — no
You move a user profile service to eventual consistency and support tickets start arriving: users update their name and the old one is still shown. Fix it without abandoning the architecture.
What the interviewer is testing Whether you know that consistency is per operation rather than per system, and whether you reach for a targeted guarantee instea
Your API gateway configuration has grown to 8,000 lines with request transformations and cross-service orchestration. Changes require a platform team ticket and take two weeks. How do you fix this?
Name the failure The gateway has become a distributed monolith — shared, centrally owned, untestable in isolation, and on the critical path of every team's deli
Your platform team of five spends nearly all its capacity upgrading control planes across 38 Kubernetes clusters. How did this happen and how do you fix it?
How it happened There is no split axis, only history. Clusters were created per project, per experiment, per team, per acquisition — each for a reason that was
Your team receives 200 pages a week and the on-call rotation has lost two engineers in six months. Fix it.
Diagnose before deleting Measure the actionability ratio : the proportion of pages that resulted in a human doing something. Recording an outcome at resolution
A release must rename a heavily used database column and ships tonight. The team proposes doing the rename in the deployment. What is wrong, and what do you propose?
What is wrong A rename is not additive. During a rolling deployment both versions of the application run simultaneously: one expects the old name, one the new.
Rollback & Forward Fix
When reversing is genuinely possible, and designing so that it usually is.
Alert Fatigue
How noise makes the real page invisible, and the structural fix.
Cutover Planning
Rehearsals, go/no-go criteria and a rollback that has been executed.
OTA Updates
Updating firmware over a flaky link, with rollback, and without bricking the device.