Search the practice set
126 questions, 454 terms and 400 topics in 20 areas.
60 results for “Debugging Distributed Systems”
Distributed Tracing
Following one logical request across every service it touches by propagating a shared trace identifier and recording timed spans.
Backpressure
A mechanism by which a component under load tells its callers to slow down, rather than accepting work it cannot complete.
Bulkhead
Partitioning resources so that exhaustion caused by one dependency or tenant cannot starve the others.
CAP Theorem
During a network partition a distributed system must choose between consistency and availability; it cannot have both.
Circuit Breaker
A proxy that stops calling a failing dependency after a failure threshold, failing fast instead, and periodically tests whether it has recovered.
Consistent Hashing
A hashing scheme where adding or removing a node remaps only a small fraction of keys, instead of nearly all of them.
Event Stream
An append-only, retained log of events that many independent consumers read at their own position, and can re-read.
Eventual Consistency
A guarantee that replicas will converge to the same value if updates stop, with no bound on how long reads may be stale.
Exponential Backoff
Increasing the wait between retries geometrically, with random jitter, so that failures do not synchronise into a stampede.
Fan-Out
One incoming request causing many outgoing ones, which multiplies both load and tail latency.
Fault Tolerance
Continuing to operate correctly despite the failure of some components, by design rather than by luck.
Graceful Degradation
Continuing to deliver reduced but useful function when a dependency fails, instead of failing the whole request.
Idempotency
The property that performing an operation many times has the same effect as performing it once.
Leader Election
The process by which a group of nodes agrees which one of them is currently in charge of a task that must not run twice.
Load Shedding
Deliberately rejecting a portion of incoming work during overload so that the remainder can be served correctly.
Message Queue
A store that holds messages until a consumer processes them, decoupling producer availability and rate from consumer availability and rate.
Monolith vs Microservices
A trade of deployment independence against distributed-systems complexity, decided by team topology far more often than by technology.
Quorum
A minimum number of nodes that must acknowledge an operation for it to count, chosen so that read and write sets are guaranteed to overlap.
Saga
A sequence of local transactions across services where each step has a compensating action that semantically undoes it if a later step fails.
Scalability
The ability to handle growing load by adding resources, ideally with cost rising no faster than the load.
Service Discovery
The mechanism by which a caller finds a currently healthy network address for a service whose instances are ephemeral.
Shuffle Sharding
Assigning each customer a random combination of workers rather than a fixed shard, so that any two customers rarely share their whole set.
Split Brain
A partition in which two halves of a cluster each believe they are authoritative, and both accept writes.
Thundering Herd
A large number of clients acting simultaneously because they were synchronised by a shared event, producing a spike that the steady-state design never sized for.
Timeout Budget
Assigning a request an overall deadline at the edge and passing the remaining time down each hop, so no service works on something already out of time.
Two-Phase Commit
A blocking protocol for atomic commit across several resources: a coordinator asks all participants to prepare, then tells them all to commit or abort.
A 43-second network partition caused GitHub over 24 hours of degraded service in 2018. How does a 43-second event become a day-long incident?
The case, as publicly reported On 21 October 2018, routine maintenance replacing failing optical equipment caused a 43 second loss of connectivity between GitHu
A card payment authorisation service runs active-active across two regions. A network partition splits them. Do you keep accepting authorisations, and what breaks either way?
What the interviewer is testing Whether you can apply CAP to a domain where the cost of each choice is concrete, and whether you recognise that "it depends" has
A downstream service slows from 50 ms to 3 s. Within two minutes every service in the request path is down, including ones that do not call it. Explain the mechanism and how you would have prevented it.
What the interviewer is testing Whether you understand that most outages are amplification, not failure — and whether you can name the specific mechanism rather
An order service must notify inventory, billing, shipping and analytics when an order is placed. Synchronous calls or events? Justify your choice per consumer.
What the interviewer is testing Whether you apply the decision per interaction rather than adopting one style globally. The framing that matters Synchronous cal
Design an order submission API that is safe when the client cannot tell whether its request succeeded. What exactly do you store, and when?
What the interviewer is testing Whether you know that "make it idempotent" is a design with specific failure modes, not a checkbox. The core design The client g
You move a user profile service to eventual consistency and support tickets start arriving: users update their name and the old one is still shown. Fix it without abandoning the architecture.
What the interviewer is testing Whether you know that consistency is per operation rather than per system, and whether you reach for a targeted guarantee instea
A nightly job occasionally runs twice, producing duplicate charges. The team proposes a distributed lock. What do you say?
The first response A lock will reduce the frequency and will not eliminate it , and if the team believes otherwise they will stop looking for the real fix. The
In the 2017 AWS S3 outage, the status page could not report the outage because it depended on S3. What does that tell you about designing status and control systems?
The case, as publicly reported On 28 February 2017, an authorised engineer running an established playbook to debug an S3 billing issue in us east 1 entered a c
In the October 2021 Meta outage, engineers reportedly struggled to physically access equipment because the access-control systems were also down. What class of dependency is that, and how do you find yours?
The case, as publicly reported On 4 October 2021, a command issued during routine backbone capacity assessment took down all connections between Meta's data cen
Debugging Distributed Systems
Localising a regression when every service reports healthy.
Distributed Systems
General material on partial failure, coordination and distributed reasoning.
Distributed Locking
Mutual exclusion across machines, and why it is harder than it looks.
Distributed Transactions
Two-phase commit, its blocking failure mode, and when it is still reasonable.
Backpressure & Flow Control
Telling callers to slow down instead of buffering into congestion collapse.
Bulkheads & Isolation
Partitioning resources so one dependency cannot starve the others.
CAP & PACELC
What you must give up during a partition, and the latency choice the rest of the time.
Circuit Breakers
Failing fast on a broken dependency, and what you fail fast to.
Clocks & Ordering
Why wall clocks lie, and how logical clocks and versions restore order.
Consensus Protocols
Raft, Paxos and quorums — what they guarantee and what they cost.
Consistency Models
Linearizable, sequential, causal, eventual, and the session guarantees between them.
Event Streaming
Retained ordered logs, consumer offsets, partitions and replay.
Failure Modes
Slow rather than down, partial, grey, and failing while reporting success.
Idempotency
Making an operation safe to repeat, because a client that times out cannot know.
Leader Election
Agreeing who is in charge, and fencing the one who no longer is.
Load Shedding
Rejecting some work deliberately so the rest can be served correctly.
Messaging & Queues
Decoupling producer from consumer, and the semantics that come with it.
Retries & Backoff
Exponential backoff, jitter, retry budgets, and how retries become the outage.
Sagas & Compensation
Replacing atomicity with semantic undo, and ordering the irreversible steps last.
Service Discovery
Finding a healthy address for something whose instances are ephemeral.
Testing Strategies
The pyramid, and the contract tests distributed systems add to it.
Timeouts & Deadlines
Per-hop timeouts that do not compose, and the deadline budget that replaces them.
Centralised vs Distributed
Shared platform leverage against team autonomy.
Distributed Tracing
Reconstructing one request's path across every service it touched.