Search the practice set
126 questions, 454 terms and 400 topics in 20 areas.
42 results for “Latency”
Tail Latency
The latency experienced by the slowest small percentage of requests, which is what users and dependent services actually feel.
Application Performance Monitoring
Instrumentation inside the application that attributes latency and errors to specific code paths, queries and dependencies.
Availability Zone
One or more physically separate data centres inside a cloud region, with independent power, cooling and network, connected by low-latency links.
Bounded Queue
A queue with a maximum depth, which converts unbounded latency growth into an explicit rejection you can control.
Cascading Timeout
The effect of independently-chosen per-hop timeouts summing to a total far longer than any caller is willing to wait.
Content Delivery Network
A geographically distributed cache that serves content from a location near the user instead of from the origin.
Context Window
The maximum number of tokens a model can attend to in one request, holding the system prompt, history, retrieved context, tools and the answer.
Edge Function
Code executed at a CDN point of presence close to the user, in a constrained, fast-starting runtime, typically to modify or route a request before it reaches an origin.
Fan-Out
One incoming request causing many outgoing ones, which multiplies both load and tail latency.
Golden Signals
The four measurements that cover most of what matters for a request-driven service: latency, traffic, errors and saturation.
Hedged Request
Sending a duplicate of a request to a second replica after a short delay and using whichever response returns first, to cut tail latency.
Little's Law Applied to Pools
Using L = λW to size connection and thread pools from measured throughput and latency rather than from a default.
Model Router
Directing each request to a model chosen by the task's difficulty, cost and latency budget, rather than sending everything to the largest model available.
Netflix's Recommendation Architecture
Netflix splits personalisation into offline, nearline and online layers so that expensive computation happens ahead of time and the request path stays fast.
Non-Functional Requirement
A requirement about how well the system must behave rather than what it must do — latency, availability, throughput, security, cost.
PACELC
An extension of CAP that also describes the normal case — if Partitioned choose Availability or Consistency, Else choose Latency or Consistency.
Read Timeout
The bound on how long a client waits for response data after a connection is established — distinct from the connect timeout, and the one that usually matters.
Serverless Cold Start
The additional latency when a function invocation must allocate and initialise a new execution environment rather than reusing a warm one.
Strong vs Eventual Consistency
A per-operation decision, not a per-system one: whether this specific read must reflect every completed write.
Synchronous vs Asynchronous Communication
Whether the caller waits for the callee's answer — decided by whether the caller's outcome depends on it, not by latency or taste.
Synchronous vs Asynchronous Replication
Whether a write is acknowledged only after a replica has it, trading write latency against the amount of data a failure can lose.
Throughput
The rate of work a system completes per unit of time — and a quantity that trades against latency rather than tracking it.
Timeout Budget
Assigning a request an overall deadline at the edge and passing the remaining time down each hop, so no service works on something already out of time.
Warm Pool
Pre-initialised instances held in a stopped or standby state so that scaling out skips boot and application warm-up.
A service autoscales on CPU. During incidents it never scales out, even as latency triples. Why, and what would you scale on instead?
Why CPU never triggers The service is latency bound, not compute bound. Its threads are waiting — on a database, a downstream API, a lock, or a connection pool
A team proposes moving personalisation to edge functions to cut latency. When does that work and when does it backfire?
The principle that decides it The edge is near the user and far from your data. Edge compute pays off for work that needs the request but not your state . The m
p99 latency on checkout tripled overnight. Dashboards look normal, no deployment went out, and every service reports healthy. How do you find it?
What the interviewer is testing Systematic diagnosis in a distributed system when the obvious signals are clean — which is the realistic version of this problem
A user in Mumbai types your URL and presses enter. Walk me through every hop until the HTML reaches their browser. Where would you look first if the page were slow?
What the interviewer is testing This is the classic architecture screening question, and it is asked because the answer's depth correlates strongly with real ex
Classify DynamoDB, Spanner and Cassandra under PACELC, and say which half of the classification you would actually design around.
The classifications Store Partition Else Why DynamoDB (default reads) PA EL Serves from any replica; eventually consistent reads avoid a quorum round trip Cassa
Netflix personalises an entire home page in the time a TV takes to draw a screen. How, given that scoring every title for every member on request is impossible?
What the interviewer is testing Whether you reach for the offline/online split — the single most reusable idea in latency critical personalisation — rather than
Latency
Distributions rather than averages, and the floors physics imposes.
Tail Latency
p99 behaviour, amplification across fan-out, and hedged requests.
Application Performance Monitoring
Attributing latency to code paths, queries and dependencies.
CAP & PACELC
What you must give up during a partition, and the latency choice the rest of the time.
DNS
Resolution, TTL behaviour, traffic steering and failover latency.
Failover
Detection latency, promotion, fencing and the cost of failing over wrongly.
Model Selection
Capability, latency, cost and the evaluation that decides between them.
Network Performance
Latency floors, bandwidth-delay product, and what no code change fixes.
Performance vs Cost
Buying latency, and knowing what the last millisecond is worth.
Quality Attributes
Availability, latency, throughput, security, cost — expressed as testable scenarios.
Queueing Theory
Why latency explodes as utilisation approaches capacity.
Throughput
Work completed per unit time, and why it trades against latency.