Search the practice set
126 questions, 454 terms and 400 topics in 20 areas.
60 results for “Cost per Request”
Cost per Request
The fully-loaded infrastructure cost of serving one request, and the unit that makes architectural cost decisions comparable.
Context Window
The maximum number of tokens a model can attend to in one request, holding the system prompt, history, retrieved context, tools and the answer.
Deadline Exceeded
The error returned when a request's overall budget expires — semantically distinct from a per-hop timeout, and a signal that must not be retried blindly.
Model Router
Directing each request to a model chosen by the task's difficulty, cost and latency budget, rather than sending everything to the largest model available.
RED Method
A minimal per-service dashboard: Rate, Errors, Duration — the request-centric view of whether users are being served.
Retry Budget
Capping retries as a proportion of overall traffic rather than as a count per request, so retries cannot multiply during the failure they are meant to survive.
Semantic Cache
Caching model responses keyed by the meaning of the request rather than by its exact text, so near-duplicate questions are served without a model call.
Serverless
A model where the provider allocates and scales compute per request, and you are billed for execution rather than for provisioned capacity.
Strangler Facade
The routing layer in front of a legacy system that decides, per capability, whether a request goes to the old implementation or the new one.
Architecture Cost Model
A calculation, made during design, of what an architecture will cost to run at expected and at peak volume.
Cost vs Reliability Trade-off
The non-linear relationship between availability and spend, which makes each additional nine roughly an order of magnitude more expensive.
Database per Service
Each service owning its own datastore, with no other service reading or writing it directly.
Egress Cost
The charge for moving data out of a cloud provider or across its zones and regions — usually the least-anticipated line on a cloud bill.
Hedged Request
Sending a duplicate of a request to a second replica after a short delay and using whichever response returns first, to cut tail latency.
Server-Side Request Forgery
Inducing a server to make an HTTP request to an attacker-chosen destination, turning it into a proxy into networks and services the attacker cannot reach directly.
Total Cost of Ownership
The full lifetime cost of a capability, including the people, operations, upgrades and exit that a licence comparison leaves out.
AI Gateway
A shared proxy in front of model providers that centralises routing, keys, quotas, caching, logging and safety policy.
Account Vending
Automated creation of new cloud accounts pre-configured with the organisation's networking, identity, logging, guardrails and cost allocation.
Admission Control
Deciding at the edge whether to accept a request at all, based on current capacity, before any work is done on it.
Application Portfolio Management
Maintaining an inventory of every application with its owner, cost, business value and technical health, and using it to decide what to invest in, replace or retire.
Backend for Frontend
A separate, narrow backend per client experience, which aggregates and reshapes downstream services for exactly that client's needs.
A team wants to build a new internal API on serverless functions. It will serve steady traffic of about 200 requests per second during business hours. What do you advise?
What the interviewer is testing Whether you can apply the serverless trade off to a specific workload rather than treating it as a default good or a default bad
A GDPR erasure request arrives for a customer. Where does their data actually live, and what makes this expensive to retrofit?
Where the data lives Longer than people expect, and enumerating it is most of the work: Primary database · read replicas · caches · search indexes · analytical
A design review presents a new event-driven platform. What cost questions do you ask before approving it?
What the interviewer is testing Whether cost is part of your architecture review or an afterthought handled by finance later, and whether you know the specific
A downstream service slows from 50 ms to 3 s. Within two minutes every service in the request path is down, including ones that do not call it. Explain the mechanism and how you would have prevented it.
What the interviewer is testing Whether you understand that most outages are amplification, not failure — and whether you can name the specific mechanism rather
A fleet of 200 instances averages 15% CPU and 85% memory. Finance wants a 40% cost reduction. What do you do?
The diagnosis The instance family is wrong. 15% CPU and 85% memory means the workload is memory bound running on general purpose or compute optimised instances,
A product catalogue page does 40,000 reads per second against a database that can serve 5,000. Walk me through the caching design, including what happens at 3 AM when the cache is empty.
What the interviewer is testing Whether you can design a cache including its failure modes, rather than saying "put Redis in front of it". The base design Cache
An order service must notify inventory, billing, shipping and analytics when an order is placed. Synchronous calls or events? Justify your choice per consumer.
What the interviewer is testing Whether you apply the decision per interaction rather than adopting one style globally. The framing that matters Synchronous cal
Design a URL shortener handling 100 million new links per month and 10 billion redirects. Where is the real difficulty?
What the interviewer is testing The classic warm up. What is being assessed is not whether you can shorten a URL — it is whether you do capacity arithmetic befo
Design an order submission API that is safe when the client cannot tell whether its request succeeded. What exactly do you store, and when?
What the interviewer is testing Whether you know that "make it idempotent" is a design with specific failure modes, not a checkbox. The core design The client g
Design the timeout configuration for a request that passes through gateway, orders, pricing and inventory. What numbers, and what rule generates them?
The rule that generates the numbers One budget at the edge, decreasing inward, with room for a retry at exactly one layer. Start from what the caller will actua
Netflix personalises an entire home page in the time a TV takes to draw a screen. How, given that scoring every title for every member on request is impossible?
What the interviewer is testing Whether you reach for the offline/online split — the single most reusable idea in latency critical personalisation — rather than
Prime Video reported a 90% cost cut by consolidating a serverless distributed service into one process. Does that mean microservices were the wrong choice, and what is the actual decision rule?
What actually happened The Prime Video Video Quality Analysis team's 2023 post describes an audio/video monitoring service built as Step Functions orchestrating
Security wants mandatory hardware keys for every login; the business says it will cost conversions. How do you resolve it architecturally rather than by picking a side?
What the interviewer is testing Whether you treat security as a fixed dial to be turned up, or as a risk proportionate design variable. Both stated positions ar
The 2019 Capital One breach chained a server-side request forgery to an over-permissive IAM role. Walk the chain, and say which single control would have contained it.
The chain, as publicly reported 1. A misconfigured web application firewall permitted server side request forgery — the application could be induced to make an
Three designs need distributed locks: a nightly report, a per-customer state machine, and a global config reload. For each, is a lock the right answer?
The nightly report — a lock is acceptable Purpose: efficiency . Two instances generating the same report wastes compute and possibly sends two emails, but nothi
You inherit an estate of roughly 400 applications, no reliable inventory, and a mandate to reduce cost and risk. What do you do in the first ninety days?
What the interviewer is testing Whether you can sequence work at portfolio scale, and whether you go for evidence before strategy. This is the enterprise archit
Your services currently trust anything inside the VPC. A security review says move to zero trust. What changes, and what will it cost you?
What the interviewer is testing Whether "zero trust" is a concrete set of changes to you, or a slogan. What actually changes Workload identity. Every service ge
Your system handles 1,000 requests per second today. Marketing says a campaign will bring 10,000 next month. What breaks first, and how do you find out?
What the interviewer is testing Whether you reason about bottlenecks systematically or start adding servers. The shape of the answer Scaling is not uniform. Som
Cost per Request
Fully-loaded per-unit cost, including the lines usually left out.
AI Observability
Logging prompts, versions, retrieved context and cost per request.
Unit Economics
Cost per request, per tenant, per transaction — the actionable number.
Licence & Vendor Costs
Per-core, per-seat and per-environment terms that shape designs.
NAT & Egress
Outbound-only connectivity, its per-gigabyte cost and its zone binding.
Polyglot Persistence
Choosing a store per workload, and the operational cost of variety.
Serverless
Scale to zero, per-request billing, cold starts and connection limits.
Strong vs Eventual Consistency
A per-operation decision, resolved by what a stale read would cost.
AI Cost Management
Token accounting, routing, caching and the context-window budget.
Architecture Cost Modelling
Pricing a design before building it, at expected and at ten times volume.
Cost & FinOps
General material on the economics of an architecture.
Cost Allocation
Tagging enforced at provisioning, and apportioning shared costs.
Cost Governance
Budgets, anomaly alerts, quotas and preventive policy.
Cost vs Reliability
Each nine costing an order of magnitude, and pricing the failure instead.
Observability Cost
Telemetry bills, cardinality control and retention tiering.
Performance vs Cost
Buying latency, and knowing what the last millisecond is worth.
Telemetry Cost
Observability bills that rival compute, and where to cut without going blind.
Total Cost of Ownership
Lifetime cost including people, operations, upgrades and exit.
Application Portfolio Management
Inventory, ownership, cost and health for every application.