Search the practice set
126 questions, 454 terms and 400 topics in 20 areas.
60 results for “AI Cost Management”
AI Gateway
A shared proxy in front of model providers that centralises routing, keys, quotas, caching, logging and safety policy.
Application Portfolio Management
Maintaining an inventory of every application with its owner, cost, business value and technical health, and using it to decide what to invest in, replace or retire.
Context Window
The maximum number of tokens a model can attend to in one request, holding the system prompt, history, retrieved context, tools and the answer.
Human in the Loop
Requiring human review or approval at a defined point in an automated flow, chosen by the reversibility and cost of the action.
Model Router
Directing each request to a model chosen by the task's difficulty, cost and latency budget, rather than sending everything to the largest model available.
Prompt Registry
A versioned store of production prompts with their model bindings, parameters and evaluation results, so a prompt change is a reviewable, traceable, reversible deployment.
Semantic Cache
Caching model responses keyed by the meaning of the request rather than by its exact text, so near-duplicate questions are served without a model call.
Architecture Cost Model
A calculation, made during design, of what an architecture will cost to run at expected and at peak volume.
Consent Management
Capturing, storing, honouring and evidencing a data subject's permissions for specific processing purposes, including withdrawal.
Cost per Request
The fully-loaded infrastructure cost of serving one request, and the unit that makes architectural cost decisions comparable.
Cost vs Reliability Trade-off
The non-linear relationship between availability and spend, which makes each additional nine roughly an order of magnitude more expensive.
Egress Cost
The charge for moving data out of a cloud provider or across its zones and regions — usually the least-anticipated line on a cloud bill.
Global Traffic Management
The layer that decides which region a given user reaches, using DNS, anycast or an edge network, and that performs regional failover.
Identity and Access Management
The system of record for principals, credentials and permissions, and the policy engine that decides what each principal may do.
Offset Management
How a consumer records its position in a stream, and the decision that determines whether processing is at-least-once or at-most-once.
Secrets Management
Storing, distributing, rotating and auditing credentials so that they never live in code, images or configuration files.
Total Cost of Ownership
The full lifetime cost of a capability, including the people, operations, upgrades and exit that a licence comparison leaves out.
Account Vending
Automated creation of new cloud accounts pre-configured with the organisation's networking, identity, logging, guardrails and cost allocation.
Blameless Postmortem
An incident review that seeks the systemic conditions that made a failure possible, explicitly excluding individual fault.
Build vs Buy
The choice between developing a capability in-house and acquiring it, decided on differentiation and total cost rather than on feature lists.
Burstable Instance
An instance that provides a low baseline CPU allocation and accrues credits while idle, spendable for short periods of full performance.
Cardinality
The number of distinct time series produced by a metric, which is the product of the distinct values of all its labels — and the main driver of monitoring cost.
Chargeback and Showback
Attributing cloud cost to the teams that generate it — either informationally (showback) or by moving it onto their budget (chargeback).
An LLM feature that worked last week now gives worse answers. Nothing was deployed. How do you find out what changed, and what should have been in place?
What the interviewer is testing Whether you treat an AI feature as a system with configuration and dependencies, or as a black box that mysteriously drifts. Wha
A design review presents a new event-driven platform. What cost questions do you ask before approving it?
What the interviewer is testing Whether cost is part of your architecture review or an afterthought handled by finance later, and whether you know the specific
A fleet of 200 instances averages 15% CPU and 85% memory. Finance wants a 40% cost reduction. What do you do?
The diagnosis The instance family is wrong. 15% CPU and 85% memory means the workload is memory bound running on general purpose or compute optimised instances,
Prime Video reported a 90% cost cut by consolidating a serverless distributed service into one process. Does that mean microservices were the wrong choice, and what is the actual decision rule?
What actually happened The Prime Video Video Quality Analysis team's 2023 post describes an audio/video monitoring service built as Step Functions orchestrating
Security wants mandatory hardware keys for every login; the business says it will cost conversions. How do you resolve it architecturally rather than by picking a side?
What the interviewer is testing Whether you treat security as a fixed dial to be turned up, or as a risk proportionate design variable. Both stated positions ar
You are asked to give an internal AI agent access to the customer database, the ticketing system and outbound email so it can resolve support tickets. What is your response?
What the interviewer is testing Whether you recognise a specific and well documented security pattern, and whether you can propose a workable design instead of
You inherit an estate of roughly 400 applications, no reliable inventory, and a mandate to reduce cost and risk. What do you do in the first ninety days?
What the interviewer is testing Whether you can sequence work at portfolio scale, and whether you go for evidence before strategy. This is the enterprise archit
Your services currently trust anything inside the VPC. A security review says move to zero trust. What changes, and what will it cost you?
What the interviewer is testing Whether "zero trust" is a concrete set of changes to you, or a slogan. What actually changes Workload identity. Every service ge
A client wants an assistant that answers questions from 50,000 internal documents which change weekly. RAG or fine-tuning? What actually determines the quality?
What the interviewer is testing Whether you understand what each technique actually does, and whether you know that RAG quality is a retrieval problem. Why RAG
A team wants to build a new internal API on serverless functions. It will serve steady traffic of about 200 requests per second during business hours. What do you advise?
What the interviewer is testing Whether you can apply the serverless trade off to a specific workload rather than treating it as a default good or a default bad
An estate has database passwords in environment variables across 200 services. Design the migration to a secrets manager.
Sequence it by risk, not by convenience Phase 0 — stop the bleeding. Secret scanning in CI and on the existing repositories, blocking new commits containing cre
AI Cost Management
Token accounting, routing, caching and the context-window budget.
AI Observability
Logging prompts, versions, retrieved context and cost per request.
Application Portfolio Management
Inventory, ownership, cost and health for every application.
Log Management
Aggregation, retention tiering, search and the cost of keeping everything.
Prompt & Version Management
Prompts as reviewed, versioned, evaluated production configuration.
Model Selection
Capability, latency, cost and the evaluation that decides between them.
AI Gateways
Centralised routing, keys, quotas, caching, logging and safety policy.
AI-Era Architecture
General material on architecting systems that include models.
Architecture Cost Modelling
Pricing a design before building it, at expected and at ten times volume.
Cost & FinOps
General material on the economics of an architecture.
Cost Allocation
Tagging enforced at provisioning, and apportioning shared costs.
Cost Governance
Budgets, anomaly alerts, quotas and preventive policy.
Cost per Request
Fully-loaded per-unit cost, including the lines usually left out.
Cost vs Reliability
Each nine costing an order of magnitude, and pricing the failure instead.
Identity & Access Management
Workload identity, roles, permission boundaries and usage-based review.
Incident Management
Command roles, severity levels and mitigation before diagnosis.
Key Management
Rotation, separation of duty, envelope encryption and crypto-shredding.
Observability Cost
Telemetry bills, cardinality control and retention tiering.
Performance vs Cost
Buying latency, and knowing what the last millisecond is worth.
Secrets Management
Runtime injection, dynamic credentials and rotation applications survive.
Telemetry Cost
Observability bills that rival compute, and where to cut without going blind.
Total Cost of Ownership
Lifetime cost including people, operations, upgrades and exit.
Agent Architectures
Loops, planning, memory and the boundaries an agent must not cross.
Build vs Buy
Differentiation, five-year TCO, and the exit cost of each option.