Portfolio ↗

Quiz

2948 questions of the kind that actually get asked — in interviews, in architecture review boards, and by the person who has to run the thing at 3 AM. Every answer states the trade-off rather than the slogan, and says when the obvious choice is the wrong one.

All areas2948 Architecture Fundamentals91 Distributed Systems101 Data Architecture90 Cloud Architecture87 Networking96 API & Integration Architecture87 Reliability & Resilience99 Observability92 Performance & Capacity Engineering100 Security Architecture105 Cost Architecture & FinOps102 Business Architecture103 Architecture Communication102 Enterprise Architecture100 Legacy Modernization102 AI-Era Architecture106 Software Architecture & Engineering93 Architecture Patterns94 Architecture Decision-Making101 The Architect's Meta-Skills102 Delivery & Release Engineering103 Platform Engineering & Developer Experience102 Testing & Quality Architecture102 Data Platform Architecture98 Streaming & Real-Time Data103 Data Governance & Semantics91 Frontend & Experience Architecture101 Edge, Mobile & IoT98 Regulatory & Data Protection Architecture99 Assurance, Audit & Model Risk98
Difficulty Any beginner intermediate advanced

2948 questions.

  1. Model Evaluation & Red-Teaming advanced

    An interviewer says — you are launching a language-model feature to 2 million users. Design the evaluation gate that decides whether it ships, and tell me what it cannot tell you. Where do you take this?

    3 min answer llmevaluationred teamrelease gate
  2. Model Evaluation & Red-Teaming advanced

    Design pre-deployment evaluation for a generative AI system in a customer-facing context.

    1 min answer evaluationred-teamingadversarialsafety
  3. Model Risk Management advanced

    A deployed model performed well in validation and its business metric has declined over four months. Nothing has been deployed. What do you investigate?

    2 min answer aimonitoringdriftmodel-risk
  4. Model Risk Management advanced

    A recommendation model is retrained monthly. The challenger is promoted automatically when it beats the champion on an offline metric computed from the last 30 days of logged interactions. It passes every month, and live engagement has been flat for a year. What is happening?

    2 min answer champion challengerfeedback loopoffline evaluationmodel risk
  5. Model Risk Management advanced

    A vendor SaaS product embeds a model that scores customers, and its output drives an automated decision in your process. Your model governance framework covers models you build. What do you do?

    2 min answer model-riskthird-partygovernanceai
  6. Model Risk Management advanced

    An independent validator asks you to reproduce a model's training run from eight months ago. Can you? What is needed?

    2 min answer mlreproducibilitygovernance
  7. Model Risk Management advanced

    An organisation deploys machine learning models in decisions affecting customers. What does model risk management require?

    1 min answer model-riskvalidationmonitoringgovernance
  8. Model Selection advanced

    A cascade sends every request to a small model and escalates to the frontier model when the small model's self-reported confidence is low. Escalation ran at 18% in week one. After a prompt edit meant to stop the small model hedging, escalation fell to 2% and cost per request fell 40%. Three weeks later a customer audit finds a run of wrong answers. What failed, and which design decision allowed it?

    3 min answer model-selectioncascaderoutingquality-signals
  9. Model Selection intermediate Multiple choice

    A client wants a model that follows their specific document formatting conventions and answers from their knowledge base. What do you recommend?

    2 min answer ragfine-tuningarchitecture
  10. Model Selection intermediate

    A content platform must choose between a large hosted model, a smaller hosted model and a self-hosted open model for a moderation task. What decides it?

    2 min answer sharechatmodel-selectioncostlatency
  11. Model Selection advanced

    A platform must choose which model serves which request class. What should drive the decision, and what changes over time?

    2 min answer model-selectionroutingevaluationcost
  12. Model Selection intermediate

    A recommendation service's model server is down at peak. Should the application fall back to an older model, precomputed candidates, popularity baselines or cached per-user results — and how is the fallback tested?

    2 min answer fallbackdegradationrecommendationsresilience