Portfolio ↗

Quiz

2741 questions of the kind that actually get asked — in interviews, in architecture review boards, and by the person who has to run the thing at 3 AM. Every answer states the trade-off rather than the slogan, and says when the obvious choice is the wrong one.

All areas2741 Architecture Fundamentals81 Distributed Systems101 Data Architecture90 Cloud Architecture87 Networking86 API & Integration Architecture78 Reliability & Resilience99 Observability92 Performance & Capacity Engineering90 Security Architecture95 Cost Architecture & FinOps92 Business Architecture93 Architecture Communication91 Enterprise Architecture91 Legacy Modernization92 AI-Era Architecture96 Software Architecture & Engineering84 Architecture Patterns84 Architecture Decision-Making91 The Architect's Meta-Skills92 Delivery & Release Engineering93 Platform Engineering & Developer Experience92 Testing & Quality Architecture102 Data Platform Architecture98 Streaming & Real-Time Data93 Data Governance & Semantics91 Frontend & Experience Architecture91 Edge, Mobile & IoT88 Regulatory & Data Protection Architecture90 Assurance, Audit & Model Risk98
Difficulty Any beginner intermediate advanced

98 questions in Assurance, Audit & Model Risk.

  1. Model Documentation intermediate

    A model makes decisions affecting people. What documentation is actually required, and who is it for?

    2 min answer gustomodel-documentationlimitationsaccountability
  2. Model Documentation intermediate Multiple choice

    A support assistant answers questions by retrieving from an internal wiki and a ticket archive and passing the result to a hosted model. Your model documentation template has a training-data section you cannot fill because you did not train the model. Which section should replace it?

    2 min answer model-documentationragcorpus-provenanceretention
  3. Model Documentation beginner

    A team is asked to produce documentation for a model that scores loan applications. They propose a technical write-up of the architecture and the training data. What is missing, who actually reads this document, and what is it not?

    2 min answer model-cardsdocumentationgovernanceintended-use
  4. Model Documentation intermediate

    What should model documentation contain, and who is it actually for?

    1 min answer model-documentationtransparencylimitationsreuse
  5. Model Documentation intermediate

    Your model documentation for a customer-facing assistant records the provider's model version it was validated against and the evaluation scores from that run. The Azure deployment of that hosted model is set to auto-update to the provider's default version. Overnight the provider's default changes. What happens next?

    3 min answer model-documentationversioningvalidationazure
  6. Model Evaluation & Red-Teaming advanced Multiple choice

    A generative assistant is about to be deployed to customer support, drafting replies that agents can edit before sending. Its retrieval corpus is the company's internal knowledge base. Which pre-deployment check most reduces the risk that actually matters here?

    2 min answer red-teamingevaluationragdeployment
  7. Model Evaluation & Red-Teaming advanced

    A team must demonstrate that a model behaves acceptably before deployment. What does evaluation need to cover, and what does red-teaming add?

    2 min answer scale-aievaluationred-teamsubgroups
  8. Model Evaluation & Red-Teaming advanced

    An interviewer says — you are launching a language-model feature to 2 million users. Design the evaluation gate that decides whether it ships, and tell me what it cannot tell you. Where do you take this?

    3 min answer llmevaluationred teamrelease gate
  9. Model Evaluation & Red-Teaming advanced

    Design pre-deployment evaluation for a generative AI system in a customer-facing context.

    1 min answer evaluationred-teamingadversarialsafety
  10. Model Risk Management advanced

    A deployed model performed well in validation and its business metric has declined over four months. Nothing has been deployed. What do you investigate?

    2 min answer aimonitoringdriftmodel-risk
  11. Model Risk Management advanced

    A recommendation model is retrained monthly. The challenger is promoted automatically when it beats the champion on an offline metric computed from the last 30 days of logged interactions. It passes every month, and live engagement has been flat for a year. What is happening?

    2 min answer champion challengerfeedback loopoffline evaluationmodel risk
  12. Model Risk Management advanced

    A vendor SaaS product embeds a model that scores customers, and its output drives an automated decision in your process. Your model governance framework covers models you build. What do you do?

    2 min answer model-riskthird-partygovernanceai