1. Outcome Measurement advanced

    A marketplace wants to A/B test a change to its matching algorithm. Why is this harder than testing a button colour?

    2 min answer experimentationmarketplaceinterferencemethod
  2. Outcome Measurement intermediate

    A nine-month consolidation retired four systems and released about 0.4 FTE of support effort in each of four teams. The programme closed as a success, verified by the delivery plan. Eleven months after the business case was signed, finance reports run cost unchanged and the £1.2m annual saving nowhere in the ledger. What failed, and which decision made it possible?

    3 min answer benefitsoutcome measurementfinancedecommissioning
  3. Outcome Measurement intermediate

    A platform ships a major architectural change and declares it successful because it was delivered on time. What should have been measured?

    2 min answer outcome-measurementhypothesisevidencearchitecture
  4. Outcome Measurement advanced

    A travel platform runs hundreds of concurrent A/B experiments as standing practice; Booking.com has built its product development around very large numbers of simultaneous experiments for years. An architecture team tests a new read-through caching layer in front of the availability service by routing 50% of sessions through it. After nine days the treatment arm shows 1.4% higher conversion at p below 0.01. Both arms are served by the same cache cluster and the same application fleet. What actually happens to the measurement and what would have to be true for the result to mean what the team thinks it means?

    3 min answer experimentationinterferencerandomisation-unitmeasurement
  5. Outcome Measurement intermediate

    An engineering organisation reports velocity, deployment frequency and story points. Leadership cannot tell whether the investment is working. What is missing?

    2 min answer chargebeeoutcomesmetricsdelivery
  6. Outcome Measurement advanced

    How would you determine whether a migration completed six months ago was worth doing?

    2 min answer outcomesmeasurementcounterfactualattribution
  7. OWASP Risks intermediate

    A marketplace hosts seller-generated content including descriptions, images and shop pages. Which common web risks are amplified, and what controls apply?

    2 min answer owaspxssuser-contentcsp
  8. OWASP Risks intermediate Multiple choice

    A penetration test reports that every object identifier in your API is a sequential integer and flags insecure direct object reference. Ownership is checked in shared middleware on every route. Three endpoints still returned another user's data: a bulk update that takes an array of identifiers, a nested comment fetch, and a CSV export. Where do you look first?

    3 min answer broken-access-controlidorobject-level-authorizationmiddleware
  9. OWASP Risks advanced

    Broken access control is consistently the top application security risk. Which architectural decisions make it structurally less likely rather than relying on review?

    2 min answer access-controlowaspstructureauthorisation
  10. Pagination & Filtering advanced

    A discussion platform paginates a feed whose ranking changes continuously as votes arrive. Offset pagination shows duplicates and gaps. What are the options, and what does each cost the product?

    2 min answer paginationcursorsrankingfeeds
  11. Pagination & Filtering intermediate

    A list endpoint returning 2 million records times out on deep pages and returns 500 for validation failures. Fix both properly.

    2 min answer paginationcursorserrorshttp-semantics
  12. Pagination & Filtering intermediate Multiple choice

    An insurance API returns policy and claim lists that can be very large. Which pagination approach should be used, and what breaks with the obvious one?

    2 min answer ackopaginationcursorsconsistency