1. Non-Functional Test Strategy advanced

    A trading platform's functional tests all pass and production fails under load. What should the non-functional test strategy contain?

    2 min answer zerodhaload-testingnfrsoak
  2. Non-Functional Test Strategy advanced

    ByteDance's Monolith paper (RecSys 2022) describes a recommendation system with minute-level parameter synchronisation whose training parameter servers are snapshotted only once a day, so a server failure loses up to a day of updates. The authors say they expected to need more frequent snapshots. What did they do instead of arguing about it, and what does that give a non-functional test strategy?

    3 min answer bytedancemonolithrecovery-objectivenfr
  3. Non-Functional Test Strategy advanced

    On 31 January 2017 a GitLab engineer removed the wrong PostgreSQL data directory, and the recovery mechanisms they believed they had turned out not to work. Roughly six hours of data was lost. Which property had never been tested, and what does that say about non-functional testing generally?

    2 min answer gitlabbackupsrestore-drilldr
  4. Non-Functional Test Strategy advanced

    Which non-functional properties can be tested pre-release, and which can only be verified in production?

    2 min answer non-functionalperformanceresilienceproduction
  5. Performance Test Design advanced

    A load test shows p99 latency of 120 ms at 2,000 requests per second. Production at the same rate shows 900 ms. Why?

    2 min answer performanceload-testingmethodology
  6. Performance Test Design advanced

    A team must load test at 10× normal traffic and a scaled-down environment tells them nothing. How is load testing in production done safely, and what does it find that nothing else does?

    2 min answer flipkartload-testingproductionisolation
  7. Performance Test Design intermediate Multiple choice

    A test must sustain 2000 requests per second against a service whose p50 is 120 ms, with 1 second of think time per virtual user between requests. Roughly how many virtual users does the generator need and what breaks first?

    2 min answer load-testinglittles-lawvirtual-usersconcurrency
  8. Performance Test Design advanced

    Alibaba has published that it rehearses Singles' Day with full-link stress testing — load generated against the production environment, using traffic shading and data isolation. Your platform faces a comparable annual peak. What does testing in production buy that a staging load test cannot, and what has to be built first?

    4 min answer alibabaload testingtesting in productionshadow data
  9. Performance Test Design advanced

    What makes a performance test predictive of production behaviour, and what is the most common design flaw?

    2 min answer performance-testingdistributionrealismbreaking-point
  10. Quality Gates intermediate

    A 12-year-old service has 4% test coverage and no tests for its core logic. It changes twice a year. Do you apply your quality gates to it?

    2 min answer legacypragmatismgovernance
  11. Quality Gates advanced

    A security vendor's release process gates sensor code behind staged rollout with canaries, and ships detection content through a separate path validated by a content checker and pushed to every customer within minutes. In July 2024 CrowdStrike's own root cause analysis describes exactly this shape, and a mismatch between 21 fields supplied and 20 fields read produced an out-of-bounds read that crashed around 8.5 million Windows hosts. Review the process. What would you remove, what would you add, and what would you leave alone?

    3 min answer crowdstrikerelease-gatesstaged-rolloutconfig-as-code
  12. Quality Gates intermediate

    An organisation's quality gates have accumulated and now block releases without reducing defects. How should the gate set be rationalised?

    2 min answer quality-gatesevidenceblockingrationalisation