practice

Use Case Risk Classification

Assigning an AI application to a risk tier based on the consequence of it being wrong, which then determines the obligations that apply.

Regulation of AI is converging on a proportionate approach: obligations scale with potential harm rather than applying uniformly. Classification is therefore the first architectural step, because it determines what the system must be built to do.

The consequence-based questions that drive it: does the system make or materially influence a decision about a person's access to employment, credit, education, housing, healthcare or justice? Does it operate on vulnerable groups? Is there meaningful human review, or is the output acted on automatically? Can an affected person contest the outcome? What is the harm if it is wrong, and is it reversible?

High-tier classification brings requirements that are design constraints rather than documentation: risk management throughout the lifecycle, data governance over training data, technical documentation, logging sufficient to trace a particular decision, human oversight that can actually intervene, and stated accuracy and robustness levels.

The two errors are symmetrical. Classifying everything as high applies expensive obligations to a spam filter and exhausts the capacity to do it properly. Classifying a consequential system as low because it is "only a recommendation" ignores that a recommendation acted on without scrutiny is a decision.