A platform must prevent harmful outputs in a user-facing AI feature. Where should guardrails sit, and what does each layer catch?

The layers

1. Input filtering. Rejects clearly abusive requests before any expensive work. Cheap, catches the obvious, and easily circumvented — so it is a first filter rather than a defence.

2. Retrieval scoping. The model can only ground its answer in content the user may see and that the platform is willing to surface. This eliminates a whole class of problem by construction rather than by detection, and it is the most reliable layer.

3. Prompt-level constraints. Instructions about scope and behaviour. Useful, and not a security control — instructions in a prompt are advisory relative to a determined adversary and to content that arrives later in the context.

4. Output filtering. Classification of the generated output before it is shown. Catches what earlier layers missed, at the cost of latency and of a decision about what to do when it fires — which must be designed rather than defaulted.

5. Structural constraints. Where the output shape can be constrained — an enumerated value, a structured object, a citation-bearing answer — the space of harmful outputs shrinks dramatically. This is far stronger than filtering free text and is under-used.

6. Human review for high-consequence outputs, with the volume made tractable by the earlier layers.

The trade that must be made explicitly

False positives versus false negatives, and the correct balance is domain-specific. A community platform blocking legitimate speech has a real cost; a platform allowing harmful content has a different one. Setting the threshold is a product and policy decision, not a technical default.

Two consequences follow: the threshold must be adjustable without deployment, and both error rates must be measured — a guardrail with no false-positive measurement is being tuned blind.

What guardrails cannot do

They cannot make an unsafe architecture safe. If the model has access to data it should not, or can invoke tools with excessive permissions, output filtering is the wrong layer to fix it. Scope and authorisation are structural; filtering is a backstop.

The operational requirements

• Appeal and correction paths, since false positives will occur and an unappealable block is a bad product.
• Logging of every guardrail activation, so patterns are visible and thresholds can be tuned with evidence.
• Versioned, evaluated guardrail changes, because a threshold change is a behaviour change to every user.
• A defined behaviour when the guardrail service is unavailable — fail closed for high-risk features, fail open for low-risk ones, decided deliberately per feature.