A content platform must choose between a large hosted model, a smaller hosted model and a self-hosted open model for a moderation task. What decides it?

What decides it

• The quality bar for this specific task, measured. Not general benchmark performance — a held-out set of your own examples with your own labels. A smaller model is frequently sufficient for a narrow, well-defined task, and moderation is narrow.
• Volume and unit cost. At high volume the cost difference between model sizes dominates every other consideration, and moderation runs on every item rather than on a sample.
• Latency requirements. Synchronous pre-publication moderation has a hard budget; asynchronous post-publication review does not, which changes the answer completely.
• Data residency and sensitivity. Sending user content to a third-party provider may be constrained by regulation or by the platform's own commitments, and that constraint can decide it regardless of the other factors.
• The operational cost of self-hosting, including GPU capacity management, model updates, and the expertise to run inference well. This is systematically underestimated and is usually larger than the licence saving for a small team.

The design that avoids committing

A cascade. A cheap fast model handles the clear cases — which for moderation is the overwhelming majority — and escalates uncertain cases to a larger model or to a human. This is usually a large cost reduction with no quality loss, because the expensive path runs on a small fraction of traffic.

It also degrades gracefully: if the large model is unavailable, the cascade can raise its escalation threshold rather than failing.

What must exist regardless

An evaluation set and a measurement harness, so that a model change is a measured decision rather than a guess. Providers release new versions, deprecate old ones and change behaviour without notice, so the question "did that change make things worse" must be answerable in an afternoon — and without a harness it is not answerable at all.

The multilingual complication

For a platform serving many languages, model quality varies enormously by language and a single choice is wrong for most of them. The evaluation set must be per language, and the answer is frequently different per language — which is an architectural requirement for per-language routing rather than a single model choice.