Coordinated Flaw Disclosure and Researcher Safe Harbour

Independent evidence is the scarcest input to any AI assurance argument, and the reason is not technical. A researcher who finds that a deployed model reliably produces a harmful output has to probe an account governed by terms of service that prohibit much of what probing requires, and publish a finding the provider would rather frame differently. The dominant risk in that work is account suspension and legal exposure, not measurement error.

Why the security playbook does not transfer

Coordinated vulnerability disclosure in software rests on assumptions that a model flaw mostly violates.

Software vulnerability disclosure assumes a patch exists and ships. Google's Project Zero works to a 90-day vendor deadline, publishing 30 days after a fix reaches users or at the deadline if none does, with a short extension when a patch is imminent (Google Project Zero, Vulnerability Disclosure FAQ). That machinery presumes a discrete defect with a discrete fix. A jailbreak class is a property of a distribution; the "fix" is a classifier update or a fine-tune that shifts a rate rather than closing a hole, and the flaw often reappears at the next release.

It assumes the flaw is binary and reproducible. A model flaw is statistical: it manifests at some rate, under some sampling temperature, for some phrasings. Two researchers running the same prompt get different results, so triage needs a rate and a confidence interval where security needs a proof of concept.

It assumes scope is clear. In software the asset boundary is a product version. For a model, a flaw found in the base model may or may not exist in the chat product, the API, a fine-tune, a distilled sibling, or a downstream integrator's deployment, and the researcher usually has access to exactly one of those.

The Coordinated Flaw Disclosure proposal adapts the process to these differences, adding extended model cards so findings have somewhere to accumulate, dynamic scope expansion for flaws that turn out to reach further than the original asset, an independent adjudication panel for disputes between reporter and provider, and automated verification to handle statistical reproduction (Cattell, Ghosh & Kaffee, 2024, Coordinated Flaw Disclosure for AI: Beyond Security Vulnerabilities, AIES, arXiv:2402.07039).

Safe harbour, and what its absence costs the evidence base

The chilling-effect argument is documented rather than speculative. A multi-author position paper at ICML 2024 argued that the terms of service and enforcement practices of major providers discourage good-faith safety testing, that researchers fear account suspension and legal action for the work and for publishing it, and that provider-run researcher access programmes are a weak substitute for independent access given their limited representation, thin funding and alignment with corporate incentives (Longpre et al., 2024, Position: A Safe Harbor for AI Evaluation and Red Teaming, PMLR 235). The proposal is a legal and technical safe harbour: a commitment not to pursue good-faith evaluation, plus account-level protection so that the research does not end in a ban.

Bounty programmes are the narrow, provider-controlled version of the same thing. Published AI safety bounties have offered rewards in the tens of thousands of dollars for universal jailbreaks in high-risk domains, with reported figures differing between sources and over time, and they typically run by application or invitation and under non-disclosure. That design buys the provider triage quality and control of the narrative; it does not produce independent evidence, because the scope, the acceptance criteria and the right to publish all sit with the party being examined.

When it breaks

Disclosure timelines have no natural length. Ninety days is calibrated to patch-and-deploy cycles. For a flaw that requires retraining, ninety days may be impossible; for one fixable by a filter update, it is an eternity during which the flaw is exploitable. Fixed deadlines are the wrong instrument, and the alternative, negotiated per-finding timelines, reintroduces the power asymmetry the norm existed to remove.

Publication is the enforcement mechanism, and it is the part most easily suppressed. Coordinated disclosure works in security because the researcher can publish at the deadline. Remove that and coordination becomes submission. Account-level dependence makes this worse than in software: the researcher needs continued access to the very system whose provider they are about to embarrass.

Harm is sometimes caused by the disclosure. A reproducible universal jailbreak in a high-uplift domain is dangerous to publish in full. Security's answer, publishing the existence and withholding the exploit, is available but weakens verification, since a withheld prompt cannot be independently reproduced.

Adjudication panels need an institution nobody has built. The proposal depends on a third party with the standing to settle a dispute over whether a finding is a flaw. Funding, composition and jurisdiction for such a body are unresolved, and in its absence the provider adjudicates its own case.

A bounty is not a safe harbour. Programmes with NDAs, invite-only entry and provider-defined scope produce findings the provider has already agreed to receive. They are a useful control and they cannot support a claim of independent scrutiny in an assurance argument.