Detecting Synthetic Media Without Watermarks

Passive detection asks whether content is machine-generated using only the content itself, with no cooperation from whoever produced it. It is the capability most people want, and it is the one with the weakest evidence base, in a way that has already produced real harm.

What the detectors use

Statistical signals in text. Generated text has lower perplexity under a language model, lower burstiness in sentence length and structure, and a distribution over token ranks that differs from human writing. Zero-shot methods such as curvature-based detection exploit the observation that generated text sits near a local maximum of the model's likelihood in a way human text does not.

Learned classifiers train on paired human and machine text. They achieve high accuracy on the distribution they were trained on and degrade sharply on a different generator, a different domain, or a different decoding configuration.

Artefacts in images and video. Frequency-domain traces of upsampling, inconsistent lighting and reflections, physiological implausibilities. Each generation of generative models removes the previous generation's artefacts, so detectors trained on one generation lose to the next.

Why deployment is different from evaluation

Reported accuracies come from balanced test sets of clean, unedited outputs from known generators. Deployment has none of those properties.

The generator is unknown and may postdate the detector. Text has been edited by a human, which is the normal case for anything consequential. Decoding parameters differ from the detector's training data. And the base rate is not fifty percent.

That last point dominates. Suppose a detector has 95 percent true positive and 95 percent true negative rates, and 1 percent of submissions are generated. Out of 10,000 submissions, 100 are generated and 95 are caught; 9,900 are human and 495 are flagged wrongly. Of 590 flagged, 495 are innocent: a precision of 16 percent. Excellent-sounding rates produce a system that is wrong five times out of six when it accuses.

The documented harm

Detectors have been shown to flag text by non-native English writers at substantially higher rates, because simpler syntax and lower lexical variety resemble the statistical signature of generated text. Deploying such a detector in an academic setting produces a disparate false accusation rate against a specific population, which is a fairness failure with individual consequences.

Several major providers of detection tools have withdrawn them citing low accuracy, and institutional guidance has moved toward not using detection scores as evidence in disciplinary processes.

When it breaks

A score is not evidence about an individual. Aggregate detection can be useful for measuring prevalence in a corpus, where errors average out. Accusing a person requires a standard the detector cannot meet, and the distinction between population-level measurement and individual determination is the one most often collapsed.

Adversaries adapt cheaply. Paraphrasing, light editing and prompting for an unusual style all reduce detector accuracy, and none costs an attacker anything meaningful. The detector is a fixed target and the generator is not.

The arms race is asymmetric. Detection must generalise to generators that do not yet exist, while generation only needs to defeat detectors that already do. That asymmetry is structural, and it is the reason provenance approaches that require producer cooperation are being pursued despite their own limitations.

Uncertainty must be communicated or it is discarded. A detector output presented as a percentage is read as a probability of guilt by people who will not read the methodology. Where such a tool is used at all, the interface has to carry the false positive rate and the population it was validated on, and most do not.