The Limits of Self-Correction

"Review your answer and correct any mistakes" is the most intuitive prompt-engineering move there is, and the least reliable. On GSM8K, CommonSenseQA and HotpotQA, prompting GPT-3.5 and GPT-4 to revise their own reasoning without external feedback lowered accuracy, because the model changed correct answers more often than it fixed wrong ones (Huang et al., ICLR 2024, Large Language Models Cannot Self-Correct Reasoning Yet, arXiv:2310.01798).

Understanding why is the difference between an agent loop that converges and one that oscillates.

The oracle leak

Several influential early self-correction results shared a methodological flaw: the loop was stopped using the ground-truth label. The model revised until it got the right answer, or the harness stopped it once it did. That protocol measures whether a correct answer exists somewhere in the model's sample space, not whether the model can find it unaided.

Remove the oracle, let the model decide when it is done, and the gains mostly disappear. Any self-correction claim should be read with one question first: what told the loop to stop?

Intrinsic versus extrinsic

The useful taxonomy is by where the feedback comes from.

Intrinsic correction uses only the model's own judgement of its own output. This is the case that fails, and the reason is not mysterious. Generating the answer and evaluating it draw on the same distribution and the same misconceptions. A model that believed the wrong step when writing believes it when reviewing. Worse, the wrong answer is now in the context, so the review is conditioned on it.

Extrinsic correction uses a signal from outside: a compiler, a test suite, a search result, a type checker, a second model with different training. This works, sometimes dramatically. Self-Refine's reported gains come overwhelmingly from settings with a real external signal (Madaan et al., NeurIPS 2023, arXiv:2303.17651), and Reflexion is explicitly built around environment feedback written into episodic memory between attempts (Shinn et al., NeurIPS 2023, arXiv:2303.11366).

The rule of thumb: self-correction works exactly as well as the verifier attached to it. With a unit test, it is powerful. With nothing, it is noise plus a bias toward change.

Why verification should be easier and often is not

The premise behind self-correction is a complexity intuition: checking is easier than solving. That holds where verification is mechanical, as with running a test, checking a proof step, or confirming an arithmetic identity. It does not hold where verification requires the same judgement as generation, which covers most natural-language tasks.

There is also an asymmetry in how the model treats its own text. Having produced an answer, the model conditions on it, and consistency pressure works against revision. This is the same mechanism behind sycophancy pointed inward: the model agrees with itself for the same reason it agrees with a confident user.

Making it work

• Attach a real verifier. Compiler, tests, schema validation, retrieval against a source. Everything else is decoration.
• Separate the critic from the generator. A different prompt, a different model, or a critic that never sees the original answer breaks the conditioning.
• Critique against explicit criteria. "Check that every numeric claim appears in the source" beats "review for mistakes", because it names what to look at.
• Bound the loop. Two rounds captures nearly all available gain in published results; more rounds mostly produce drift.
• Require a stated reason for change. Forcing the model to name the defect before rewriting suppresses gratuitous edits.

When it breaks

Confidence is uncalibrated on the second pass. A model asked to rate its own revision reports high confidence whether or not it improved, so self-assessed confidence cannot be used as the stopping rule.

Correcting correct answers is the dominant cost. Track the two rates separately. A loop with a 30 percent fix rate on wrong answers and a 15 percent break rate on right ones is net negative on any task where most answers start correct.

Reasoning models blur the boundary. Long-CoT models revise inside a single generation, with backtracking learned through RL against a verifiable reward rather than prompted. That is a different mechanism from prompted intrinsic self-correction, and the negative results above do not automatically transfer to it. What does transfer is the dependence on a verifier: the RL training that produced the behaviour had one, even if inference does not.