LoRA is usually explained as a budget approximation of full fine-tuning: same destination, less memory. Two lines of evidence say that framing is wrong. LoRA learns less, forgets less, and reaches a structurally different solutio…
Three 7B models that each scored under 30% on Japanese maths were averaged into one that scored 52%. No gradient was computed. Weight-space arithmetic is the cheapest capability gain in the field and the easiest one to fool yours…
A prompt optimiser that never touches a weight has been reported to beat GRPO by around six points using up to 35 times fewer rollouts. That result only makes sense once you stop treating the prompt as writing and start treating …
Most teams reach for fine-tuning when they should reach for retrieval. The cost is not just dollars - it is months of confused engineers trying to figure out why the model started lying again.