Pretraining economics, optimisers, RL post-training and scaling behaviour.
A 2-billion-parameter model that trades blows with one ten times its size is not an accident of architecture. It is the product of a teacher pouring its full probability distribution into a student, token by token.
Human preference labels are the most expensive ingredient in a modern aligned model. Constitutional AI replaced most of them with a written document and a model judging itself, and the idea quietly took over the alignment stack.
For years the field raced to build bigger models. Then a 2022 study showed most of those giants were undertrained, and quietly redrew the map of how to spend a compute budget.
For eight years Adam owned every serious pretraining run. Then a competitive-speedrun optimizer that orthogonalizes its own gradient updates scaled from a 124M-parameter toy to a trillion-parameter model with zero loss spikes. He…
Replace the reward model with a function that simply checks the answer, and a frontier reasoning model falls out of pure reinforcement learning. The catch is what 'checkable' quietly assumes, and what the model learns to exploit.
DPO is the right default for almost every preference-tuning project in 2026, but the cases where PPO still wins are sharper and more common than the simplicity pitch admits.
DeepSeek shipped a 1.6T-parameter model in April 2026 with native Ascend inference support while OpenAI committed $500B to Stargate. Both can be right, but only one of them is a moat.
GRPO weights every token by its own importance ratio, and on long responses that single-sample estimator quietly poisons the gradient until the model collapses. GSPO moves the ratio up to the whole sequence, and Qwen3's largest m…