123 long-form pieces on architecture, training, inference, agents and safety.
In 2002 Jon Kleinberg proved that no clustering function can satisfy three properties almost everyone would ask for. Every algorithm is therefore a definition of what a cluster is, and every validation index is another definition…
Human preference labels are the most expensive ingredient in a modern aligned model. Constitutional AI replaced most of them with a written document and a model judging itself, and the idea quietly took over the alignment stack.
An agent that can act for eight hours still thinks inside a window that holds a few hundred thousand tokens. The gap between those two numbers is where most agent failures now live, and closing it has become its own engineering d…
A million-token window promises perfect recall of everything you feed it. Controlled tests on 18 frontier models show recall degrading steadily, unevenly, and well before the window fills, a pattern researchers now call context rot.
In a 2025 randomized trial, experienced developers using AI tools took 19% longer to finish their tasks while believing they had been 20% faster. Twenty-six years of human-AI interaction research explain the gap: an assistant's v…
Autoregressive models write left to right, one token at a time. Diffusion language models reveal a whole sequence at once and sharpen it over a handful of steps. That single change rewrites the latency math, and in 2025 it stoppe…
Every attention head learns three weight matrices for query, key, and value. A 2026 study trained models up to 1.2B parameters to ask which of them are actually load-bearing, and found that keys and values can share one projectio…
Plug a random forest into a causal regression and the confidence interval tightens around the wrong number, because regularisation bias shrinks more slowly than the standard error. Double machine learning fixes this with two devi…
A static agent pipeline is a diagram you draw before you know the task. A dynamic workflow is a program the agent writes once it sees the task, then runs deterministically. The difference reshapes how fan-out, verification, and c…
A 250-question benchmark carries a standard error of about three percentage points. Most of the model comparisons published on top of such benchmarks cannot distinguish the models they are comparing. Evaluations are experiments, …
Weight quantisation, KV cache eviction, prompt compression and distillation are treated as four separate engineering disciplines with four separate literatures. They are one problem: choosing a point on a rate-distortion curve. S…
A 75% idle GPU is usually not a compute problem. Across the configurations one VLDB study measured, DNN training spent between 10% and 70% of every epoch blocked on I/O, and the storage system was not busy. The data path has its …