Teams tune the embedding model and the vector database, then wonder why answers are still wrong. Both are the least important parts of the stack. Retrieval is a two-stage ranking problem, and information retrieval solved the shap…
A 700-draw run of the eight-schools model reported R-hat of 1.01 and put the 2.5% quantile of the between-school spread at 0.89, while the exact posterior holds almost 18% of its mass below that value. Calling a sampler takes one…
In 2023 GPT-4 scored 97% on a classic cause-and-effect benchmark and 62% on one that hands it the causal graph and asks it to compute. Both results fit a theorem proved in 2020: data from one rung of Pearl's ladder almost never d…
Embedding models are chosen from a leaderboard more often than from an experiment, and the leaderboard now publishes training splits for its own test sets. Between contamination, task-family averaging and geometry no benchmark me…
Run exact search over a trained translation model and, for more than half of a standard test set, the highest-scoring output is the empty string. Beam search works only because it fails to find what the model actually prefers. Th…
On one synthetic honeypot evaluation, Claude Sonnet 4.5 said out loud that it suspected it was being tested in 80 to 100 percent of transcripts, against under 10 percent for its predecessor. When the internal representations behi…
In 2019 a careful team could reproduce only 7 of 18 neural recommenders from top venues, and 6 of those 7 lost to nearest-neighbour heuristics. The models were not the problem. The protocol was: untuned baselines, metrics compute…
o3, R1 and Claude extended thinking are a real capability shift on a narrow slice of tasks. They are also being shoved into product surfaces that punish every property reasoning models exhibit - and the bill is starting to arrive.
A dead Atlantic salmon, scanned in 2009, showed 16 'active' voxels at p below 0.001; every procedure that controlled an error rate across the family found none. This is the argument over what that error rate should be, from Holm …
A language model eats trillions of tokens scraped for free. The largest open robot dataset is 527 skills gathered by hand across 21 institutions. That asymmetry, not model capacity, is what makes robot learning hard, and it expla…
In 2019 a masked language model recalled facts almost as well as a relation extractor with an oracle entity linker, and people asked whether knowledge graphs were finished. They were not, but the reason is narrower than their adv…
Two hundred dollars of API calls pulled more than ten thousand verbatim training examples out of ChatGPT. Memorisation is not a defect that better engineering removes; it scales log-linearly with everything the field is scaling. …