124 long-form pieces on architecture, training, inference, agents and safety.
You add a KL penalty to an RLHF objective, log it, and it prints minus 0.03. KL divergence is provably non-negative, and nothing is broken. One formula does four different jobs in modern machine learning, and almost every confusi…
Training a frontier model is a one-time expense measured in millions. Serving it to users is a perpetual expense measured in billions of tokens per day. For most organisations, inference is already the larger number, and the gap …
In the fall of 1960 Rudolf Kalman presented a paper at NASA Ames that engineers found hard to grasp; by early 1961 it was navigating simulated spacecraft around the Moon. The same recursion now computes the exact likelihood of ev…
In 2023 GPT-4 scored 97% on a classic cause-and-effect benchmark and 62% on one that hands it the causal graph and asks it to compute. Both results fit a theorem proved in 2020: data from one rung of Pearl's ladder almost never d…
Deep learning took images in three years and text in five. It has been attacking tabular data since 2016 and still has not won. The reason is not compute or architecture; it is three specific inductive biases that make a tree the…
Embedding models are chosen from a leaderboard more often than from an experiment, and the leaderboard now publishes training splits for its own test sets. Between contamination, task-family averaging and geometry no benchmark me…
A fraud model can score perfect recall offline and block nothing in production, because its training join looked a few hours into the future. Feature stores exist to enforce one contract: a training row may only see what the serv…
A 512-token model could read a paragraph. A million-token model can read a novel. The journey between those two numbers is a story about quadratic cost, clever approximations, and what it means for a machine to "remember."
The protocols that let agents read your email, query your database, and post to Slack were the most-adopted infrastructure in AI over the past year. They also handed attackers a way in that no input filter can close, because the …
Run exact search over a trained translation model and, for more than half of a standard test set, the highest-scoring output is the empty string. Beam search works only because it fails to find what the model actually prefers. Th…
On one synthetic honeypot evaluation, Claude Sonnet 4.5 said out loud that it suspected it was being tested in 80 to 100 percent of transcripts, against under 10 percent for its predecessor. When the internal representations behi…
A guardrail is a classifier sandwich wrapped around your model, and every layer you add buys coverage with latency and false refusals. Here is how the layer actually works, what it costs, and where it breaks.