123 long-form pieces on architecture, training, inference, agents and safety.
In 2015 a Google paper catalogued the ways machine learning systems rot, and a decade of MLOps tooling set out to pay that debt down. It paid down the debt that lives in pipelines and artefacts, and left the debt that lives in ju…
The same 1024x1024 screenshot costs 576 visual tokens in one model and 2,880 in another, and the cheap one cannot read the text. The rule that turns pixels into tokens is the least examined hyperparameter in the multimodal stack,…
For years the field raced to build bigger models. Then a 2022 study showed most of those giants were undertrained, and quietly redrew the map of how to spend a compute budget.
XGBoost, LightGBM and CatBoost minimise the same objective with the same kind of tree. What separates them is bookkeeping: XGBoost turned two sums of derivatives into a split score, LightGBM made those sums cheap, and CatBoost ma…
Most agent frameworks treat state as an afterthought, tacking memory onto a chain of LLM calls. LangGraph inverts the design: the graph is the state machine, every node transition is checkpointed, and cycles are first-class citiz…
In 1951 Claude Shannon estimated the entropy of English by having people guess the next letter. In 2023 a 70-billion-parameter language model compressed a gigabyte of Wikipedia to 8.3% of its size, beating every compressor ever p…
Chain-of-thought made models reason out loud, one word at a time. A new line of work lets them reason in the silent space between words, trading auditability for compute that does not have to be spelled out.
A 36.7-million-parameter graph network produces a 10-day global forecast in under a minute and beats the world's best physics-based system on 90 percent of verification targets. It also cannot start without that system's data ass…
An unbiased estimate of every policy's value does not give you an unbiased choice of policy. The moment an optimiser searches over importance-weighted estimates, it goes looking for the estimator's noise, and the history of learn…
Every joint-embedding method has the same trivial solution available to it: map every input to the same vector. The history of self-supervised learning is the history of preventing that, and the methods that look most different f…
LoRA is usually explained as a budget approximation of full fine-tuning: same destination, less memory. Two lines of evidence say that framing is wrong. LoRA learns less, forgets less, and reaches a structurally different solutio…
Microsoft has renamed its AI development platform three times in three years, from Azure AI Studio to Azure AI Foundry to Microsoft Foundry. Behind the branding churn is a genuinely ambitious consolidation: 1,900+ models, a manag…