63 long-form pieces on architecture, training, inference, agents and safety.
LoRA is usually explained as a budget approximation of full fine-tuning: same destination, less memory. Two lines of evidence say that framing is wrong. LoRA learns less, forgets less, and reaches a structurally different solutio…
Microsoft has renamed its AI development platform three times in three years, from Azure AI Studio to Azure AI Foundry to Microsoft Foundry. Behind the branding churn is a genuinely ambitious consolidation: 1,900+ models, a manag…
Every frontier LLM released since 2023 uses some form of Mixture of Experts. The idea is deceptively simple: build a model with hundreds of billions of parameters but activate only a fraction for each token. The result is dense-m…
Before MCP, connecting five agents to twenty tools meant writing a hundred bespoke adapters. A protocol turns that multiplication into an addition, and the math is the whole story.
A single agent burns about 4x the tokens of a chat. A multi-agent system burns about 15x. The whole question of orchestration is whether the extra agents buy you accuracy worth more than that bill, and four named patterns decide …
Every token an LLM generates forces it to reload the keys and values of every token that came before. Multi-Head Latent Attention rewrites that trade by caching one compressed vector instead of dozens of separate heads, cutting m…
For eight years Adam owned every serious pretraining run. Then a competitive-speedrun optimizer that orthogonalizes its own gradient updates scaled from a 124M-parameter toy to a trillion-parameter model with zero loss spikes. He…
A short experimental note claims that orthogonalizing an mLSTM's matrix memory at read time, borrowing Muon's Newton-Schulz step, sharpens noisy associative recall exactly where the baseline is failing. We walk through the mechan…
A GPU loaded with a 13B model can have most of its KV-cache memory sitting idle while requests queue for capacity. PagedAttention and continuous batching reclaim that memory, and the throughput follows.
A tokenizer decides in advance how many bits of compute every piece of text deserves. The Byte Latent Transformer throws that decision out and lets the entropy of the raw bytes allocate compute instead, matching Llama 3 at 8B par…
Fifty-seven percent of organizations now run AI agents in production, yet quality remains the top deployment barrier. This is the field guide for the engineers shipping those systems: frameworks compared, patterns catalogued, fai…
Most teams treat prompt caching as a billing hack. The deeper consequence is that you can now ship product patterns - persistent agents, large-context tools, system-prompt-heavy UX - that were latency-prohibitive eighteen months ago.