All writing

123 long-form pieces on architecture, training, inference, agents and safety.

All Model Architecture19 Training & Alignment22 Inference & Serving18 Agents & Orchestration11 Reasoning & Evaluation26 Safety, Security & Governance7 Platforms & Practice20

Model Architecture 25 min

Mixture of Experts: How Frontier LLMs Achieve Scale Without Proportional Cost

Every frontier LLM released since 2023 uses some form of Mixture of Experts. The idea is deceptively simple: build a model with hundreds of billions of parameters but activate only a fraction for each token. The result is dense-m…

moe architecture scaling inference ∑ ◫

Platforms & Practice 21 min

Model Context Protocol: How One JSON-RPC Standard Collapsed the M×N Integration Problem

Before MCP, connecting five agents to twenty tools meant writing a hundred bespoke adapters. A protocol turns that multiplication into an addition, and the math is the whole story.

mcp agents protocols tool-use ∑ ◫

Training & Alignment 27 min

Model Merging: Why Averaging Weights Works, and Where the Free Lunch Ends

Three 7B models that each scored under 30% on Japanese maths were averaged into one that scored 52%. No gradient was computed. Weight-space arithmetic is the cheapest capability gain in the field and the easiest one to fool yours…

model-merging fine-tuning task-vectors peft ∑ ◫

Agents & Orchestration 19 min

Multi-Agent Orchestration Patterns: When Coordination Beats One Agent, and When It Just Multiplies Cost

A single agent burns about 4x the tokens of a chat. A multi-agent system burns about 15x. The whole question of orchestration is whether the extra agents buy you accuracy worth more than that bill, and four named patterns decide …

multi-agent agents orchestration llm-systems ∑ ◫

Model Architecture 22 min

Multi-Head Latent Attention: How DeepSeek Compressed the KV Cache Without Losing Quality

Every token an LLM generates forces it to reload the keys and values of every token that came before. Multi-Head Latent Attention rewrites that trade by caching one compressed vector instead of dozens of separate heads, cutting m…

llm-architecture attention-mechanisms deepseek kv-cache ∑ ◫

Training & Alignment 21 min

Muon and MuonClip: The Optimizer That Broke Adam's Monopoly on LLM Pretraining

For eight years Adam owned every serious pretraining run. Then a competitive-speedrun optimizer that orthogonalizes its own gradient updates scaled from a 124M-parameter toy to a trillion-parameter model with zero loss spikes. He…

optimization llm-training muon adamw ∑ ◫

Model Architecture 15 min

Orthogonalizing Memory Reads: A Muon Trick for Noisy Recurrent Recall

A short experimental note claims that orthogonalizing an mLSTM's matrix memory at read time, borrowing Muon's Newton-Schulz step, sharpens noisy associative recall exactly where the baseline is failing. We walk through the mechan…

recurrent-models mlstm xlstm associative-recall ∑ ◫

Inference & Serving 21 min

PagedAttention and Continuous Batching: How vLLM Stopped Wasting Your GPU

A GPU loaded with a 13B model can have most of its KV-cache memory sitting idle while requests queue for capacity. PagedAttention and continuous batching reclaim that memory, and the throughput follows.

inference vllm kv-cache gpu-serving ∑ ◫

Model Architecture 22 min

Patches Over Tokens: How the Byte Latent Transformer Kills the Tokenizer

A tokenizer decides in advance how many bits of compute every piece of text deserves. The Byte Latent Transformer throws that decision out and lets the entropy of the raw bytes allocate compute instead, matching Llama 3 at 8B par…

byte-latent-transformer tokenization llm-architecture dynamic-patching ∑ ◫

Agents & Orchestration 25 min

Production Agent Engineering: The Pro-Code Playbook

Fifty-seven percent of organizations now run AI agents in production, yet quality remains the top deployment barrier. This is the field guide for the engineers shipping those systems: frameworks compared, patterns catalogued, fai…

agents frameworks langgraph crewai ∑ ◫

Inference & Serving 4 min

Prompt caching is not just a cost optimisation - it changes what apps you can build

Most teams treat prompt caching as a billing hack. The deeper consequence is that you can now ship product patterns - persistent agents, large-context tools, system-prompt-heavy UX - that were latency-prohibitive eighteen months ago.

caching inference latency economics

Training & Alignment 22 min

Prompts as Programs: What Changes When You Optimise the Prompt Instead of the Weights

A prompt optimiser that never touches a weight has been reported to beat GRPO by around six points using up to 35 times fewer rollouts. That result only makes sense once you stop treating the prompt as writing and start treating …

prompt-engineering llm optimization evaluation ∑ ◫

Browse by topic