All writing

123 long-form pieces on architecture, training, inference, agents and safety.

All Model Architecture19 Training & Alignment22 Inference & Serving18 Agents & Orchestration11 Reasoning & Evaluation26 Safety, Security & Governance7 Platforms & Practice20

Safety, Security & Governance 24 min

Proving What Is Real: Content Credentials, Watermarks, and the Limits of Detection

In October 2025 The Washington Post uploaded an AI-generated video carrying signed Content Credentials to eight social platforms, and not one kept the credential. Watermarks survive that trip but not a determined paraphrase, and …

model-provenance-and-watermarking watermarking eu-ai-act regulation ∑ ◫

Platforms & Practice 24 min

Query Understanding Is the Hidden Half of Search: From Broder's Taxonomy to LLM Query Rewriting

An LLM-written pseudo-document lifts BM25 by 15 nDCG@10 points on TREC DL 2019, and turns a 16 ms search into one that waits over two seconds for the model. Query understanding decides what the engine searches for before ranking …

query-understanding information-retrieval retrieval rag ∑ ◫

Training & Alignment 21 min

RL from Verifiable Rewards: Training Models on Answers That Can Be Checked

Replace the reward model with a function that simply checks the answer, and a frontier reasoning model falls out of pure reinforcement learning. The catch is what 'checkable' quietly assumes, and what the model learns to exploit.

rlvr reinforcement-learning reasoning grpo ∑ ◫

Training & Alignment 10 min

RLHF vs DPO in production: what we learned shipping both

DPO is the right default for almost every preference-tuning project in 2026, but the cases where PPO still wins are sharper and more common than the simplicity pitch admits.

rlhf dpo alignment post-training

Platforms & Practice 24 min

Reading Its Own Handwriting: Position Bias, Feedback Loops and Counterfactual Correction

Move a document from rank 5 to rank 1 and its click-through rate jumps without its relevance changing at all. Train on those clicks and the model learns position, which the previous ranker chose, which means it learns to imitate …

ranking recommender-systems learning-to-rank counterfactual ∑ ◫

Platforms & Practice 10 min

Retrieval Is a Ranking Problem: Why Your RAG System Doesn't Need a Better Embedding Model

Teams tune the embedding model and the vector database, then wonder why answers are still wrong. Both are the least important parts of the stack. Retrieval is a two-stage ranking problem, and information retrieval solved the shap…

rag retrieval reranking chunking ∑ ◫

Inference & Serving 24 min

Running Language Models on a Phone: Memory Bandwidth, NPUs, and the Few-Billion-Parameter Ceiling

Phones ship NPUs rated in trillions of operations per second, yet the speed at which a reply appears is set by how fast LPDDR memory can hand a couple of gigabytes of weights to the processor, over and over. This post derives the…

on-device-and-edge-ai on-device-ai llm-inference memory-bandwidth ∑ ◫

Inference & Serving 21 min

Sleep-Time Compute: Thinking About the Context Before the Question Arrives

Test-time compute makes models smarter but slower and pricier. Sleep-time compute moves the thinking to before the user asks, cutting the compute needed for the same accuracy by roughly 5x and the cost per query by 2.5x when cont…

inference test-time-compute agents reasoning ∑ ◫

Inference & Serving 23 min

Speculative Decoding: How a Small Draft Model Makes Large Language Models Think Faster

Autoregressive decoding forces even the largest language models to emit one token at a time, bottlenecked by memory bandwidth rather than compute. Speculative decoding breaks this sequential constraint by letting a lightweight dr…

speculative-decoding llm-inference parallel-decoding draft-model ∑ ◫

Platforms & Practice 3 min

Stop fine-tuning for facts: the silent productivity tax of mis-matched tools

Most teams reach for fine-tuning when they should reach for retrieval. The cost is not just dollars - it is months of confused engineers trying to figure out why the model started lying again.

rag retrieval fine-tuning architecture

Inference & Serving 25 min

Structured Generation and Constrained Decoding: Making LLMs Predictable

Language models generate text one token at a time by sampling from a probability distribution over their entire vocabulary. Constrained decoding intervenes at that sampling step, masking out every token that would violate a targe…

structured-output constrained-decoding JSON-mode FSM ∑ ◫

Reasoning & Evaluation 18 min

Test-Time Compute: How Reasoning Models Buy Intelligence by the Token

A 7B model that thinks for thirty seconds can outscore a 70B model that answers instantly. Test-time compute is the lever that makes this possible, and it is quietly rewriting the economics of model design.

reasoning test-time-compute inference reinforcement-learning ∑ ◫

Browse by topic