Weight quantisation, KV cache eviction, prompt compression and distillation are treated as four separate engineering disciplines with four separate literatures. They are one problem: choosing a point on a rate-distortion curve. S…
Two servers run the same model on the same GPUs. One reports 4,200 tokens per second and is unusable; the other reports 2,600 and feels instant. Throughput is a property of the server, latency is a property of the request, and th…
LoRA is usually explained as a budget approximation of full fine-tuning: same destination, less memory. Two lines of evidence say that framing is wrong. LoRA learns less, forgets less, and reaches a structurally different solutio…
A thousand identical requests to the same model at temperature zero produced eighty different completions. The cause is not random seeds, not sampling, and not GPU atomics. It is that reduction kernels change their summation orde…