Tagged “parallel-decoding”

2 posts.

All Model Architecture19 Training & Alignment22 Inference & Serving18 Agents & Orchestration11 Reasoning & Evaluation26 Safety, Security & Governance7 Platforms & Practice20

Inference & Serving 21 min

Diffusion Language Models: Writing Text by Denoising, Not Predicting the Next Token

Autoregressive models write left to right, one token at a time. Diffusion language models reveal a whole sequence at once and sharpen it over a handful of steps. That single change rewrites the latency math, and in 2025 it stoppe…

diffusion-models llm parallel-decoding generative-ai ∑ ◫

Inference & Serving 23 min

Speculative Decoding: How a Small Draft Model Makes Large Language Models Think Faster

Autoregressive decoding forces even the largest language models to emit one token at a time, bottlenecked by memory bandwidth rather than compute. Speculative decoding breaks this sequential constraint by letting a lightweight dr…

speculative-decoding llm-inference parallel-decoding draft-model ∑ ◫