Tagged “grpo”

3 posts.

All Model Architecture19 Training & Alignment22 Inference & Serving18 Agents & Orchestration11 Reasoning & Evaluation26 Safety, Security & Governance7 Platforms & Practice20

Training & Alignment 21 min

RL from Verifiable Rewards: Training Models on Answers That Can Be Checked

Replace the reward model with a function that simply checks the answer, and a frontier reasoning model falls out of pure reinforcement learning. The catch is what 'checkable' quietly assumes, and what the model learns to exploit.

rlvr reinforcement-learning reasoning grpo ∑ ◫

Training & Alignment 25 min

Why Policy Gradients Need a Baseline: Variance, Trust Regions, and the Road to PPO

Add 1,000 to every reward in an environment. The optimal policy is unchanged, and the expected policy gradient is unchanged — but the variance of the estimator you actually compute goes up by four orders of magnitude. Every advan…

rl policy-gradients reinforce ppo ∑ ◫

Training & Alignment 21 min

Why Token-Level RL Collapses: GSPO and Sequence-Level Importance Sampling

GRPO weights every token by its own importance ratio, and on long responses that single-sample estimator quietly poisons the gradient until the model collapses. GSPO moves the ratio up to the whole sequence, and Qwen3's largest m…

reinforcement-learning rlhf gspo grpo ∑ ◫