Tagged “policy-gradients”

2 posts.

All Model Architecture20 Training & Alignment22 Inference & Serving21 Agents & Orchestration11 Reasoning & Evaluation27 Safety, Security & Governance8 Platforms & Practice22

Training & Alignment 24 min

Learning From the Log: Off-Policy Policy Learning, From IPS to Counterfactual Risk Minimisation

An unbiased estimate of every policy's value does not give you an unbiased choice of policy. The moment an optimiser searches over importance-weighted estimates, it goes looking for the estimator's noise, and the history of learn…

policy-learning-and-ope off-policy-evaluation counterfactual importance-sampling ∑ ◫

Training & Alignment 25 min

Why Policy Gradients Need a Baseline: Variance, Trust Regions, and the Road to PPO

Add 1,000 to every reward in an environment. The optimal policy is unchanged, and the expected policy gradient is unchanged — but the variance of the estimator you actually compute goes up by four orders of magnitude. Every advan…

rl policy-gradients reinforce ppo ∑ ◫