Bandits & Exploration
Regret, UCB, Thompson sampling, contextual bandits, and exploration under a budget.
7concepts
68flashcards
50minutes of reading
- 01 Best-Arm Identification When the goal is to find the best option with confidence rather than to earn reward while searching, the right objective is simple regret, the problem splits into fixed-confidence and fixed-budget versions, and elimination-style algorithms need samples in proportion to the sum of inverse squared gaps.
- 02 Regret and the Exploration-Exploitation Tradeoff What regret measures, why the logarithmic lower bound says exploration can never stop entirely, and how the bandit setting differs from both supervised learning and full reinforcement learning.