Bayesian Methods
Priors and posteriors, MCMC and HMC, variational inference, Gaussian processes and model comparison.
7concepts
145flashcards
55minutes of reading
- 01 Bayesian Model Comparison: Bayes Factors, PSIS-LOO and WAIC Bayes factors compare models by how well their priors predicted the data and are acutely sensitive to prior width, while PSIS-LOO and WAIC estimate out-of-sample predictive accuracy from the posterior and come with diagnostics that say when to distrust them.
- 02 Gaussian Processes Placing a prior over functions rather than parameters, which gives exact posterior uncertainty that grows away from the data, at a cubic cost that determines where they are usable.
- 03 MCMC and Hamiltonian Monte Carlo Why sampling from an unnormalised posterior is possible at all, why random-walk proposals fail in high dimensions, and how using gradient information turns a random walk into directed motion.
- 04 Variational Inference and the ELBO Turning integration into optimisation by fitting a tractable distribution to the posterior, and the specific bias that comes from minimising the reverse KL divergence.