Bayesian Methods
Priors and posteriors, MCMC and HMC, variational inference, Gaussian processes and model comparison.
7concepts
145flashcards
55minutes of reading
- 01 Hierarchical Models and Partial Pooling A hierarchical model lets groups borrow strength from each other by learning how different they are, which turns the choice between separate estimates and one pooled estimate into a parameter the data decides.
- 02 Posterior Predictive Checks and the Bayesian Workflow A posterior predictive check simulates replicated data from the fitted model and asks whether they look like the real data in the ways that matter, and it is one step in a workflow that also checks priors, computation and the model's ability to recover known parameters.
- 03 Priors, Conjugacy and the Posterior How a prior functions as pseudo-data, why conjugate families make the posterior a closed-form update, and why "uninformative" priors are informative on some scale.
- 04 Bayesian Model Comparison: Bayes Factors, PSIS-LOO and WAIC Bayes factors compare models by how well their priors predicted the data and are acutely sensitive to prior width, while PSIS-LOO and WAIC estimate out-of-sample predictive accuracy from the posterior and come with diagnostics that say when to distrust them.
- 05 Gaussian Processes Placing a prior over functions rather than parameters, which gives exact posterior uncertainty that grows away from the data, at a cubic cost that determines where they are usable.
- 06 MCMC and Hamiltonian Monte Carlo Why sampling from an unnormalised posterior is possible at all, why random-walk proposals fail in high dimensions, and how using gradient information turns a random walk into directed motion.
- 07 Variational Inference and the ELBO Turning integration into optimisation by fitting a tractable distribution to the posterior, and the specific bias that comes from minimising the reverse KL divergence.