Probabilistic ML & Bayesian Methods

What you’ll be able to do

1. LO1Write a model as a generative story and a factorised joint distribution, and carry out exact Bayesian inference with conjugate priors, including posteriors, posterior predictive distributions, credible intervals and hierarchical shrinkage.
2. LO2Derive Bayesian linear regression and its predictive distribution, separate noise from parameter uncertainty, and use the model evidence to set hyperparameters and compare models, knowing where it misleads.
3. LO3Derive the Laplace approximation, apply it to Bayesian logistic regression with its moderated predictive distribution, and recognise posteriors for which a Gaussian approximation fails.
4. LO4Derive the evidence lower bound from the inference side, apply EM to latent-variable and missing-data models, and fit mean-field and reparameterised variational approximations, explaining the biases the direction of the KL divergence introduces.
5. LO5Implement Metropolis-Hastings, Gibbs sampling and Hamiltonian Monte Carlo from their correctness conditions, and diagnose convergence and efficiency with trace plots, split R-hat and effective sample size.
6. LO6Build Gaussian process regression from kernels, fit its hyperparameters by maximising the marginal likelihood, and use it as the surrogate in Bayesian optimisation with expected improvement and confidence-bound acquisitions.
7. LO7Quantify predictive uncertainty in neural networks with MC dropout, deep ensembles and last-layer Laplace, separate aleatoric from epistemic uncertainty, and evaluate it with proper scoring rules, calibration error, selective prediction and out-of-distribution detection.
8. LO8Express a model as a probabilistic program and choose and justify an inference method (exact, Laplace, variational or MCMC) for a given model, data size and decision, reporting uncertainty honestly.

Chapters

6 chapters · each section ends with a short check

1. 01Bayesian inference: priors, posteriors and conjugacy
2. 02Bayesian linear models: regression, evidence and the Laplace approximation
3. 03Latent variables, EM and variational inference
4. 04Markov chain Monte Carlo and probabilistic programming
5. 05Gaussian processes and Bayesian optimisation
6. 06Uncertainty in deep learning

Graded labs

1. L1

   Conjugate updates, Bayesian linear regression and the Laplace approximation

   After chapter 2 · about 5 hours · Synthetic school districts · Course own work, CC BY 4.0 · Runs on a CPU

2. L2

   EM for missing data and robust regression, and variational logistic regression

   After chapter 3 · about 5 hours · Diabetes dataset · BSD-3-Clause (scikit-learn) · Runs on a CPU

3. L3

   MCMC from scratch: Metropolis, Gibbs, HMC and the diagnostics that keep you honest

   After chapter 4 · about 5 hours · Breast Cancer Wisconsin · BSD-3-Clause (scikit-learn) · Runs on a CPU

4. L4

   Gaussian processes and Bayesian optimisation

   After chapter 5 · about 5 hours · Seeded 2-D response surface · Course own work, CC BY 4.0 · Runs on a CPU

5. L5

   Uncertainty for neural networks: MC dropout, deep ensembles and last-layer Laplace

   After chapter 6 · about 6 hours · Optical recognition of handwritten digits · BSD-3-Clause (scikit-learn) · Runs on a CPU

Before you start

How it’s taught

You read short sections written for the page. Each ends with a check, and in-browser exercises let you practise. Graded labs run on your own cloud lab machine, CPU-friendly.

No videos. No live classes. No AI tutor.

How it works→