Blog

Formulating probabilistic machine learning and optimisation concepts through mathematics.

Stochastic gradient and Hamiltonian Monte Carlo
23 min
Bayesian Inference
MCMC
Statistics

A complete mathematical formulation of Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) from physical dynamics to sampling.

Nov 19, 2023
Expectation - Maximisation algorithm and its applications in finite mixture models
24 min
Expectation-Maximization
Clustering
Latent Variables

A deep dive into the Expectation-Maximization (EM) algorithm and its application in Gaussian and Multinomial mixture models.

Jul 17, 2022
Bias - variance decomposition
23 min
Machine Learning
Statistics

Extending the square-loss bias-variance decomposition framework to Bregman divergence, 0-1 loss, and KL divergence.

May 3, 2022
From hyper-parameter optimisation to meta-learning
17 min
Meta-Learning
Transfer Learning
Optimisation

Formulating meta-learning explicitly through empirical Bayes and bi-level hyper-parameter optimisation.

Nov 22, 2021
Outer product approximation of Hessian matrix
7 min
Optimisation
Machine Learning
Linear Algebra

Deriving the Gauss-Newton optimisation method and the properties of the Gauss-Newton approximation matrix.

Apr 12, 2021
PAC-Bayes bounds for generalisation error
7 min
Statistical Learning Theory
Generalisation

A comprehensive, step-by-step mathematical proof of the classic PAC-Bayesian generalisation error bounds.

Dec 26, 2020
VAE: normalising constant matters
8 min
Deep Learning
Generative Models
Variational Inference

Why balancing reconstruction vs. KL divergence with a β weight is often just a workaround for neglecting the normalising constant.

Nov 24, 2020
No matching items
Back to top