Blog

Stochastic gradient and Hamiltonian Monte Carlo

This post is to introduce the formulation of stochastic gradient descent as a Monte Carlo sampling to approximate the posterior of the variables of interest.

Expectation - Maximisation algorithm and its applications in finite mixture models

Missing data and latent variables are frequently encountered in various machine learning and statistical inference applications. A common example is the finite mixture…

Bias - variance decomposition

Bias and variance decomposition is one of the key tools to understand machine learning. However, conventional discussion about bias - variance decomposition revolves around…

From hyper-parameter optimisation to meta-learning

Meta-learning, also known as learn-how-to-learning, has been being studied from 1980s (Schmidhuber 1987; Naik and Mammone 1992), and recently attracted much attention from…

Outer product approximation of Hessian matrix

Hessian matrix is heavily studied in the optimization community. The purpose is to utilize the second order derivative to optimize a function of interest (also known as…

PAC-Bayes bounds for generalisation error

Properly approaximately correct (PAC) learning is a part of statistical machine learning which has been a fundamental course for most of graduate programs in machine…

VAE: normalising constant matters

Variational auto-encoder (VAE) is one of the most popular generative models in machine learning nowadays. However, the rapid development of the field has made many machine…