Welcome to Probabilita ML

This website is dedicated to introducing the foundational motivations and key concepts of various probabilistic machine learning techniques. The aim is to facilitate a comprehensive understanding of how learning algorithms are derived and formulated through some level of mathematics, particularly probability theory, linear algebra, and multivariate calculus. While this content sounds boring and may not encompass several recently developed advanced techniques, these foundational principles are expected to remain central to the field of machine learning, despite the rapid advancements of the field.

What is machine learning?

Machine learning has been trendy, especially after the Image-Net challenge 2012 with AlexNet improving the benchmark significantly compared to non-deep learning approaches. There are several definitions of machine learning that could be found on the internet. Here, machine learning is defined in a simple term:

\begin{aligned} \text{machine learning} & = \text{solving inverse problems}. \end{aligned}

What is an inverse problem then? In simple terms, it is to work out unknown parameters from observations of a system of interest. For example:

• Forward problem: given f(x) = x^{2} + 1, one can easily calculate y_{i} = f(x_{i}) for different values of x_{i}.
• Inverse problem: given a set of observations \{(x_{i}, y_{i})\}_{i = 1}^{N}, how to find the function f that satisfies: y_{i} = f(x_{i}).

Although the inverse problem may easily be solved for the example above, the difficulty to find f increases with the complexity of f. In general, the inverse problem is far more difficult than the forward problem.

What is probabilistic machine learning?

Probabilistic machine learning is to model the data generation process, for example, through some graphical models, then use the observed data to infer the posterior of the model’s parameter.

Why is the name probabilita?

This is derived from the Latin word probabilitas, meaning probability.