keyword
Bayesian learning
Bayesian learning is a statistical approach to machine learning that applies Bayes theorem to update probability distributions over model parameters or hypotheses as new data is observed. Rather than identifying a single optimal set of parameters through deterministic optimization, this paradigm begins with a prior probability distribution representing initial beliefs and transforms it into a posterior distribution conditioned on the observed evidence. Predictions are formed by integrating over the entire posterior distribution, which provides a principled framework for quantifying epistemic uncertainty, incorporating domain knowledge, and preventing overfitting.
2 items
Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods
Eyke Hüllermeier, Willem Waegeman
Why you should read this
Clarifies the critical distinction between irreducible data randomness and reducible model ignorance, providing a comprehensive framework for quantifying both aleatoric and epistemic uncertainty to build safer, more reliable machine learning systems.
The notion of uncertainty is of major importance in machine learning and constitutes a key element of machine learning methodology. In line with the statistical tradition, uncertainty has long been perceived as almost synonymous with standard probability and probabilistic predictions. Yet, due to the steadily increasing relevance of machine learning for practical applications and related issues such as safety requirements, new problems and challenges have recently been identified by machine learning scholars, and these problems may call for new methodological developments. In particular, this includes the importance of distinguishing between (at least) two different types of uncertainty, often referred to as aleatoric and epistemic. In this paper, we provide an introduction to the topic of uncertainty in machine learning as well as an overview of attempts so far at handling uncertainty in general and formalizing this distinction in particular.
Added
2026-09-16
Bayesian Learning via Stochastic Gradient Langevin Dynamics
Max Welling, Yee Whye Teh
Why you should read this
Introduces Stochastic Gradient Langevin Dynamics, a scalable framework that enables Bayesian posterior sampling on large datasets by injecting balanced Gaussian noise into mini-batch gradient updates without requiring full-dataset Metropolis-Hastings acceptance steps.
In this paper we propose a new framework for learning from large scale datasets based on iterative learning from small mini-batches. By adding the right amount of noise to a standard stochastic gradient optimization algorithm we show that the iterates will converge to samples from the true posterior distribution as we anneal the stepsize. This seamless transition between optimization and Bayesian posterior sampling provides an in-built protection against overfitting. We also propose a practical method for Monte Carlo estimates of posterior statistics which monitors a “sampling threshold” and collects samples after it has been surpassed. We apply the method to three models: a mixture of Gaussians, logistic regression and ICA with natural gradients.
Added
2026-09-12