Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

single-model deep uncertainty

Single-model deep uncertainty refers to the methodology and practice of estimating predictive confidence and uncertainty within a single deep neural network architecture rather than combining outputs from multiple distinct models. Unlike deep ensembles or multi-network Bayesian approaches that require training, storing, and running several independent networks, single-model approaches quantify uncertainty directly from one model instance. This can be achieved through techniques such as Monte Carlo dropout, evidential learning, quantile regression, or deterministic distance-aware representations paired with Gaussian process output layers. By evaluating prediction reliability, calibration, and sensitivity to out-of-distribution data within an individual network, single-model deep uncertainty aims to provide trustworthy risk assessment and distributional shift detection while minimizing the computational and memory overhead associated with multi-model ensembles during training and inference.

1 item

A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness

A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness

Jeremiah Zhe Liu, Shreyas Padhy, Jie Ren, Zi Lin, Yeming Wen, Ghassen Jerfel, Zachary Nado, Jasper Snoek, Dustin Tran, Balaji Lakshminarayanan

Why you should read this

Proposes Spectral-normalized Neural Gaussian Process (SNGP), a method that combines spectral normalization with a Gaussian process output layer to achieve distance-aware, high-quality uncertainty estimation in a single deterministic network without the computational overhead of deep ensembles.

Uncertainty quantification is a key component of modern machine learning systems. A popular approach for quantifying predictive uncertainty on neural network models is deep ensembles. However, training multiple independently initialized networks leads to prohibitive computational costs at both train time and test time. Alternatively, single-model approaches such as MC-dropout and its variants have been proposed but typically underperform relative to their ensemble-based counterparts. In this work we propose a simple method which brings together distance-awareness and spectral normalization for improving robustness to distributional shift while retaining model calibration. We demonstrate significant gains over baseline methods on benchmark tasks including out-of-distribution detection.

Added

2026-10-02