Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

distance awareness

Distance awareness is the property of a machine learning model that enables it to reflect how far a given input sample lies from the training data distribution. Under this property, a model assigns higher predictive uncertainty to test inputs that are distant from the training manifold, preventing it from making overly confident errors on unfamiliar data. Achieving distance awareness typically requires preserving geometric distances throughout the network architecture so that hidden representations do not collapse distinct inputs together, alongside output layers whose uncertainty estimates monotonically track the distance to training examples. This capability is fundamental for dependable uncertainty quantification, robust calibration under distributional shift, and reliable out-of-distribution detection.

1 item

A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness

A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness

Jeremiah Zhe Liu, Shreyas Padhy, Jie Ren, Zi Lin, Yeming Wen, Ghassen Jerfel, Zachary Nado, Jasper Snoek, Dustin Tran, Balaji Lakshminarayanan

Why you should read this

Proposes Spectral-normalized Neural Gaussian Process (SNGP), a method that combines spectral normalization with a Gaussian process output layer to achieve distance-aware, high-quality uncertainty estimation in a single deterministic network without the computational overhead of deep ensembles.

Uncertainty quantification is a key component of modern machine learning systems. A popular approach for quantifying predictive uncertainty on neural network models is deep ensembles. However, training multiple independently initialized networks leads to prohibitive computational costs at both train time and test time. Alternatively, single-model approaches such as MC-dropout and its variants have been proposed but typically underperform relative to their ensemble-based counterparts. In this work we propose a simple method which brings together distance-awareness and spectral normalization for improving robustness to distributional shift while retaining model calibration. We demonstrate significant gains over baseline methods on benchmark tasks including out-of-distribution detection.

Added

2026-10-02