Built independently by an author, for readers. Read the story and support ChapterPal

keyword

out-of-distribution examples

Out-of-distribution examples are data points presented to a machine learning model that originate from a probability distribution different from the one used during training, such as inputs belonging to unknown classes or featuring conditions the model has never encountered. Because standard predictive models operate under the assumption that test data follows the same distribution as the training data, encountering out-of-distribution examples often causes models to produce incorrect predictions with mistakenly high confidence. Accurately identifying and flagging these unfamiliar inputs through out-of-distribution detection is critical for assessing model robustness, preventing unexpected failures, and ensuring safety in real-world deployments.

1 item

Natural Adversarial Examples

Natural Adversarial Examples

Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, Dawn Song

OrganizationsUniversity of California BerkeleyUniversity of ChicagoUniversity of Washington

Why you should read this

Introduces ImageNet-A and ImageNet-O, two benchmarks of unmodified real-world images that expose shared blind spots in computer vision models by causing severe performance drops without synthetic pixel perturbations.

We introduce two challenging datasets that reliably cause machine learning model performance to substantially degrade. The datasets are collected with a simple adversarial filtration technique to create datasets with limited spurious cues. Our datasets' real-world, unmodified examples transfer to various unseen models reliably, demonstrating that computer vision models have shared weaknesses. The first dataset is called ImageNet-A and is like the ImageNet test set, but it is far more challenging for existing models. We also curate an adversarial out-of-distribution detection dataset called ImageNet-O, which is the first out-of-distribution detection dataset created for ImageNet models. On ImageNet-A a DenseNet-121 obtains around 2% accuracy, an accuracy drop of approximately 90%, and its out-of-distribution detection performance on ImageNet-O is near random chance levels. We find that existing data augmentation techniques hardly boost performance, and using other public training datasets provides improvements that are limited. However, we find that improvements to computer vision architectures provide a promising path towards robust models.

Added

2026-09-17