Regularized Evolution for Image Classifier Architecture Search
Esteban RealA. AggarwalYanping HuangQuoc V. Le
Demonstrates that a simple evolutionary algorithm using aging tournament selection finds high-performing image classifiers faster than reinforcement learning, producing the state-of-the-art AmoebaNet architecture.
Researchers have long relied on expert hand-design to create high-accuracy convolutional neural networks for image classification, a process that is slow and difficult to scale. Automated architecture search offers a way to accelerate discovery, yet evolutionary methods had consistently produced models inferior to both human designs and those found by reinforcement learning. This paper set out to test whether a simple modification to evolutionary search could close that gap while remaining computationally practical.
The authors introduced aging evolution, a variant of tournament selection in which the oldest model in a fixed-size population is removed at each step rather than the weakest performer. They applied this algorithm, along with two basic mutations, inside the established NASNet search space. Controlled experiments evaluated 20 000 architectures on CIFAR-10 using identical training code and hardware for evolution, reinforcement learning, and random search; the resulting architectures were then enlarged and retrained on ImageNet.
Evolution reached competitive accuracy earlier than reinforcement learning and matched its final quality after the full search budget. The best architecture found, AmoebaNet-A, achieved 82.8 % top-1 accuracy on ImageNet at roughly 87 million parameters—on par with the best reinforcement-learning result of similar size—and set a new state-of-the-art of 83.9 % top-1 when scaled to 469 million parameters. Models discovered by evolution also required fewer floating-point operations than those found by reinforcement learning at comparable accuracy. Aging proved advantageous over standard tournament selection across multiple small-scale settings.
These outcomes show that evolution can be both simpler to implement and faster under resource constraints than reinforcement learning, while still producing deployable classifiers that exceed prior human-designed and automatically discovered networks. The approach therefore provides a practical alternative when compute budgets are limited or when rapid early progress matters. Further work could test the method on larger or different search spaces, quantify speed advantages more precisely, and examine whether the discovered connectivity patterns generalize to other vision tasks. Results are tied to the NASNet space and the CIFAR-10/ImageNet regime; broader validation would strengthen confidence in wider applicability.
- Paper: Neural Architecture Search with Reinforcement Learning, Barret Zoph et al. (2016). This foundational paper introduces neural architecture search with reinforcement learning, directly establishing the methodology and search spaces that the source paper adapts and improves upon through evolutionary algorithms.
- Paper: Learning Transferable Architectures for Scalable Image Recognition, Barret Zoph et al. (2018). Reading this work on NASNet provides critical context on cell-based architecture search and reinforcement learning baselines that the source paper evaluates and surpasses using regularized evolution.
- Paper: EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks, Mingxing Tan et al. (2019). This book extends architectural exploration by introducing compound scaling methods that build upon the efficient building blocks discovered by automated search techniques like the source paper's AmoebaNet.
