Learning to Optimize: A Primer and A Benchmark
Tianlong ChenXiaohan ChenWuyang ChenHoward HeatonJialin LiuZhangyang WangWotao Yin
Presents a comprehensive survey and standardized benchmarking framework for learning-to-optimize methods in continuous optimization, providing practical categorizations, open challenges, and the open-source Open-L2O package for reproducible evaluation.
Modern industrial and engineering systems frequently solve repetitive continuous optimization problems across domains such as medical imaging, signal processing, and machine learning model training. Traditional optimization algorithms are manually designed around worst-case mathematical theories, which often results in slow iterative runtimes and suboptimal performance on specific data distributions. Learning to Optimize (L2O) has emerged as a data-driven paradigm that trains machine learning models to discover or refine optimization update rules automatically. The article evaluates the state of this emerging field, introduces a comprehensive taxonomy of techniques, and benchmarks leading L2O approaches to determine their practical performance and reliability across representative optimization tasks.
The article systematically analyzes two main classes of learned optimizers: model-free methods, which learn black-box update rules using neural networks such as recurrent architectures, and model-based methods, which embed machine learning components into established analytical algorithms via algorithm unrolling or plug-and-play operators. To resolve inconsistent evaluation standards across prior studies, the authors established the Open-L2O benchmark platform. They conducted standardized empirical evaluations spanning three representative testbeds: convex sparse optimization, non-convex landscape minimization using the generalized Rastrigin function, and multi-layer neural network training across unseen architectures and dataset shifts.
The benchmark results reveal three primary findings. First, model-based L2O methods substantially outperform both classical solvers and model-free techniques when problem structure is available; for example, unrolled sparse solvers reached target precision in approximately 16 iterations compared to hundreds or thousands of steps needed by standard iterative algorithms. Second, model-free L2O methods exhibited inconsistent gains; while population-based swarm methods navigated complex non-convex landscapes more effectively than standard gradient techniques, standard model-free optimizers failed to match basic analytical solvers on structured convex tasks. Third, model-free optimizers demonstrated poor generalization and instability when deployed on longer iteration horizons, higher problem dimensions, or unseen neural network architectures, frequently diverging due to truncation bias and optimizer training artifacts.
These findings imply that learned optimization is not yet a universal replacement for classic general-purpose solvers, but it offers significant speedups and computational cost reductions for specialized, repetitive tasks. Model-based approaches represent a practical sweet spot by combining theoretical algorithmic structure with data-driven tuning, making them well-suited for high-throughput operational pipelines like image reconstruction. Conversely, purely model-free optimizers carry high operational risks due to memory bottlenecks during training and erratic performance on tasks that deviate from the training distribution.
Organizations evaluating L2O should focus near-term adoption on model-based methods for well-structured inverse problems while treating general model-free optimizers as exploratory. Future efforts should prioritize hybrid architectures, memory-efficient training procedures to handle large-scale models, and automated safeguarding mechanisms that fall back to classic algorithms when out-of-distribution inputs are detected. Because theoretical guarantees remain largely restricted to specialized model-based settings and empirical testing was limited to moderate problem scales, stakeholders should exercise caution and conduct thorough distributional validation before integrating learned optimizers into mission-critical systems.
- Paper: Learning to learn by gradient descent by gradient descent, Marcin Andrychowicz et al. (2016). This foundational paper introduced training recurrent neural networks via gradient descent to act as learned optimizers, establishing the core framework surveyed and benchmarked in the primer.
- Paper: Optimization as a Model for Few-Shot Learning, Sachin Ravi et al. (2017). This work pioneered using meta-learning and LSTM-based optimization updates for fast model adaptation, providing key conceptual foundations for learning-to-optimize methods.
- Paper: Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon, Yoshua Bengio et al. (2018). This comprehensive review surveys integrating machine learning into combinatorial optimization, serving as essential prior literature on the broader learning-based optimization paradigm.
- Paper: OptNet: Differentiable Optimization as a Layer in Neural Networks, Brandon Amos et al. (2017). This paper establishes methods for embedding differentiable optimization problems into deep neural networks, providing important groundwork for differentiable architectures used in learning to optimize.
- Paper: Neural Combinatorial Optimization with Reinforcement Learning, Irwan Bello et al. (2016). This seminal paper demonstrated how neural architectures trained with reinforcement learning can discover optimization policies, forming an early milestone in the learning-to-optimize domain.
- Paper: Optimization Methods for Large-Scale Machine Learning, Léon Bottou et al. (2016). This survey provides fundamental theory and computational trade-offs of classical continuous optimization methods that learning-to-optimize approaches aim to accelerate.
- Paper: Discovered Policy Optimisation, Chris Lu et al. (2022). This work applies meta-learning to automatically discover policy optimization algorithms with theoretical grounding, directly advancing the algorithmic discovery theme surveyed in the primer.
- Paper: Learning to Search Feasible and Infeasible Regions of Routing Problems with Flexible Neural k-Opt, Yining Ma et al. (2023). This paper extends learned search heuristics by introducing a flexible neural k-opt policy that navigates both feasible and infeasible spaces in routing optimization.
- Paper: Aligning Optimization Trajectories with Diffusion Models for Constrained Design Generation, Giorgio Giannone et al. (2023). This work explores generating constrained engineering designs by aligning diffusion model trajectories with physical optimization paths, providing a novel application of learned optimization dynamics.
- Paper: Diffusion Models for Black-Box Optimization, Siddarth Krishnamoorthy et al. (2023). This study advances data-driven optimization by casting offline black-box optimization as a conditional generative process using diffusion models.
- Paper: Lower Bounds and Accelerated Algorithms for Bilevel Optimization, Kaiyi Ji et al. (2023). This research provides theoretical lower bounds and accelerated algorithms for bilevel optimization, addressing core complexity challenges present in meta-optimization frameworks.
- Paper: Pre-trained Gaussian Processes for Bayesian Optimization, Zi Wang et al. (2024). This article applies data-driven pre-training to Gaussian processes for Bayesian optimization, operationalizing prior knowledge reuse across optimization tasks.
