Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

model-based policy search

Model-based policy search is an approach in reinforcement learning where an agent learns or utilizes a predictive model of environment dynamics to optimize a control policy. Rather than relying solely on direct trial-and-error interactions in the real environment, this methodology constructs an internal model of state transitions and rewards to simulate future trajectories, evaluate outcomes, and compute policy improvements. By leveraging these simulated experiences or calculating analytic gradients through the learned dynamics model, model-based policy search substantially enhances sample efficiency, enabling agents to acquire effective control strategies with significantly fewer real-world trials than model-free alternatives.

1 item

Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models

Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models

Kurtland Chua, Roberto Calandra, Rowan McAllister, Sergey Levine

OrganizationsUniversity of California Berkeley

Why you should read this

Proposes the PETS algorithm, which combines deep neural network ensembles with trajectory sampling to close the asymptotic performance gap between model-based and model-free reinforcement learning while dramatically reducing the required training samples.

Model-based reinforcement learning (RL) algorithms can attain excellent sample efficiency, but often lag behind the best model-free algorithms in terms of asymptotic performance. This is especially true with high-capacity parametric function approximators, such as deep networks. In this paper, we study how to bridge this gap, by employing uncertainty-aware dynamics models. We propose a new algorithm called probabilistic ensembles with trajectory sampling (PETS) that combines uncertainty-aware deep network dynamics models with sampling-based uncertainty propagation. Our comparison to state-of-the-art model-based and model-free deep RL algorithms shows that our approach matches the asymptotic performance of model-free algorithms on several challenging benchmark tasks, while requiring significantly fewer samples (e.g., 8 and 125 times fewer samples than Soft Actor Critic and Proximal Policy Optimization respectively on the half-cheetah task).

Added

2026-09-24