keyword
atomic actions
In artificial intelligence, robotics, and reinforcement learning, atomic actions are the most fundamental, indivisible primitive operations that an agent directly executes within an environment at a discrete time step. Unlike high-level tasks, abstract goals, or temporally extended macro-actions that consist of multi-step plans, an atomic action cannot be broken down further within the decision-making framework of the agent. These low-level control commands serve as the foundational building blocks from which hierarchical planning systems and decision policies construct complex behaviors, enabling an embodied or simulated agent to alter its environment state and interact with its surroundings.
2 items
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
Tejas D. Kulkarni, Karthik R. Narasimhan, Ardavan Saeedi, Joshua B. Tenenbaum
Why you should read this
Proposes hierarchical-DQN, a framework that integrates multi-timescale value functions with intrinsic motivation to solve sparse-reward reinforcement learning problems like Montezuma's Revenge by decoupling goal selection from action execution.
Learning goal-directed behavior in environments with sparse feedback is a major challenge for reinforcement learning algorithms. The primary difficulty arises due to insufficient exploration, resulting in an agent being unable to learn robust value functions. Intrinsically motivated agents can explore new behavior for its own sake rather than to directly solve problems. Such intrinsic behaviors could eventually help the agent solve tasks posed by the environment. We present hierarchical-DQN (h-DQN), a framework to integrate hierarchical value functions, operating at different temporal scales, with intrinsically motivated deep reinforcement learning. A top-level value function learns a policy over intrinsic goals, and a lower-level function learns a policy over atomic actions to satisfy the given goals. h-DQN allows for flexible goal specifications, such as functions over entities and relations. This provides an efficient space for exploration in complicated environments. We demonstrate the strength of our approach on two problems with very sparse, delayed feedback: (1) a complex discrete stochastic decision process, and (2) the classic ATARI game `Montezuma's Revenge'.
Added
2026-09-25
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
Wenlong Huang, Pieter Abbeel, Deepak Pathak, Igor Mordatch
Why you should read this
Demonstrates that large language models can act as zero-shot planners for embodied agents by decomposing high-level natural language instructions and semantically mapping them into executable, environment-admissible actions.
Can world knowledge learned by large language models (LLMs) be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g. "make breakfast"), to a chosen set of actionable steps (e.g. "open fridge"). While prior work focused on learning from explicit step-by-step examples of how to act, we surprisingly find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into mid-level plans without any further training. However, the plans produced naively by LLMs often cannot map precisely to admissible actions. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions. Our evaluation in the recent VirtualHome environment shows that the resulting method substantially improves executability over the LLM baseline. The conducted human evaluation reveals a trade-off between executability and correctness but shows a promising sign towards extracting actionable knowledge from language models. Website at this https URL
Added
2026-09-24