DeformGen: Dynamics-Based Topology Augmentation for Deformable Manipulation Policy Learning
Zili LinWenyao ZhangYuyang ZhangZekun QiJunyan LinHanxin ZhuJiaolong YangZhibo ChenYao MuXiaokang Yang
Presents DeformGen, a dynamics-based augmentation framework that expands demonstration datasets for deformable object manipulation by forward-simulating physically plausible states and warping robot trajectories to match deformed geometries.
Training robotic systems to manipulate deformable items—such as garments, ropes, and flexible packages—is essential for advancing industrial automation, household robotics, and healthcare assistance. However, training these robots requires vast volumes of demonstration data, which are prohibitively costly, time-consuming, and difficult to collect manually. While traditional data augmentation techniques expand rigid-object demonstrations using simple geometric shifts, these methods fail on soft objects because material deformations alter local contact points and physical dynamics in complex ways.
To address this data bottleneck, the article introduces and evaluates DeformGen, a framework designed to automate the generation of diverse, physically realistic demonstrations for deformable object manipulation starting from just a single human example. The framework aims to expand initial object shapes and adapt corresponding robotic motion trajectories without violating physical laws.
DeformGen operates in two high-level stages. First, it generates diverse, physically plausible starting configurations by applying randomized physical disturbances via simulated robotic contact and letting physics-engine simulations naturally settle the object into stable, non-rigid states. Second, it adapts the original robot manipulation path to the newly deformed shape through spatial deformation-field warping, which adjusts gripper position and orientation to match local material shifts while preserving the broader task path. The authors evaluated this approach in high-fidelity simulation environments across three representative deformable tasks (rope routing, toy packing, and cloth folding) using four widely used robotic learning architectures.
Across the evaluated setups, policies trained on DeformGen-augmented data dramatically outperformed models trained on a single demonstration, where success rates were nearly zero (1.3% to 2.5%). DeformGen achieved average task success rates between 37.3% and 59.0% across the architectures, generally outperforming rigid-style augmentation baselines. Topological state expansion proved crucial, as policies trained on diverse physical shapes generalized far better than those trained only on rigid shifts. Furthermore, scaling the amount of synthetically generated data steadily enhanced performance, lifting average success rates from approximately 20–37% with 100 trajectories up to 61–63% with 750 trajectories.
These findings demonstrate that scalable, automated demonstration generation for soft materials is technically viable when physical dynamics and local geometry are jointly accounted for. This reduces the dependency on labor-intensive physical data collection, lowering deployment costs and development timelines for complex robotics applications without sacrificing performance on simpler, rigid object orientations.
Organizations developing robotic manipulation systems should consider integrating dynamics-based synthetic augmentation pipelines into their simulation workflows. However, before deploying these systems in production, teams should conduct real-world pilot studies to assess physical deployment risks. Future developmental efforts should focus on multi-robot or dual-arm coordination and expand testing to complex non-rigid tasks like dough manipulation or surgical tissue handling.
The findings are supported by consistent trends across multiple model architectures and thousands of simulation runs. Readers should exercise caution regarding the sim-to-real transfer gap, as all evaluations occurred within simulated environments, and trajectory synthesis success rates can drop on severe object deformations.
- Paper: Diffusion policy: Visuomotor policy learning via action diffusion, Cheng Chi et al. (2023). Read this first to understand diffusion-based visuomotor policies for learning manipulation actions from demonstrations, the policy-learning context DeformGen augments.
No sufficiently relevant recommendations were found.
