PlayWorld: Learning Robot World Models from Autonomous Play

Action-conditioned video models offer a promising path to building general-purpose robot simulators that can improve directly from data. Yet, despite training on large-scale robot datasets, current state-of-the-art video models still struggle to predict physically consistent robot-object interactions that are crucial in robotic manipulation. To close this gap, we present PlayWorld, a simple, scalable, and fully autonomous pipeline for training high-fidelity video world simulators from interaction experience. In contrast to prior approaches that rely on success-biased human demonstrations, PlayWorld is the first system capable of learning entirely from unsupervised robot self-play, enabling naturally scalable data collection while capturing complex, long-tailed physical interactions essential for modeling realistic object dynamics. Experiments across diverse manipulation tasks show that PlayWorld generates high-quality, physically consistent predictions for contact-rich interactions that are not captured by world models trained on human-collected data. We further demonstrate the versatility of PlayWorld in enabling fine-grained failure prediction and policy evaluation, with up to 40% improvements over human-collected data. Finally, we demonstrate how PlayWorld enables reinforcement learning in the world model, improving policy performance by 65% in success rates when deployed in the real world.

💡 Research Summary

**
PlayWorld tackles a fundamental bottleneck in robot manipulation: the inability of current action‑conditioned video world models to faithfully predict contact‑rich dynamics. Existing models are typically trained on human‑demonstration datasets that are heavily biased toward successful task executions, resulting in limited exposure to diverse object interactions, failure modes, and rare contact events. Consequently, when such models are queried under novel policies, prediction errors quickly compound, leading to physically implausible rollouts that undermine policy evaluation and reinforcement learning (RL) in the real world.

The authors propose a data‑centric solution: collect massive, diverse interaction data through fully autonomous robot “play”. The system consists of two cooperating components. First, a Vision‑Language Model (VLM) observes the current multi‑camera scene and generates natural‑language instructions (e.g., “push the red block forward”, “stack the blue cylinder”). To encourage diversity, the VLM is prompted to perturb verbs, objects, and descriptors, thereby producing a wide range of semantically grounded tasks without any reward engineering. Second, a Vision‑Language‑Action (VLA) policy, pre‑trained on language‑conditioned manipulation, executes these instructions. The VLA is deliberately exposed to perturbed commands, which induces varied contact dynamics and expands the state‑action visitation distribution far beyond that of human demonstrations. A lightweight safety filter enforces joint limits and triggers automatic “reset” commands when objects drift toward the robot’s reachability boundary, enabling continuous, unsupervised data collection for up to eight hours per night.

The resulting dataset, D_play, contains synchronized multi‑view RGB streams (overhead and wrist cameras), proprioceptive states, and action commands across thousands of episodes. Compared to human‑collected datasets, D_play exhibits substantially higher diversity in contact events, object state transitions, and failure modes, while remaining scalable because no manual labeling or scene‑specific engineering is required.

For modeling, the authors adopt a pre‑trained Stable Video Diffusion (SVD) backbone, which provides strong spatial‑temporal attention and can be conditioned per‑frame on action embeddings. The model predicts three camera views simultaneously, mitigating partial observability. Training uses the standard diffusion loss on noisy latent predictions, fine‑tuned for two days on a cluster of eight H200 GPUs with batch size 64.

A key technical challenge is the long‑tailed nature of the play data: most transitions are trivial free‑space motions, while rare but crucial contact interactions are under‑represented. To address this, PlayWorld employs a curriculum learning schedule that automatically rates sample difficulty (e.g., based on motion magnitude, contact detection) and feeds easier samples early, gradually increasing the proportion of hard, contact‑rich examples. This balanced exposure prevents the model from over‑fitting to dominant patterns and improves its ability to capture subtle dynamics.

Extensive experiments across several manipulation benchmarks (e.g., block stacking, object pushing, drawer opening) demonstrate that PlayWorld‑trained models achieve markedly higher physical consistency scores—up to 30 % improvement over baselines trained on human data. Failure prediction accuracy improves by up to 40 %, enabling more reliable policy ranking. Crucially, the authors integrate the world model into a model‑based RL loop: imagined rollouts are used to fine‑tune the policy, and the updated policy is deployed on the real robot. This pipeline yields a 65 % increase in real‑world success rates compared to the original pre‑trained policy, confirming that the richer dynamics captured by autonomous play translate into tangible performance gains.

A scaling study further shows that performance continues to rise with up to five times more play data, whereas models trained on human demonstrations saturate much earlier, highlighting the superior scalability of autonomous play.

Limitations include the VLA policy’s sensitivity to language perturbations, which can generate inefficient motions; the conservative safety filter that may restrict exploration; and the computational cost of diffusion‑based video generation, which hinders real‑time inference. Future work is suggested to develop more robust language‑action grounding, efficient multi‑scale diffusion architectures, and domain adaptation techniques to bridge any remaining sim‑to‑real gaps.

Overall, PlayWorld establishes that fully autonomous, language‑guided robot play is a viable and powerful source of training data for high‑fidelity video world models, opening a path toward scalable, data‑driven simulation that can directly improve real‑world robotic manipulation.