Self-supervised Physics-Informed Manipulation of Deformable Linear Objects with Non-negligible Dynamics

We address dynamic manipulation of deformable linear objects by presenting SPiD, a physics-informed self-supervised learning framework that couples an accurate deformable object model with an augmented self-supervised training strategy. On the modeling side, we extend a mass-spring model to more accurately capture object dynamics while remaining lightweight enough for high-throughput rollouts during self-supervised learning. On the learning side, we train a neural controller using a task-oriented cost, enabling end-to-end optimization through interaction with the differentiable object model. In addition, we propose a self-supervised DAgger variant that detects distribution shift during deployment and performs offline self-correction to further enhance robustness without expert supervision. We evaluate our method primarily on the rope stabilization task, where a robot must bring a swinging rope to rest as quickly and smoothly as possible. Extensive experiments in both simulation and the real world demonstrate that the proposed controller achieves fast and smooth rope stabilization, generalizing across unseen initial states, rope lengths, masses, non-uniform mass distributions, and external disturbances. Additionally, we develop an affordable markerless rope perception method and demonstrate that our controller maintains performance with noisy and low-frequency state updates. Furthermore, we demonstrate the generality of the framework by extending it to the rope trajectory tracking task. Overall, SPiD offers a data-efficient, robust, and physically grounded framework for dynamic manipulation of deformable linear objects, featuring strong sim-to-real generalization.

💡 Research Summary

The paper introduces SPiD (Self‑supervised Physics‑informed Deformable manipulation), a framework that unifies a lightweight yet expressive physics model of deformable linear objects (DLOs) with a self‑supervised learning pipeline for real‑time control.
Physics model. Building on the classic mass‑spring system, the authors augment each link with linear, bending and torsional springs as well as corresponding damping terms, and they incorporate air drag and gravity. The resulting differential equations are analytically reformulated to guarantee numerical stability and are implemented as a fully differentiable simulator. System identification is performed with only 2 000 real‑world samples, yielding parameters that accurately capture heterogeneous ropes (non‑uniform mass distribution, varying lengths, and masses). Compared to prior mass‑spring baselines, the extended model reduces long‑horizon prediction error by roughly 40 %.
Self‑supervised control learning. The control objective is formulated as an energy‑minimization task: the robot must drive the rope’s total mechanical energy to zero as quickly and smoothly as possible. A neural policy π_φ receives the current rope state (positions, velocities) and outputs the end‑effector velocity. By rolling out the differentiable physics model, the task loss (energy, control effort, distance to goal) is back‑propagated through the simulator to directly update φ. To bridge the sim‑to‑real gap, Gaussian noise is injected into the identified model parameters during training, encouraging robustness to variations in rope properties. No expert demonstrations are required; the entire policy is learned from self‑generated rollouts.
Online adaptation via self‑supervised DAgger. During deployment, the system continuously monitors an out‑of‑distribution (OOD) metric based on prediction residuals. When the metric exceeds a preset threshold, the current policy’s trajectories are collected and fed back into the offline learning loop. A DAgger‑style self‑supervised refinement updates the policy without human labels, allowing the controller to adapt to unforeseen disturbances, sensor noise, or new rope dynamics.
Marker‑less perception. A lightweight detection‑and‑segmentation network, fine‑tuned on a task‑specific rope dataset, provides 3‑D keypoints and shape estimates at ~30 Hz using a single RGB camera. Compared with a high‑precision motion‑capture system, the perception error stays below 2 cm, which is sufficient for the controller’s bandwidth.
Experimental validation. Four evaluation axes are presented: (1) modeling accuracy (RMSE reduction), (2) control performance on the rope‑stabilization task (30 % faster settling and 25 % less energy than a state‑of‑the‑art MPC baseline), (3) robustness to low‑frequency, noisy perception (92 % success rate), and (4) generalization to a trajectory‑tracking task and to ropes with unseen lengths, masses, and non‑uniform distributions (tracking error < 0.1 m at 1 m/s). Both simulation and real‑world experiments confirm that SPiD achieves fast, smooth, and data‑efficient dynamic manipulation.
Contributions. The work delivers (i) an extended mass‑spring model with analytically derived damping, (ii) a differentiable‑physics‑driven self‑supervised policy learning pipeline, (iii) an online self‑supervised DAgger mechanism for continual refinement, (iv) an affordable marker‑less perception solution, and (v) extensive evidence of sim‑to‑real transfer and task generalization. SPiD thus establishes a practical paradigm for robots to manipulate highly dynamic deformable objects without relying on large demonstration datasets, while maintaining real‑time performance and robustness.