A Mixed Reality System for Robust Manikin Localization in Childbirth Training

Opportunities for medical students to gain practical experience in vaginal births are increasingly constrained by shortened clinical rotations, patient reluctance, and the unpredictable nature of labour. To alleviate clinicians’ instructional burden and enhance trainees’ learning efficiency, we introduce a mixed reality (MR) system for childbirth training that combines virtual guidance with tactile manikin interaction, thereby preserving authentic haptic feedback while enabling independent practice without continuous on-site expert supervision. The system extends the passthrough capability of commercial head-mounted displays (HMDs) by spatially calibrating an external RGB-D camera, allowing real-time visual integration of physical training objects. Building on this capability, we implement a coarse-to-fine localization pipeline that first aligns the maternal manikin with fiducial markers to define a delivery region and then registers the pre-scanned neonatal head within this area. This process enables spatially accurate overlay of virtual guiding hands near the manikin, allowing trainees to follow expert trajectories reinforced by haptic interaction. Experimental evaluations demonstrate that the system achieves accurate and stable manikin localization on a standalone headset, ensuring practical deployment without external computing resources. A large-scale user study involving 83 fourth-year medical students was subsequently conducted to compare MR-based and virtual reality (VR)-based childbirth training. Four senior obstetricians independently assessed performance using standardized criteria. Results showed that MR training achieved significantly higher scores in delivery, post-delivery, and overall task performance, and was consistently preferred by trainees over VR training.

💡 Research Summary

The paper addresses the growing shortage of hands‑on obstetric training opportunities for medical students by introducing a mixed‑reality (MR) system that merges tactile interaction with a physical childbirth manikin and real‑time virtual guidance. The hardware platform consists of a Meta Quest head‑mounted display (HMD) augmented with an external RGB‑D camera mounted on the headset. A custom “eye‑to‑hand” calibration aligns the camera’s coordinate frame with that of the headset, enabling spatially accurate passthrough video and depth capture without external computing resources.

Localization of the manikin is performed through a two‑stage coarse‑to‑fine pipeline. In the coarse stage, planar fiducial markers are rigidly attached to the maternal manikin. The markers are detected in the RGB‑D stream, and using the fixed transforms between headset‑camera (T_HC) and marker‑manikin (T_FM), the 6‑DoF pose of the maternal manikin (T_HM) is computed in real time. This provides a reliable delivery region of interest (ROI). In the fine stage, the neonatal head—made of deformable silicone and therefore unsuitable for markers—is registered using point‑cloud alignment. A pre‑scanned 3‑D model of the neonatal head is matched to the live depth data via an Iterative Closest Point (ICP) algorithm constrained within the ROI, yielding a precise pose for the infant head. The combined pose information allows virtual guiding hands to be overlaid on the physical manikin with anatomical accuracy, letting trainees follow expert trajectories while feeling the real haptic feedback of the manikin.

All processing, including marker detection, pose estimation, and ICP, runs on the Quest’s onboard GPU, eliminating the need for external servers and preserving battery life. Bench‑top experiments show average static localization errors below 3 mm and stable tracking in over 95 % of frames during dynamic simulated deliveries.

To evaluate educational impact, a large‑scale user study enrolled 83 fourth‑year medical students who were randomly assigned to either the MR condition (physical manikin with virtual guidance) or a fully virtual reality (VR) condition using the same Quest hardware. Four senior obstetricians independently scored participants on three criteria: delivery technique, post‑delivery handling, and overall task performance. The MR group achieved significantly higher scores across all criteria (12 %–18 % improvement). Post‑session questionnaires indicated that participants found the MR experience more immersive, more informative, and more motivating than the VR alternative.

The authors claim four primary contributions: (1) the first MR‑based childbirth training system that preserves tactile realism while providing adaptive visual guidance; (2) integration of an external RGB‑D camera with a consumer‑grade Quest headset for real‑time, spatially aligned passthrough; (3) a novel coarse‑to‑fine localization strategy that combines multi‑marker alignment for the maternal manikin with point‑cloud registration for the neonatal head; and (4) a large‑scale empirical evaluation demonstrating the superiority of MR over VR for obstetric skill acquisition.

Limitations include the focus on normal vaginal delivery only; the system has not yet been tested on emergency or complicated delivery scenarios, and long‑term robustness of marker detection and point‑cloud alignment under varying lighting or occlusion conditions remains to be studied. Future work will extend the framework to cover a broader range of obstetric emergencies, explore marker‑less tracking methods, and conduct longitudinal studies to assess skill retention. Overall, the paper presents a compelling case that mixed reality can bridge the gap between high‑fidelity tactile simulation and scalable, data‑driven instructional support in medical education.