Toward Accurate and Accessible Markerless Neuronavigation

Neuronavigation is widely used in biomedical research and interventions to guide the precise placement of instruments around the head to support procedures such as transcranial magnetic stimulation. Traditional systems, however, rely on subject-mounted markers that require manual registration, may shift during procedures, and can cause discomfort. We introduce and evaluate markerless approaches that replace expensive hardware and physical markers with low-cost visible and infrared light cameras incorporating stereo and depth sensing combined with algorithmic modeling of the facial geometry. Validation with $50$ human subjects yielded a median tracking discrepancy of only $2.32$ mm and $2.01°$ for the best markerless algorithms compared to a conventional marker-based system, which indicates sufficient accuracy for transcranial magnetic stimulation and a substantial improvement over prior markerless results. The results suggest that integration of the data from the various camera sensors can improve the overall accuracy further. The proposed markerless neuronavigation methods can reduce setup cost and complexity, improve patient comfort, and expand access to neuronavigation in clinical and research settings.

💡 Research Summary

Neuronavigation is a cornerstone technology for precise brain‑targeted interventions such as transcranial magnetic stimulation (TMS), EEG electrode placement, and image‑guided neurosurgery. Conventional systems rely on infrared cameras that track passive retro‑reflective markers affixed to the subject’s head. While this approach delivers sub‑millimeter accuracy, it suffers from several practical drawbacks: the need for manual marker placement and registration, potential marker drift caused by skin movement, patient discomfort, and high equipment cost that limits widespread adoption.

In response, the authors propose a fully markerless neuronavigation solution that replaces specialized hardware with two consumer‑grade Azure Kinect DK devices (each integrating an RGB camera and a time‑of‑flight depth sensor). The system exploits three complementary sensing modalities: (i) monocular RGB pose estimation using a Perspective‑n‑Point (PnP) algorithm, (ii) stereo RGB triangulation, and (iii) direct depth‑sensor point‑cloud registration. For each modality, they evaluate two variants: with and without a statistical head prior derived from a large‑scale facial shape model. This yields six distinct tracking pipelines.

Hardware and Calibration
The two Kinects are hardware‑synchronized and spatially calibrated using a standard checkerboard procedure. Intrinsic parameters (focal length, principal point, distortion) are obtained for each RGB camera, while extrinsic transformations between the left and right devices are computed to enable stereo reconstruction. The depth sensor’s point cloud is aligned to the RGB frame using the device’s factory‑provided registration.

Algorithmic Pipeline
Facial landmark detection is performed with MediaPipe, which provides 468 2D landmarks per frame. An empirical ablation study identifies a robust subset (e.g., eye corners, nose tip, mouth corners) that balances coverage and stability; this subset is used throughout the experiments.

Monocular RGB: The selected 2D landmarks are paired with a 3D facial template (either a generic mean face or a personalized statistical model). A PnP solver minimizes the reprojection error, yielding the SE(3) transform from the head coordinate frame to the camera frame.

Stereo RGB: Corresponding landmarks in the left and right RGB images are matched, and triangulation (using the calibrated baseline) reconstructs 3D landmark positions. These 3D points are then fed into the same PnP formulation to estimate head pose.

Depth: The depth map is converted to a point cloud; facial regions are segmented using the 2D landmark mask. An Iterative Closest Point (ICP) algorithm aligns the observed point cloud to the 3D facial model, directly providing the head pose.

Statistical Head Prior
A principal‑component analysis (PCA) model built from thousands of 3D face scans defines a low‑dimensional shape space. During pose estimation, the reconstructed landmarks are projected onto this space, enforcing anatomically plausible configurations and reducing sensitivity to occlusions or expression‑induced deformations.

Experimental Protocol
Fifty adult volunteers participated in a controlled session where the markerless system and a commercial NDI Polaris Vicra (the clinical gold standard that tracks retro‑reflective markers) operated simultaneously. For each captured frame, the Euclidean distance between the two systems’ translation estimates (mm) and the angular difference between their rotation matrices (degrees) were computed. Both median and inter‑quartile ranges are reported to capture the distribution of errors.

Results
The best performing configuration—depth‑sensor tracking combined with the statistical head prior—achieved a median positional error of 2.32 mm and a median angular error of 2.01°. Stereo RGB with prior yielded 3.1 mm / 2.8°, and monocular RGB with prior gave 3.6 mm / 3.2°. Without the prior, errors increased by roughly 15–25 % across all modalities. The system operated at ~35 fps (≈28 ms per frame), satisfying real‑time requirements for TMS navigation.

Discussion
These accuracy figures, while modestly higher than the sub‑millimeter performance of marker‑based setups, fall well within the tolerances accepted for TMS (typically ≤3 mm, ≤3°). The approach dramatically reduces hardware cost (≈ $400 per Kinect versus several thousand dollars for dedicated infrared trackers) and eliminates the need for adhesive markers, improving patient comfort and simplifying workflow. Limitations include sensitivity to ambient lighting for RGB‑based pipelines, depth sensor noise at the extremes of its range, and occasional tracking loss during rapid head motions or extreme facial expressions. The study focused on quasi‑static conditions; future work must assess long‑duration stability during actual clinical procedures.

Conclusion and Future Directions
The paper demonstrates that a low‑cost, markerless neuronavigation system built from off‑the‑shelf RGB‑D cameras can achieve clinically acceptable accuracy. By systematically comparing three sensing modalities and the impact of statistical shape priors on a sizable cohort, the authors provide a solid benchmark for the field. Future research avenues include: (1) integrating deep‑learning based landmark detectors robust to illumination changes, (2) extending the framework to multi‑subject or multi‑instrument tracking, (3) validating the system in real TMS or neurosurgical workflows, and (4) pursuing regulatory clearance and integration with existing clinical navigation platforms.