Standardized Evaluation of Automatic Methods for Perivascular Spaces Segmentation in MRI -- MICCAI 2024 Challenge Results

Perivascular spaces (PVS), when abnormally enlarged and visible in magnetic resonance imaging (MRI) structural sequences, are important imaging markers of cerebral small vessel disease and potential indicators of neurodegenerative conditions. Despite their clinical significance, automatic enlarged PVS (EPVS) segmentation remains challenging due to their small size, variable morphology, similarity with other pathological features, and limited annotated datasets. This paper presents the EPVS Challenge organized at MICCAI 2024, which aims to advance the development of automated algorithms for EPVS segmentation across multi-site data. We provided a diverse dataset comprising 100 training, 50 validation, and 50 testing scans collected from multiple international sites (UK, Singapore, and China) with varying MRI protocols and demographics. All annotations followed the STRIVE protocol to ensure standardized ground truth and covered the full brain parenchyma. Seven teams completed the full challenge, implementing various deep learning approaches primarily based on U-Net architectures with innovations in multi-modal processing, ensemble strategies, and transformer-based components. Performance was evaluated using dice similarity coefficient, absolute volume difference, recall, and precision metrics. The winning method employed MedNeXt architecture with a dual 2D/3D strategy for handling varying slice thicknesses. The top solutions showed relatively good performance on test data from seen datasets, but significant degradation of performance was observed on the previously unseen Shanghai cohort, highlighting cross-site generalization challenges due to domain shift. This challenge establishes an important benchmark for EPVS segmentation methods and underscores the need for the continued development of robust algorithms that can generalize in diverse clinical settings.

💡 Research Summary

This paper presents a comprehensive overview of the Enlarged Perivascular Spaces (EPVS) Segmentation Challenge held at MICCAI 2024, detailing its motivation, dataset composition, organization, participating methods, and results. EPVS are small fluid‑filled spaces surrounding cerebral vessels that become visible on structural MRI when enlarged; they are increasingly recognized as imaging biomarkers for cerebral small‑vessel disease, cognitive decline, and neurodegeneration. Manual quantification is labor‑intensive and suffers from inter‑rater variability, motivating the development of automated segmentation tools.

The challenge provided a multi‑site, multi‑scanner dataset comprising 100 fully annotated training scans, 50 validation scans, and 50 hidden test scans. The data were collected from three geographic regions—United Kingdom (University of Edinburgh), Singapore (SG70 and MACC cohorts), and China (Shanghai TCM cohort)—and included co‑registered T1‑weighted, T2‑weighted, and T2‑FLAIR volumes. All EPVS masks were generated following the STRIVE protocol, covering the whole brain parenchyma and focusing on the centrum semiovale (CSO) and basal ganglia (BG) regions. Notably, the Shanghai cohort featured larger slice spacing and higher in‑plane resolution, deliberately introducing a domain shift to assess generalization.

The challenge was structured in three phases (training, validation, test) with a strict Docker‑based submission pipeline and publicly released evaluation scripts (Dice similarity coefficient, absolute volume difference, recall, precision). Seven teams submitted fully automated pipelines, most of which were built on U‑Net variants but incorporated diverse innovations:

• The winning team employed a MedNeXt architecture with a dual 2D/3D strategy, allowing the model to handle varying slice thicknesses by first extracting 2D features and then aggregating them in a 3D context.
• Another top method integrated cross‑attention Transformers to fuse multi‑modal information (T1, T2, FLAIR) more effectively, improving detection of low‑contrast EPVS.
• Several teams used ensemble approaches, combining multiple U‑Net models trained at different scales and applying majority voting to reduce false positives.
• One group attempted domain adaptation via Cycle‑GAN style transfer from the Shanghai images to the other sites, but observed only modest gains.

Overall performance on the hidden test set was encouraging for the “seen” sites: average Dice scores ranged from 0.62 to 0.68, absolute volume differences were between 15 % and 22 %, and recall/precision balanced around 0.70/0.68. However, all methods suffered a marked drop on the unseen Shanghai cohort, with Dice falling below 0.45 and volume errors exceeding 30 %. This stark degradation underscores the challenge of domain shift in neuroimaging, especially for tiny structures like EPVS that are highly sensitive to voxel size, contrast, and acquisition protocol.

Beyond the quantitative results, the challenge contributed several community resources: the full multi‑modal dataset (subject to data‑use agreements), a synthetic EPVS dataset for pre‑training, open‑source evaluation code, and publicly available Docker containers for each submission. These assets aim to foster reproducibility and enable future researchers to benchmark new algorithms against a standardized reference.

In conclusion, the EPVS Challenge establishes the first large‑scale, multi‑site benchmark for automated EPVS segmentation, revealing both the promise of modern deep‑learning architectures and their current limitations in cross‑site generalization. The authors recommend future work focus on robust domain‑adaptation techniques, weak‑supervision or semi‑automatic labeling to alleviate annotation bottlenecks, and tighter integration of quantitative EPVS metrics with clinical outcomes. Such advances will be essential for translating EPVS quantification from research settings into routine clinical practice.