Baseline Method of the Foundation Model Challenge for Ultrasound Image Analysis

Ultrasound (US) imaging exhibits substantial heterogeneity across anatomical structures and acquisition protocols, posing significant challenges to the development of generalizable analysis models. Most existing methods are task-specific, limiting their suitability as clinically deployable foundation models. To address this limitation, the Foundation Model Challenge for Ultrasound Image Analysis (FM_UIA2026) introduces a large-scale multi-task benchmark comprising 27 subtasks across segmentation, classification, detection, and regression. In this paper, we present the official baseline for FM_UIA2026 based on a unified Multi-Head Multi-Task Learning (MH-MTL) framework that supports all tasks within a single shared network. The model employs an ImageNet-pretrained EfficientNet–B4 backbone for robust feature extraction, combined with a Feature Pyramid Network (FPN) to capture multi-scale contextual information. A task-specific routing strategy enables global tasks to leverage high-level semantic features, while dense prediction tasks exploit spatially detailed FPN representations. Training incorporates a composite loss with task-adaptive learning rate scaling and a cosine annealing schedule. Validation results demonstrate the feasibility and robustness of this unified design, establishing a strong and extensible baseline for ultrasound foundation model research. The code and dataset are publicly available at \href{https://github.com/lijiake2408/Foundation-Model-Challenge-for-Ultrasound-Image-Analysis}{GitHub}.

💡 Research Summary

The paper introduces the official baseline for the Foundation Model Challenge for Ultrasound Image Analysis (FM_UIA 2026), a large‑scale benchmark that aggregates 27 heterogeneous tasks—including 12 segmentation, 9 classification, 3 detection, and 3 regression subtasks—into a single unified model. The authors propose a Multi‑Head Multi‑Task Learning (MH‑MTL) framework that shares a common encoder while providing task‑specific routing to distinct heads.

Architecture: The backbone is an ImageNet‑pretrained EfficientNet‑B4, chosen for its balanced scaling of depth, width, and resolution. The encoder outputs a hierarchy of feature maps (C1‑C5). To capture multi‑scale context, a Feature Pyramid Network (FPN) is attached as a decoder, producing a fused representation (P_out) with 128 channels at ¼ of the input resolution (256×256 → 64×64).

Task routing: A deterministic switch based on the task identifier directs the flow either to a “Global Branch” (for classification and regression) or a “Dense Branch” (for segmentation and detection). The Global Branch pools the deepest feature C5 with Global Average Pooling, applies dropout (0.2), and feeds a fully‑connected head. Classification logits are passed through softmax; regression outputs a vector of normalized (x, y) coordinates for keypoints. The Dense Branch uses P_out and applies lightweight convolutional heads: segmentation projects P_out to K class masks followed by up‑sampling; detection adopts an anchor‑free grid approach where each cell predicts a 5‑dimensional vector (objectness score + bounding‑box offsets).

Losses: The training objective is a composite loss selected per batch: Dice loss for segmentation, cross‑entropy for classification, mean‑squared error for regression, and a focused detection loss consisting of binary cross‑entropy on objectness plus L1 on bounding‑box coordinates, applied only at the ground‑truth grid cell (λ = 8 balances the terms).

Training protocol: Images are resized to 256×256, augmented with random brightness/contrast and Gaussian noise via Albumentations. The model is trained for 50 epochs using AdamW with a cosine‑annealing learning‑rate schedule. The backbone learning rate is set to 1e‑4, while task‑specific heads use a higher rate of 1e‑3 to accelerate convergence.

Results: On the official validation set (unseen domains), the baseline achieves:

• Classification: mean AUC 0.9155, mean F1 0.7896, mean MCC 0.6766.
• Segmentation: mean Dice 0.7543, mean Hausdorff Distance 81.18 px.
• Detection: mean IoU 0.2641.
• Regression: mean radial error (MRE) 67.43 px (computed on original resolution).

The strong performance on classification and segmentation demonstrates that the shared EfficientNet encoder captures high‑level semantic cues and sufficient mid‑level detail for organ delineation. However, detection suffers from low IoU, reflecting difficulty in precisely localizing small lesions with the coarse grid and limited spatial resolution. Regression error is inflated by the resolution mismatch between training (256×256) and evaluation (original high‑resolution images).

Discussion: The baseline validates the feasibility of a single network handling diverse ultrasound tasks, but also highlights key limitations. Precise localization and fine‑grained measurement require higher‑resolution decoders, more sophisticated detection heads (e.g., transformer‑based or multi‑scale anchor‑free designs), and possibly multi‑stage refinement. Moreover, the deterministic routing avoids negative transfer but may still constrain task‑specific feature specialization; dynamic or attention‑guided routing could alleviate this.

Future directions proposed include: (1) integrating high‑resolution feature pyramids or HRNet‑style decoders, (2) employing self‑supervised or semi‑supervised pre‑training on massive unlabeled ultrasound corpora to improve robustness to speckle noise and device variability, (3) exploring adaptive loss weighting or curriculum learning to balance tasks with disparate data volumes, and (4) extending the framework with transformer‑based encoders that can model long‑range dependencies crucial for certain diagnostic views.

In summary, this work delivers a solid, extensible baseline for the FM_UIA 2026 challenge, offering the community a common starting point for building truly generalist ultrasound foundation models that can be further refined with advanced multi‑scale, task‑adaptive, and self‑supervised techniques.