Enhancing Hyperspectral Image Prediction with Contrastive Learning in Low-Label Regime

Self-supervised contrastive learning is an effective approach for addressing the challenge of limited labelled data. This study builds upon the previously established two-stage patch-level, multi-label classification method for hyperspectral remote sensing imagery. We evaluate the method’s performance for both the single-label and multi-label classification tasks, particularly under scenarios of limited training data. The methodology unfolds in two stages. Initially, we focus on training an encoder and a projection network using a contrastive learning approach. This step is crucial for enhancing the ability of the encoder to discern patterns within the unlabelled data. Next, we employ the pre-trained encoder to guide the training of two distinct predictors: one for multi-label and another for single-label classification. Empirical results on four public datasets show that the predictors trained with our method perform better than those trained under fully supervised techniques. Notably, the performance is maintained even when the amount of training data is reduced by $50%$. This advantage is consistent across both tasks. The method’s effectiveness comes from its streamlined architecture. This design allows for retraining the encoder along with the predictor. As a result, the encoder becomes more adaptable to the features identified by the classifier, improving the overall classification performance. Qualitative analysis reveals the contrastive-learning-based encoder’s capability to provide representations that allow separation among classes and identify location-based features despite not being explicitly trained for that. This observation indicates the method’s potential in uncovering implicit spatial information within the data.

💡 Research Summary

This paper addresses the pervasive problem of limited labeled data in hyperspectral remote sensing by proposing a two‑stage framework that leverages self‑supervised contrastive learning (CL) to pre‑train a lightweight encoder and then fine‑tunes two separate classifiers for multi‑label and single‑label patch‑level prediction. In the first stage, unlabeled hyperspectral patches are augmented through spectral and spatial transformations to generate paired views. A contrastive loss (NT‑Xent) encourages the encoder’s representations of the two views to be close while pushing apart representations of different patches. The encoder, together with a small projection head, learns discriminative embeddings that capture both spectral signatures and implicit spatial context without any manual supervision.

In the second stage, the pre‑trained encoder is either frozen or jointly fine‑tuned with two downstream heads: a sigmoid‑based multi‑label classifier trained with binary cross‑entropy, and a softmax‑based single‑label classifier trained with categorical cross‑entropy. The architecture is deliberately streamlined, avoiding deep backbones such as ResNet, which reduces computational overhead and facilitates end‑to‑end adaptation of the encoder to the downstream tasks.

Experiments are conducted on four public hyperspectral datasets (Indian Pines, Pavia University, Salinas, and KSC). For each dataset, the authors vary the proportion of labeled samples from 10 % to 100 % and compare against three baselines: fully supervised training, a conventional self‑supervised encoder (ResNet‑50 backbone), and an auto‑encoder trained without contrastive objectives. The proposed method consistently outperforms the baselines in overall accuracy (OA) and mean F1‑score. Notably, when the labeled set is reduced by 50 % or more, the performance drop is less than 2 % points, demonstrating strong label efficiency.

Qualitative analysis using t‑SNE visualizations shows that the contrastive‑trained encoder forms well‑separated clusters that respect both class identity and spatial locality, a property absent in purely supervised encoders. The authors also present case studies where the encoder implicitly captures location‑based cues, enabling better discrimination of spectrally similar but spatially distinct materials.

Key contributions include: (1) a simple yet effective two‑stage CL pipeline tailored for hyperspectral patch classification under scarce labels, (2) empirical evidence that a lightweight encoder can rival or surpass deeper supervised models, (3) systematic evaluation of robustness to label reduction, and (4) insight into how contrastive learning uncovers hidden spatial information without explicit supervision.

Limitations are acknowledged: the contrastive pre‑training requires large batch sizes and considerable GPU memory, the augmentation policy may need domain‑specific tuning, and the current work focuses solely on patch‑level classification, leaving pixel‑wise segmentation and object detection as future extensions. The authors suggest exploring memory‑efficient CL variants (e.g., MoCo with a memory bank), automated augmentation search, and applying the learned representations to downstream tasks such as semantic segmentation and change detection.