Self-supervised Hyperspectral Image Restoration using Separable Image Prior

Supervised learning with a convolutional neural network is recognized as a powerful means of image restoration. However, most such methods have been designed for application to grayscale and/or color images; therefore, they have limited success when applied to hyperspectral image restoration. This is partially owing to large datasets being difficult to collect, and also the heavy computational load associated with the restoration of an image with many spectral bands. To address this difficulty, we propose a novel self-supervised learning strategy for application to hyperspectral image restoration. Our method automatically creates a training dataset from a single degraded image and trains a denoising network without any clear images. Another notable feature of our method is the use of a separable convolutional layer. We undertake experiments to prove that the use of a separable network allows us to acquire the prior of a hyperspectral image and to realize efficient restoration. We demonstrate the validity of our method through extensive experiments and show that our method has better characteristics than those that are currently regarded as state-of-the-art.

💡 Research Summary

This paper tackles two fundamental challenges that have limited the performance of deep‑learning‑based hyperspectral image (HSI) restoration: the scarcity of large, labeled training datasets and the prohibitive computational cost of processing images with hundreds of spectral bands. The authors propose a novel framework that combines a self‑supervised learning strategy with a separable convolutional neural network (CNN) specifically designed to exploit the intrinsic low‑rank structure of HSIs.

Self‑supervised learning from a single degraded image
Instead of relying on external clean references, the method automatically generates training pairs from the single degraded HSI itself. By adding synthetic noise or applying a random sampling mask, the network receives a noisy input and a target that is the original degraded image. This “noise‑to‑noise” paradigm is reminiscent of Noise2Void and Deep Image Prior, but it avoids the over‑smoothing of DIP and the locality‑dependence issues of N2V. Consequently, the network can be trained solely on the available data, eliminating the need for costly data acquisition.

Separable CNN architecture
HSIs exhibit strong spectral correlation: a pixel’s spectrum can be approximated by a few endmember signatures, leading to a low‑rank representation in the spectral dimension. To capture this property efficiently, the authors replace conventional 3‑D convolutions with a two‑step separable design: (1) depth‑wise 2‑D convolutions applied independently to each spectral channel, and (2) point‑wise 1‑×‑1 convolutions that mix information across channels. This reduces the parameter count from O(K³·M·L) to O(K²·M + M·L) while preserving the ability to model spatial‑spectral interactions. The network thus remains lightweight enough to train on a single HSI patch yet expressive enough to learn the structural prior of hyperspectral data.

Experimental validation
The authors evaluate the approach on three representative restoration tasks: (a) hole‑filling (inpainting), (b) denoising of Gaussian and Poisson noise, and (c) reconstruction from compressed measurements. In the hole‑filling experiment, a separable 8‑layer CNN achieved a mean PSNR of 39.13 dB, surpassing a dense 8‑layer CNN (37.8 dB) despite using far fewer parameters. For denoising, the proposed method outperformed classical model‑based techniques such as BM4D and FastHyDe, as well as recent deep‑learning baselines, by better preserving spectral fidelity while suppressing noise. In compressed‑sensing reconstruction, the separable network recovered spectral signatures with higher accuracy than both non‑separable deep models and optimization‑based priors.

Theoretical insight
The paper argues that the set of images representable by a separable network (S_Φ) is a subset of those representable by a dense network, but the low‑rank nature of HSIs ensures that the optimal solution lies within S_Φ. Hence, the separable architecture not only reduces training difficulty but also yields a more stable convergence to a high‑quality solution. The authors also emphasize that the separable CNN itself encodes a structural prior: its architecture inherently favors smooth spatial variations and low‑rank spectral structures, eliminating the need for handcrafted regularizers such as total variation or low‑rank matrix penalties.

Broader impact and future work
Because the method requires only a single degraded HSI, it is immediately applicable to remote‑sensing scenarios where ground‑truth data are unavailable or expensive to obtain. The separable design is also transferable to other high‑dimensional imaging modalities (e.g., medical multispectral scans). Future directions suggested include extending the framework to more complex degradation models (atmospheric scattering, sensor non‑linearity), integrating physics‑based forward models into the loss function, and implementing hardware‑friendly versions for real‑time onboard processing.

In summary, the paper makes two key contributions: (1) a practical self‑supervised training pipeline that eliminates the dependence on large labeled datasets, and (2) a lightweight separable CNN that leverages the intrinsic low‑rank spectral structure of HSIs, achieving restoration quality comparable to or better than state‑of‑the‑art methods while dramatically reducing computational burden. This work therefore establishes a new, efficient paradigm for hyperspectral image restoration.