Adversarial generalization of unfolding (model-based) networks

Unfolding networks are interpretable networks emerging from iterative algorithms, incorporate prior knowledge of data structure, and are designed to solve inverse problems like compressed sensing, which deals with recovering data from noisy, missing observations. Compressed sensing finds applications in critical domains, from medical imaging to cryptography, where adversarial robustness is crucial to prevent catastrophic failures. However, a solid theoretical understanding of the performance of unfolding networks in the presence of adversarial attacks is still in its infancy. In this paper, we study the adversarial generalization of unfolding networks when perturbed with $l_2$-norm constrained attacks, generated by the fast gradient sign method. Particularly, we choose a family of state-of-the-art overaparameterized unfolding networks and deploy a new framework to estimate their adversarial Rademacher complexity. Given this estimate, we provide adversarial generalization error bounds for the networks under study, which are tight with respect to the attack level. To our knowledge, this is the first theoretical analysis on the adversarial generalization of unfolding networks. We further present a series of experiments on real-world data, with results corroborating our derived theory, consistently for all data. Finally, we observe that the family’s overparameterization can be exploited to promote adversarial robustness, shedding light on how to efficiently robustify neural networks.

💡 Research Summary

The paper provides the first rigorous theoretical analysis of the adversarial generalization properties of deep unfolding networks (DUNs) when applied to compressed sensing (CS) problems. DUNs are constructed by “unfolding’’ iterative optimization algorithms—specifically ADMM—into neural network layers, preserving the interpretability and structure of the original algorithm while allowing trainable parameters. The authors focus on the state‑of‑the‑art ADMM‑DAD model, which shares a single over‑complete sparsifying transform (W\in\mathbb{R}^{N\times n}) ((N>n)) across all (L) layers, thereby operating in an over‑parameterized regime.

The central contribution is a novel bound on the adversarial Rademacher complexity (ARC) of the network under (l_{2})‑norm constrained Fast Gradient Sign Method (FGSM) attacks. The authors first prove that the perturbed final decoder (h_{L}^{W}(y+\delta)) is Lipschitz continuous with respect to (W); the Lipschitz constant scales linearly with the number of layers (L) and with the spectral‑norm bound (\sqrt{\beta}) of the transform. Leveraging this continuity, they bound the ARC via covering numbers, obtaining a complexity term of order (O\big(p,L\log(\beta(1+\epsilon))\big)), where (p=N/n) measures over‑completeness and (\epsilon) is the FGSM attack magnitude. This bound improves upon prior work that depended on per‑layer spectral norms and yielded (O(\epsilon\sum_{k=1}^{L}\beta_{k})) bounds; the new result is tighter both in the dependence on (\epsilon) and on the depth (L).

From the ARC bound, the authors derive an adversarial generalization error bound for the DUN: with high probability the error scales roughly as (p,L\log(1+\epsilon)). Crucially, the bound predicts that increasing the over‑completeness (p) reduces the complexity term, suggesting that over‑parameterization can be harnessed to improve robustness—a counter‑intuitive insight given the usual trade‑off between model size and generalization.

Empirical validation is performed on three datasets: MNIST, CIFAR‑10, and a real‑world MRI reconstruction task. The authors compare ADMM‑DAD (over‑complete, shared (W)) against a baseline DUN that learns an orthogonal (non‑over‑complete) transform. Experiments include standard training, adversarial training with FGSM, and test‑time FGSM attacks of varying (\epsilon). Results show: (1) the measured adversarial generalization gap closely follows the theoretical bound; (2) larger (N) (i.e., higher (p)) yields a flatter degradation curve as (\epsilon) increases, confirming the predicted robustness benefit; (3) the over‑complete model consistently outperforms the orthogonal baseline in both reconstruction accuracy and robustness across all attack levels.

The paper concludes with practical design recommendations for safety‑critical applications such as medical imaging: (i) employ over‑complete sparsifying transforms to exploit the robustness‑enhancing effect of over‑parameterization; (ii) share parameters across layers to keep the total number of trainable weights manageable while retaining depth; (iii) use ARC‑based theoretical analysis during model selection to guarantee adversarial generalization. By bridging interpretability, efficiency, and provable security, this work opens a pathway for deploying model‑based neural architectures in domains where both accuracy and robustness are non‑negotiable.