A Geometric Probe of the Accuracy-Robustness Trade-off: Sharp Boundaries in Symmetry-Breaking Dimensional Expansion

The trade-off between clean accuracy and adversarial robustness is a pervasive phenomenon in deep learning, yet its geometric origin remains elusive. In this work, we utilize Symmetry-Breaking Dimensional Expansion (SBDE) as a controlled probe to investigate the mechanism underlying this trade-off. SBDE expands input images by inserting constant-valued pixels, which breaks translational symmetry and consistently improves clean accuracy (e.g., from $90.47%$ to $95.63%$ on CIFAR-10 with ResNet-18) by reducing parameter degeneracy. However, this accuracy gain comes at the cost of reduced robustness against iterative white-box attacks. By employing a test-time \emph{mask projection} that resets the inserted auxiliary pixels to their training values, we demonstrate that the vulnerability stems almost entirely from the inserted dimensions. The projection effectively neutralizes the attacks and restores robustness, revealing that the model achieves high accuracy by creating \emph{sharp boundaries} (steep loss gradients) specifically along the auxiliary axes. Our findings provide a concrete geometric explanation for the accuracy-robustness paradox: the optimization landscape deepens the basin of attraction to improve accuracy but inevitably erects steep walls along the auxiliary degrees of freedom, creating a fragile sensitivity to off-manifold perturbations.

💡 Research Summary

The paper investigates the long‑observed trade‑off between clean accuracy and adversarial robustness by introducing a controllable input transformation called Symmetry‑Breaking Dimensional Expansion (SBDE). SBDE expands each image by inserting rows and columns of constant‑valued pixels at regular intervals, thereby breaking the translational symmetry of the data. This symmetry breaking acts as an external field in the parameter space, reducing the degeneracy of equivalent minima and providing more informative gradients during training. As a result, models trained on SBDE‑augmented data achieve substantially higher clean accuracy; for example, a ResNet‑18 on CIFAR‑10 improves from 90.47 % to 95.63 % without any changes to the training pipeline aside from the input expansion.

While SBDE raises clean performance, the authors discover that it simultaneously creates extremely steep loss gradients along the newly introduced auxiliary dimensions (the inserted constant pixels). During training, these auxiliary pixels are fixed to a constant value, but the loss function is not penalized for deviations along these axes. Consequently, the loss surface becomes highly anisotropic: it is deep and flat around the natural data manifold (the original pixels) but rises sharply when moving away from the manifold in the auxiliary directions. Iterative white‑box attacks such as PGD, APGD, BIM, and AutoAttack exploit precisely these steep directions, concentrating almost the entire perturbation budget on the auxiliary coordinates while leaving the visible image virtually unchanged. Visualizations of adversarial examples confirm that the perturbations are almost entirely confined to the inserted rows/columns.

To verify that the vulnerability originates from the auxiliary dimensions, the authors propose a simple test‑time mask projection operator Π. Π resets all auxiliary pixels to their training constant before feeding the perturbed image to the classifier. This operation projects the adversarial point back onto the hyperplane defined by the constant auxiliary values, effectively removing the displacement along the steep loss walls. Empirically, applying Π after an attack restores robust accuracy to the level of a naturally trained model, demonstrating that the loss landscape’s sharp boundaries in the auxiliary subspace are the root cause of the accuracy‑robustness trade‑off.

The paper conducts extensive ablations across different constant values (0, 0.2, 0.3) and cyclic insertion patterns (0.2‑GapCycle, 0.3‑GapCycle). In all cases, SBDE improves clean accuracy while degrading robustness, and Π consistently recovers robustness, indicating that the phenomenon is structural rather than an artifact of a particular hyper‑parameter choice. The experimental setup modifies the first convolutional layer of ResNet‑18 to accommodate the larger spatial dimensions (7×7 kernel, stride 2) and removes the initial max‑pooling layer. Training follows standard practice (200 epochs, cosine‑annealed learning rate, SGD with momentum 0.9, mixed‑precision). All attacks are performed in the expanded input space, and Π is applied only at inference time, ensuring that the defense does not rely on gradient obfuscation.

The authors interpret these findings as a concrete geometric explanation of the accuracy‑robustness paradox. By breaking symmetry, the optimizer can find deeper minima, which translates into higher clean accuracy. However, the same process narrows the basin of attraction along the auxiliary axes, erecting “sharp walls” that are highly sensitive to off‑manifold perturbations. This anisotropic loss landscape explains why improving accuracy inevitably incurs a robustness penalty when the model is allowed to exploit additional degrees of freedom that are not constrained during training.

The work contributes to the broader discussion on the nature of non‑robust features and the geometry of decision boundaries. It suggests that future defenses should aim to control the curvature of the loss surface not only on the data manifold but also in any auxiliary dimensions introduced by preprocessing or architectural choices. Potential directions include regularizing the Hessian along auxiliary axes, designing symmetric expansions that preserve isotropy, or explicitly penalizing steep gradients in off‑manifold directions. Overall, the paper provides a clear, experimentally validated geometric mechanism linking accuracy gains to the emergence of sharp loss boundaries, thereby deepening our understanding of the fundamental trade‑off between performance and security in deep neural networks.