Algebraic Robustness Verification of Neural Networks

We formulate formal robustness verification of neural networks as an algebraic optimization problem. We leverage the Euclidean Distance (ED) degree, which is the generic number of complex critical points of the distance minimization problem to a classifier’s decision boundary, as an architecture-dependent measure of the intrinsic complexity of robustness verification. To make this notion operational, we define the associated ED discriminant, which characterizes input points at which the number of real critical points changes, distinguishing test instances that are easier or harder to verify. We provide an explicit algorithm for computing this discriminant. We further introduce the parameter discriminant of a neural network, identifying parameters where the ED degree drops and the decision boundary exhibits reduced algebraic complexity. We derive closed-form expressions for the ED degree for several classes of neural architectures, as well as formulas for the expected number of real critical points in the infinite-width limit. Finally, we present an exact robustness certification algorithm based on numerical homotopy continuation, establishing a concrete link between metric algebraic geometry and neural network verification.

💡 Research Summary

The paper introduces a novel algebraic framework for formal robustness verification of neural networks by treating decision boundaries as algebraic varieties and measuring verification complexity through the Euclidean Distance (ED) degree. Robustness is reformulated as the problem of computing the minimal Euclidean distance γ between a test input ξ and the decision boundary separating the predicted class c from any other class c′. This distance problem is expressed as a constrained polynomial optimization whose Karush‑Kuhn‑Tucker (KKT) conditions yield a square system of polynomial equations. By focusing on the relaxed problem that drops inequality constraints, the authors obtain a system whose generic number of complex critical points is the ED degree of the hypersurface defined by the logit difference fθ,c(x)−fθ,c′(x)=0.

The ED degree is a geometry‑invariant that depends only on the network architecture and generic parameters, providing an architecture‑dependent notion of verification complexity independent of optimization heuristics. To make this notion operational, two discriminants are defined: (i) the ED discriminant, a locus in input space where the number of real critical points changes, thereby classifying test instances as easy or hard to certify; (ii) the parameter discriminant, a hypersurface in parameter space where the ED degree drops, indicating a reduction in algebraic complexity of the decision boundary.

Closed‑form expressions for the ED degree are derived for several common architectures. For wide fully‑connected polynomial networks with layer degrees d₁,…,d_L, the generic ED degree equals (∏_{l=1}^L d_l)·(n+1). For bottleneck architectures, the degree scales with the bottleneck dimension. In the infinite‑width limit, the expected number of real critical points is obtained via the Kac‑Rice formula, and matches predictions from Neural Network Gaussian Processes (NNGP). These results give practitioners a way to anticipate verification difficulty from architectural choices.

On the algorithmic side, the paper presents an exact certification method based on numerical homotopy continuation. Starting from a simple start system, solution paths are tracked to the target KKT system, guaranteeing that all isolated complex solutions are found. Real solutions that satisfy the feasibility constraints yield the exact minimal distance γ, and thus a provably correct robustness certificate whenever the algebraic problem is tractable. The authors provide an open‑source implementation and empirical evaluations showing that instances lying on the parameter discriminant are dramatically easier to certify, confirming the theoretical predictions.

Overall, the work bridges metric algebraic geometry and neural network verification, offering a principled, architecture‑aware measure of verification complexity and an exact, albeit computationally intensive, certification algorithm. It complements existing SAT/SMT, mixed‑integer programming, and abstract‑interpretation approaches by exposing the intrinsic algebraic difficulty of robustness verification, and opens new avenues for designing networks with provable robustness properties.