Robustness of Neural Networks: A Probabilistic and Practical Approach

Neural networks are becoming increasingly prevalent in software, and it is therefore important to be able to verify their behavior. Because verifying the correctness of neural networks is extremely challenging, it is common to focus on the verification of other properties of these systems. One important property, in particular, is robustness. Most existing definitions of robustness, however, focus on the worst-case scenario where the inputs are adversarial. Such notions of robustness are too strong, and unlikely to be satisfied by-and verifiable for-practical neural networks. Observing that real-world inputs to neural networks are drawn from non-adversarial probability distributions, we propose a novel notion of robustness: probabilistic robustness, which requires the neural network to be robust with at least $(1 - \epsilon)$ probability with respect to the input distribution. This probabilistic approach is practical and provides a principled way of estimating the robustness of a neural network. We also present an algorithm, based on abstract interpretation and importance sampling, for checking whether a neural network is probabilistically robust. Our algorithm uses abstract interpretation to approximate the behavior of a neural network and compute an overapproximation of the input regions that violate robustness. It then uses importance sampling to counter the effect of such overapproximation and compute an accurate estimate of the probability that the neural network violates the robustness property.

💡 Research Summary

The paper addresses the growing need to verify neural network behavior in modern software systems. Existing robustness definitions focus on worst‑case, adversarial perturbations, which are often too stringent and computationally infeasible for practical, non‑malicious deployments. To bridge this gap, the authors introduce probabilistic robustness, a property that leverages the fact that real‑world inputs are drawn from an underlying probability distribution D.

Formally, a network f is said to be (δ, k, ε)‑probabilistically robust if, for a pair of inputs (x, x′) sampled from D and conditioned on ‖x‑x′‖ ≤ δ, the Lipschitz condition ‖f(x)‑f(x′)‖ ≤ k·‖x‑x′‖ holds with probability at least 1 − ε. This definition replaces the universal quantification of local and global robustness with a high‑probability guarantee, making the property both globally defined and tunable via ε. The requirement that the output be continuous (e.g., a real‑valued vector) is justified because classifiers can be interpreted as producing probability distributions over classes.

To verify this property, the authors propose an algorithm that combines abstract interpretation and importance sampling. The steps are:

1. Product Network Construction – Create a product program pf by duplicating the original network f. The product network receives a pair (x, x′) and independently computes (f(x), f(x′)). This transformation turns the hyper‑property (relation between two executions) into a single‑trace property on pf.

2. Abstract Interpretation – Define the negated Lipschitz condition φ = ¬(‖f(x′)‑f(x)‖ ≤ k·‖x′‑x‖). Using a powerset‑polyhedra abstract domain, the abstract interpreter computes an over‑approximation poly of the set of input pairs that violate the property. Each element of poly is a polyhedron in the joint input space.

3. Importance Sampling – For each polyhedron p ∈ poly, the algorithm draws samples from a biased distribution that concentrates on p, evaluates whether the concrete violation occurs, and re‑weights the results to estimate the original probability under D. This step corrects the conservatism introduced by the abstract over‑approximation and yields a statistically sound estimate of the violation probability.

4. Decision – The estimated probabilities from all polyhedra are summed. If the total exceeds ε, the network fails the probabilistic robustness check (output F); otherwise it passes (output T).

The method offers several advantages. By allowing ε to be set by the user, it provides a flexible trade‑off between verification effort and robustness guarantees. Abstract interpretation ensures soundness: no violating input pair is missed, only potentially spurious ones are added. Importance sampling then reduces the conservatism, delivering a tight estimate with far fewer samples than naïve Monte‑Carlo would require.

However, the approach has notable limitations. It assumes knowledge of the input distribution D; when D is unknown, one must resort to a surrogate distribution, which may misrepresent real data. The requirement of continuous outputs excludes raw categorical classifiers unless they are transformed (e.g., via soft‑max), potentially introducing additional approximation error. The polyhedral abstract domain can become overly coarse in high‑dimensional spaces, leading to many large polyhedra and consequently higher sampling costs. Finally, the paper provides limited empirical evaluation, leaving open questions about scalability to large‑scale convolutional or transformer models.

In summary, the paper contributes a novel, probabilistic notion of neural‑network robustness and a practical verification pipeline that blends static analysis with statistical estimation. It moves robustness verification from an impractical worst‑case paradigm toward a realistic, distribution‑aware framework, opening avenues for future work on distribution learning, tighter abstract domains, and extensive benchmarking on modern deep‑learning architectures.