Graph Attention Network for Node Regression on Random Geometric Graphs with Erdős--Rényi contamination

Graph attention networks (GATs) are widely used and often appear robust to noise in node covariates and edges, yet rigorous statistical guarantees demonstrating a provable advantage of GATs over non-attention graph neural networks~(GNNs) are scarce. We partially address this gap for node regression with graph-based errors-in-variables models under simultaneous covariate and edge corruption: responses are generated from latent node-level covariates, but only noise-perturbed versions of the latent covariates are observed; and the sample graph is a random geometric graph created from the node covariates but contaminated by independent Erdős–Rényi edges. We propose and analyze a carefully designed, task-specific GAT that constructs denoised proxy features for regression. We prove that regressing the response variables on the proxies achieves lower error asymptotically in (a) estimating the regression coefficient compared to the ordinary least squares (OLS) estimator on the noisy node covariates, and (b) predicting the response for an unlabelled node compared to a vanilla graph convolutional network~(GCN) – under mild growth conditions. Our analysis leverages high-dimensional geometric tail bounds and concentration for neighbourhood counts and sample covariances. We verify our theoretical findings through experiments on synthetically generated data. We also perform experiments on real-world graphs and demonstrate the effectiveness of the attention mechanism in several node regression tasks.

💡 Research Summary

The paper tackles a node‑regression problem in which each node i possesses an unobserved latent covariate vector (x_i\in\mathbb{R}^d) and a scalar response (y_i = x_i^\top\beta + \varepsilon_i). The analyst only observes a noisy version (z_i = x_i + \eta_i) (with fixed‑variance Gaussian measurement error) and a graph that is the union of two independent components: (1) a random geometric (dot‑product) graph where an edge ((i,j)) exists if (x_i^\top x_j) exceeds a threshold (t_n), and (2) an Erdős–Rényi (ER) graph with edge probability (p_n). The ER component injects “structural noise” by adding spurious edges that do not reflect similarity in the latent space. Both the dimension (d) and the number of nodes (n) grow to infinity, while the signal‑to‑noise ratios of the covariates, measurement error, and response noise remain constant.

Baseline shortcomings.
If one simply regresses (y) on the observed noisy features (Z), the classical attenuation bias of errors‑in‑variables yields an inconsistent estimator (\hat\beta_Z). If one instead uses a vanilla graph convolutional network (GCN) that averages raw neighbour features, the presence of many ER edges can dominate the neighbourhood, causing the aggregated signal to be heavily diluted. Consequently, both OLS on (Z) and GCN‑based prediction can be far from optimal.

Proposed method – a task‑specific discrete attention GAT.
The authors design a two‑layer attention mechanism that constructs a denoised proxy (\lambda_i) for each latent covariate (x_i). The key ideas are:

1. Coordinate splitting. Each observed vector (z_i) is split into two disjoint blocks: the top (\lceil d/2\rceil) coordinates (z_i^{(1)}) and the bottom (\lfloor d/2\rfloor) coordinates (z_i^{(2)}).

2. Screening (attention) step. Using one block (say (z^{(1)})), the algorithm computes binary attention weights
   \