GraDE: A Graph Diffusion Estimator for Frequent Subgraph Discovery in Neural Architectures

Finding frequently occurring subgraph patterns or network motifs in neural architectures is crucial for optimizing efficiency, accelerating design, and uncovering structural insights. However, as the subgraph size increases, enumeration-based methods are perfectly accurate but computationally prohibitive, while sampling-based methods are computationally tractable but suffer from a severe decline in discovery capability. To address these challenges, this paper proposes GraDE, a diffusion-guided search framework that ensures both computational feasibility and discovery capability. The key innovation is the Graph Diffusion Estimator (GraDE), which is the first to introduce graph diffusion models to identify frequent subgraphs by scoring their typicality within the learned distribution. Comprehensive experiments demonstrate that the estimator achieves superior ranking accuracy, with up to 114% improvement compared to sampling-based baselines. Benefiting from this, the proposed framework successfully discovers large-scale frequent patterns, achieving up to 30$\times$ higher median frequency than sampling-based methods.

💡 Research Summary

The paper tackles the problem of discovering frequent subgraph motifs in neural architecture graphs, a task that is crucial for performance optimization, automated design, and structural analysis of deep learning models. Traditional approaches fall into two categories: exact enumeration (e.g., ESU) and sampling‑based methods (e.g., Rand‑ESU, NRS). Enumeration guarantees perfect frequency counts but becomes computationally infeasible as the subgraph size k grows because the search space expands exponentially (NP‑hard). Sampling methods scale better but suffer from severe sparsity when k is large, leading to high variance in frequency estimates and a dramatic loss of discovery capability.

GraDE (Graph Diffusion Estimator) proposes a third way by leveraging recent graph diffusion generative models as a surrogate for subgraph frequency. The central insight (Observation 4.1) is that the generative probability p₀(G₀) assigned by a diffusion model trained on a representative set of k‑node subgraphs Dₖ correlates strongly with the true relative frequency RFₖ(G₀) of that subgraph in the whole dataset. Consequently, p₀(G₀) can be used as a scoring function that reflects how “typical” a candidate subgraph is under the learned distribution.

The framework consists of three stages. First, a subgraph sampling module collects a uniform, representative training set Dₖ using existing samplers (Rand‑ESU, ARS, NRS, etc.). The authors treat this module as pluggable, allowing any unbiased sampler to be used. Second, an estimator is constructed by training a graph diffusion model on Dₖ. The paper focuses on a DisCo‑based implementation (DisCo‑E) but also mentions DiGress and DeFoG variants. The diffusion process is modeled as a continuous‑time Markov chain (CTMC) over a discrete state space that encodes node and edge types. Forward transition rates are factorized across nodes and edges with a common noise schedule β(t). The reverse process, required for likelihood computation, is derived via detailed balance and approximated as a Poisson process, yielding a closed‑form expression for short time intervals (Equation 8). Monte‑Carlo sampling of forward‑reverse trajectories provides an unbiased estimator of p₀(G₀).

Third, the estimated scores are fed into a beam search. Starting from size‑1 subgraphs, candidates are iteratively expanded; at each step only the top‑N subgraphs according to the GraDE score are retained. This pruning dramatically reduces the combinatorial explosion, turning what would be an exponential search into a near‑linear one with respect to k.

Empirical evaluation spans four synthetic NAS‑Bench datasets (101, 201, 301, NLP) and a real‑world “Young” architecture collection. The authors compare GraDE against exact enumeration (where feasible) and several sampling baselines. Two main metrics are reported: (1) ranking accuracy measured by Spearman’s rank correlation between estimated scores and true frequencies, and (2) discovery capability measured by the median relative frequency of the motifs found. GraDE achieves up to a 114 % improvement in Spearman correlation over sampling baselines, especially in low‑sample regimes, and discovers motifs whose median frequency is up to 30× higher for large k (≥ 8). While exact enumeration still yields slightly higher absolute accuracy, GraDE’s runtime is orders of magnitude lower, making it practical for real‑world architecture repositories.

The paper also discusses limitations. The quality of the diffusion estimator depends on the representativeness of the sampled training set; biased or insufficient samples could lead to a skewed model. Training a diffusion model itself incurs non‑trivial GPU cost, although this cost is amortized across many downstream searches. Currently a separate model is trained for each subgraph size k; future work could explore multi‑scale diffusion models that share parameters across k to further reduce training overhead.

In conclusion, GraDE introduces a novel use of graph diffusion generative models as frequency surrogates, bridging the gap between the accuracy of enumeration and the scalability of sampling. By integrating the estimator with a beam search, the method achieves both computational feasibility and high discovery capability, opening new avenues for motif mining in large‑scale neural architecture graphs. Future directions include adaptive sampling strategies, multi‑scale diffusion architectures, and incorporation of domain‑specific priors into the beam search to further boost efficiency and robustness.