Graph-based Nearest Neighbors with Dynamic Updates via Random Walks

Approximate nearest neighbor search (ANN) is a common way to retrieve relevant search results, especially now in the context of large language models and retrieval augmented generation. One of the most widely used algorithms for ANN is based on constructing a multi-layer graph over the dataset, called the Hierarchical Navigable Small World (HNSW). While this algorithm supports insertion of new data, it does not support deletion of existing data. Moreover, deletion algorithms described by prior work come at the cost of increased query latency, decreased recall, or prolonged deletion time. In this paper, we propose a new theoretical framework for graph-based ANN based on random walks. We then utilize this framework to analyze a randomized deletion approach that preserves hitting time statistics compared to the graph before deleting the point. We then turn this theoretical framework into a deterministic deletion algorithm, and show that it provides better tradeoff between query latency, recall, deletion time, and memory usage through an extensive collection of experiments.

💡 Research Summary

The paper addresses a critical gap in the widely‑used Hierarchical Navigable Small World (HNSW) index: while HNSW efficiently supports insertions and queries, it lacks a proper deletion mechanism. Existing work either leaves deleted vectors as “tombstones” (preserving graph connectivity but inflating memory use and query latency) or applies heuristic patching strategies (local reconnect, FreshDiskANN, global reconnect) that suffer from high deletion time, reduced recall, or difficult parallelization.

To overcome these limitations, the authors re‑interpret HNSW search as a stochastic process. Instead of the deterministic greedy step “move to the nearest neighbor”, they define a “softmax walk”: from the current vertex u, the next vertex v is sampled with probability proportional to exp(−r²·‖q−v‖²), where q is the query and r > 0 is a temperature‑like parameter. Empirically, for sufficiently large r, the softmax walk reproduces the greedy path while providing a clean probabilistic model. This model treats the HNSW graph as a query‑dependent weighted graph, where each directed edge u→v carries weight w(u,v)=exp(−r²·‖q−v‖²).

Within this framework, deletion is cast as a random‑walk sparsification problem: after removing a vertex p, we must reconnect its neighborhood N(p) so that the hitting‑time distribution (expected steps for a walk to reach any target) remains unchanged. The authors first compute exact edge weights for all potential new edges among N(p) that preserve the softmax transition probabilities. Then they apply a randomized sparsification scheme that samples edges proportionally to these weights, guaranteeing that the expected hitting times of the new graph match those of the original.

To obtain a practical algorithm, the random sampling is replaced by a deterministic “top‑k” selection: for each vertex we keep the k edges with highest weights and discard the rest. This deterministic variant, named SPatch, runs in O(|N(p)| log |N(p)|) time per deletion and requires no global re‑insertion or expensive parallel operations. Crucially, SPatch maintains the same local connectivity rules used by HNSW insertions, ensuring that the overall graph structure remains navigable.

The authors evaluate SPatch on several large‑scale datasets (text, image, multimodal) under massive deletion scenarios (30 %–70 % of points removed). Compared against five baselines—tombstone, no‑patch, local reconnect, FreshDiskANN, and global reconnect—SPatch consistently achieves:

• Recall ≥ 95 %, comparable to or better than all baselines.
• Query latency increase ≤ 10 % even at high deletion ratios, whereas tombstone latency grows 2–3×.
• Deletion time up to 2× faster than FreshDiskANN and global reconnect for batch deletions.
• Memory usage proportional to the remaining points, reducing overall footprint by 30 %–50 % relative to tombstones.

Table 1 in the paper summarizes these trade‑offs, showing that SPatch simultaneously satisfies the four key desiderata for a deletion algorithm: low memory, fast deletion, high recall, and low query latency.

The theoretical contribution lies in linking HNSW navigation to a random‑walk model and proving that preserving hitting times suffices to retain search quality after deletions. The practical contribution is a simple, deterministic edge‑pruning procedure that can be dropped into existing HNSW implementations without major code changes.

Future work suggested includes automatic tuning of the temperature parameter r, extending the framework to other graph‑based ANN structures (e.g., DiskANN, Navigable Small World), and exploring multi‑query concurrent deletions. Overall, the paper delivers a well‑grounded, experimentally validated solution that fills a long‑standing void in dynamic ANN indexing.