Spectral Sparsification and Regret Minimization Beyond Matrix Multiplicative Updates

In this paper, we provide a novel construction of the linear-sized spectral sparsifiers of Batson, Spielman and Srivastava [BSS14]. While previous constructions required $\Omega(n^4)$ running time [BSS14, Zou12], our sparsification routine can be implemented in almost-quadratic running time $O(n^{2+\varepsilon})$. The fundamental conceptual novelty of our work is the leveraging of a strong connection between sparsification and a regret minimization problem over density matrices. This connection was known to provide an interpretation of the randomized sparsifiers of Spielman and Srivastava [SS11] via the application of matrix multiplicative weight updates (MWU) [CHS11, Vis14]. In this paper, we explain how matrix MWU naturally arises as an instance of the Follow-the-Regularized-Leader framework and generalize this approach to yield a larger class of updates. This new class allows us to accelerate the construction of linear-sized spectral sparsifiers, and give novel insights on the motivation behind Batson, Spielman and Srivastava [BSS14].

💡 Research Summary

The paper presents a new algorithmic framework for constructing linear‑size spectral sparsifiers—graphs that approximate the Laplacian of an input graph up to a (1+ε) factor—while dramatically reducing the running time from the previously known quartic‑time constructions. The classic Batson‑Spielman‑Srivastava (BSS) method achieves optimal edge count O(n/ε²) but requires Ω(n⁴) time; Spielman‑Srivastava’s randomized approach is faster but uses O(n log n/ε²) edges. The authors bridge these two lines of work by recasting spectral sparsification as a regret‑minimization problem over density matrices (positive semidefinite matrices with trace 1).

In the full‑information online setting, a player repeatedly selects a density matrix Xₖ, observes a loss matrix Fₖ, and incurs loss ⟨Fₖ, Xₖ⟩. The goal is to keep cumulative regret against the best fixed matrix U small. This setting is naturally solved by the Follow‑the‑Regularized‑Leader (FTRL) framework: Xₖ₊₁ = argmin_{Z∈Δ} { w(Z) + α∑_{j≤k}⟨Fⱼ, Z⟩ }, where w(·) is a strongly convex regularizer. The well‑known matrix multiplicative‑weights update (MWU) corresponds to choosing the entropy regularizer w(X)=Tr X(log X−I).

The key innovation is to replace the entropy regularizer with a family of ℓ_{1‑1/q} regularizers: w_q(X)=−(q/(q−1))·Tr X^{1‑1/q} for any integer q≥2. This change modifies the Bregman divergence and, crucially, the “width” term in the regret bound. Instead of a global ℓ_∞ bound on the loss matrices, the bound now involves a local norm ‖Fₖ‖{Xₖ}=max_i |f{k,i}|√{x_{k,i}}, which can be dramatically smaller in the sparsification context where the effective resistance of edges may be as large as n. The “diameter” term grows from log n (entropy case) to √n, but the overall regret remains O(√{n·T}) when the local norm is small, matching the behavior needed for BSS‑type guarantees.

By setting q=2 the authors recover an algorithm essentially equivalent to BSS, but with a formulation that admits efficient implementation. For larger constant q, the update rule simplifies to Xₖ₊₁ ∝ (I − α∑_{j≤k}Fⱼ)^{−q}, which can be computed using fast matrix‑multiplication techniques and avoids expensive matrix exponentials. The resulting algorithm constructs a (1+ε) spectral sparsifier with at most O(√q·n/ε²) edges in time O(m n^{1+1/q}/ε⁵). Choosing q as a modest constant (e.g., q=4 or 8) yields an almost‑quadratic running time O(n^{2+δ}) for any small δ>0, a substantial improvement over prior Ω(n⁴) bounds.

The paper also extends the method to general sums of rank‑1 PSD matrices, achieving similar edge‑count guarantees with an almost‑cubic runtime O(n^{3+1/q}) for the matrix case. Further applications include sparsifying high‑rank PSD sums, weakly‑weighted graph sparsification (κ‑spectral sparsifiers), and hypergraph or SDP sparsification, all benefiting from the same regret‑minimization perspective.

Technical contributions include:

1. Formal derivation of the matrix‑valued FTRL update for ℓ_{1‑1/q} regularizers, together with tight regret bounds that capture the local‑norm width term.
2. A “twisted” mirror‑descent analysis that handles the non‑smoothness of the ℓ_{1‑1/q} regularizer and yields the required potential‑function decrease.
3. Efficient implementation details: how to maintain the cumulative loss matrix, compute the required matrix powers, and perform edge sampling based on approximate effective resistances.
4. Proof that the BSS potential function is a special case of the ℓ_{1‑1/q} potential, thereby providing a unifying view of deterministic and randomized sparsifiers.

In summary, the authors demonstrate that spectral sparsification can be viewed as a specific instance of online regret minimization over density matrices. By exploiting the flexibility of the FTRL framework and selecting appropriate regularizers, they obtain deterministic sparsifiers with optimal edge counts and dramatically faster runtimes, opening the door to practical large‑scale applications of spectral sparsification in graph algorithms, numerical linear algebra, and beyond.