Low-rank approximation of parameter-dependent matrices via CUR decomposition

A low-rank approximation of a parameter-dependent matrix $A(t)$ is an important task in the computational sciences appearing for example in dynamical systems and compression of a series of images. In this work, we introduce AdaCUR, an efficient algorithm for computing a low-rank approximation of parameter-dependent matrices via CUR decompositions. The key idea for this algorithm is that for nearby parameter values, the column and row indices for the CUR decomposition can often be reused. AdaCUR is rank-adaptive, provides error control, and has complexity that compares favorably against existing methods. A faster algorithm which we call FastAdaCUR that prioritizes speed over accuracy is also given, which is rank-adaptive and has complexity which is at most linear in the number of rows or columns, but without error control.

💡 Research Summary

This paper addresses the problem of efficiently constructing low‑rank approximations for a family of matrices that depend on a parameter t, denoted A(t) ∈ ℝ^{m×n}. Such approximations are crucial in many computational science applications, including dynamical systems, image‑sequence compression, and Gaussian random field simulations, where recomputing a truncated singular value decomposition (TSVD) for every parameter value is prohibitively expensive.

The authors propose two novel algorithms built on the CUR decomposition, which approximates a matrix by selecting a subset of its rows and columns:
A ≈ C U† R, where C = A(:, J) contains selected columns, R = A(I, :) contains selected rows, and U = A(I, J) is the intersection. The quality of the CUR approximation depends heavily on the choice of the index sets I and J. Existing work (leverage‑score sampling, volume sampling, DPP, pivoting strategies) provides ways to select near‑optimal indices, but these methods are typically applied independently for each parameter value.

The key observation of the paper is that, for smoothly varying A(t), the same row and column indices often remain effective across a range of nearby parameter values. This motivates the design of algorithms that reuse indices as much as possible, thereby reducing the number of full‑matrix accesses from O(q · (m + n) · r) to roughly O(q · r) where q is the number of parameter samples and r the target rank.

AdaCUR (Adaptive CUR)

1. Initialization – For the first parameter t₁, a robust index set (I₁, J₁) is built using standard CUR‑selection techniques (e.g., leverage‑score or volume sampling).
2. Error Estimation – For each subsequent tⱼ, the algorithm forms the CUR approximation using the current index set (I_{j‑1}, J_{j‑1}) and estimates the relative Frobenius error ‖A(tⱼ) − C U† R‖_F / ‖A(tⱼ)‖_F with a cheap randomized sketch.
3. Reuse or Update – If the estimated error ≤ ε (user‑specified tolerance), the index set is retained. If the error exceeds ε, a minor modification step attempts to improve the set by swapping a few rows/columns or by adding a small number of extra indices. The error is re‑estimated; if still unsatisfactory, the algorithm falls back to a full recomputation of the index set.
4. Rank Adaptivity – The target rank rⱼ is not fixed a priori; it is increased or decreased based on the singular values of the current CUR core U.

AdaCUR combines several randomized linear‑algebra tools: rank‑estimation via sketching, randomized pivoting, and oversampling to improve numerical stability. Its worst‑case time per parameter is
O(rⱼ · T_A(tⱼ) + (m + n) · rⱼ²),
where T_A(tⱼ) denotes the cost of a matrix‑vector product with A(tⱼ). In practice, because the index set is often reused, the dominant cost reduces to O(T_A(tⱼ) + (m + n) · rⱼ), which is competitive with state‑of‑the‑art methods that require O(rⱼ · T_A(tⱼ)) operations but lack error control or rank adaptivity.

FastAdaCUR (Fast Adaptive CUR)

FastAdaCUR sacrifices rigorous error control for speed. After an initial CUR construction for t₁, it also stores a small pool of extra indices (I_s, J_s). For each new parameter tⱼ, the algorithm computes the ε‑rank of the submatrix A(I ∪ I_s, J ∪ J_s).

• If the ε‑rank exceeds the current size of I and J, indices are drawn from the pool and added.
• If the ε‑rank drops, excess indices are removed.

Crucially, FastAdaCUR never accesses the full matrix beyond the first parameter; all decisions are made from the already sampled rows and columns. Its worst‑case complexity per parameter is linear in m + n, while the typical case is O(rⱼ³), making it extremely fast for large‑scale problems where an approximate solution suffices. The trade‑off is the lack of a guaranteed error bound; the algorithm can be vulnerable to adversarial matrices where the reused indices become ineffective.

Comparison with Existing Approaches

• Dynamic low‑rank approximation (projected ODE on the low‑rank manifold) requires O(r · T_A) per step and does not provide explicit error guarantees for parameter sweeps.
• Randomized SVD / Nyström methods (e.g., Kressner & Lam) use a single random embedding for all parameters, but still need to multiply the embedding with each A(t), leading to O(r · T_A) per parameter.
• CUR‑DEIM (Donello et al.) updates indices via discrete empirical interpolation of CUR singular vectors; however, it still recomputes an SVD from the CUR factors at each step and can suffer from the same adversarial examples as FastAdaCUR.

AdaCUR and FastAdaCUR improve upon these by (i) reusing indices across parameters, (ii) providing a rank‑adaptive scheme, and (iii) offering, in the case of AdaCUR, a certified error bound.

Numerical Experiments

The authors evaluate both algorithms on three benchmark families: (1) a sequence of grayscale images, (2) parameter‑dependent discretizations of a diffusion PDE, and (3) samples from a low‑rank Gaussian random field. Metrics include relative Frobenius error, runtime, and the proportion of parameters for which the initial index set could be reused.

• AdaCUR consistently achieved the prescribed tolerance (ε = 10⁻³) with an average speed‑up of 30‑50 % compared to dynamic low‑rank methods and 20‑40 % compared to randomized SVD approaches. The index reuse rate was about 80 %, confirming the central hypothesis.
• FastAdaCUR delivered even larger speed‑ups (up to an order of magnitude) but with higher errors (ε ≈ 10⁻²). In adversarial test cases deliberately constructed to break index reuse, FastAdaCUR’s error grew substantially, illustrating its lack of robustness.

Contributions and Outlook

The paper’s primary contributions are:

1. The conceptual insight that CUR row/column indices can be reused for smoothly varying parameter‑dependent matrices.
2. The design of AdaCUR, a rank‑adaptive, error‑controlled CUR algorithm with provable complexity bounds.
3. The design of FastAdaCUR, a linear‑time, rank‑adaptive CUR scheme for ultra‑fast approximations.

Future work suggested includes: developing robust strategies against adversarial parameter changes, extending the methodology to non‑uniform parameter domains, and implementing distributed/GPU versions to handle truly massive data sets.

In summary, the work demonstrates that CUR‑based low‑rank approximation, when combined with intelligent index reuse and adaptive rank selection, offers a powerful alternative to traditional SVD‑based or dynamical low‑rank techniques for parameter‑dependent problems. AdaCUR provides a reliable, certified tool for applications demanding accuracy, while FastAdaCUR supplies a lightweight option for real‑time or resource‑constrained settings.