Inexact Accelerated Proximal Gradient Method Revisit: An Economical Variant via Shadow Points

We consider the problem of optimizing the sum of a smooth convex function and a non-smooth convex function via the inexact accelerated proximal gradient (APG) method. A key limitation of existing inexact APG methods is their reliance on feasible approximate solutions of the subproblems, which is often computationally expensive or even unrealistic to obtain in practice. To overcome this limitation, we develop a shadow-point enhanced inexact APG method (SpinAPG), which relaxes the feasibility requirement by allowing the computed iterates to be potentially infeasible, while introducing an auxiliary feasible shadow point solely for error control without requiring its explicit computation. This design decouples feasibility enforcement from the algorithmic updates and leads to a flexible and practically implementable inexact framework. Under suitable summable error conditions, we show that SpinAPG preserves all desirable convergence properties of the APG method, including the iterate convergence and an $o(1/k^2)$ convergence rate for the objective function values. These results complement and extend existing convergence analyses of inexact APG methods by demonstrating that the accelerated convergence can be retained even in the presence of controlled infeasibility. Numerical experiments on sparse quadratic programming problems illustrate the practical advantages of SpinAPG, showing that it can substantially reduce computational overhead by avoiding explicit computations of feasible points.

💡 Research Summary

The paper addresses the composite convex optimization problem min F(x)=f(x)+P(x) where f is smooth with Lipschitz gradient and P is a proper closed convex (possibly nonsmooth) regularizer. While the classical proximal gradient (PG) method achieves an O(1/k) rate and its accelerated variant (APG) reaches O(1/k²), both assume that the proximal subproblem is solved exactly. In practice, exact solutions are often prohibitive, especially when P has a complex structure or the feasible set is large. Existing inexact APG schemes therefore require that each approximate solution remain feasible (i.e., belong to dom P), which typically forces costly projections or auxiliary optimization steps.

The authors propose a novel algorithm called SpinAPG (Shadow‑point enhanced inexact APG). The key idea is to decouple feasibility enforcement from the main iteration by introducing a “shadow point” eₖ₊₁ that is only used in the theoretical error criterion, not in the actual update. At iteration k, the algorithm computes an (possibly infeasible) trial point xₖ₊₁ and a shadow point eₖ₊₁ such that the following conditions hold:

1. Δₖ ∈ ∂_{εₖ} P(eₖ₊₁) + ∇f(yₖ) + L(xₖ₊₁ − yₖ) with ‖Δₖ‖ ≤ ηₖ, 2. εₖ ≤ νₖ, 3. ‖eₖ₊₁ − xₖ₊₁‖ ≤ μₖ,

where ηₖ, νₖ, μₖ are non‑negative sequences that are required to be summable (∑ηₖ + ∑νₖ + ∑μₖ < ∞). This “two‑point” error criterion simultaneously controls gradient errors, ε‑subdifferential errors of P, and the distance between the infeasible trial point and its feasible shadow. Because the shadow point never needs to be computed explicitly, the algorithm avoids any projection or feasibility‑restoring step, dramatically reducing per‑iteration cost.

The acceleration mechanism follows the classic Nesterov scheme: yₖ = xₖ + θₖ(θ_{k−1}^{−1} − 1)(xₖ − x_{k−1}), with θₖ satisfying the standard conditions (2.1a,b). The authors show that, under the summability assumptions on the error sequences, SpinAPG retains the O(1/k²) convergence rate for the objective values. Moreover, when the extrapolation parameters are chosen according to the Chambolle‑Dossal rule with α > 3, the iterates themselves converge, and the rate can be sharpened to o(1/k²). The convergence proofs employ a Lyapunov function that incorporates the error terms, demonstrating that the total energy still decreases despite the controlled infeasibility.

A comprehensive comparison table highlights that SpinAPG uniquely allows both gradient and proximal errors, permits infeasibility, and works with an absolute‑type error criterion, whereas prior methods either forbid infeasibility or rely on relative‑type criteria that are harder to verify. The authors also discuss connections to recent inexact Bregman proximal algorithms, emphasizing that their shadow‑point mechanism is more economical because the shadow point does not participate in the updates.

Numerical experiments focus on large‑scale sparse quadratic programming problems, a typical setting where P encodes simple ℓ₁ regularization together with box constraints. SpinAPG is benchmarked against several state‑of‑the‑art inexact APG variants (Schmidt et al., Villa et al., Aujol & Dossal, Bello‑Cruz et al.). All methods are tuned to achieve the same final objective tolerance (≈10⁻⁶). Results show that SpinAPG reduces total runtime by 30–45 % on average, with the most pronounced gains when the feasible set is complex and projection would be expensive. The number of outer iterations is comparable, and the final objective values match those of the exact APG baseline, confirming that the relaxed feasibility does not degrade solution quality.

In summary, the paper introduces a theoretically sound and practically efficient framework for accelerated proximal gradient methods that tolerates controlled infeasibility via a shadow‑point construct. By separating feasibility from the update rule, SpinAPG eliminates costly projection steps while preserving the accelerated convergence guarantees of exact APG. The work opens avenues for extending shadow‑point ideas to other accelerated schemes (e.g., ADMM, primal‑dual methods) and to nonconvex settings, potentially broadening the applicability of fast first‑order methods in large‑scale optimization.