Convergence Analysis of Continuous-Time Distributed Stochastic Gradient Algorithms

In this paper, we propose a new framework to study distributed optimization problems with stochastic gradients by employing a multi-agent system with continuous-time dynamics. Here the goal of the agents is to cooperatively minimize the sum of convex objective functions. When making decisions, each agent only has access to a stochastic gradient of its own objective function rather than the real gradient, and can exchange local state information with its immediate neighbors via a time-varying directed graph. Particularly, the stochasticity is depicted by the Brownian motion. To handle this problem, we propose a continuous-time distributed stochastic gradient algorithm based on the consensus algorithm and the gradient descent strategy. Under mild assumptions on the connectivity of the graph and objective functions, using convex analysis theory, the Lyapunov theory and Ito formula, we prove that the states of the agents asymptotically reach a common minimizer in expectation. Finally, a simulation example is worked out to demonstrate the effectiveness of our theoretical results.

💡 Research Summary

The paper introduces a novel continuous‑time framework for distributed stochastic optimization over multi‑agent networks. Each agent i holds a local state vector x_i(t)∈ℝ^m and seeks to minimize the global objective f(x)=∑_{i=1}^n f_i(x), where each f_i is convex, twice continuously differentiable, and L‑smooth. Unlike deterministic settings, agents have access only to noisy gradients modeled as
∇̃f_i(x_i(t)) = ∇f_i(x_i(t)) dt + g_i(t) dB_i(t),
where B_i(t) is a standard Brownian motion and g_i(t) is a bounded noise intensity. Agents exchange information over a time‑varying directed communication graph G(t) that is assumed to be (δ,T_c)‑strongly connected and weight‑balanced.

The proposed algorithm combines a consensus term with a stochastic gradient descent term:
dx_i(t) = Σ_{j=1}^n a_{ij}(t)(x_j(t)−x_i(t)) dt − η_t ∇̃f_i(x_i(t)) dt,
where a_{ij}(t) are the adjacency weights and η_t>0 is a non‑increasing step size. By discretizing (Euler scheme) the authors show that this dynamics is the continuous‑time counterpart of a standard distributed stochastic gradient descent with Gaussian‑like noise.

Key technical contributions include:

1. A rigorous stochastic analysis using Itô calculus. Lemma 1 provides the Itô formula for a twice‑differentiable test function; Lemma 3 bounds the p‑th moment of stochastic integrals; Lemma 4 supplies integral inequalities needed for handling the decaying step size.
2. Decomposition of the system into the average state \bar{x}(t)= (1/n)∑_i x_i(t) and deviation terms x_i(t)−\bar{x}(t). Lemma 5 gives explicit bounds on the deviations in terms of the graph’s exponential consensus rate (λ^t) and the accumulated step sizes.
3. Construction of a Lyapunov function that captures both optimality error (‖\bar{x}(t)−x^*‖^2) and consensus error (∑_i‖x_i(t)−\bar{x}(t)‖^2). Applying the Itô formula to this Lyapunov function and taking expectations yields a differential inequality that can be integrated using the step‑size schedule η_t = β (t+1)^{‑a}.

The main result (Theorem 1) states that under Assumptions 1‑4 (graph connectivity, bounded gradients, L‑smoothness, bounded noise intensity) and with η_t = β (t+1)^{‑a} for some ½ < a ≤ 1, the expected suboptimality satisfies:

• For ½ < a < ¾:  E