Online convex optimization for robust control of constrained dynamical systems

This article investigates the problem of controlling linear time-invariant systems subject to time-varying and a priori unknown cost functions, state and input constraints, and exogenous disturbances. We combine the online convex optimization framework with tools from robust model predictive control to propose an algorithm that is able to guarantee robust constraint satisfaction. The performance of the closed loop emerging from application of our framework is studied in terms of its dynamic regret, which is proven to be bounded linearly by the variation of the cost functions and the magnitude of the disturbances. We corroborate our theoretical findings and illustrate implementational aspects of the proposed algorithm by a numerical case study on a tracking control problem of an autonomous vehicle.

💡 Research Summary

The paper addresses the challenging problem of controlling linear time‑invariant (LTI) systems when the cost function, state and input constraints, and external disturbances are all time‑varying and a priori unknown. Classical model predictive control (MPC) can handle constraints and disturbances, while online convex optimization (OCO) excels at dealing with unknown, changing cost functions with low computational overhead. The authors propose a novel framework that fuses OCO with robust MPC techniques to obtain a controller that guarantees robust constraint satisfaction and provides provable performance guarantees in terms of dynamic regret.

Problem setting
The system dynamics are given by
 x_{t+1}=A x_t + B u_t + w_t, \tilde{x}_t = x_t + v_t,
with state constraints x_t∈X, input constraints u_t∈U, and bounded disturbance and measurement‑noise sets w_t∈W, v_t∈V. At each time step a convex cost L_t(x,u) is revealed; the functions are Lipschitz continuous on the feasible set and, after a linear feedback transformation K, become α‑strongly convex with L‑Lipschitz gradients. The goal is to design an online algorithm that, using only the noisy measurement \tilde{x}_t and past cost information, generates a control sequence that respects the constraints for all t and minimizes the cumulative performance gap (dynamic regret) with respect to the optimal time‑varying steady‑state solution of the constrained optimal control problem.

Algorithmic construction

1. Stabilizing feedback – Choose a matrix K such that A_K = A + B K is Schur stable. This enables the construction of a minimal robust positively invariant (RPI) set P* for the closed‑loop system under bounded disturbances.
2. Constraint tightening – The original constraint sets are shrunk by the RPI set: X̃ = X ⊖ P, Ũ = U ⊖ K P. The set of admissible steady‑states S = { (x,u) | x = G_K u, x∈X̃, u+Kx∈Ũ } is convex, compact, and guarantees that any trajectory starting inside S⊕P stays inside X and U for all future times, regardless of disturbances.
3. μ‑step prediction – Using the current noisy state \tilde{x}t and the μ‑step input sequence computed at the previous iteration, a prediction of the future state ˆx{μ,t} = A_K^μ \tilde{x}t + S_c σ(ˆu{μ,t-1}) is formed, where S_c is the controllability matrix of the stabilized system and μ ≥ μ* is a horizon long enough to guarantee controllability.
4. Online gradient descent (OGD) – An intermediate point ˆz_t = (ˆx_{μ,t}, ˆu_{s,t-1}) is constructed and then projected onto the tightened steady‑state set \bar S while taking a gradient step on the revealed cost:
    ˆζ_t = Π_{\bar S}