First Order Methods For Globally Optimal Distributed Controllers Beyond Quadratic Invariance

First Order Metho ds F or Globally Optimal Distributed Con trollers Bey ond Quadratic In v ariance Luca F urieri and Mary am Kamgarp our ∗ Abstract W e study the distributed Linear Quadratic Gaussian (LQG) con trol problem in discrete-time and finite-horizon, where the controller dep ends linearly on the history of the outputs and it is required to lie in a giv en subspace, e.g. to p ossess a certain sparsit y pattern. It is well- kno wn that this problem can b e solv ed with conv ex programming within the Y oula domain if and only if a condition kno wn as Quadratic Inv ariance (QI) holds. In this pap er, we first sho w that given QI sparsit y constraints, one can directly descend the gradient of the cost function within the domain of output-feedback controllers and conv erge to a global optimum. Note that con vergence is guaranteed despite non-con vexit y of the cost function. Second, we c haracterize a class of Uniquely Stationary (US) problems, for which first-order metho ds are guaranteed to con verge to a global optimum. W e show that the class of US problems is strictly larger than that of strongly QI problems and that it is not included in that of QI problems. W e refer to Figure 1 for details. Finally , we prop ose a tractable test for the US property . 1 In tro duction The safe and efficient operation of emerging netw ork ed dynamical systems, such as the smart grid and autonomous v ehicles, relies on the decision making of multiple in teracting agen ts. Con trolling these systems optimally is challenged b y an inherent lac k of information ab out the systems internal v ariables, p ossibly due to priv acy concerns, geographic distance or the high cost of implemen ting a reliable comm unication net work. The classical works of [ 1 , 2 ] highligh ted that, giv en information constrain ts, ev en simple instances of the Linear Quadratic Gaussian (LQG) con trol problem can result in highly intractable optimization tasks. A v ast amount of literature has fo cused on approaching the distributed LQG problem and its v arian ts with conv ex programming in the Y oula parameter [ 3 ]. This enables utilizing efficient off- the-shelf softw are for numerical computation. A main c hallenge inherent to this approach is that the distributed con trol problem admits an exact con vex reform ulation if and only if the information constrain ts and the system dynamics in teract in a Quadratically In v arian t (QI) manner [ 4 , 5 ]. This limitation sev erely restricts the class of problems for which optimal distributed controllers can b e computed in a tractable w ay . A v ariet y of approximation metho ds and alternativ e con troller implemen tations hav e henceforth b een devised to deal with the non-QI cases, based both on conv ex programming and nonlinear optimization. How ev er, these approac hes cannot compute a globally optimal sparse output-feedbac k controller in general. W e refer the reader to [ 6 – 11 ] for a collection of recen t results. ∗ This research w as gratefully funded by the European Union ER C Starting Gran t CONENE. Luca F urieri and Mary am Kamgarpour are with the Automatic Con trol Lab oratory , Department of Information T echnology and Elec- trical Engineering, ETH Z ¨ uric h, Switzerland. E-mails: { furieril, mkamgar } @control.ee.ethz.ch . The source co des of our sim ulations are av ailable upon request. A preliminary v ersion of this pap er app eared on 21.08.2019 as a preprint at https://www.researc h-collection.ethz.ch/handle/20.500.11850/359817 with a different title. 1 The recent y ears hav e witnessed a rapid gro wth of interest in dev eloping learning-based, model- free techniques for optimal con trol problems. Sp ecifically , some scenarios en vision an unkno wn blac k-b o x system, for whic h an optimal b eha vior is obtained by observing the system’s output tra jectories in resp onse to different controllers and iteratively improving the control p olicy . In these cases, optimizing within the Y oula domain is impractical b ecause one is unable to reco ver the disturbance tra jectories from the observ ed output tra jectories for an unknown dynamical system. Therefore, mo del-free scenarios motiv ate optimizing directly within the domain of output-feedbac k con trollers, for instance, b y devising gradien t-descent based methods. Con v ergence of these metho ds to a global optim um w as recen tly pro v en for the LQR problem in the non-distributed case [ 12 – 16 ]. When carrying on these metho ds to the distributed controller case, ho wev er, one can in general only guarantee con v ergence to a stationary p oin t, which ma y not b e a global optimum [ 15 , 17 , 18 ]. F or the infinite-horizon and static-controller cases, this is mainly due to the set of stabilizing distributed controllers b eing disconnected in general [ 19 ]. T o the b est of the authors’ kno wledge, classes of distributed con trol problems solv able to global optimalit y with first-order methods are y et to be c haracterized, and a connection with the QI notion is y et to be established. F urthermore, a condition that is more general than QI for global optimality certificates has not b een found yet. Indeed, the QI notion is closely link ed to using con vex programming; this pap er w as driv en b y the in tuition that less restrictiv e conditions for global optimalit y might exist b y instead using first-order optimization metho ds directly in the domain of output-feedbac k con trollers. W e will show that this in tuition indeed holds true. Motiv ated as ab o ve, we in vestigate first-order metho ds for the distributed LQG problem in discrete-time and finite-horizon. Our contributions are as follows. First, w e sho w that given QI sparsit y constraints, one can descend the gradient of the generally non-conv ex cost function in the output-feedback domain and alwa ys conv erge to a globally optimal distributed controller. W e foresee that this metho d will enable devising learning-based p olicy gradien t approaches for distributed control in future w orks. Second, we c haracterize a new class of Uniquely Stationary (US) con trol problems, which can b e solv ed to global optimalit y using first-order metho ds. W e sho w that ev ery strongly QI problem is US and that there are instances of US problems whic h are neither strongly QI or QI. W e refer to Figure 1 for the details. Pap er structur e: Section 2 in tro duces the necessary notation and background. Section 3 con tains our first result about global optimalit y giv en strong QI and a numerical example. Section 4 establishes our results on first-order metho ds for certificates of global optimality strictly b ey ond QI. W e conclude the pap er in Section 5. 2 Bac kground and Problem Statemen t W e start this section b y pro viding the necessary notation. W e then pro ceed with stating the distributed LQG problem and reviewing useful results ab out disturbance-feedback con trol p olicies and quadratic inv ariance. 2.1 Notation W e use R to denote the set of real n um b ers. The ( i, j )-th element in a matrix Y ∈ R m × n is referred to as Y i,j . W e use I n to denote the identit y matrix of size n × n , 0 m × n to denote the zero matrix of size m × n . Whenever the subscripts are omitted, the dimensions are implied b y the con text. The symbols Im( M ) and Ker( M ) denote the range and the kernel of the linear op erator asso ciated with matrix M . W e write M = blkdg( M 1 , . . . , M n ) to denote a blo c k-diagonal matrix where the blo c ks are the matrices M 1 , . . . , M n . F or a symmetric matrix M = M T w e write M  0 (resp. 2 M  0) if and only if it is p ositiv e definite (resp. p ositiv e semidefinite), that is its eigenv alues are strictly p ositiv e (resp. non-negativ e). F or tw o matrices M , P of an y dimensions M ⊗ P denotes the Kroneck er pro duct and for t wo matrices of equal dimensions M  P denotes the Hadamard pro duct 1 . F or any matrix K ∈ R m × n , v ec( K ) ∈ R mn is a vector obtained b y stacking the columns of K into a single column. Giv en a binary matrix X ∈ { 0 , 1 } m × n , w e define the associated sp arsity subsp ac e as Sparse( X ) := { Y | Y i,j = 0 for all i, j suc h that X i,j = 0 } . Similarly , giv en Y ∈ R m × n , w e define X = Struct( Y ) as the binary matrix suc h that X i,j = 0 if Y i,j = 0 and X i,j = 1 otherwise. Let X , ˆ X ∈ { 0 , 1 } m × n and Z ∈ { 0 , 1 } n × p b e binary matrices. W e adopt the following con v entions: X + ˆ X := Struct( X + ˆ X ), X Z := Struct( X Z ), X ≤ ˆ X if and only if X i,j ≤ ˆ X i,j ∀ i, j . The Euclidean norm of a vector v ∈ R n is denoted b y k v k 2 2 = v T v and the F rob enius norm of a matrix M ∈ R m × n is denoted by k M k 2 F = T race( M T M ). Giv en a matrix K ∈ R m × n and a contin uously differentiable function J : R m × n → R we define ∇ J ( K ) as the m × n matrix suc h that ∇ J ( K ) i,j = ∂ J ( K ) ∂ K i,j . F or a v ector v ∈ R n and a function f : R n → R w e denote the gradient b y ∇ f ( v ) ∈ R n and the Hessian b y ∇ 2 f ( v ) ∈ R n × n . Given a subspace K ⊆ R m × n w e denote its orthogonal complement as K ⊥ . The sym b ol N ( µ, Σ) denotes the normal distribution with exp ected v alue µ ∈ R n and co v ariance matrix Σ ∈ R n × n  0, and x ∼ N ( µ, Σ) indicates that x ∈ R n follo ws the distribution N ( µ, Σ). F or a subspace K ⊆ R n , Π K ( · ) denotes the pro jection op erator on K . 2.2 Problem Setup W e consider time-v arying linear systems in discrete-time x t +1 = A t x t + B t u t + w t , (1) y t = C t x t + v t , where x t ∈ R n is the system state at time t affected b y additiv e noise w t ∼ N (0 , Σ w t ) with x 0 ∼ N ( µ 0 , Σ 0 ) , y t ∈ R p is the output at time t affected by additive noise v t ∼ N (0 , Σ v t ) and u t ∈ R m is the con trol input at time t . W e assume that Σ 0 , Σ w t  0 and Σ v t  0 for all t . W e consider the evolution of (1) in finite-horizon for t = 0 , . . . N , where N ∈ N . By defining the matrices A = blkdg( A 0 , . . . , A N ), B =  blkdg( B 0 , . . . , B N -1 ) 0 n × mN  , C =  blkdg( C 0 , . . . , C N -1 ) T 0 n × pN  T , and the v ectors x =  x T 0 . . . x T N  T ∈ R n ( N +1) , y =  y T 0 . . . y T N − 1  T ∈ R pN , u =  u T 0 . . . u T N − 1  T ∈ R mN , w =  x T 0 w T 0 . . . w T N − 1  T ∈ R n ( N +1) and v =  v T 0 . . . v T N − 1  T ∈ R pN , and the shift matrix Z =  0 n × nN 0 n × n I nN 0 nN × n  , w e can write the system (1) compactly as x = P 11 w + P 12 u , y = Cx + v , (2) 1 ( M  P ) i,j = M i,j P i,j 3 where P 11 = ( I − ZA ) − 1 and P 12 = ( I − ZA ) − 1 ZB . In this pap er w e consider output-feedback p olicies of the form u = Ky , K ∈ K , (3) where K is a subspace that 1) ensures causalit y of the feedbac k policy b y forcing to 0 those en tries of K corresponding to future outputs, 2) ma y encode arbitrary time-v arying spatio-temp oral sparsit y constrain ts for distributed control as p er [ 20 ], and 3) can imp ose that the control policy is memory- less and time-indep enden t in the sense that K = I N ⊗ K for some K ∈ R m × p . Our goal is to compute K ∈ K that minimizes the exp ected v alue of a quadratic cost in the states and the inputs: J ( K ) := E w , v " N − 1 X t =0  x T t M t x t + u T t R t u t  + x T N M N x N # , (4) where M t  0 and R t  0 for every t . Remark 1 The problem of minimizing (4) is known as the Linear Quadratic Gaussian (LQG) problem. It is w ell-kno wn that a time-in v arian t and memory-less control policy (commonly denoted as static ) of the form u t = K y t ac hieves global optimalit y when N → ∞ and there are no subspace constrain ts to comply with. F or the finite-horizon and/or constrained cases, a time-v arying con trol p olicy with memory (commonly denoted as dynamic ) ac hiev es higher p erformance in general. In this pap er, w e therefore consider dynamic linear p olicies as in (3). F rom (2)-(3) we deriv e the closed-lo op equations: x = ( I − P 12 K C ) − 1 ( P 11 w + P 12 Kv ) , y = C ( I − P 12 K C ) − 1 P 11 w + ( I − CP 12 K ) − 1 v , (5) u = KC ( I − P 12 K C ) − 1 P 11 w + K ( I − CP 12 K ) − 1 v . By defining M = blkdg( M 0 , M 1 , . . . , M N ), R = blkdg( R 0 , . . . R N − 1 ), Σ w = blkdg(Σ 0 , Σ w 0 , . . . , Σ w N − 1 ), Σ v = blkdg(Σ v 0 , . . . , Σ v N − 1 ), µ w =  µ T 0 0 . . . 0  T the cost function (4) can thus be written as J ( K ) =     M 1 2 ( I − P 12 K C ) − 1 P 11 Σ 1 2 w     2 F +     M 1 2 P 12 K ( I − CP 12 K ) − 1 Σ 1 2 v     2 F +     R 1 2 K ( I − CP 12 K ) − 1 CP 11 Σ 1 2 w     2 F +     R 1 2 K ( I − CP 12 K ) − 1 Σ 1 2 v     2 F (6) +    M 1 2 ( I − P 12 K C ) − 1 P 11 µ w    2 2 +    R 1 2 K ( I − CP 12 K ) − 1 CP 11 µ w    2 2 . A deriv ation of J ( K ) as per (6) is rep orted in the App endix. Remark 2 Note that J ( K ) is a multiv ariate p olynomial in the entries of K . Indeed, one can v erify ( I − CP 12 K ) − 1 = N X i =0 ( CP 12 K ) i , due to the fact that each p × p block on the diagonal of CP 12 K is the zero matrix b y construction, and hence ( CP 12 K ) i = 0 pN × pN for ev ery i ≥ N + 1. 4 T o summarize, in this pap er we are interested in solving the follo wing optimization problem P K : Problem P K min K ∈K J ( K ) , whic h might be non-conv ex due to J b eing non-conv ex in K in general. 2.3 Disturbance-feedbac k strategies The classical wa y to deal with the non-con vexit y of J ( K ) is to parametrize the output-feedback p olicy u = Ky in terms of an equiv alen t disturbance-feedback p olicy u = QCP 11 w + Qv [ 20 , 21 ]. Suc h parametrization is akin to the Y oula p ar ametrization [ 3 ]. Similarly to [ 20 , 21 ], w e hav e the follo wing result, whose pro of is rep orted in the App endix. Lemma 1 L et us define function ˜ J : R mN × pN → R as ˜ J ( Q ) =     M 1 2 ( I + P 12 QC ) P 11 Σ 1 2 w     2 F +     M 1 2 P 12 QΣ 1 2 v     2 F +     R 1 2 QCP 11 Σ 1 2 w     2 F (7) +     R 1 2 QΣ 1 2 v     2 F +    R 1 2 QCP 11 µ w    2 2 +    M 1 2 ( I + P 12 QC ) P 11 µ w    2 2 . L et h : R mN × pN → R mN × pN b e the bije ction define d as h ( Q , CP 12 ) = ( I + QCP 12 ) − 1 Q . The fol lowing facts hold. 1. ˜ J ( Q ) is strictly c onvex and quadr atic in Q . 2. ˜ J ( h − 1 ( K , CP 12 )) = J ( K ) for al l K ∈ R mN × pN . 3. ˜ J ( Q ) = J ( h ( Q , CP 12 )) for al l Q ∈ R mN × pN . In other words, the nonlinear c hange of co ordinates induced by h allo ws expressing the non- con vex cost function J ( K ) in (6) as the conv ex function ˜ J ( Q ) in (7). Last, we c haracterize the follo wing prop ert y of J ( K ) to b e exploited in Section 3 and Section 4. The corresp onding pro of is rep orted in the App endix. Lemma 2 L et K 0 ∈ R mN × pN and define the sublevel set of J ( K 0 ) as L := { K | J ( K ) ≤ J ( K 0 ) } . The sublevel set L is b ounde d for any K 0 . 2.4 Quadratic inv ariance Since ˜ J is con v ex and it corresponds to J up to a nonlinear change of coordinates, one may exploit ˜ J for con vex computation of constrained con trollers. In particular, if and only if a prop ert y denoted as Quadratic Inv ariance (QI) holds [ 4 , 5 ], one can solv e a conv ex program in Q that is equiv alen t to P K . F or our finite-horizon setting, it is conv enien t to review the notions of QI and strong QI and recall the corresp onding con vexit y result from [ 20 ]. 5 Definition 1 A subspace K ⊂ R mN × pN is QI with resp ect to CP 12 if and only if KCP 12 K ∈ K , ∀ K ∈ K . and it is str ongly QI with resp ect to CP 12 if and only if K 1 CP 12 K 2 ∈ K , ∀ K 1 , K 2 ∈ K . Note that a general subspace is QI if it is strongly QI, but not vice-v ersa; instead, a sparsit y subspace Sparse( S ) is QI if and only if it is strongly QI [ 4 ]. Now, notice that by Lemma 1 our original problem P K is equiv alen t to min Q ∈ h − 1 ( K , CP 12 ) ˜ J ( Q ) . (8) The QI result in finite-horizon is that problem (8) is con v ex if and only QI holds. W e refer to [ 4 , 5 , 20 ] for details. Theorem 1 (QI) The fol lowing t hr e e statements ar e e quivalent. 1. The set h − 1 ( K , CP 12 ) = − h ( K , CP 12 ) is c onvex. 2. K is QI with r esp e ct to CP 12 . 3. h − 1 ( K , CP 12 ) = K . It follo ws from Theorem 1 that problem P K is equiv alent to a conv ex program, and in particular equiv alen t to min Q ∈K ˜ J ( Q ) , (9) if and only if QI holds. As we ha ve observ ed in Section 1, if the system mo del was unkno wn and w e only had blac k-b o x sim ulation access to the cost function, w e w ould not b e able to optimize within the Q domain due to the mapping h b eing unknown. Moreov er, it would b e highly desirable to step b ey ond the long-standing QI limitation, whic h is inheren t to using conv ex programming in the Q domain. Motiv ated as abov e, the rest of the paper dev elops a first-order gradien t-descent metho d to solv e P K to global optimality directly in the K domain. 3 First-order metho d for globally optimal sparse controllers giv en QI In this section w e fo cus our atten tion on sparsity subspace constraints for the synthesis of distributed con trollers complying with arbitrary information structures [ 20 ]. F or a sparsit y constraint K ∈ Sparse( S ), the set of stationary p oin ts for problem P K is defined as follows: Definition 2 Consider problem P K with K = Sparse( S ). A con troller K ∈ Sparse( S ) is a station- ary p oin t for P K if and only if ∇ J ( K ) ∈ Sparse( S ) ⊥ = Sparse( S c ) , (10) where S c is the binary matrix that has a 0 wherev er S has a 1, and a 1 wherever S has a 0. 6 In general, a stationary p oin t as in (10) could b e a lo cal minim um, a lo cal maximum or a saddle p oin t for P K . In the next lemma, we show that the set of stationary p oin ts for P K corresp onds to that of stationary p oin ts for problem (9) when strong QI holds. The pro of is mainly based on [ 21 , Lemma 1]. W e rep ort it in the App endix for completeness. Lemma 3 Supp ose that the subsp ac e K is str ongly QI with r esp e ct to CP 21 , and let K ∈ K . Also define Q = h − 1 ( K , CP 12 ) . We have that ∇ ˜ J ( Q ) ∈ K ⊥ ⇐ ⇒ ∇ J ( K ) ∈ K ⊥ . Notice that since any QI sparsity subspace is also strongly QI [ 4 ], Lemma 3 holds for all the arbitrary QI information structures characterized in [ 20 ]. 3.1 Global optimality of gradient-descen t By exploiting Lemma 3 our first result establishes that if Sparse( S ) is QI with resp ect to CP 12 , an y stationary p oint of P K is a global optimum. Theorem 2 Supp ose that Sparse( S ) is QI with r esp e ct to CP 12 and let K ? ∈ Sparse( S ) b e a stationary p oint of J ( K ) . Then, K ? ∈ arg min K ∈ Sparse( S ) J ( K ) . Pro of By Theorem 1, P K is equiv alent to (9). Since problem (9) is conv ex, ev ery Q ? ∈ Sparse( S ) suc h that ∇ ˜ J ( Q ? ) ∈ Sparse( S c ) (that is, Q ? is a stationary p oint) is a global optimum and th us ac hieves the optimal cost J ? . Let K ? = h ( Q ? , CP 12 ). Now remember that Sparse( S ) is QI if and only if it is strongly QI [ 4 ]. By Lemma 3 ∇ J ( K ? ) ∈ Sparse( S c ), and hence K ? is a stationary point for J ( K ). Since ˜ J ( Q ) = J ( h ( Q , CP 12 )) for ev ery Q by definition, we ha v e that J ( K ? ) = ˜ J ( Q ? ) = J ? and th us K ? is optimal. By Lemma 3, there can b e no other stationary p oin t K ∈ Sparse( S c ) suc h that J ( K ) = J > J ? ; otherwise, Q = h − 1 ( K , PC 12 ) w ould also b e a stationary p oin t for problem (9) with cost J > J ? , whic h is a con tradiction due to (9) b eing con v ex. Remark 3 Theorem 2 trivially generalizes to any subspace constrain t K that is strongly QI, as the key Lemma 3 holds for any strongly QI subspace. In Theorem 2, we decided to sp ecialize the result to the most common case of sparsity constrain ts in the interest of clarit y . Theorem 2 leads to a fundamen tal insight: under QI sparsity constrain ts, if w e can find an y stationary p oin t of the generally non-conv ex function J ( K ), this p oin t is certified to b e a globally optimal solution to P K . Based on this observ ation, we develop a gradien t-descent metho d that solv es P K to global optimality for QI sparsit y constrain ts. Theorem 3 Supp ose Sparse( S ) is QI with r esp e ct to CP 12 . L et K 0 ∈ Sparse( S ) b e an initial output-fe e db ack c ontr ol p olicy, and c onsider the iter ation K t +1 = K t − η t ∇ J ( K t )  S . (11) Then, K t ∈ Sparse( S ) for every t and ther e exists η t for every t such that lim t →∞ J ( K t ) = J ? , wher e J ? is the optimal value of pr oblem P K . 7 The proof of Theorem 3 uses Lemma 2 and the follo wing four Lemmas. The pro ofs of Lemmas 4, 5 and 6 can b e found in [ 22 , Theorem 3.2], [ 22 , Lemma 3.1] and [ 23 , Prop osition 5.7] resp ectively . W e pro ve Lemma 7 in the App endix. Lemma 4 L et f : R n → R b e b ounde d b elow and c onsider the iter ation x t +1 = x t − η t ∇ f ( x t ) , (12) wher e η t satisfies the Wolfe c onditions: f ( x t − η t ∇ f ( x t )) ≤ f ( x t ) − c 1 η t k∇ f ( x t ) k 2 2 , (13) − ∇ f ( x t − η t ∇ f ( x t )) T ∇ f ( x t ) ≥ − c 2 k∇ f ( x t ) k 2 2 , (14) for some 0 < c 1 < c 2 < 1 and every t . L et f b e c ontinuously differ entiable in an op en set U c ontaining the sublevel set L = { x : f ( x ) ≤ f ( x 0 ) } , wher e x 0 is the starting p oint of the iter ation (12). Assume that ∇ f is Lipschitz c ontinuous on U . Then, lim t →∞ ∇ f ( x t ) = 0 . Lemma 5 Supp ose f : R n → R is c ontinuously differ entiable and b ounde d b elow. Then, for any 0 < c 1 < c 2 < 1 , ther e exist intervals of step lengths satisfying the Wolfe c onditions (13)-(14). Lemma 6 L et f : R n → R b e twic e c ontinuously differ entiable on an op en c onvex set U ⊆ R n , and supp ose that ∇ 2 f is b ounde d on U . Then, ∇ f is Lipschitz c ontinuous on U . Lemma 7 L et f : R n → R b e a c ontinuously differ entiable function and K ⊆ R n b e a subsp ac e. L et f : R r → R b e define d as f ( α ) = f ( M α ) , wher e Im( M ) = K and r is the dimension of K . Then: 1. min x ∈K f ( x ) = min α ∈ R r f ( α ) . 2. ∇ f ( α ) = 0 ⇐ ⇒ ∇ f ( M α ) ∈ K ⊥ . W e are now ready to pro v e Theorem 3. Pro of (Theorem 3) Denote k = vec( K ) and let f : R mpN 2 → R b e the function such that f ( k ) = J ( K ) for every K ∈ R mN × pN . Clearly , if k 0 = v ec( K 0 ) the iterations of (11) are equiv alent to those of k t +1 = k t − η t ∇ f ( k t )  vec( S ) = k t − η t Π Sparse(vec( S )) ( ∇ f ( k t )) . (15) No w let M b e such that its columns are an orthonormal basis of Sparse(vec( S )). Consider the iteration α t +1 = α t − η t ∇ f ( α t ) , (16) where f is suc h that f ( α ) = f ( M α ) for every α . Let k 0 = M α 0 and supp ose that k t = M α t . Then, b y (15), (16) and noting that M T M = I and ∇ f ( α t ) = M T ∇ f ( M α t ): k t +1 = M α t − η t M ( M T M ) − 1 M T ∇ f ( M α t ) = M α t +1 . W e conclude by induction that v ec( K t ) = M α t for every t . Let us c ho ose η t satisfying (13)- (14). Notice that, acc ording to Lemma 5, a choice for η t exists for every t b ecause f is contin uously differen tiable and b ounded b elo w by 0. By Lemma 2 w e obtain that the sublevel set L = { α | f ( α ) ≤ 8 f ( α 0 ) } is b ounded. Consider an open, conv ex and b ounded set U that con tains L . Since f ( α ) is a m ultiv ariate polynomial, every entry of its Hessian matrix ∇ 2 f ( α ) is also a m ultiv ariate polynomial and is thus b ounded on U . By Lemma 6, w e deduce that ∇ f is Lipschitz contin uous. By Lemma 4, lim t →∞ ∇ f ( α t ) = 0 . (17) Let α ? = lim t →∞ α t and K ? ∈ Sparse( S ) the corresp onding output-feedback controller ac- cording to vec( K ? ) = M α ? . Since ∇ f ( α ? ) = 0, by Lemma 7 ∇ J ( K ? ) ∈ Sparse( S c ) and hence ∇ J ( K ? )  S = 0, that is, iteration (11) conv erges to a stationary p oin t of P K . By Theorem 2, K ? is a globally optimal solution for P K b ecause Sparse( S ) is QI with resp ect to CP 12 . A c hoice for η t satisfying the W olfe conditions (13)-(14) can be found b y using, for instance, the bisection algorithm rep orted in [ 23 , Prop osition 5.5], whic h alw ays con v erges in a finite n umber of iterations. W e conclude this section b y providing a n umerical example. 3.2 Numerical example Motiv ated b y the example system of [ 4 ], we consider system (1) and the cost function (6) with A t =    1 . 6 0 0 0 0 0 . 5 1 . 6 0 0 0 2 . 5 2 . 5 − 1 . 4 0 0 − 2 1 − 2 0 . 1 0 0 2 0 − 0 . 5 1 . 1    , and B t = I , C t = I , M t = I , R t = I , Σ w = I , Σ v = I , µ 0 =  1 − 1 2 − 3 3  T . W e set a horizon of N = 3. Our goal is to compute a controller K with a given sparsit y that minimizes the cost (6). Sp ecifically , w e aim to solv e P K with K = Sparse( S ) and S = T ⊗ S , where T i,j = 1 if j ≤ i and T i,j = 0 otherwise, and S =    0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 1    . The total n um b er of scalar decision v ariables is | S | = | S | N ( N +1) 2 = 30. It is easy to verify that Sparse( S ) is QI with resp ect to CP 12 , for example, b y using the binary test [ 20 , Theorem 1]. By direct computation of the Hessian through the Symbolic Math T o olb o x V er. 7.1 a v ailable in MA TLAB [ 24 ] w e v erify that J ( K ) is not con vex on K 2 . Despite this non-con vexit y , we know b y Theorem 3 that the gradient-descen t iteration (11) will conv erge to a global optimum of P K for t → ∞ thanks to the QI prop ert y . 3.2.1 Numerical results The gradien t-descent iteration (11) w as implemented in MA TLAB with the stepsize b eing chosen according to the bisection algorithm of [ 23 , Prop osition 5.5]. The iteration (11) was initialized from a v ariet y of randomly selected initial distributed con trollers. Sp ecifically , for eac h en try ( i, j ) suc h that S i,j = 1 w e selected the entry ( K 0 ) i,j uniformly at random in the in terv al [ − 10 , 10], and set ( K 0 ) i,j = 0 otherwise. In all instances, w e conv erged to a cost of 796 . 5627 within up to 2 sp ecifically we verify ∇ 2 f (0) 6 0, where f ( α ) = f ( M α ), the columns of M are an orthonormal basis of K and f (v ec( K )) = J ( K ) 9 700 iterations, with a run time of appro ximately 2 seconds. The stopping criterion was selected as max |∇ J ( K t )  S | < 5 · 10 − 5 . T o v alidate the global optimalit y result, w e also solved the corresp onding conv ex program (9) in Q with MOSEK [ 25 ], called through MA TLAB via Y ALMIP [ 26 ], and obtained a minimum cost of 796 . 5627. A t this p oin t, it is natural to ask a follo w-up question: is the QI/strong QI prop ert y ne c essary to guaran tee con v ergence of gradien t-descen t to a globally optimal distributed controller? In the follo wing section, we pro vide a negative answ er. 4 Unique Stationarit y: Global Optimalit y Bey ond QI In this section, w e consider general subspace constraints K ∈ K . W e define the notion of unique stationarit y (US) and show that it allows to step b eyond the QI notion in obtaining global optimality certificates with first-order metho ds. W e further pro vide initial results on verifying the US property in a tractable wa y . 4.1 Unique stationarity generalizes QI W e define unique stationarity of problem P K as follo ws. Definition 3 Consider pr oblem P K subje ct to a subsp ac e c onstr aint K ∈ K . We say that P K is Uniquely Stationary (US) if and only if: ∇ J ( K ) ∈ K ⊥ = ⇒ K ∈ arg min K ∈K J ( K ) . (18) First, it is easy to see that the class of US problem is at least as large as that of strongly QI problems. Corollary 1 (Theorem 2) Supp ose that K is str ongly QI . Then, P K is US . Pro of If K is a stationary p oin t of P K then Π K ( ∇ J ( K )) = 0. By Theorem 2 and Remark 3, K is a global optimum. Hence, US as p er (18) holds. Second, w e extend the global con vergence result of Theorem 3 from strongly QI to US problems. Prop osition 1 Supp ose that P K is US . L et K 0 ∈ K and c onsider the iter ation K t +1 = K t − η t Π K ( ∇ J ( K t )) . (19) Then, K t ∈ K for every t and ther e exists η t for every t such that lim t →∞ J ( K t ) = J ? , wher e J ? is the optimal value of pr oblem P K . Pro of The pro of mirrors that of Theorem 3 by selecting M such that its columns are an orthonor- mal basis of K in pro ving that (19) con verges to a stationary point. Since P K is US, ev ery stationary p oin t is optimal. In other w ords, every US problem can b e solved to global optimality with pro jected gradient- descen t. Third, we c haracterize a US problem that is neither strongly QI nor QI. 10 4.1.1 Example US b ey ond QI Consider the system (1) and the cost function (6) with A t =  1 2 − 1 − 3  , B t = I , C t = I , Σ 0 = I , M t = I , R t = I , Σ w t = 0 , Σ v = I , ∀ t = 0 , 1 , 2 , and µ 0 =  0 1  T where w e set a horizon of N = 2. The con troller K is sub ject to b eing in the form K = I N ⊗ K for some K ∈ R 2 × 2 . In other w ords, we consider a static-controller u t = K y t in finite-horizon. Note that in the finite-horizon setup it is not necessary to require that ( A + B K ) is Hurwitz, since the finite-horizon cost J ( K ) is finite for every K , as opp osed to the infinite-horizon cases of [ 15 , 19 ]. Additionally , w e require that K is decentralized, or equiv alen tly K ∈ Sparse( I 2 ). In summary , we enforce K ∈ K = { K = I N ⊗ diag( a, b ) , a, b ∈ R } . By computing K CP 12 K for a generic K ∈ K it is easy to v erify that K is neither strongly QI or QI with resp ect to CP 12 . Hence, a conv ex program equiv alen t to P K in the Q domain do es not exist b y Theorem 1. Nonetheless, we prov e that P K is US and can thus b e solved to global optimality with gradien t-descent. Pr o of of US: F or any K ∈ K we v erify J ( K ) = f ( a, b ) = 4 a 4 + 8 a 3 + 28 a 2 + 18 ab − 38 a + 6 b 4 − 42 b 3 + 149 b 2 − 216 b + 166 . The expression ab o ve can be obtained b y using the Sym b olic Math T o olb o x in MA TLAB [ 24 ]. The Hessian is ∇ 2 f ( a, b ) =  48 a 2 + 48 a + 56 18 18 72 b 2 − 252 b + 298  . W e v erify that 48 a 2 + 48 a + 56 = 12(2 a + 1) 2 + 44 > 0 for all a ∈ R and det  ∇ 2 f ( a, b )  = 24(1 + 2 a ) 2  36 ( b − 1 . 75) 2 + 38 . 75  + 198(7 − 4 b ) 2 + 3086 > 0 , ∀ a, b ∈ R . It follows that ∇ 2 f ( a, b )  0 for all a, b ∈ R , and hence J ( K ) is con vex on K . W e conclude that P K is US, despite not b eing QI. The globally optimal con troller K ? = I N ⊗ diag(0 . 2752 , 1 . 1354) is found on av erage in 11 iterations of (19) with the tw o free v ariables of K 0 ∈ K randomly selected in [ − 10 , 10], stepsize as p er [ 23 , Proposition 5.5] and stopping criterion max | Π K ( ∇ J ( K t )) | < 5 · 10 − 5 . Last, w e summarize the main result of this section as follo ws. A corresp onding visualization is rep orted in Figure 1. Theorem 4 The class of US pr oblems is b oth 1. strictly lar ger than the class of str ongly QI pr oblems, 2. not include d in the class of QI pr oblems. Pro of Every strongly QI problem is US b y Corollary 1. W e hav e shown an instance of a US problem which is neither strongly QI or QI. This prov es that the class of US problems is b oth strictly larger than strongly QI problems and not included in QI problems. 11 An y K K is QI K is strongly QI ( i n c l . S p ar s e ( S )i s Q I ) K is US ? K = Sparse( S )i sQ I AAACJnicbVDLSgNBEJyN7/iKevQyGIR4CbtRMBdB8KJ4UTQmkITQO+nEwdkHM72iLPsffoJf4VVP3kQ8CH6Ku2sOmlinoqqbri43VNKQbX9Yhanpmdm5+YXi4tLyymppbf3KBJEW2BCBCnTLBYNK+tggSQpboUbwXIVN9+Yo85u3qI0M/Eu6D7HrwdCXAymAUqlXqnU8oGsBKj5NDjqEdxRfhKANJpXccQfxRbLzY3Bp+PlJ0iuV7aqdg08SZ0TKbISzXumz0w9E5KFPQoExbccOqRuDJikUJsVOZDAEcQNDbKfUBw9NN85/S/h2ZIACHqLmUvFcxN8bMXjG3HtuOpnlNeNeJv7ntSMa1Lux9MOI0BfZIZIK80NGaJmWhrwvNRJBlhy59LkADUSoJQchUjFKWyymfTjj30+Sq1rV2a3WzvfKh/VRM/Nsk22xCnPYPjtkx+yMNZhgD+yJPbMX69F6td6s95/RgjXa2WB/YH19A6E9ppQ= example AAACAHicbVC7TgJBFJ3FF+ILtbSZSEysyC6aaEliY4mJPBJYyd3hghNmH5m5ayAbGr/CVis7Y+ufWPgv7uIWCp7q5Jz7PF6kpCHb/rQKK6tr6xvFzdLW9s7uXnn/oGXCWAtsilCFuuOBQSUDbJIkhZ1II/iewrY3vsr89gNqI8PglqYRuj6MAjmUAiiV7nqEE0pwAn6kcNYvV+yqPQdfJk5OKixHo1/+6g1CEfsYkFBgTNexI3IT0CRFOq/Uiw1GIMYwwm5KA/DRuMn86hk/iQ1QyCPUXCo+F/F3RwK+MVPfSyt9oHuz6GXif143puGlm8ggigkDkS0iqXC+yAgt0ziQD6RGIsguRy4DLkADEWrJQYhUjNN8SmkezuL3y6RVqzpn1drNeaVey5MpsiN2zE6Zwy5YnV2zBmsywTR7Ys/sxXq0Xq036/2ntGDlPYfsD6yPbzH2l4Q= Figure 1: Problems in the blue region can b e solv ed with conv ex programming (QI problems). Problems in the green region can b e solv ed with gradient-descen t (US problems). The US region includes the case of QI sparsity constraints. The green circle stands for the explicit example w e provided. Differen t metho ds are needed to solv e problems in the red region; how ever, whether the red region contains any problem at all is an op en question. The question mark indicates problems whose existence is y et to b e verified. W e remark that the notion of US genuinely extends QI in terms of pro viding global optimalit y certificates for distributed control. This might sound surprising at first. T o grasp this fact, notice that QI is metho d-sp ecific, in the sense that it is only necessary for global optimalit y certificates when one uses conv ex programming in the Q domain [ 5 ]. On the con trary , w e hav e sho wn that P K migh t b e uniquely stationary and even conv ex in the original K co ordinates despite b eing non-con vex in the Q domain. This observ ation and Corollary 1 allow stepping b eyond the QI limitations, from conv exit y in Q to unique stationarity in K ! 4.2 T ests for unique stationarit y Note that the US prop ert y , while ha ving a theoretical in terest, might not b e useful in practice in the form (18). This is because, in general, one can only pro ve (18) b y knowing the set of global optima. F or this reason, it is necessary to identify sufficient conditions for US. While noting that more general tests should be env isioned in future research, w e pro vide our initial results. A first test of US given sparsit y constrain ts follows naturally from Corollary 2: Corollary 2 Supp ose that K = Sparse( S ) and L et ∆ = Struct( CP 12 ) . Then S∆S ≤ S = ⇒ P K is US . Pro of By [ 20 , Theorem 1], Sparse( S ) is strongly QI with respect to CP 12 if and only if S∆S ≤ S . Hence, P K is US as a consequence of Theorem 2. Notice that S∆S ≤ S is verified in p olynomial time in m , p and N . A second sufficient test for US b ey ond QI is to chec k whether J ( K ) is conv ex on K . Prop osition 2 L et f : R mpN 2 → R b e such that f (vec( K )) = J ( K ) and f : R r → R b e such that f ( α ) = f ( M α ) wher e the c olumns of M ar e an orthonormal b asis of K and r is the dimension of K . L et ∇ 2 g f ( α ) ∈ R g × g denote the submatrix of ∇ 2 f ( α ) obtaine d by r emoving its last r − g r ows and c olumns. Then det  ∇ 2 g f ( α )  > 0 ∀ α , ∀ g = 1 , . . . , r = ⇒ P K is US . Pro of By definition f is con vex if and only if J is conv ex on K . The function f is conv ex if and only ∇ 2 f ( α )  0 for ev ery α , or equiv alen tly , the determinan t of eac h principal minor of ∇ 2 f ( α ) is p ositiv e for every α . 12 Notice that det  ∇ 2 g f ( α )  is a p olynomial for every g . Deciding p ositivit y of m ultiv ariate p olyno- mials is NP-hard in general, but it can b e p erformed in finite time [ 27 ]. When det  ∇ 2 g f ( α )  is a Sum-of-squares (SOS) for ev ery g , as in the example w e provided, then the US prop ert y can be decided in p olynomial time with standard techniques [ 28 ]. 5 Conclusions W e hav e addressed con vergence to a global optimum of first-order metho ds for the distributed discrete-time LQG problem in finite-horizon. If the strong QI property holds, a pro jected gradient- descen t algorithm is guaran teed to con v erge to a global optim um. Moreov er, we ha ve c haracterized the class of uniquely stationary (US) problems, for whic h pro jected gradient-descen t conv erges to a global optimum. W e ha ve pro ved that the class of US problems is strictly larger than strongly QI problems and not included in QI problems. Our results indicate that first-order methods in the K domain are sup erior to conv ex programming in the Q domain in terms of generalit y of their global optimalit y certificates and allo w stepping b ey ond the long-standing QI limitation [ 4 ]. Additionally , first-order metho ds can b e used to learn globally optimal distributed controllers when the system and the cost function are unknown, as w as recen tly sho wn in [ 12 , 14 , 18 ] for the non-distributed case. W e en vision that future work will discuss application of our methods to learning-based distributed con trol. This work initiates the researc h for no v el classes of constrained and distributed con trol problems, for whic h a test of the US prop ert y beyond QI and beyond testing con v exity of P K in the K domain is av ailable. F or instance, one could study under whic h conditions J ( K ) is gr adient dominate d on K [ 29 ]. In the finite-horizon setting considered here, it is important to either confirm or disprov e the existence of non-US problems, indicated by “?” in Figure 1. This insight would further adv ance the comprehension of the mathematical challenges inherent to linear distributed control. Last, it is important to address the infinite-horizon and contin uous-time cases. In infinite-horizon, [ 19 ] provided explicit examples of problems that are non-US due to the set of distributed static stabilizing controllers b eing disconnected; it is in teresting to explore whether dynamic con trollers can mitigate this issue. Ac kno wledgemen ts W e thank Tyler Summers and Iln ura Usmano v a for useful discussions. App endix Deriv ation of the cost function J ( K ) Note that the cost (4) is equiv alent to J ( K ) = E w , v  x T Mx + u T Ru  . (20) No w consider the con trol input u = Ky . The closed-lo op state, output and input tra jectories are giv en in (5), where x and u are expressed as a function of w and v . Substitute (5) in to (20). By using the fact that for any matrix X we ha v e E w  w T Xw  = T race( XΣ w ) + µ T w X µ w , 13 E v  v T Xv  = T race( XΣ v ) , E w , v  w T Xv  = 0 , and remem b ering that k X k 2 F = T race( X T X ) w e obtain the expression (6). Pro of of Lemma 1 1) By using several relationships to compute deriv atives with resp ect to matrices from [ 30 ] and the fact that vec( AX B ) = ( B T ⊗ A )vec( X ) we obtain that ∇  v ec  ∇ ˜ J ( Q )  = 2  ( Σ v + CP 11 Σ w P T 11 C T ) | {z }  0 ⊗ ( R + P T 12 MP 12 ) | {z }  0 + CP 11 µ w µ T w P T 11 C T | {z }  0 ⊗ ( R + P T 12 P 12 ) | {z }  0   0 , b ecause R , Σ v  0 and M , Σ w  0 b y h yp othesis. It follo ws that ˜ J ( Q ) is a quadratic form that is strictly conv ex. The statemen ts 2) and 3) follow from direct computation by exploiting the definition of the function h . Pro of of Lemma 2 Since ˜ J ( Q ) is strictly conv ex by Lemma 1, its sublev el set ˜ L := { Q | ˜ J ( Q ) ≤ J ( K 0 ) } is b ounded for an y K 0 [ 31 , Ch. 9.1.2]. Since ˜ J ( Q ) = J ( h ( Q , CP 12 )) for ev ery Q w e hav e L = h ( ˜ L , CP 12 ). No w notice that h ( Q , CP 12 ) = N X i =0 ( − 1) i ( QCP 12 ) i Q , (21) b ecause eac h m × m blo c k of QCP 12 is the zero matrix b y construction. Hence, ev ery en try of matrix h ( Q , CP 12 ) is a multiv ariate p olynomial in Q , that is, a con tinuous function. W e conclude that L is b ounded if and only if ˜ L is b ounded. Since ˜ L is b ounded for an y K 0 , the result follows. Pro of of Lemma 3 ⇐ ) In the in terest of readabilit y , in this pro of w e omit the second argumen t of the function h ( · , · ), whic h is assumed to alw ays be fixed to CP 12 . Assume that ∇ J ( K ) ∈ K ⊥ , but ∇ ˜ J ( Q ) 6∈ K ⊥ . Then, there exists ˜ Q ∈ K with ˜ Q 6 = 0 with: lim  → 0 ˜ J ( Q +  ˜ Q ) − ˜ J ( Q )  = k 6 = 0 . Equiv alen tly , since h ( · ) is inv ertible, lim  → 0 ˜ J ( h − 1 ( h ( Q +  ˜ Q ))) − ˜ J ( Q )  = lim  → 0 J ( h ( Q +  ˜ Q )) − J ( K )  = k 6 = 0 . No w, using a first-order T a ylor expansion we ha ve h ( Q +  ˜ Q ) = ( I + ( Q +  ˜ Q ) CP 12 ) − 1 ( Q +  ˜ Q ) (22) = ( I + QCP 12 +  ˜ QCP 12 ) − 1 ( Q +  ˜ Q ) = h ( I + QCP 12 ) − 1 −  ( I + QCP 12 ) − 1 ˜ QCP 12 ( I + QCP 12 ) − 1 + O (  2 ) i ( Q +  ˜ Q ) = K +  ˜ K + O (  2 ) , 14 where ˜ K = ( I + QCP 12 ) − 1 ( ˜ Q − ˜ QCP 12 ( I + QCP 12 ) − 1 Q ) . (23) By (21) and by applying the strong QI property we deduce that ˜ K ∈ K . By substituting the ab o ve deriv ations in to (22): lim  → 0 J ( K +  ˜ K + O (  2 )) − J ( K )  = lim  → 0 J ( K +  ˜ K ) − J ( K )  = k 6 = 0 . Since ˜ K ∈ K and is non-null due to ˜ Q 6 = 0, this con tradicts ∇ J ( K ) ∈ K ⊥ . ⇒ ) can b e pro ven analogously . Pro of of Lemma 7 Since Im( M ) = K , minimizing f on R r is equiv alen t to minimizing f on K . Hence, the first p oin t holds by definition of f . F or the second point, we hav e ∇ f ( α ) = M T ∇ f ( M α ) b y the deriv ativ e c hain rule. W e deduce that ∇ f ( α ) = 0 if and only if ∇ f ( M α ) ∈ Ker( M T ) = Im( M ) ⊥ = K ⊥ . References [1] H. S. Witsenhausen, “A coun terexample in sto chastic optim um control,” SIAM Journal on Contr ol , v ol. 6, no. 1, pp. 131–147, 1968. [2] C. H. Papadimitriou and J. Tsitsiklis, “In tractable problems in control theory ,” SIAM j. on c ontr. and opt. , vol. 24, no. 4, pp. 639–654, 1986. [3] D. Y oula, H. Jabr, and J. Bongiorno, “Modern Wiener-Hopf design of optimal con trollers–P art I I: The m ultiv ariable case,” IEEE T r ans. on A ut. Contr. , vol. 21, no. 3, pp. 319–338, 1976. [4] M. Rotko witz and S. Lall, “A ch aracterization of conv ex problems in decen tralized con trol,” IEEE T r ans. on A ut. Contr. , vol. 51, no. 2, pp. 274–286, 2006. [5] L. Lessard and S. Lall, “Quadratic inv ariance is necessary and sufficient for con vexit y ,” in A meric an Contr ol Confer enc e (ACC), 2011 . IEEE, 2011, pp. 5360–5362. [6] L. F urieri, Y. Zheng, A. Papac hristodoulou, and M. Kamgarp our, “Sparsity inv ariance for con vex design of distributed con trollers,” arXiv pr eprint arXiv:1906.06777 , 2019. [7] Y.-S. W ang, N. Matni, and J. C. Do yle, “A system lev el approac h to controller synthesis,” IEEE T r ans. on A ut. Contr. , 2019. [8] G. F azelnia, R. Madani, A. Kalbat, and J. Lav aei, “Con vex relaxation for optimal distributed con trol problems,” IEEE T r ans. on Aut. Contr. , vol. 62, no. 1, pp. 206–221, 2017. [9] Y. W ang, J. A. Lop ez, and M. Sznaier, “Con vex optimization approac hes to information structured decen tralized control,” IEEE T r ans. on Aut. Contr. , 2018. [10] K. Dvijotham, E. T o doro v, and M. F azel, “Conv ex structured con troller design in finite hori- zon,” IEEE T r ans. on Contr. of Netw. Sys. , vol. 2, no. 1, pp. 1–10, 2015. [11] F. Lin, M. F ardad, and M. R. Jov anovic, “Augmented Lagrangian approach to design of structured optimal state feedbac k gains,” IEEE T r ans. on Aut. Contr. , vol. 56, no. 12, pp. 2923–2929, 2011. 15 [12] M. F azel, R. Ge, S. M. Kak ade, and M. Mesbahi, “Global conv ergence of p olicy gradient metho ds for the linear quadratic regulator,” arXiv pr eprint arXiv:1801.05039 , 2018. [13] B. Gra v ell, Y. Guo, and T. Summers, “Sparse optimal con trol of netw orks with multiplicativ e noise via p olicy gradien t,” arXiv pr eprint arXiv:1905.13547 , 2019. [14] B. Grav ell, P . Moha jerin Esfahani, and T. Summers, “Global con v ergence of p olicy gradient metho ds for the linear quadratic regulator,” arXiv pr eprint arXiv:1905.13547 , 2019. [15] J. Bu, A. Mesbahi, M. F azel, and M. Mesbahi, “LQR through the lens of first order methods: Discrete-time case,” arXiv pr eprint arXiv:1907.08921 , 2019. [16] M. Hesameddin, Z. Armin, S. Mahdi, and M. Jo v ano vic, “Global exponential con vergence of gradien t metho ds ov er the noncon v ex landscap e of the linear quadratic regulator,” Confer enc e on De cision and Contr ol (CDC) , 2019. [17] K. M ˚ artensson and A. Ran tzer, “Gradient metho ds for iterativ e distributed con trol synthesis,” in Confer enc e on De cision and Contr ol (CDC) . IEEE, 2009, pp. 549–554. [18] S. Hassan-Moghaddam, M. R. Jo v ano vi´ c, and S. Meyn, “Data-driv en proximal algorithms for the design of structured optimal feedback gains,” in Americ an Contr ol Confer enc e (A CC) . IEEE, 2019, pp. 5846–5850. [19] H. F eng and J. La v aei, “On the exp onen tial n umber of connected comp onen ts for the feasible set of optimal decentralized control problems,” in A meric an Contr ol Confer enc e (A CC) , 2019, p. 8. [20] L. F urieri and M. Kamgarp our, “Unified approach to conv ex robust distributed con trol giv en arbitrary information structures,” IEEE T r ans. on Aut. Contr. , 2019. [21] M. Colombino, R. Smith, and T. Summers, “Mutually quadratically inv ariant information structures in tw o-team sto c hastic dynamic games,” IEEE T r ans. on Aut. Contr. , v ol. 63, no. 7, pp. 2256–2263, 2017. [22] J. No cedal and S. W right, Numeric al optimization . Springer Science & Business Media, 2006. [23] F. J. Arag´ on, M. A. Gob erna, M. A. L´ opez, and M. M. Rodr ´ ıguez, Nonline ar optimization . Springer, 2019. [24] MA TLAB R2016b and Symb olic Math T o olb ox V ersion 7.1 . Natic k, Massach usetts: The MathW orks Inc., 2016. [25] MOSEK Aps, “The MOSEK optimization to olb o x for MA TLAB manual. Version 8.1.” 2017. [26] J. L¨ ofberg, “Y ALMIP : A T o olbox for Modeling and Optimization in MA TLAB,” in In Pr o c. of the CA CSD Conf. , T aip ei, T aiwan, 2004. [27] E. Beck er, V. Po w ers, and T. W ormann, “Deciding p ositivit y of real p olynomials,” Contem- p or ary Mathematics , v ol. 253, pp. 19–24, 2000. [28] P . A. Parrilo, “Structured semidefinite programs and semialgebraic geometry metho ds in ro- bustness and optimization,” Ph.D. dissertation, California Institute of T ec hnology , 2000. 16 [29] B. T. Poly ak, “Gradient metho ds for the minimisation of functionals,” USSR Computational Mathematics and Mathematic al Physics , vol. 3, no. 4, pp. 864–878, 1963. [30] K. B. Petersen, M. S. Pedersen et al. , “The matrix co okb ook,” T e chnic al University of Den- mark , v ol. 7, no. 15, p. 510, 2008. [31] S. Bo yd and L. V anden b erghe, Convex optimization . Cam bridge universit y press, 2004. 17