Joint Cross-layer Optimization in Real-Time Networked Control Systems

1 Joint Cross-layer Optimizati on in Real-T ime Netw orke d Control Sy stems Markus Kl ¨ ugel ∗ , Mohammad H. Mamduhi ‡ , Onur A yan ∗ , Mikhail V ilgelm ∗ , Karl H. Johans son ‡ Sandra Hirche † , W olfgang K ellerer ∗ ∗ Chair of Communication Networks, T echnic a l Univ ersity of Munich † Chair of Information-Oriented Control, T ec h nical Universit y o f Mun ic h ‡ Di vision o f De cision a nd C o ntrol Sy stems, KTH Royal Institute of T e chnology , Stockho lm { markus.kluegel, onur .aya n, mikhail.vilgelm, hirche, wolfgang.kellerer } @tum.de, { mamduhi, kallej } @kth.se Abstract —Network ed control system (NCS) refer to a set of control l oops that are closed over a communication n etwork. In this article, the joint operation of control and network- ing f or NCS is inv estigated wh erein the network serv es th e sensor -to-controller communication links for multiple stochas- tic linear ti me-in var iant (L TI) sub -systems. Th e sen sors sample packets based on th e observed plant state, whi ch th ey send ov er a shared multi-hop network. T he network has limited communication resour ces, which need to be assigned to competing links to support proper control loop operation. In this set-u p, we fo rmulate an opti mization problem to minimize the weighted- sum linear -quadratic-Gaussian (LQG) cost of all loops, taking into account the admissible sampling, control, congestion control and scheduling policies. Under some mild assumptions on the sampling frequencies of the contr ol loops and the communication network, we find the joint optimal solut i on to be giv en by a certainty equivalence control with threshold-based samplin g policy , as well as a back-pressure type schedul er with a simpl e pass-through congestion control. The interface between network and control loops is iden tified to be the buffer state of the sensor node, wh ich can be interpreted as network price for samplin g a packet from control perspective . W e validate our theoretical claims by simulating NCSs comprising of mul tiple L TI stochastic control loops communicating ov er a two -hop cellul ar network. I . I N T RO D U C T I O N Motivation Machine-to -machine (M2M) and internet of th ings (IoT) are en vision ed as driving, re venue-gen erating app lications for the near fu ture of c o mmunic a tion n e tworks. They includ e a wide r ange of applicatio ns in vertical dom a ins, e.g., smart grids, vehicular comm unications and industrial automation . While current n etworks were primarily d esigned to suppo r t high-r a te , h uman-d riven applications such as video strea ming, web-browsing, or file tr ansfer, curr e nt research focuses on wider ra n ge of heterogeneo us requ irements from bo th hum an- and machine-d riven applications. Many M2M application s in volve commu nicating senso rs, actuato rs or in general con- trol loo ps that are closed over a network. Studie s sho w the r esulting con trol perf ormanc e within these app lications is tig htly cou pled with p erform ance of the com munication system. Howev er , the exact r e lationships is non-tr i vial an d not yet fully un derstood . T o efficiently support M 2M applicatio n s, the interp lay be- tween contro l perf ormance and the und erlying co mmunic a tio n This work has been funded by the German Researc h Foundation (DFG) under the grant number 315177489 as part of the SPP 1914 (CPN). system ca pabilities h as to be precisely stud ied. In this line o f work until very recently , two rath e r independe nt perspe c ti ves have been dominant amo ng the control and commun ication societies: while from contro l perspective, th e commu n ication network capab ilities are typically abstracted as max imum rate, delay and packet loss pro p erties, par allel ap proach e s from the commun ication co mmunity abstrac t control app lications by their requ irements on rate, delay and p acket loss. This leads ev entually to a sepa rate design of control and commun ication yet ign o ring their n on-trivial coup ling. It is sho wn in a variety of recent works th at joint design of communication and contr ol systems in networked control system (NCS) provides flexible networking algo rithms and improves con trol per forman ce. A. Con tributions In this work, we in vestigate optimal joint design of net- work and contr ol strategies by minimizin g th e weighted sum linear-quadratic-Gau ssian ( L QG) cost of mu ltiple stoch astic linear time-in variant ( L TI) control loo ps th at share a comm u- nication n etwork. W e tackle the task in a cross-lay e r fashion [1], optimizin g over all po ssible sampling and co ntrol p oli- cies, as well as over congestion co ntrol ( CC) and sch eduling strategies. W e u se a g e n eralized system mo del that allows application to a variety of networks, in particular to wireline ethernet, cellula r, ad -hoc networks and satellite c o mmunica - tion. Th e results can be applied to single- hop or m u lti-hop networks. T o th e best of our knowledge, the combinatio n of multi-loop control with multi-ho p networks is novel; fu rther we are among th e first to consider N CS in an interdisciplinary fashion of th is depth and generalizab ility . W e show ways for interactio n betwe e n network an d contro l b eyond explicit rate and delay constraints, which we see as critical point for en suring the r ight level of compatibility and d ecouplin g among both discipline s. T o limit the delay effect, we focus on rea l-time com munication in wh ich data tran sp ort latency is n egligib le fro m contro l per spectiv e du e to co mmunicatio n sampling period s being much finer than those of con trol loops. The remaining problem still poses challeng es as network resources need to be traded off among the control loops, whereas each loo p need s to adapt to the offered network resources.Ou r major contributions are: • Performan ce optimization of NCS consisting o f stochastic L TI systems over a shared network in cross-layer fashion over all sampling, co n trol, CC and schedulin g policies. 2 • Identification o f buffer status as a natural interface be- tween network and control loops. • Applying non-tr i vial decompo sition meth ods to show that certainty eq uiv a le n ce contr o l, thresho ld-based sampling policy , and back- p ressure schedu lin g achieve optimality . B. R elated W ork The problem of joint communication and control de sign in NCS h as been a n in tensiv e research subje c t in the contr ol commun ity . There exist two general app roaches to consider commun ication in NCSs: Treat given pr otocol and medium as con straints [2] or consider tran sm issions as an additional cost [3] . The form er approach is more commo n as it is aligned with the lay ering p rinciple o f sy stem d e sig n [ 1], however the latter is m o re powerful in term s of join t optimization . From the con trol side, the optimal design is studied in [4], in particular it is shown u n der which c onditions the c ertainty equiv alence co n troller is op timal. Stability and performa n ce of feed back con trol under d elay and p acket drop outs are discussed in [5], [6]. I n [2] an o ptimal control strategy with constrain t on resources and p acket loss is dev eloped, whereas [7] studies op timal control when a joint transmission constraint enfor ces a tr ade-off amo ng networked con trol loops. Authors in [ 8] also pro p osed an LQG optimal delay-de penden t sampling when usag e of co mmunicatio n resources is costly . From the com munication side, cen tralized ap proach es fo r resource allocation in NCSs have been dev eloped in [9]–[11]. The au thors in [9 ], [10] c onsider rate schedu lin g, whereas [11] deal with user scheduling proble m s. A well-known approa ch to user schedu ling, Maximum Error First (MEF) Try-once- Discard, h as b een p resented in [1 2], [13]. The auth ors prop ose to gr eedily schedule the contro l sub- system with th e highest control er ror first. An extension of the MEF app r oach using finite horizon mo del-based prediction is introd uced in [14]. Sev eral works exist fo r design o f decentralized Medium Access Control (MAC) protocols, i.e., CSMA/CA- or ALOHA-based protoco ls [15]–[18], with a p plications to W ireless LAN ( WLAN) systems [19]–[21]. In addition, data link layer for NCS has been studie d for op timal power allocation [22], and modulation an d coding schemes choice [23]. Giv en th e complexity of the joint design of NCS, state-of- the-art typically restricts the scenario to single- hop ne tworks and a specific MA C layer [3], [24]. Other com munication layers or m ultiple hops are n ot con sidered in the optimization , howe ver , perfor mance evaluation case studies with multi-ho p networks are av ailable in th e literatu re, e.g., [25]. In this article we consider a multi-loop NCS supported by a multi-hop network and study the joint optim iz a tio n over the sampling and contro l laws associated with co ntrol, as well as the CC and scheduling laws of the co mmunicatio n network. Our fr amew ork is applicable to both d ecentralized and cen- tralized MAC pr o tocols, includ ing ethern e t, cellular ne twork s, ad-hoc n e tworks and satellite commun ication. While building on existing works, e.g., [7], ou r results are significantly mor e general. Further, we quantify a natur al interface between control and n etworking, that are gi ven a prio ri in related w orks. C i E i P i S i r i [ τ ] Y i [ τ ] N 1 N 2 N 3 N 4 u i k x i k ˆ x i k δ i k Z i Q , A s i t i Fig. 1. Schematic of the problem setup with one control loop and a four- node multi -hop netwo rk. The decisio n variab les include sampling, congest ion control , tran smission control and feedback control. C. Outlin e In the rem ainder of this article, NCS model an d op timization preliminar ie s are presented in Sec tio n II. Problem statem ent and solution are explained in Section II I. Simulation results are shown in Section IV and th e article is co ncluded in Section V. I I . S Y S T E M M O D E L & P R E L I M I NA R I E S Consider a mu ltip le-loop NCS as schematically shown in Fig. 1. It con sists of L control loops, with th e set of all loop s denoted by L = { 1 , . . . , L } , where sensor-to-controller links are closed over a multi-h o p com munication n etwork. A. Con tr ol Model Loop i ∈ L consists of a physical process P i , sensor S i and a contr ol un it includ ing an estimato r E i and a feedback controller C i . All processes follow L T I dyn amics disturb ed by exogenou s stochastic inp u ts. Th e p rocess P i is described by x i k +1 = A i x i k + B i u i k + w i k , (1) where x i k ∈ R n i is the system state of loop i at time-step k , u i k ∈ R m i is the contro l input, an d A i ∈ R n i × n i , B i ∈ R n i × m i represent the system an d control matr ic e s, respe c ti vely . The exogenou s distur bance w i k ∈ R n i takes rand om values at each time k accordin g to a z ero-mean Gaussian distribution with covariance Z i and is assumed to b e an i.i.d . process f or all i ∈ L and k ∈ { 0 , 1 , . . . } . W e, m oreover , a ssum e that the in itial values x i 0 ’ s, i ∈ L , are i.i.d. f rom distributions with symmetric density functio ns ar ound their respective means E [ x i 0 ] . The disturbanc e s w i k ’ s are presum ed to be indep endent f rom the initial states x i 0 ’ s, for all i and all k . W e assume that all contro l loops i ∈ L e volve in discr ete time with sampling periods T i , i.e., time - step k refers to th e tim e instant k · T i for the i th loop. Quality of contro l for the i th control loo p is measu red in infinite time-hor izon, b y the following LQG cost function J i : J i = lim s up K →∞ 1 K E ( K − 1 X k =0 ( x i k ) T Q i x x i k + ( u i k ) T Q i u u i k ) , (2) where Q i x and Q i u are positive semi-d efinite and po siti ve definite matrices of appro priate dim ensions, r espectiv ely . The sen so r-controller link is closed over the commu nication network. The co ntrol un it of each sub-system i ∈ L includ es a 3 controller C i that gen erates, at a time k , the contro l sign al u i k , and an estimator E i that computes state estimation ˆ x i k in case x i k is no t accessible. W e assume that sensor s measure p erfect copies o f th eir corresp onding sub-system’ s state inf ormation . The control input u i k is generated accordin g to a con trol law ξ i = { ξ i k } , described by cau sal map pings ξ i k from th e i th - loop o b servation history at time k to the respe c ti ve con trol input. W e a llow th e contr ol law to depen d on the com plete observation history o f the recei ved in formatio n at the control side. In addition, we do not restrict the control in puts to remain constant in between successful tran smissions. Th e com b ination of loca l con trol laws is deno ted by ξ = { ξ 1 , . . . , ξ L } ∈ Ξ , where Ξ d e notes the set of all admissible con trol laws. Each sensor has a sampler attached to d ecide when to transmit the curren t state information to the con troller . The sampling decision is d enoted by δ i k ∈ { 0 , 1 } , where δ i k = 1 indicates that x i k is transmitted, and δ i k = 0 ind ic a te s otherw ise. The tran smission indu ces a network packet of r i k informa tio n units 1 , which is forwarded to the n e twork fo r transpo rt to the controller 2 . The sampling decision is th e o u tcome of a sampling law ϕ i , which m aps th e histor y o f state observa- tions into δ i k . Samp ling laws are all aggregated in a vector ϕ = { ϕ 1 , . . . , ϕ L } ∈ Φ , and Φ is the set of a ll adm issible laws. B. Network Model The network is a set of nodes N = { 1 , 2 , . . . } , each representin g a tran smitting or receiving device. Each sensor is attached to a sou rce node s i ∈ N and each contr oller to a target nod e t i ∈ N . I n general, there might be loop s that share a sou rce or target nod e, such th at we define the set U n = { i ∈ L : s i = n } o f loops that have a specific n ode n as their sou rce n ode. Furth er , there might be nodes that are neith e r source, nor target but can forward data . Each loop is associated with a dedicated p a th Z i = { ( s i , n i 1 ) , ..., ( n i l − 1 , t i ) } , i.e . , a sequence of links ( n i d − 1 , n i d ) ∈ N × N connecting the source an d target nodes, over which its correspon d ing data is transported . The pa th is kept fixed and determ ined a priori, therefor e routin g is not part of the given problem. The node s are co upled by a link state m atrix Q ∈ Q , w h ere each element Q mn denotes the link state for transmission s from node m to n and Q is the set of states that Q may assume. For example , in a wireless commun ication system with single antennas Q mn ∈ [0 , 1 ] could be th e attenuation coefficient between m and n , wh ile Q would be the chann el matrix ta ken from the set Q = [0 , 1] |N | of possible chann el matrices. Each node has a set of av ailable actions to ch oose to tran sm it da ta. The combined action s of all nodes are den oted by the matrix A ∈ A , wh ere elemen t A mn denotes the a c tion of link ( m, n ) and A is the set o f all valid matrices. Th e com bination of an action an d lin k state leads to the data transmission. The maximum amount o f data that c an be tra nsmitted for a defined action is descr ibed by the r ate fun ction R : Q × A 7→ R |N |×|N | + , which is a map ping from th e link and action sp aces Q × A to the node-by-n ode tr ansmission rate R mn . Each elemen t R mn indicates the amount of data that is transmitted from node m 1 Information units can, e.g., be bits, Bytes or service data units (SDUs). 2 W e generally assume time-vary ing r i k may differ among loops. In many cases we can make the v alid assumption that r i k = r i ∀ k , or r i k = r ∀ i, k . to nod e n , expressed in appr opriate dimension of in formatio n units p er slot. W e assume n o packet losses occur . W e discuss this in mor e d etail in Section II-C. Assume th at the network ope r ates in a time-slotted fashion with slot τ of width T τ , i.e., τ ref ers to the time inter val (( τ − 1) T τ , τ T τ ] . W e m a ke th e assumption that T τ ≪ T i , ∀ i , i.e., the network opera tes significan tly faster than the control loop s. By this assum ption, data tran sportation can be considered to be delay-fr ee fro m the con trol loo p per sp ectiv e, although in re ality it m ight req uire m u ltiple com m unication slots due to q u euing or retransmission s. Again, we discuss the impact and necessity of the real-time assum ption in Section II-C. W e assume that th e link state Q chang es fro m slot to slot acc o rding to a stationary p r ocess but remain s constant in state Q [ τ ] at slot τ . Furth er , in each slot a single action choice A c an be made, which is denoted b y A [ τ ] . The combinatio n of action an d link state leads to an amount o f R [ τ ] = R ( Q [ τ ] , A [ τ ]) information units b eing transmitted among th e nod es. W e assume the max imum ach iev ab le rate is finite for any link, i.e., max A , Q k R ( Q [ τ ] , A [ τ ]) k ∞ < ∞ . Each node is assumed to hav e an infinite length transmission buf fer whose traffic passes th rough the nod e. The b uffer is used to store data originatin g from the corre sponding sensor for relaying. W e denote the buffer back -log a t slot τ , the set of all back-log s for loop i , and all b ack-logs fo r a ll loo ps b y B i n [ τ ] , B i [ τ ] = { B i n [ τ ] ∀ n ∈ N } , and B [ τ ] = { B i [ τ ] ∀ i ∈ L} , respectively . A sou rce node s i is assumed to have an infinitely large CC buffer with the back-lo g deno ted by Y i [ τ ] . Th is buf fer stores data outpu t of the sam pler until it is pushed into the transmission buf fer . T e chnically , the CC buffer re sides in the transport layer and corr esponds to, e.g., th e in p ut buffer to a T ransmission Con trol Protocol (TCP) socket, whereas the transmission buffer is in the MA C layer at th e o utgoing network interface chip. T ransferrin g data from the CC buffer to tr ansmission b uffer is done based on a CC mechanism. Remark 1: T h e describ ed network m odel has been devel- oped in [ 26] for cro ss-layer optimization. By ad justing the link state and action sets, a variety of co mmunicatio n system models can be realized within the sketched mode l in cluding wireline networks [26, Ex. 2.1], wireless network with chann el variation and power co ntrol [26, Ex . 2 . 5], ad- hoc network s [26, Ex. 2. 6], satellite downlinks [26, Ex. 2.4], and cellular networks with adap tive modulation and coding [27, Ch. 2.2. 3]. Define K i [ τ ] = { k : ( τ − 1) T τ < k T i ≤ τ T τ } as the set of control time-steps o f loop i that fall into a n etwork slot. Then , the amou nt o f input data r i [ τ ] arriving in to th e CC buf fer of source node s i in slot τ can be expressed as r i [ τ ] = X k ∈K i [ τ ] δ i k r i k . (3) Note that r i [ τ ] can be interpreted as the arrival process to the CC buf f er of node s i , the exact statistics of which d epend on the sampling mech anism ϕ i and the differences in time scales of the con trol and communication systems. Having th e assumption that T τ is much shorter that T i , r i [ τ ] = 0 hold s in most of the slots. In general, what we essen tially req uire is that r i = E τ  r i [ τ ]  remains finite. This is not restrictiv e as T i and the samp ling rates o f all con trol loops ar e finite. Th erefore, 4 T i T τ is non-z e ro and finite. By the Lind ley’ s recursion [28, Ch. 1], the e volution of the CC buffer back-log becomes Y i [ τ ] =  Y i [ τ − 1] + r i [ τ ] − µ i s i [ τ ]  + , (4) where µ i s i [ τ ] is the admission d ecision at n o de s i , i.e., th e data amount p u shed f rom i ’ s CC buffer into no de s i ’ s transmission queue in slot τ , with the notation [ · ] + , max {· , 0 } . The admission can be inter p reted as service process to the CC buf fer, with expected service rate µ i s i = E τ  µ i s i [ τ ]  . At the same time , th e served amoun t of data acts as a r riv al to the MAC la y er tran smission buffer . De fin e the amount o f data originatin g from lo op i , tha t has b een tran smitted over link ( m, n ) in slot τ , by R i mn [ τ ] . Then the following must hold: X i ∈L R i mn [ τ ] ≤ R mn [ τ ] ∀ ( m, n ) , τ . (5) Further, for n otational consistency , define arriv al processes µ i n [ τ ] for any com b ination of i , n , such that µ i n [ τ ] = 0 ∀ τ if n 6 = s i . T hen, th e back - log of the tran smission buffer f or loo p i on node n , i.e. B i n [ τ ] , ev olves in time according to 3 : B i n [ τ ] = " B i n [ τ − 1] + X m ∈N R i mn [ τ ] + µ i n [ τ ] − X o ∈N R i no [ τ ] # + . (6) Due to th e assum ed ro uting on a single path, a node is eith er a source, in wh ic h case the inco ming rates R i mn [ τ ] m u st b e zero in all slo ts, or o therwise µ i n [ τ ] = 0 , ∀ τ . W e further assume that B t i [ τ ] = 0 ∀ τ , as all data will be for warded towards the upper lay ers, i.e., to the con troller, without consider able delay . Giv en th is model, in each slot the actions A [ τ ] are chosen accordin g to a schedulin g law π ∈ Π , where π is a mapp ing from the history of link states H Q [ τ ] = { ..., Q [ τ − 1] , Q [ τ ] } and q u eue ba ck-logs H B [ τ ] = { ..., B [ τ − 1 ] , B [ τ ] } to an ac- tion out of the set A , and Π is th e set of all possible s chedulin g laws. Similarly , the admissions µ i n [ τ ] are chosen a c c ording to a CC law ψ i , used b y loop i . All emp loyed CC laws are gathered in the vector ψ ∈ Ψ , which is o ne out o f a set of p ossible laws Ψ . Similar to π , e ach ψ i is a mappin g from the history of CC buf fer back-logs H Y i [ τ ] = { ..., Y i [ τ − 1] , Y i [ τ ] } to an amount of admitted data. Both π an d ψ c a n , but do not hav e to, incorpo rate the ef fect of repo r ting de lays by depend ing o n out-dated b u ffer status and the effect of estimation inacc uracies by including ran domn e ss into the decisions. Further, π can be realized in a cen tralized m a nner or distributi vely at the nodes. C. Model J ustifica tion Sev eral simp lifying assumptions ar e made in the mo del, that requ ire further justification. Fir st, all tran smissions are assumed to be error-free. As has be e n co mpreh e n si vely discussed in [26, Ch. 2.4 .3], this simplification does not se verely limit the results as lon g as lost data is re - injected into the network by an error-recovery protoco l, such as automatic repeat req uest ( ARQ). Re-injection then can be modeled by an equivalent rate reduction of the chan nel, which falls into the sketched model. Second, we assume that end-to-en d d ata transport is fi- nalized within a single control step T i for all lo ops. The 3 For tracta bility , we assume that data arri vals in a time slot happen just after transmissions, i.e., recei ved data cannot be transmitted in the same slot. reason for this assum p tion is that delays can create a fe e dback effect that is no t yet fully u n derstood . In fact, in c r eased delays can lead to more requ e sts for data transmission b y the contro l loops, which in turn can incre a se th e delay ev en further due to qu e uing effects. Nev ertheless we expec t that our results transfer to the d e lay-affected case with some add itio nal modification s an d find a reasonab le p erform a nce verified even for the de la y -affected case in o ur simu latio n results. The targeted problem rem ains challeng in g e ven with th ese simplifications. The main challenge arises from the question which control loop s sho uld transmit while ne twork can only serve D < L tr ansmissions per time. Further , what if network has co mplex physical and MAC layers, ran domly varying channels and relays data in multi-ho p fashion, such that D is unk nown or time-depend ent. Other challenges are when control loop s are no t synchronou s an d hav e partial informatio n about e ach other , or the network is not control-aware. D. Bu ffer Stability , Capacity and T ranspo rt Capacity Define R i mn := E τ { R i mn [ τ ] } as the time - av erage of R i mn [ τ ] . W e write similarly for r i and µ i n . Let the buffers in the network be modeled as q ueuing system s with the inpu t data as arriv al p r ocess and th e outg o ing d ata as service pro cess. Fr o m queuing theo r y , we take the definition of q u eue stability [2 6], [29], for a qu eue with back- log B i n [ τ ] , whic h is defined by lim sup N →∞ 1 N E  X N τ =1 B i n [ τ ]  < ∞ . (7) If th e expectatio n s of the ar riv al an d d eparture pr o cesses are r i and µ i s i , resp e cti vely , then und er some loose admissibility assumptions [26, Def. ’ s 3 .4-3.5 ], Y i [ τ ] is stable iff r i < µ i s i [26, Lemma 3.6]. This n otion can be extended to a network , which we call stable if all network q ueues are stable [26, Def. 3.2]. A n e twork is stable in the queu ing sense iff [29] µ i n + X m ∈N R i mn < X o ∈N R i no ∀ i, n. (8) If a network is un stab le th en a nod e with a bo ttlen eck link exists. Assume that we n eed to operate the network to meet av erage p er-link rate targets R mn , ∀ m, n . Then we ca n d efine the feasible region for all targets, i.e., the Network Capacity C , [30]. Let the actio ns A [ τ ] in each slot be chosen acc o rding to a sched uling law π ∈ Π . Then from [26], C is defined as C = { R : R mn ≥ 0 , R nn = 0 ∀ m, n ; ∃ π ∈ Π : R mn ≤ E τ { R mn ( Q [ τ ] , A [ τ ]) } ∀ m, n } , and is the set of expected per-link ra tes that can be provided by at least on e scheduling law . The first row ensures that all rates are non-negative an d there is no self- commun ication. An importan t prop erty is that C is a con vex set [26]. Here we take a transpor t-layer pe rspective, in which the exact ro utes and MAC layer p rocedur es are no t of interest. Giv en that all p aths Z i are fixed, we exten d the MAC layer network ca pacity to wards a tran sport-laye r capacity; the T rans- port Capa city Λ . Define th e set o f valid per-loop link rates R i as R i =  R i : R i mn ≥ 0; R i mn = 0 ∀ ( m, n ) / ∈ Z i  . 5 Considering that µ i n is an expected end-to- end data rate, the transport capacity is defined as in [26]: Λ =  µ ≥ 0 : µ i n = 0 ∀ n 6 = s i , (9a) ∃ R ∈ C an d R i ∈ R i , (9b) s.t. X i ∈L R i mn ≤ R mn ∀ ( m, n ) (9c) µ i n + X m ∈N R i mn ≤ X o ∈N R i no o , (9d) where 0 is a zero vector of compatib le d imension a n d ≥ de- notes elementwise comparison. V ector µ contains the served, expected end-to -end rate for each comm unication flow i . T h e constraint (9a) en sures tha t r a te s are on ly pr ovided to flows that originate at th e respective node, while (9b) d emands that all MA C layer a verage rates are chosen from the netw ork capacity and h e nce achiev able by a scheduling law , as well as th a t the per-link rate assignments com ply with the ch osen routes. The third con straint (9c) ensures th a t the loop rates co mply with the link rates, an d (9 d) ensures network stability 4 . No te that C and R i are conve x , and so are th e co nstraints ( 9 a)–(9d), h ence Λ is con vex as it is form ed by the intersection of conve x sets. Giv en th e transport ca p acity , we can demand that all loo p rates r i should be ser ved by the n e twork b y simp ly ensuring r i ≤ µ i s i ∀ i ; µ ∈ Λ . (10) This for mulation ab stracts away the technical complexity of including the u n derlying n etwork and MA C layers explicitly . I I I . P RO B L E M S TA T E M E N T & A S S E S S M E N T W e now co n struct o ur target o ptimization problem, so-called global o ptimization pr oblem (GOP). The go al is to ma x imize control p erforma nce in f orm of a weigh ted cost making use of both c o ntrol an d network paramete r s. It is formulated as GOP: min ϕ,ξ ,ψ ,π X i ∈L w i J i , s.t. ϕ ∈ Φ , ξ ∈ Ξ , ψ ∈ Ψ , π ∈ Π . ( 11) GOP m inimizes the weighted sum o f local LQG co sts, over all possible samp ling, control, CC and scheduling laws . A. De c omposition of Contr ol and Networking The GOP considers joint optimization over network in g and control parameter s. As this constitutes an impr actical so lution from the global per spectiv e, we aim at finding w ays to separate different aspects from o ne another . T o do so, we interpret J i = J i ( ϕ, ξ , ψ , π ) as set-fun ction of the co nsidered policies. Then , we can u se the gen e ral eq uality [3 1] min ϕ,ξ ,ψ ,π X i ∈L w i J i = min ψ ,π v ( ψ , π ) , (12) where v ( ψ , π ) = inf Φ × Ξ X i ∈L w i J i ( ϕ, ξ , ψ , π ) . (13 ) That is, f or fixed CC mechanism ψ and sched u ling law π , the cost function s ar e o ptimized over the possible sampling and control laws ϕ and ξ , re sp ectiv ely . The mapp ing o f { ψ , π } to its minimum weighted su m cost, i. e., gi ven th e optimal ϕ and 4 Networ k stabilit y condition is formally introdu ced in (8). Comparing it w ith the constraint (9d), “ ≤ ” is an approximation of “ < ” to ease the de vel opment. T echnicall y , we w ould ha ve to ad d an a rbitrari ly small ε > 0 to the left hand side and take the limit ε → 0 , which leads to the same results. ξ , can be interpreted as set-fu nction v ( ψ , π ) over Ψ × Π , and the network par ameters are optimized in the o uter loo p. The problem (12) is denoted the master pr oblem , and (13) the p rimal pr ob lem . Note that technically , we would h av e to restrict { ψ , π } to a set that render s the prim al problem feasible. Howe ver, as it is an unco nstrained problem , the feasible sets are simply the intro duced sets Ψ and Π , respe cti vely . W e can b ring this abstract f o rmulation into a m ore technic a l form that re sides on transport layer . For this, we leverage results that have been derived in [7], wh ere the pro blem min ϕ,ξ X i ∈L J i s.t. X i ∈L r i ≤ c (14) is co nsidered. It is argued that the average r a te r i ( ϕ, ξ ) and cost J i ( ϕ, ξ ) depend o n the c h osen p a ir { ϕ , ξ } . Hen ce th e region of all feasible tu ples ( J i , r i ) can be con structed as J i =  ( J i ∗ , r i ∗ ) : ∃ ϕ, ξ s.t. J i ( ϕ, ξ ) = J i ∗ ; r i ( ϕ, ξ ) = r i ∗  . Note tha t J i is shown to be conv ex [ 7]. Then , the Pareto curve J i ( r i ) = inf { J i ∗ : ( J i ∗ , r i ) ∈ J i } (15 ) represents a con vex func tio n taking the rate as inp ut to d eter- mine the optimal cost. Th e curve J i ( r i ) th en corr esponds to the cost unde r a n optimal choice ϕ , ξ such that r i ( ϕ, ξ ) = r i ∗ . Adopting this k nowledge, we can fur th er re-state the master problem by referring to the fea sible region of end-to -end rates that can be served by a combinatio n ( ψ , π ) of CC and scheduling . Th is region is exactly the transport capac ity Λ . Then, the m aster p roblem simply red uces to min µ , r X i ∈L w i J i ( r i ) s.t. r i ≤ µ i s i ∀ i ; µ ∈ Λ . (16) Note that, technically , in problem (16), we do not directly o pti- mize over the networking and contr ol policies, but rather over a set of feasible end-to -end rate s and achievable cost values for which suc h policies exist. Howe ver, we c a n d e duce the optimal policies f r om the argum e nt minimizing the pro blem. B. Op timal Co ntr ol Due to the similarity o f ( 14) a nd (16), the results o f [7] are directly transferable to solve the primal p roblem. As effect of the sum-stru cture in th e objective, for given fixed µ i s i , the primal p roblem can be decomposed in to per-loop pr oblems as min ϕ i ,ξ i w i J i ( r i ) s.t. r i ≤ µ i s i . (17) Inspired by [7], we stud y the structural prop erties o f th e optimal control an d event-triggered samplin g policy . In fact, (17) can b e solved b y scalarization appro ach u sing a fixed La- grange multiplier λ i ≥ 0 , yielding the un constrained prob lem min ϕ i ,ξ i w i J i ( r i ) + λ i  r i − µ i s i  ˆ = min ϕ i ,ξ i w i J i ( r i ) + λ i r i , where λ i is referr ed to as “co mmunica tio n price”. For fix ed λ i , the op timal con trol policy is derived as a certainty equiv alence controller in combinatio n with a model-b ased estimator . For time-varying λ i , an adap ta tio n mod e l is p r oposed in [7] tha t propo ses an adaptiv e pricing model by use of a g radient ascent on the dual problem . The results of this article are extend able to th e adaptive p ricing f ollowing the ideas o f [ 7]. 6 It is observed fro m the relaxed problem (16) that the optimal policies ξ i and ϕ i are solely ch aracterized by the tran smission rate r i and th e cost J i of sub -system i . Hence, we will searc h for the feasible region of the pairs ( J i , r i ) with respe c t to the dominatin g class of co ntrol and samp ling strategies to form the Pareto op timal policies. Having the class o f dominatin g strategies fo und, we can th en confine our search within this class of narr owed d own adm issible stra tegies to solve the optimization problem (16) withou t loo sing op timality . It is discussed in [4] tha t any p a ir o f policies ( ξ i , ϕ i ) with the certainty equiv alence contro ller is dominating . Hence, the optimal control p olicy can be expressed as the causal map ping of the observation history stor ed at the controller side, i.e., u i k = ξ i, ∗ k ( Z i k ) = − K i ∗ E [ x i k | Z i k ] , (18) where Z i k denotes the observation history at the co ntrol side from the initial time un til tim e k , an d E [ x i k | Z i k ] den o tes the optimal state estimation at time k at con tr ol side gi ven Z i k . The optimal control g ain K i ∗ can then be compu ted as follows K i ∗ = ( B i T P i B i + Q i u ) − 1 B i T P i A i , (19) where P i solves the succeedin g algeb raic Riccati equation P i = Q i x + A i ⊤  P i − P i B i ( Q i u + B i ⊤ P i B i ) − 1 B i ⊤ P i  A i . W e denote the binary variable γ i k as the delivery ind ic a tor , i.e., if δ i k = 1 , th e n γ i k = 1 indicates th at x i k is successfully delivered at tim e -step k , and oth erwise if γ i k = 0 . Clearly , γ i k = 0 if δ i k = 0 . T he optima l state estimation , given the informa tio n set Z i k and E [ x i 0 ] , can th e n b e expressed as E [ x i k | Z i k ] = ( x i k , δ i k γ i k = 1 , E [ x i k | Z i k − 1 ] , otherwise , (20) where, from ( 1) and (18), we ha ve E [ x i k | Z i k − 1 ] = E [ A i x i k − 1 − B i K i ∗ E [ x i k − 1 | Z i k − 1 ] + w i k | Z i k − 1 ] = ( A i − B i K i ∗ ) E [ x i k − 1 | Z i k − 1 ] . (21) The variable γ i k in (2 0) represen ts th e effects of n e twork actions o n transmitted d ata packets. Assumin g that the com mu- nication n etwork has much fin e r slotted periods comp ared to the control sampling periods, and also assum ing tha t there are no (net) packet losses, it h olds that γ i k = δ i k , i.e., any packet is delivered within the correspond ing contr ol sampling slot if it is transmitted, and hence δ i k γ i k = δ i k in ( 20). In fact, we assume that the tim e d uration required for the entire d ata transportation including contention r esolution, queuing, r etransmissions, a n d node allo cation, is negligible f rom the control pe rspective. This assumption remains valid for a wide range of NCS applications w h erein the dynamics of the processes are not super fast, e.g. sma r t homes, traffic control, electric power flow , d istrict heating systems and ty pical chem ic a l pr o cess control. In these ap plications, the requ ired sampling ra tes of the dyn amical pr o cesses are typ ic a lly 1 0 Hz or lower , while today’ s com m unication tech nolog y can guaran tee muc h faster data handling and transmission with negligible de la y ( < 1 ms). This ide alization leads to expressing the estimation process as in (20) where δ i k γ i k is b e in g sub stituted by δ i k . W e define e i k | k − 1 = x i k − E [ x i k | Z i k − 1 ] as the estimation err or computed at the sampling unit of sub-system i at time-step k given the informa tio n Z i k − 1 . No te that e i k | k − 1 is com puted befo re the sampler decides on δ i k . Therefore, using (1), (20) and δ i k γ i k = δ i k and kn owing that δ i k = 0 results in E [ x i k | Z i k ] = E [ x i k | Z i k − 1 ] , the dynamics of the one-step ahead estimation error becomes e i k +1 | k =  1 − δ i k  A i e i k | k − 1 + w i k . (22) Remark 2: T o derive th e optimal sampling po licy we use the results presented in [7], [32] and assume that for the event- based remote estimation the optimal e vent-trigger ed samplin g law is symme tr ic, if: 1) the noise distribution is zero-mean , unimod al and symmetr ic , and, 2 ) distribution of x i 0 is sym- metric ar ound E [ x i 0 ] , ∀ i . Despite some efforts in, e.g. [ 3 3], [34 ] that hint the o ptimality of symmetric event-triggered samp lin g laws for h ig her dimensions, this result is for m ally p roved only for first-order L TI systems [35]. However , it is discussed in [36] that even if the optimal event-triggered sampling law is not symmetric f or higher or der systems, then an extra b ia s term will be ad d ed to the optim al estimator wh ich has n o compro mising effect on deriv ation of our following results. Employing (18), ( 20) and δ i k γ i k = δ i k , the local L QG cost function J i in (2) can be equ i valently expressed as follows: J i = T r  P i Z i  +lim sup K →∞ 1 K E " K − 1 X k =0  1 − δ i k  e i T k | k − 1 Q i e e i k | k − 1 # where Q i e = K i ⊤ ∗  Q i u + B i ⊤ P i B i  K i ∗ , and Z i is the covari- ance m atrix of the correspond in g noise process with realization w i k . Th e o ptimal samp ling law ϕ i, ∗ k can then be compu ted as ϕ i, ∗ k ( e i k | k − 1 ) = arg min ϕ i ∈ Φ i J i ( e i k | k − 1 ) , (23) where Φ i denotes the class of all adm issible local sam pling policies. Accordin g to the discu ssions in [7], (23) can be solved via scalarization method [37] u sin g a fixed L agrange multiplier λ i ≥ 0 , yielding an unco nstrained problem . He nce, the P areto curve of the pair ( J i , r i ) can be char acterized as min ϕ i, ∗ k { J i + λ i r i } . (24) T o solve the prob lem (24), we need to charac terize the triggering law φ i k . Fro m (1), (1 8) and (22) one ca n simply derive the closed-lo op dy namics o f a sub -system i as follows: x i k +1 = ( A i − B i K i ∗ ) x i k + (1 − δ i k ) B i K i ∗ e i k | k − 1 + w i k . (25) According to (25), h aving the stabilizing gain u i k , th e process state K i ∗ is stable a n d contro lled by the estimatio n error state e i k | k − 1 . Hence, to d e sign the triggerin g law , we confine our search to error-depen dent po licies. T o introd uce the class of admissible triggering laws Φ i , assume an arbitrary variable M i ( λ i ) ∈ R + , such that ϕ i, ∗ k ( e i k | k − 1 ) = 0 for k e i k | k − 1 k 2 ≤ M i ( λ i ) , and ϕ i, ∗ k ( e i k | k − 1 ) = 1 for k e i k | k − 1 k 2 > M i ( λ i ) . Since M i ( λ i ) can be selected f reely , i.e. very small o r very large, it does n ot impo se any ad ditional constrain t o n the class of admissible scheduling policies. Unde r this assum ption, the problem (24) can be solved by duality usin g value iter ation [37], [38]. Mo reover , acco rding to [7], the optimal ev ent- triggered sam pling law ϕ i, ∗ k ( e i k | k − 1 ) : R + ∪ { 0 } 7→ { 0 , 1 } is a 7 stationary map ping fr om the estimation e r ror to a transmission decision. Finally , the optimal sampling of s ub-system i at time- step k can be expressed as δ i k = 1 {k e i k | k − 1 k 2 > M i ( λ i ) } . C. Dual Decomposition W e now target network optimization using the formulation in (16). The employed approach is a du al d e composition with on-line adaptation of the Lag range mu ltip liers. Remin d that both J i ( r i ) and Λ ar e conve x, h ence so is the pro blem (16). Further, Slaters’ con straint q ualification [31, Ch. 5.2 .3] is guaran tee d to h old fo r any gi ven positiv e rate. This re sults in zero duality gap an d stro ng duality then ho lds. By explicitly formu latin g the constraints tha t define Λ , the GOP becomes: min µ , r P i ∈L w i J i ( r i ) (26a) s.t. r i ≤ µ i s i ; (26b) r i , µ i s i ≥ 0 ∀ i ; µ i n = 0 ∀ n 6 = s i ; (26c) − − − − − − − − − − − − − µ i n + P m ∈N R i mn ≤ P o ∈N R i no ∀ n, i ; ( 2 6d) P i ∈L R i mn ≤ R mn ∀ ( m, n ); (26e) R ∈ C , R i ∈ R i ∀ i. (26f) As indicated b y the dashe d line, the functionality o f th e p rob- lem covers two networkin g layers. The optimizing fun ction (26a) and the first two con straints (26 b), (2 6 c) reside in the transport laye r, because the impacted variables are the rates injected b y the con trol sampler a n d the CC admission r ates µ i s i . Con straints ( 26d)–(26f) reside on the MA C la y er , as they constrain the per-link rates. When explicitly relaxing only the constraints (26b) and (26d), the Lagran g ian functio n becomes L ( r , µ , R i , λ , q ) = X i ∈L  w i J i ( r i ) + λ i ( r i − µ i s i )  + X n ∈N X i ∈L q i n µ i n + X m ∈N R i mn − X o ∈N R i no ! . V ariables λ i and q i n are non-n egati ve Lagrangian multipliers, respectively . W e can now co nstruct the Karush-Kuhn - T ucker (KKT) condition [31, Ch. 5.5.3 ] for µ i n by demanding ∂ L/ ∂ µ i n = 0 , which y ields that λ i = q i s i . Using this, bo th , µ an d λ can b e eliminated from the Lagrang ia n , which n ow is: L ( r , R i , q ) = X i ∈L  w i J i ( r i ) + q i s i r i  (27) + X n ∈N X i ∈L q i n X m ∈N R i mn − X o ∈N R i no ! . Let Ω =  r , R i : ∃ R s.t. (26 c), (2 6 e), (2 6f) h old  be the domain of L ( r , R i , q ) over the p rimal variables r and R i . Then, the d ual function re d uces to Θ( q ) = inf ( r , R i ) ∈ Ω L ( r , R i , q ) . Due to strong d uality , the GOP can be so lved in the dual domain, i.e., by solv ing max q ≥ 0 Θ( q ) , which is a conve x problem . While we cannot provid e a closed-fo rm expression for the d ual function , we can ev a lu ate its value fo r g iv en q by solving a minimization p roblem on L ( r , R i , q ) . For fixed multipliers q , the Lag rangian ( 2 7) ha s tw o additive p arts th at depend on different variables r and R i . Thus, both parts can be optimized ind ependen tly of each o th er . In r , the optimization problem decompo ses into inde penden t pr oblems o f th e fo rm: min r i ≥ 0 w i J i ( r i ) + q i s i r i , (28) which is an unconstrained pro blem fo r a single loop . T h e solution for (28) is in vestigated in [ 7] and is discussed above. By re-order ing the secon d sum of L ( r , R i , q ) , co n verting the minimizatio n into a maximizatio n and e xplicitly stating the constraints of Ω , the optimization problem in R i becomes: max R i , R P m ∈N P n ∈N P i ∈L R i mn  q i m − q i n  (29) s.t. P i ∈L R i mn ≤ R mn ∀ ( m, n ); (30) R ∈ C , R i ∈ R i ∀ i. (31) Parts o f this prob lem can be solved analytically . N o te that the majority of R i mn are zero d ue to the constraint set R i , which constrains comm unication to the rou ted path of each loop . For fixed R mn , the pr oblem d ecomposes into indepen dent problem s for each link ( m, n ) . In particular , a weighted sum of those per-loop rates R i mn that are allowed to use link ( m, n ) needs to be maximized . Clearly , if q i m < q i n , the m aximum value is obtain ed by R i mn = 0 . Among all tho se lo ops i for which q i m ≥ q i n , the op timal so lution is rea d ily solved as R i mn = ( R mn , if i = arg max i { q i m − q i n } 0 , else . (32) If sev eral links are tied on th e a rg max , a rand om link may b e chosen. T echnica lly , th e solu tion o f this sub-prob lem pr ovides a flow prioritization d ecision, as it defines to w h ich loop i the data r ate of a link is assigne d . Th e multipliers then play the role of pr ioritization factors. W e can now define W i mn :=  q i m − q i n  + ; W mn := max i  W i mn  . (33) Using the optima l values and d e fin itions, the stated problem has the simp le r expression max R ∈C X m ∈N X n ∈N W mn R mn . ( 34) This is a MAC layer weig h ted sum- rate ( WSR) max imization, a pro blem that is well recogn ized in networkin g (e.g. , [39]– [43]). The explicit solutio n to th is p roblem varies dependin g on the exact system pro perties, i.e., with the action set A and link state set Q , respecti vely . Howe ver, it is well in vestigated under various system assumptions. W e co nclude that given q , the value of Θ( q ) can be obtained by so lv ing a co mbination of optimal co ntrol, flo w sched u ling an d WSR max imization. Giv en this development, we can cho ose the optimal multi- pliers by maximizin g the dual function with a gr a d ient ascent [31], [44]. As the optimal v alues o f the Lagrang e -variables satisfy λ i = q i s i ∀ i , it is only necessary to update q . Using the formu latio n in (27), this resu lts in the update fo rmula q i n [ x + 1 ] := " q i n [ x ] + θ " X m ∈N R i mn + X i ∈U n r i − X o ∈N R i no ## + (35) 8 Algorithm 1 Back-p r essure Solutio n 1: Pre - design ξ i ∀ i using 2: ξ i, ∗ k ( Z i k ) = − K i ∗ E [ x i k | Z i k ] accordin g to (19)-(21) 3: Pre - design M i ( λ ) ∀ i (e.g ., with value iteratio n [ 45]) 4: Cho ose θ > 0 5: for τ = 1 , ..., ∞ do 6: for all i ∈ L , k ∈ K i [ τ ] do 7: u i k = ξ i, ∗ k ( Z i k ) 8: δ i k = 1 {k e i k | k − 1 k 2 > M i ( θB s i [ τ ]) } 9: end f or 10: for all ( m, n ) ∈ N × N do 11: for all i : ( m, n ) ∈ Z i do 12: W i mn [ τ ] :=  B i n [ τ ] − B i m [ τ ]  + 13: end f or 14: W mn [ τ ] := max i { W i mn [ τ ] } 15: i ∗ mn := arg max i { W i mn [ τ ] } 16: end f or 17: A [ τ ] := ar g max A ∈A P m ∈N P n ∈N W mn R mn ( Q [ τ ] , A ) 18: Assign resulting rates to i ∗ mn ∀ ( m, n ) 19: end for where x ∈ N + is the iter ation and θ > 0 is a fixed step-size. From conve x optimizatio n th e o ry [31], [44], it is known that for approp riately ch osen fixed step-sizes, gradient ascent algorithm s con verge towards a vicin ity of the optimal solutions from arb itr ary initial po ints. The size of the vicinity and co n- vergence sp e ed thereby depe n ds on the step- size and smaller step-sizes r e su lt in a smaller vicinity b ut slower c o n vergence. By com paring (35) with ( 6) and ch o osing q i n [0] := θB i n [0] ∀ n, l , it b ecomes clear that q i n [ x ] := θB i n [ x ] , i.e., q i n reflects the behavior of the r espectiv e MAC layer queue. D. Op timal Networking For the optimal CC and sched uling strategies ψ and π , a ny value of µ i n satisfies the KKT condition b ecause th e optimal multipliers are given by λ i = q i s i ∀ i . From the co mplemen tary slackness conditions, it mu st ho ld at op tim ality that λ i  r i − µ i s i  = 0 ∀ i . (36) W e further deduc e th a t at optim ality , λ i > 0 because J i ( r i ) is decreasing in r i . Let λ i = 0 be an optim a l multip lier , then r i could be ever increased to red uce the Lagran gian (27) further, which contradicts optimality of a ny r i and leads to infeasibility due to viola tio n of the rate constrain t. Hence λ i > 0 must hold, leading to µ i s i = r i by (36), i.e. , the op timal admitted rate matches the injected on e. Hence th ere is no need for CC in the giv en f ormulation . Indeed , as the sampling st rategy is network - aware, it implicitly performs the task of CC. Th erefore, using an optimal samp ling strategy , the be st CC policy is to pass throug h any traffic towards the MAC-layer immediately . E. R esulting Algorithm The o p timization algo rithm, i.e. Algorithm 1, is a comb ina- tion of certa in ty e quiv alent contr ol, thresh old based sampling and b ack-pre ssur e schedu ler [1], [26], [ 30]: As shown in lines 2 an d 7, each loop uses the certain ty equiv alent contr ol law 0 20 40 60 80 100 120 140 1 60 18 0 200 0 5 10 15 20 Cost λ i Threshold M i ( λ i ) A i = 0 . 75 A i = 1 . 25 Fig. 2. Sampling threshold M i as a function of λ i , giv en A i ∈ { 0 . 75 , 1 . 25 } in (19)–(21). Further, a threshold map p ing M i : R + 7→ R + from each Lagr ange m ultiplier λ i to a threshold M i ( λ i ) is designed. This can be don e with value iteratio n [ 45], using the erro r as sy stem state and fixed λ i as transmission cost, [7]. In each co ntrol step, the samplers u se a threshold p olicy (line 8 ) where a scaled version of th e MAC back -log, θ B s i [ τ ] , is used as p rice value. On the network side each nod e n ∈ N determines the W i mn [ τ ] =  B i n [ τ ] − B i m [ τ ]  + of all contr o l loops whose tra ffic passes th rough them. The values ar e then used to solve a back-pr e ssure scheduling prob lem in line 17 and the resulting r ates are assigned to th e loops with maximum differential back- log on each n ode. I V . S I M U L A T I O N R E S U LT S W e present a simulation study with th e back-pressur e solu- tion f rom Algor ithm 1. W e assume |L| = { 2 , 4 , 6 , . . . , 46 } control lo ops over a two-hop wireless network, in which all packets gen erated at the sou rce n odes are forwarded to their destinations th rough a central b ase station n o de. T wo trans- mission ch annels are av ailable f o r each uplink and d ownlink hops to transport the d ata. W e assume that th e b ase station knows the queue bac k log lengths and channel q ualities through buf fers status r e ports and cha n nel sou nding, based on which it enforce s schedules according to a back-p ressure sched uling law . T o create diversity of CC mechanisms am ong loop s, consider two classes of scala r plants with A i ∈ { 0 . 75 , 1 . 25 } . Let the numb er of stable p lants ( A i = 0 . 75 ) be equal with that of unstable o nes ( A i = 1 . 2 5 ), B i = 1 , and w i ∼ N (0 , 1 ) , ∀ i ∈ L . Packet size and channel qu a lity is equal fo r all loop s and each link can accommo date two users simu ltaneously . W e consider equa l sampling p eriod among all co ntrol loops, an d ten times faster commu n ication T k = T i = 10 · T τ , ∀ i ∈ L . Fig. 2 illustrates the threshold values f o r both classes of loops for Q i x = 1 . 0 and Q i u = 0 , ∀ i . From ( 1 9) together with Q i u = 0 , th e optimal control bec o mes a deadbeat contro ller with the optim a l g ain K i ∗ = A i . The values are obtained by using the value iter ation to solve an average cost-per-stage problem for each fixed λ i , following [7]. Note that as shown in Section I II-C, threshold values of the i -th loop λ i are propo rtional to the curre nt backlog in the M A C layer q ueue, i.e., λ i = q i s i = θ B i s i . A high er queu e backlo g le a d s to a higher thresh old value which re d uces the chance of a packet 9 0 10 20 30 40 0 0 . 5 1 L r r r 0 . 75 r 1 . 25 Fig. 3. Solid line: av erage pack et injection rate per system, i.e., r . Dashed lines r 0 . 75 and r 1 . 25 : av erage rate of stab le and un stable plan ts, respecti vely . 0 10 20 30 4 0 0 1 2 3 L B [p ackets] B 1 . 25 B 0 . 75 B Fig. 4. Solid line : ave rage queue backlog of the MA C buf fer per system, i.e., B . Dashed lines B 0 . 75 and B 1 . 25 : av erage queue backlog of stable and unstable plants, respecti vely . V ertic al error bars with 95% confidence interv als. injection at the source, effectiv ely leading to a CC mechan ism. Further, it is ob served that rate re d uction is mo re ag gressiv e for stable plants due to the increased th reshold. Figs. 3 and 4 p resent th e interplay between the qu eue backlog ( q ueued pa ckets) at the source an d the resulting av erage rate r admitted at the sour ce in terms of packets per slot. In a d dition, the a verages over all stable and un stable loops are sho wn. Confidence inte r vals f or 95 % co n fidence level are included but mostly smaller than the line wid th in the figures. As the nu mber of lo o ps L incre ases fro m 2 to 20, we o bserve that the average rate r per loop stays c o nstant. The initial rate being equal to 1 . 0 corresponds to the normalized sam p ling rate of control loops and im plies that all of the gen erated p ackets have been successfully delivered to the resp e c ti ve r e c ipient. As the number of lo o ps exceed s 20 , wh ich is a lso the maximum number o f transmission oppo rtunities a vailable in samplin g period T k , the CC mechanism redu c es th e packet admission rate. The system beco m es unstab le when L = 4 6 . From this point on, due to d iv ergence of error in un stable loops, the CC alg orithm starts to inject packets into MAC queue with full sam pling rate, i.e. , r = 1 . 0 , e ffectively overloadin g the network. Hence, the packet injection stops completely for th e stable loo p s with A i = 0 . 75 , as th eir error nor m hardly exceeds the resp e cti ve threshold values. In order to av oid visual clu tter , 0 10 2 0 3 0 4 0 0 10 20 L A verag e Delay [time-steps] d 1 . 25 d 0 . 75 d Fig. 5. The solid line illustrate s the ave rage delay per system, i.e., d . Dashed lines d 0 . 75 and d 1 . 25 sho w the dela y of stable and unstable pla nts, respect i vely . V ertica l error bars represent 95% confidence interv als. 0 10 20 30 40 5 10 15 L A verag e Cost J J 1 . 25 J 0 . 75 J Fig. 6. Solid li ne illustrates the av erage quadratic cost per system, i.e., J . Dashed lines J 0 . 75 and J 1 . 25 sho w the avera ge rate of stable and unstable plants, respecti ve ly . V ertic al error bars represent 95% confidence interv als. we exclude the results for L = 46 in Figs. 4 , 5 and 6. From Fig. 4 we can see th at togeth er with the admission control, back-p ressure solutio n as a MA C p olicy succeed s limiting queu e backlog length u p to 4 4 loops. Since Alg orithm 1 selects the user with maximu m queue backlog on e a c h link, av erage qu eue b acklog stays close fo r bo th plant classes for L ≤ 44 . Fr o m the results illustrated in Fig .s 3 and 4, we can conclud e that the u nstable loops, i.e., A i = 1 . 25 , constitute the m ajority of the ong oing traffic as n etwork congestion increases. Th is is e vident fr om Figure 5 as well which sh ows the average delay in time-steps. Even thoug h packet in je c tion rate of the unstable loop s is mu ch hig her than the stable ones, due to prior ization o f larger queue backlog , th ey are pr ovided lower end-to-end delay . The resulting contr o l cost is illustrated in Fig. 6. It shows that for higher numb er of loop s, th e control cost of th e unstable loo ps is h igher e ven though th ey are served higher rates and lower d elays. This follows fro m the higher sensiti vity of u n stable lo ops to p acket dr ops and delays [46]. V . C O N C L U S I O N S In this work we in vestigate a joint system d esign for NCS, from bo th, co ntrol and ne tworking p erspectives. W e f o rmulate the problem of minimizing the weigh ted sum LQG cost f or 10 stochastic L TI systems in a multi-ho p network with gen eral- ized MAC layer capa b ilities. Optimization is perfo rmed over the set of ad missible sampling, co ntrol, cong estion control and scheduling strategies. W e perf orm decomp osition of the global problem into a “primal” contr ol and a “master” networkin g problem , which ar e coupled throu gh Lagr angian multipliers. The primal problem is solved for fixed multiplier s and leads to a certainty e quiv alence control togeth er with a thre sh old based sampling po licy . A dual optimization is used for the master problem, leadin g to a back-pressure ty p e scheduler with a simple p a ss-th rough con gestion control. Interestingly , the La- grange multiplier s ar e shown to be related to qu eue back-lo gs, i.e., the local back-log could be used a s “co mmunicatio n p r ice” for the correspo nding control loop. The resulting structure was applied and imp lemented to a two-hop cellular network , where the observations consistently verified the theoretical r esults. R E F E R E N C E S [1] M. Chiang, S. H. Low, A. R. Calderb ank, and J. C. Doyle, “Layering as optimiza tion decomposit ion: A mathematical theory of network archit ectures, ” IEEE Proce edings , vol. 95, no. 1, pp. 255–312, 2007. [2] P . Bommanna v ar and T . Basa r , “Optimal control with limit ed control actio ns and lossy transmissions, ” in 47th IEE E Confe ren ce on Decision and Contr ol . IE EE, 2008, pp. 2032–2037. [3] A. Molin and S. Hirch e, “On lqg joint opti mal sched uling and control under communication constraints, ” in Proce edings of the 48th IEEE Confer ence on Decision and Contr ol , 2009, pp. 5832–5838. [4] ——, “On the optimali ty of certainty equiv alence for e vent -triggere d control systems, ” IEEE T rans. Autom. Contr ol , vol. 58, no. 2, pp. 470– 474, 2013. [5] M. S. Branick y , S. M. P hilips, and W . Zhang, “Scheduling and Feedback Co-Design for Network ed Control Systems, ” in Pr oceedi ngs of the 41st IEEE Conferen ce on Deci sion and Contr ol , 2002, pp. 1211–1217. [6] W . P . M. H. Heemels, A. R. T eel, N. va n de W ouw , and D. Ne ˇ si ´ c, “Netw orked Control Systems with Communication Constraint s: T rade- of fs between Transmissio n Interv als and Delays, ” in Pr oceedi ngs of the Eur opean Contr ol Confer ence . EUCA, 2009, pp. 4296–4301. [7] A. Molin and S. Hirche, “Pric e-based adapti ve scheduling in multi-loo p control systems with resource c onstraint s, ” IEEE T rans. Autom. Contr ol , vol. 59, no. 12, pp. 3282–3295, 2014. [8] D. Maity, M. H. Mamduhi, S. Hi rche, K. H. Johansson, a nd J. S. Baras, “Optimal lqg cont rol under delay-depen dent costly informatio n, ” IEEE Contr ol Systems Lette rs , vol. 3, no. 1, pp. 102–107, 2019. [9] L . X iao, M. Johansson, H. Hindi, S. Boyd, and A. Goldsmit h, “Joint optimiza tion of communica tion rates and linear systems, ” IEE E T rans. Autom. Contr ol , vol. 48, no. 1, pp. 148–153, 2003. [10] A. Molin and S. Hirche, “Optimal design of decentrali zed ev ent- triggere d controlle rs for lar ge-scal e systems with contenti on-based com- municati on, ” in 2011 50th IEE E Confer ence on Decision and Contr ol and Eur opean Contr ol Confer ence , Dec 2011, pp. 4710–4716. [11] M. H. Mamduhi , D. T oli ´ c, and S. Hirche, “Robust Event-Ba sed Data Scheduli ng for Resou rce Constrained Network ed Control Systems, ” in 2015 American Contr ol Confe re nce (ACC) , 2015, pp. 4695–4701. [12] G. C. W alsh and H. Y e, “Sched uling of Network ed Control Systems, ” IEEE Contro l Syst. Mag. , vol. 10, no. 3, pp. 57–65, 2001. [13] G. C. W alsh, H. Y e, and L. G. Bushnell, “Stabilit y Analysis of Net- work ed Control Systems, ” IEEE T rans. Contr ol Syst. T echnol . , vol . 10, no. 3, pp. 438–446, 2002. [14] R. Schoeff auer and G. Wun der , “Predicti ve netw ork control and through- put sub-opti mality of max weight, ” in 2018 Eur opean Confer ence on Network s and Communications (E uCNC) . IEEE, 2018, pp. 1–6. [15] M. H. Mamduhi, D. T oli ´ c, and S. Hirc he, “Dec entrali zed eve nt-based scheduli ng for shared-resource netwo rked control systems, ” in Contr ol Confer ence (ECC), 2015 Eur opean . IEEE, 2015, pp. 947–953. [16] M. V ilgelm, M. H. Mamduhi, W . Kel lerer , and S. Hirche, “Adapti ve Decent ralize d MA C for Event-T riggered Network ed Control Systems, ” in Proce edings of the 19th International Confe re nce on Hybrid Systems: Computati on and Contr ol (HSCC) . A CM, 2016, pp. 165–174. [17] K. Ga tsis, A . Ribeiro, and G. J. Pappas, “Control-A ware Random Access Communicat ion, ” in 2016 ACM/ IEEE 7th Internat ional Confer ence on Cyber -Physical Systems (ICCPS) . IEEE , 2016, pp. 1–9. [18] M. Bal aghi, D. J. Antune s, M. H. Mamduhi , and S. Hirch e, “ An opti mal LQG control ler for stochast ic e vent -triggere d scheduling over a lossy communicat ion netw ork, ” 7th IF A C W orkshop on Dist ribute d Estimat ion and Contr ol in Net worke d Systems , vol. 51, no. 23, pp. 58 – 63, 2018. [19] G. Boggia and P . Camarda, “T ow ard wireless networ ked control systems: an expe rimental study on rea l-time communications in 80 2.11 W LANs, ” Systems, 2008. WFCS , 2008. [20] S. Misra, P . V . Krishna, V . Saritha, H. Agarwal, L . Shu, and M. S. Obaidat , “Efficient medium acc ess control for cybe r-physi cal s ystems with heterogeneou s netw orks, ” IEEE Syst. J. , vol. 9, no. 1, pp. 22–30, 2015. [21] M. Eisen, M. M. Rashid, K. Gatsis, D. Cav alcanti, N. Himayat, and A. Ribeiro , “Control aware radio resource allocat ion in lo w latenc y wireless control systems, ” arX iv pr eprint arXiv:1811.00409 , 2018. [22] D. E. Quev edo, A. Ahl ´ en, and J. Ostergaard, “Energy effic ient state estimati on with wireless sensors through the use of predicti ve power control and coding, ” IEEE T rans. Signal Pr ocess. , vol. 58, no. 9, pp. 4811–4823, 2010. [23] X. Liu and A. Goldsmith, “W ireless network design for distrib uted control , ” in Pro ceedin gs of the IEEE Confer ence on Decision and Contr ol . Citeseer , 2004, pp. 2823–2829. [24] J. Wu , Q. Jia, K. H. Johansson, and L. Shi, “Event-base d sensor data scheduli ng: Tra de-of f between communicati on rate and estimation qualit y , ” IE EE T rans. Autom. Cont r ol , vol. 58, no. 4, pp. 1041–1046, 2013. [25] B. L i, Y . Ma, T . W estenbr oek, C. Wu, H. Gonzalez, an d C. Lu, “W irele ss routing and control: a cyber -physical case s tudy , ” in Cyber-Physica l Systems, ACM/IEEE 7th Internati onal Confe re nce on , 2016, pp. 1–10. [26] L . Georgi adis, M. J. Neely , and L. T assiulas, “Resource allocation and cross-laye r control in wireless networks, ” F oundations and T rends in Network ing , vol . 1, no. 1, pp. 1–144, 2006. [27] M. Kl ¨ ugel , “Operation and control of device -to-de vice communication in cellu lar networks, ” P h. D. dissertation, TU Munich, 2019. [28] S. As mussen, A pplied P r obabil ity and Queues . Springer NY , 2003. [29] M. J. Neely , “Stochastic network optimization with applicatio n to communicat ion and queueing systems, ” Synthesis Lectu re s on Commu- nicati on Netw orks , vol. 3, no. 1, pp. 1–211, 2010. [30] X. Lin, N. B. Shrof f, and R. Srikant, “ A tut orial on cross-layer opti- mizatio n in wireless networks, ” IEEE J. Sel. A re as Commun. , v ol. 24, no. 8, pp. 1452–14 63, Aug 2006. [31] S. Boyd and L. V andenber ghe, Con vex Optimization . New Y ork, NY , USA: Cambridge Uni ver sity P ress, 2004. [32] C. Ramesh, H. Sandber g, and K. H. Johansson, “Design of state-based schedule rs for a net work of control l oops, ” IEEE T rans. Autom. Cont r ol , vol. 58, no. 8, pp. 1962–1975, Aug 2013. [33] A. Nayyar , T . Baar , D. T eneketzi s, and V . V . V eerav alli, “Optimal strate gies for communication and remote estimation with an energy harve sting sensor , ” IEEE Tr ans. Autom. Contr ol , vol. 58, no. 9, pp. 2246–2260, Sep. 2013. [34] M. Rabi , G. V . Moustakides, and J. S. Baras, “ Adapti ve sampling for linea r state estimati on, ” SIAM J . Contr ol Optim. , vol. 50, no. 2, pp. 672–702, Mar . 2012. [35] G. M. Lipsa and N. C. Martins, “Remote state estimation with commu- nicat ion costs for first-order lti systems, ” IEEE T rans. Aut om. Cont r ol , vol. 56, no. 9, pp. 2013–2025, Sep. 2011. [36] A. Molin and S. Hirche, “Struct ural characteri zation of optimal ev ent- based controlle rs fo r linear stochastic systems, ” in 49th IE EE Confer ence on Decision and Contr ol (CDC) , Dec 2010, pp. 3227–3233. [37] O. Hern ´ andez-Lerma and R. Romera, “The scalarizati on approach to multiobj ecti ve mark ov controlp roblems: Why does it work?” Appl. Math. Optim. , vol. 50, no. 3, pp. 279–293, Oct 2004. [38] O. Hern ´ andez-Lerma, Adaptive Marko v Contr ol P r ocesses . Berlin, Heidelb erg: Springer-V erlag, 2001. [39] M. Kobaya shi and G. Caire, “ An iterat i ve water- filling algorith m for maximum weighted sum-rate of gaussian mimo-bc, ” IEEE J. Sel. Are as Commun. , vol. 24, no. 8, pp. 1640–1646, 2006. [40] S. S. Christensen, R. Agarwal, E. D. Carv alho, and J. M. Ciof fi, “W eighte d sum-rate m aximiza tion using weighted mmse for mimo-bc beamforming design, ” IEE E T rans. W ireless Commun. , vol. 7, no. 12, pp. 4792–4799, Dece mber 2008. [41] I. C. W ong and B. Ev ans, “Optimal downlink OFDMA resource allo- catio n with line ar complexity to maximize er godic rates, ” IEEE T rans. W ire less Commun. , vol. 7, no. 3, pp. 962–971, 2008. [42] I. C. W ong, O. Oteri , and W . Mccoy, “Optimal resource allocati on in uplink sc-fdma systems, ” IEE E T rans. W irele ss Commun. , vol. 8, no. 5, pp. 2161–2165, May 2009. 11 [43] C. W . T an, M. Chiang, and R. Srikant, “Maximizi ng sum rate and minimizin g m se on multi user downlin k: Optimality , fast algorithms and equi valen ce via max-min sinr , ” IEEE T rans. Signal Proce ss. , vol. 59, no. 12, pp. 6127–6143, Dec 2011. [44] P . P . Marino, Optimizati on of Computer Networks: Modeling and Algo- rithms: a Hands-on Approac h . J ohn W ile y & Sons, 2016. [45] D. P . Bertsekas, Dynamic Pr ogramming and Opti mal Contro l, V ol. II , 3rd ed. Athena Scient ific, 2007. [46] W . Zhang, M. S. Branicky , and S. M. Phillips, “Stability of networke d control systems, ” IEE E Contro l Syst. Mag. , vol. 21, no. 1, pp. 84–99, Feb 2001.