Network Estimation and Packet Delivery Prediction for Control over Wireless Mesh Networks

Net w o rk Estimation and P ack et Delivery Prediction fo r Control over Wireless Mesh Net w o rks The w o rk w as supp o rted b y the EU p roject F eedNetBack, the Sw edish Resea rch Council, the Sw edish Strategic Resea rch F oundation, the Sw edish Governmental Agency fo r Innovation Systems, and the Knut and Alice W allenb erg F oundation. PHOEBUS CHEN, CHITHRUP A RAMESH, AND KARL H. JOHANSSON Sto ckholm 2010 A CCESS Linnaeus Centre Automatic Control Scho ol of Electrical Engineering KTH Ro yal Institute of T echnology SE-100 44 Sto ckholm, Sw eden TRIT A-EE:043 Abstract Muc h of the current theory of netw ork ed control systems uses simple point-to-point communication mo dels as an abstraction of the underlying net work. As a result, the controller has very limited information on the net work conditions and p erforms sub optimally . This work models the underlying wireless multihop mesh net work as a graph of links with transmission success probabilities, and uses a recursive Bay esian estimator to provide pac ket deliv ery predictions to the con troller. The predictions are a join t probability distribution on future pac ket deliv ery sequences, and th us capture correlations betw een successive pac k et deliveries. W e lo ok at finite horizon LQG control ov er a lossy actuation c hannel and a perfect sensing channel, b oth without delay , to study ho w the con troller can compensate for predicted netw ork outages. 1 In tro duction Increasingly , control systems are op erated ov er large-scale, net work ed infrastructures. In fact, several companies to da y are introducing devices that communicate o ver lo w-p ow er wireless mesh netw orks for industrial automation and pro cess control [1, 2]. While wireless mesh net w orks can connect con trol pro cesses that are physically spread out o ver a large space to sa ve wiring costs, these netw orks are difficult to design, pro vision, and manage [3, 4]. F urthermore, wireless communication is inherently unreliable, in tro ducing pac ket losses and delays, whic h are detrimen tal to con trol system p erformance and stabilit y . Researc h in the area of Net w orked Control Systems (NCSs) [5] addresses how to design control systems which can account for the lossy , dela y ed communication channels introduced by a netw ork. T raditional tasks in control systems design, lik e stability/performance analysis and controller/estimator synthesis, are revisited, with netw ork mo dels providing statistics ab out pack et losses and delays. In the pro cess, the studies highlight the b enefits and dra wbac ks of different system arc hitectures. F or example, Figure 1 depicts the general system architecture of a net w orked con trol system o ver a mesh net work prop osed by Robinson and Kumar [6]. A fundamen tal architecture problem is how to choose the b est lo cation to place the controllers, if they can b e placed at an y of the sensors, actuators, or comm unication rela y nodes in the net w ork. One insigh t from Sc henato et al. [7] is that if the con troller can know whether the control pack et reaches the actuator, e.g., we place the controller at the actuator, then the optimal LQG con troller and estimator can b e designed separately (the separation principle). Figure 1: A netw orked con trol system ov er a mesh netw ork, where the controllers can b e lo cated on an y no de. T o gain more insights on how to architect and design NCSs, tw o limitations in the approach of many current 1 NCS researc h studies need to be addressed. The first limitation is the use of simple mo dels of pack et delivery o v er a point-to-point link or a star net work top ology to represen t the netw ork, which are often m ultihop and more complex. The second limitation is the treatment of the netw ork as something designed and fixed a priori b efore the design of the control system. V ery little information is passed through the interface b etw een the net w ork and the con trol system, limiting the in teraction b etw een the tw o “lay ers” to tune the con troller to the net work conditions, and vice v ersa. 1.1 Related W orks Sc henato et al. [7] and Is hii [8] study stability and controller synthesis for different control system architectures, but they both model net works as i.i.d. Bernoulli processes that drop pac kets on a single link. The information passed through the interface b etw een the netw ork and the control system is the pack et drop probability of the link, whic h is assumed to b e kno wn and fixed. Seiler and Sengupta [9] study stability and H ∞ con troller synthesis when the net w ork is mo deled as a pac k et-dropping link describ ed by a t w o-state Mark o v c hain (Gilb ert-Elliott mo del), where the information passed through the net work-con troller in terface are the transition probabilities of the Mark o v c hain. Elia [10] studies stability and the synthesis of a stabilizing con troller when the net w ork is represen ted b y an L TI system with sto c hastic disturbances mo deled as parallel, indep endent, multiplicativ e fading channels. Some related w ork in NCSs do use models of multihop net w orks. F or instance, work on consensus of m ulti-agent systems [11] typically study ho w the connectivit y graph(s) provided b y the netw ork affects the conv ergence of the system, and is not focused on modeling the links. Robinson and Kumar [6] study the optimal placement of a con troller in a m ultihop net w ork with i.i.d. Bernoulli pack et-dropping links, where the pack et drop probability is kno wn to the controller. Gupta et al. [12] study how to optimally process and forward sensor measuremen ts at eac h no de in a multihop netw ork for optimal LQG control, and analyze stability when pack et drops on the links are mo deled as spatially-indep endent Bernoulli, spatially-indep endent Gilb ert-Elliott, or memoryless spatially- correlated pro cesses. 1 V aragnolo et al. [13] compare the p erformance of a time-v arying Kalman filter on a wireless TDMA mesh netw ork under unicast routing and constrained flo o ding. The netw ork mo del describ es the routing top ology and schedule of an implemen ted communication proto col, TSMP [14], but it assumes that transmission successes on the links are spatially-indep endent and memoryless. Both Gupta et al. [12] and V aragnolo et al. [13] are concerned with estimation when pack et drops o ccur on the sensing c hannel, and the estimators do not need to kno w net w ork parameters lik e the pac k et loss probabilit y . 1.2 Con tributions Our approach is a step tow ard using more sophisticated, multihop netw ork mo dels and passing more information through the interface betw een the con troller and the netw ork. Similar to Gupta et al. [12], w e mo del the netw ork routing top ology as a graph of independent links, where transmission success on eac h link is described b y a t wo-state Mark o v chain. The netw ork model consists of the routing top ology and a global TDMA transmission schedule. Suc h a minimalist net work model captures the essence of how a netw ork with bursty links can hav e correlated pac k et deliveries [15], whic h are particularly bad for control when they result in bursts of pack et losses. Using this mo del, w e propose a netw ork estimator to estimate, without loss of information, the state of the netw ork giv en the past pack et deliveries. 2 The netw ork estimate is translated to a joint probability distribution predicting the success of future pack et deliveries, whic h is passed through the netw ork-controller interface so the controller can comp ensate for unav oidable netw ork outages. The netw ork estimator can also b e used to notify a netw ork manager when the net w ork is brok en and needs to b e reconfigured or repro visioned, a direction for future research. Section 2 describes our plant and net work mo dels. W e propose t wo netw ork estimators, the Static Indep enden t links, Hop-b y-hop routing, Scheduled (SIHS) net work estimator and the Gilbert-Elliott Independent links, Hop- b y-hop routing, Sc heduled (GEIHS) netw ork estimator in Section 3. Next, we design a finite-horizon, F uture- P ac ket-Deliv ery-optimized (FPD) LQG controller to utilize the pac ket deliv ery predictions provided b y the net work 1 Here, “spatially” means “with resp ect to other links.” 2 Strictly speaking, we obtain the probabilit y distribution on the states of the net work, not a single point estimate. 2 Figure 2: A control lo op for plant P with the netw ork on the actuation channel. The netw ork estimator ˆ N passes pac k et deliv ery predictions f ν k + H − 1 k to the FPD controller C , with past pack et delivery information obtained from the net w ork N ov er an ackno wledgement (ACK) c hannel. estimators, presen ted in Section 4. Section 5 pro vides an example and simulations demonstrating how the GEIHS net w ork estimator com bined with the FPD controller can pro vide b etter performance than a classical LQG con troller or a controller assuming i.i.d. pack et deliveries. Finally , Section 7 describ es the limitations of our approach and future w ork. 2 Problem F orm ulation This pap er studies an instance of the general system architecture depicted in Figure 1, with a single con trol lo op con taining one sensor and one actuator. One netw ork estimator and one controller are placed at the sensor, and we assume that an end-to-end ackno wledgement (ACK) that the controller-to-actuator pack et is delivered is alwa ys receiv ed at the netw ork estimator, as shown in Figure 2. F or simplicity , w e assume that the plant dynamics are significan tly slow er than the end-to-end pack et delivery deadline, so that we can ignore the delay in tro duced by the net w ork. The general problem is to jointly design a net work estimator and con troller that can optimally control the plan t using our prop osed SIHS and GEIHS netw ork models. In our problem setup, the controller is only concerned with the past, presen t, and future pack et delivery sequence and not with the detailed behavior of the netw ork, nor can it affect the behavior of the net w ork. Therefore, the net work estimation problem decouples from the con trol problem. The information passed through the netw ork-controller in terface is the pac k et deliv ery sequence, sp ecifically the join t probabilit y distribution describing the future pac k et deliv ery predictions. 2.1 Plan t and Netw ork Mo dels The state dynamics of the plan t P in Figure 2 is giv en b y x k +1 = Ax k + ν k B u k + w k , (1) where A ∈ R ` × ` , B ∈ R ` × m , and w k are i.i.d. zero-mean Gaussian random v ariables with co v ariance matrix R w ∈ S ` + , where S ` + is the set of ` × ` p ositive semidefinite matrices. The initial state x 0 is a zero-mean Gaussian random v ariable with cov ariance matrix R 0 ∈ S ` + and is m utually indep endent of w k . The binary random v ariable ν k indicates whether a pack et from the controller reac hes the actuator ( ν k = 1) or not ( ν k = 0), and eac h ν k is indep enden t of x 0 and w k (but the ν k ’s are not indep enden t of eac h other). 3 Figure 3: The pack et containing the con trol input u k is generated righ t b efore time slot t k . The pack et may be in transit through the netw ork in the shaded time slots, until righ t b efore time slot t 0 k . Thus, time t k is aligned with the b eginning of the time slot. Let the discrete sampling times for the control system b e indexed by k , but let the discrete time for schedule time slots (describe d below) be indexed by t . The time slot interv als are smaller than the sampling in terv als. The time slot when the control pack et at sample time k is generated is denoted t k , and the deadline for receiving the con trol pac ket at the receiv er is t 0 k . W e assume that t 0 k ≤ t k +1 for all k . Figure 3 illustrates the relationship betw een t and k . The model of the TDMA wireless mesh net work ( N in Figure 2) consists of a routing top ology G , a link mo del describing how the transmission success of a link ev olves ov er time, and a fixed rep eating sc hedule F ( T ) . The SIHS net w ork mo del and the GEIHS netw ork mo del only differ in the link model. Each of these comp onents will b e describ ed in detail b elow. The routing top ology is describ ed b y G = ( V , E ), a connected directed acyclic graph with the set of v ertices (no des) V = { 1 , . . . , M } and the set of directed edges (links) E ⊆ { ( i, j ) : i, j ∈ V , i 6 = j } , where the num b er of edges is denoted E . The source node is denoted a and the sink (destination) node is denoted b . Only the destination no de has no outgoing edges. A t any moment in time, the links in G can b e either b e up (succeeds if attempt to transmit pack et) or down (fails if attempt to transmit pac k et). Thus, there are 2 E p ossible top ology realizations ˜ G = ( V , ˜ E ), where ˜ E ⊆ E represen ts the edges that are up. 3 A t time t k , the actual state of the topology is one of the top ology realizations but it is not kno wn to the netw ork estimator. With some abuse of terminology , we define G ( k ) to b e the random v ariable represen ting the state of the top ology at time t k . 4 This pap er considers the netw ork under tw o link mo dels, the static link mo del and the Gilb ert-Elliott (G-E) link mo del. Both netw ork mo dels assume all the links in the netw ork are indep endent. The static link model assumes the links do not switc h betw een being up and down while pac kets are sen t through the net work. Therefore, the sequence of top ology realizations o ver time is constan t. While not realistic, it leads to the simple netw ork estimator in Section 3.1 for p edagogical purp oses. The a priori transmission success probability of link l = ( i, j ) is p l . The G-E link mo del represents each link l by the tw o-state Marko v chain shown in Figure 4. A t each sample time k , a link in state 0 (down) transitions to state 1 (up) with probability p u l , and a link from state 1 transitions to state 0 with probability p d l . 5 The steady-state probability of being in state 1, which w e use as the a priori probabilit y of the link b eing up, is p l = p u l / ( p u l + p d l ) . The fixed, rep eating schedule of length T is represented b y a sequence of matrices F ( T ) = ( F (1) , F (2) , . . . , F ( T ) ), where the matrix F ( t − 1 (mo d T )+1) represen ts the links sc heduled at time t . The matrix F ( t ) ∈ { 0 , 1 } M × M is defined from the set F ( t ) ⊆ E con taining the links scheduled for transmission at time t . W e assume that no des can only unicast pack ets, meaning that for all no des i , if ( i, j ) ∈ F ( t ) then for all v 6 = j, ( i, v ) 6∈ F ( t ) . F urthermore, a 3 Symbols with a tilde ( ˜ · ) denote v alues that can b e tak en on b y random v ariables, and can b e the arguments to probabilit y distribution functions (p dfs). 4 Strictly speaking, G ( k ) is a function mapping even ts to the set of all topology realizations, not to the set of real num b ers. 5 W e can easily instead use a G-E link mo del that adv ances at each time step t , but it would make the following exposition and notation more complicated. 4 Figure 4: Gilb ert-Elliott link mo del no de holds onto a pack et if the transmission fails and can retransmit the pack et the next time an outgoing link is sc heduled (hop-b y-hop routing). Thus, the matrix F ( t ) has en tries F ( t ) ij =      1 if ( i, j ) ∈ F ( t ) , or if i = j and ∀ v ∈ V , ( i, v ) 6∈ F ( t ) 0 otherwise. An exact description of the netw ork consists of the sequence of top ology realizations ov er time and the schedule F ( T ) . Assuming a top ology realization ˜ G , the links that are scheduled and up at any giv en time t are represented b y the matrix ˜ F ( t ; ˜ G ) ∈ { 0 , 1 } M × M , with en tries ˜ F ( t ; ˜ G ) ij =      1 if ( i, j ) ∈ F ( t ) ∩ ˜ E , or if i = j and ∀ v ∈ V , ( i, v ) 6∈ F ( t ) ∩ ˜ E 0 otherwise. (2) Define the matrix ˜ F ( t,t 0 ; ˜ G ) = ˜ F ( t ; ˜ G ) ˜ F ( t +1; ˜ G ) · · · ˜ F ( t 0 ; ˜ G ) , such that en try ˜ F ( t,t 0 ; ˜ G ) ij is 1 if a pac ket at no de i at time t will be at no de j at time t 0 , and is 0 otherwise. Since the destination b has no outgoing links, a pac k et sen t from the source a at time t reaches the destination b at or b efore time t 0 if and only if ˜ F ( t,t 0 ; ˜ G ) ab = 1. T o simplify the notation, let the function δ κ indicate whether the pac ket delivery ˜ ν ∈ { 0 , 1 } is consisten t with the topology realization ˜ G , assuming the pac k et w as generated at t κ , i.e., δ κ ( ˜ ν ; ˜ G ) = ( 1 if ˜ ν = ˜ F ( t κ ,t 0 κ ; ˜ G ) ab 0 otherwise. (3) The function assumes the fixed rep eating schedule F ( T ) , the pack et generation time t κ , the deadline t 0 κ , the source a , and the destination b are implicitly kno wn. 2.2 Net work Estimators As shown in Figure 2, at eac h sample time k the netw ork estimator ˆ N takes as input the previous pack et delivery ν k − 1 , estimates the topology realization using the net work mo del and all past pac ket deliveries, and outputs the join t probabilit y distribution of future pack et deliveries f ν k + H − 1 k . F or clarit y in the following exposition, let V κ ∈ { 0 , 1 } b e the v alue tak en on b y the pac ket delivery random v ariable ν κ at some past sample time κ . Let the vector V k − 1 0 = [ V 0 , . . . , V k − 1 ] denote the history of pack et deliveries at sample time k , the v alues taken on by the vector of random v ariables ν k − 1 0 = [ ν 0 , . . . , ν k − 1 ]. Then, f ν k + H − 1 k ( ˜ ν H − 1 0 ) = P ( ν k + H − 1 k = ˜ ν H − 1 0 | ν k − 1 0 = V k − 1 0 ) (4) is the prediction of the next H pack et deliveries, where ν k + H − 1 k = [ ν k , . . . , ν k + H − 1 ] is a vector of random v ariables represen ting future pac k et deliv eries and ˜ ν H − 1 0 ∈ { 0 , 1 } H . 5 Figure 5: Graphical mo del describing the netw ork estimation problem. ν k is the measurement output v ariable at time k , and G ( k ) is the hidden state of the net w ork. The SIHS and GEIHS netw ork estimators only differ in the netw ork mo dels. The parameters of the netw ork mo dels — top ology G , schedule F ( T ) , link probabilities { p l } l ∈E or { p u l , p d l } l ∈E , source a , sink b , pack et generation times t k , and deadlines t 0 k — are known a priori to the netw ork estimators and are left out of the conditional probabilit y expressions. In Section 3, we will use the probabilit y distribution on the top ology realizations (our net work state estimate), P ( G ( k ) = ˜ G | ν k − 1 0 = V k − 1 0 ) , to obtain f ν k + H − 1 k from V k − 1 0 and the net w ork mo del. 2.3 FPD Controller The FPD con troller ( C in Figure 2) optimizes the control signals to the statistics of the future pac ket deliv ery sequence, derived from the past pac k et delivery sequence. W e choose the optimal control framew ork b ecause the cost function allows us to easily compare the FPD controller with other controllers. The control p olicy op erates on the information set I C k = { x k 0 , u k − 1 0 , ν k − 1 0 } . (5) The con trol p olicy minimizes the linear quadratic cost function E [ x T N Q 0 x N + P N − 1 n =0 x T n Q 1 x n + ν n u T n Q 2 u n ] , (6) where Q 0 , Q 1 , and Q 2 are p ositive definite weigh ting matrices and N is the finite horizon, to get the minimum cost J = min u 0 ,...,u N − 1 E [ x T N Q 0 x N + P N − 1 n =0 x T n Q 1 x n + ν n u T n Q 2 u n ] . Section 4 will show that the resulting arc hitecture separates into a netw ork estimator and a controller whic h uses the p df f ν k + H − 1 k supplied b y the net w o rk estimator ( ˆ N in Figure 2) to find the control signals u k . 3 Net w ork Estimation and P ac k et Deliv ery Prediction W e will use recursive Bay esian estimation to estimate the state of the netw ork, and use the netw ork state estimate to predict future pack et deliv eries. Figure 5 is the graphical model / hidden Marko v mo del [16] describing our recursiv e estimation problem. 6 3.1 SIHS Netw ork Estimator The steps in the SIHS net work estimator are deriv ed from (4). W e introduce new notation for conditional p dfs (i.e., α k , β k , Z k ), which will b e used later to state the steps in the estimator compactly . 6 First, express f ν k + H − 1 k ( ˜ ν H − 1 0 ) as P ( ν k + H − 1 k = ˜ ν H − 1 0 | ν k − 1 0 = V k − 1 0 ) | {z } f ν k + H − 1 k ( ˜ ν H − 1 0 ) = X ˜ G P ( ν k + H − 1 k = ˜ ν H − 1 0 | G ( k − 1) = ˜ G ) | {z } α k ( ˜ ν H − 1 0 ; ˜ G ) · P ( G ( k − 1) = ˜ G | ν k − 1 0 = V k − 1 0 ) | {z } β k ( ˜ G ) , where w e use the relation P ( ν k + H − 1 k = ˜ ν H − 1 0 | G ( k − 1) = ˜ G, ν k − 1 0 = V k − 1 0 ) = P ( ν k + H − 1 k = ˜ ν H − 1 0 | G ( k − 1) = ˜ G ) . This relation states that giv en the state of the net work, future pack et deliv eries are indep endent of past pack et deliv eries. The expression P ( ν k + H − 1 k = ˜ ν H − 1 0 | G ( k − 1) = ˜ G ) indicates whether the future pack et deliv ery sequence ˜ ν H − 1 0 is consisten t with the graph realization ˜ G , meaning P ( ν k + H − 1 k = ˜ ν H − 1 0 | G ( k − 1) = ˜ G ) = H − 1 Y h =0 δ k + h ( ˜ ν h ; ˜ G ) , where Q is the and operator (sometimes denoted V ). The netw ork state estimate at sample time k from past pac k et deliv eries is β k ( ˜ G ) and is obtained from the net w ork state estimate at sample time k − 1, since P ( G ( k − 1) = ˜ G | ν k − 1 0 = V k − 1 0 ) | {z } β k ( ˜ G ) = δ k − 1 ( V k − 1 ; ˜ G ) z }| { P ( ν k − 1 = V k − 1 | G ( k − 1) = ˜ G ) · β k − 1 ( ˜ G ) z }| { P ( G ( k − 1) = ˜ G | ν k − 2 0 = V k − 2 0 ) P ( ν k − 1 = V k − 1 | ν k − 2 0 = V k − 2 0 ) | {z } Z k . (7) Here, P ( G ( k − 1) = ˜ G | ν k − 2 0 = V k − 2 0 ) = P ( G ( k − 2) = ˜ G | ν k − 2 0 = V k − 2 0 ) = β k − 1 ( ˜ G ) for the static link mo del because G ( k − 1) = G ( k − 2) = G (0) . Again, w e used the indep endence of future pac ket deliveries from past pack et deliveries giv en the net w ork state, P ( ν k − 1 = V k − 1 | G ( k − 1) = ˜ G, ν k − 2 0 = V k − 2 0 ) = P ( ν k − 1 = V k − 1 | G ( k − 1) = ˜ G ) . Note that P ( ν k − 1 = V k − 1 | G ( k − 1) = ˜ G ) can only b e 0 or 1, indicating whether the pac ket delivery is consisten t with the graph realization. Finally , P ( ν k − 1 = V k − 1 | ν k − 2 0 = V k − 2 0 ) is the same for all ˜ G , so it is treated as a normalization constan t. A t sample time k = 0, when no pack ets hav e b een sent through the netw ork, β 0 ( ˜ G ) = P ( G (0) = ˜ G ), whic h is expressed in (8d) b elo w. This equation comes from the assumption that all links in the netw ork are indep endent. T o summarize, the SIHS Network Estimator and Packet Delivery Pr e dictor is a recursive Bay esian estimator where the measuremen t output step consists of f ν k + H − 1 k ( ˜ ν H − 1 0 ) = X ˜ G α k ( ˜ ν H − 1 0 ; ˜ G ) · β k ( ˜ G ) (8a) α k ( ˜ ν H − 1 0 ; ˜ G ) = H − 1 Y h =0 δ k + h ( ˜ ν h ; ˜ G ) , (8b) 6 A semicolon is used in the conditional p dfs to separate the v alues b eing conditioned on from the remaining argumen ts. 7 and the inno v ation step consists of β k ( ˜ G ) = δ k − 1 ( V k − 1 ; ˜ G ) · β k − 1 ( ˜ G ) Z k (8c) β 0 ( ˜ G ) =   Y l ∈ ˜ E p l     Y l ∈E \ ˜ E 1 − p l   , (8d) where α k and β k are functions, Z k is a normalization constant suc h that P ˜ G β k ( ˜ G ) = 1, and the functions δ k + h and δ k − 1 are defined b y (3). 3.2 GEIHS Netw ork Estimator F or compact notation in the probability expressions b elow, we use V k − 1 0 in place of ν k − 1 0 = V k − 1 0 and only write the random v ariable and not its v alue ( ˜ · ) . The deriv ation of the GEIHS netw ork estimator is similar to the previous deriv ation, except that the state of the netw ork evolv es with every sample time k . Since all the links in the net w ork are indep endent, the probability that a giv en top ology ˜ G 0 at sample time k − 1 transitions to a top ology ˜ G after one sample time is giv en b y Γ( ˜ G ; ˜ G 0 ) = P ( G ( k ) | G ( k − 1) ) =   Y l 1 ∈ ˜ E 0 ∩ ˜ E 1 − p d l 1     Y l 2 ∈ ˜ E 0 \ ˜ E p d l 2     Y l 3 ∈ ˜ E \ ˜ E 0 p u l 3     Y l 4 ∈E \ ( ˜ E 0 ∪ ˜ E ) 1 − p u l 4   . (9) First, express f ν k + H − 1 k ( ˜ ν H − 1 0 ) as P ( ν k + H − 1 k | V k − 1 0 ) | {z } f ν k + H − 1 k ( ˜ ν H − 1 0 ) = X G ( k + H − 1) P ( ν k + H − 1 | G ( k + H − 1) ) | {z } δ k + H − 1 ( ˜ ν H − 1 ; ˜ G H − 1 ) · P ( ν k + H − 2 k , G ( k + H − 1) | V k − 1 0 ) | {z } α H − 1 | k − 1 ( ˜ ν H − 2 0 , ˜ G H − 1 ) , where for h = 2 , . . . , H − 1 P ( ν k + h − 1 k , G ( k + h ) | V k − 1 0 ) | {z } α h | k − 1 ( ˜ ν h − 1 0 , ˜ G h ) = X G ( k + h − 1) P ( G ( k + h ) | G ( k + h − 1) ) | {z } Γ( ˜ G h ; ˜ G h − 1 ) · P ( ν k + h − 1 | G ( k + h − 1) ) | {z } δ k + h − 1 ( ˜ ν h − 1 ; ˜ G h − 1 ) · P ( ν k + h − 2 k , G ( k + h − 1) | V k − 1 0 ) | {z } α h − 1 | k − 1 ( ˜ ν h − 2 0 , ˜ G h − 1 ) . (10) When h = 1, replace P ( ν k + h − 2 k , G ( k + h − 1) | V k − 1 0 ) in (10) with P ( G ( k ) | V k − 1 0 ) = β k | k − 1 ( ˜ G ). The v alue β k | k − 1 ( ˜ G ) comes from P ( G ( k ) | V k − 1 0 ) | {z } β k | k − 1 ( ˜ G ) = X G ( k − 1) P ( G ( k ) | G ( k − 1) ) | {z } Γ( ˜ G ; ˜ G 0 ) · P ( G ( k − 1) | V k − 1 0 ) | {z } β k − 1 | k − 1 ( ˜ G 0 ) . The v alue β k − 1 | k − 1 ( ˜ G 0 ) comes from (7), with β k replaced b y β k − 1 | k − 1 and β k − 1 replaced b y β k − 1 | k − 2 . Finally , β 0 |− 1 ( ˜ G ) = P ( G (0) ), where all links are indep endent and hav e link probabilities equal to their steady-state proba- bilit y of b eing in state 1, and is expressed in (11f) b elow. T o summarize, the GEIHS Network Estimator and Packet Delivery Pr e dictor is a recursiv e Bay esian estimator. The measuremen t output step consists of f ν k + H − 1 k ( ˜ ν H − 1 0 ) = X ˜ G H − 1 δ k + H − 1 ( ˜ ν H − 1 ; ˜ G H − 1 ) · α H − 1 | k − 1 ( ˜ ν H − 2 0 , ˜ G H − 1 ) , (11a) where the function α H − 1 | k − 1 is obtained from the follo wing recursiv e equation for h = 2 , . . . , H − 1: α h | k − 1 ( ˜ ν h − 1 0 , ˜ G h ) = X ˜ G h − 1 Γ( ˜ G h ; ˜ G h − 1 ) · δ k + h − 1 ( ˜ ν h − 1 ; ˜ G h − 1 ) · α h − 1 | k − 1 ( ˜ ν h − 2 0 , ˜ G h − 1 ) , (11b) 8 with initial condition α 1 | k − 1 ( ˜ ν 0 0 , ˜ G 1 ) = X ˜ G Γ( ˜ G 1 ; ˜ G ) · δ k ( ˜ ν 0 ; ˜ G ) · β k | k − 1 ( ˜ G ) . (11c) The prediction and inno v ation steps consist of β k | k − 1 ( ˜ G ) = X ˜ G 0 Γ( ˜ G ; ˜ G 0 ) · β k − 1 | k − 1 ( ˜ G 0 ) (11d) β k − 1 | k − 1 ( ˜ G ) = δ k − 1 ( V k − 1 ; ˜ G ) · β k − 1 | k − 2 ( ˜ G ) Z k − 1 (11e) β 0 |− 1 ( ˜ G ) =   Y l ∈ ˜ E p l     Y l ∈E \ ˜ E 1 − p l   , (11f ) where α h | k − 1 , β k − 1 | k − 1 , and β k | k − 1 are functions, Z k − 1 is a normalization constant suc h that P ˜ G β k − 1 | k − 1 ( ˜ G ) = 1, and the functions δ κ (for the differen t v alues of κ ab o v e) and Γ are defined b y (3) and (9), resp ectively . 3.3 P ack et Predictor Complexit y The netw ork estimators are trying to estimate netw ork parameters using measurements collected at the b order of the netw ork, a general problem studied in the field of netw ork tomograph y [17] under v arious problem setups. One of the greatest c hallenges in netw ork tomography is getting go o d estimates with low computational complexity estimators. Our prop osed netw ork estimators are “optimal” with resp ect to our mo dels in the sense that there is no loss of information, but they are computationally exp ensiv e. Prop ert y 1. The SIHS network estimator describ e d by the set of e quations (8) takes O ( E 2 E ) to initialize and O (2 H 2 E ) to up date the network state estimate and pr e dictions at e ach step. The GEIHS network estimator describ e d by the set of e quations (11) takes O ( E 2 2 E ) to initialize and O (2 H 2 2 E ) for e ach up date step. Pr o of. Let D = max k t 0 k − t k . W e assume that con verting ˜ G to the set of links that are up, ˜ E , tak es constant time. Also, one can simulate the path of a pack et by looking up the scheduled and successful link transmissions instead of m ultiplying matrices to ev aluate ˜ F ( t κ ,t 0 κ ; ˜ G ) ab , so computing δ κ for each graph ˜ G only tak es O ( D ). The computational complexities b elow assume that the p dfs can b e represen ted by matrices, and multiplying an ` × m matrix with a m × n matrix tak es O ( `mn ). SIHS p acket delivery pr e dictor c omplexity: Computing α k in (8b) takes O ( D 2 H 2 E ) and computing β k in (8c) takes O ( D 2 E ), since there are 2 E graphs and 2 H pac k et delivery prediction sequences. Computing f ν k + H − 1 k in (8a) takes O ( D 2 H 2 E ). The SIHS pac ket deliv ery predictor up date step is the aggregate of all these computations and takes O ( D 2 H 2 E ). The initialization step of the SIHS pac ket delivery predictor is just computing β 0 in (8d), whic h takes O ( E 2 E ). GEIHS p acket d elivery pr e dictor c omplexity: Computing α h | k − 1 in (11b) takes O ( D 2 E + 2 E 2 h + 2 2 E 2 h ) and computing α 1 | k − 1 in (11c) takes O ( D 2 E + 2 E +1 + 2 2 E +1 ), so computing all of them tak es O ( D H 2 E + 2 H (2 E + 2 2 E )), or just O ( D H 2 E + 2 H 2 2 E ). Computing β k | k − 1 in (11d) tak es O (2 2 E ), and computing β k − 1 | k − 1 in (11e) tak es O ( D 2 E ). Computing f ν k + H − 1 k in (11a) tak es O ( D 2 E + 2 H 2 E ). The GEIHS pac k et deliv ery predictor update step is the aggregate of all these computations and tak es O ( D H 2 E + 2 H 2 2 E ). Computing Γ in (9) takes O ( E 2 2 E ), and computing β 0 |− 1 in (11f) takes O ( E 2 E ). The initialization step of the GEIHS pac k et deliv ery predictor is the aggregate of these computations and tak es O ( E 2 2 E ). If we assume that the deadline D is short enough to be considered constant, we get the computational com- plexities giv en in Prop ert y 1. 9 A goo d direction for future researc h is to find low er complexit y , “suboptimal” net work estimators for our problem setup, and compare them to our “optimal” net w ork estimators. 3.4 Discussion Our net work estimators can easily be extended to incorporate additional observ ations besides past pac ket deliv eries, suc h as the pack et delay and pac ket path traces. The latter can b e obtained by recording the state of the links that the pac k et has tried to tra v erse in the pac k et payload. The function δ k − 1 in (8c) and (11e) just needs to b e replaced with another function that returns 1 if the the received observ ation is consisten t with a netw ork topology ˜ G , and 0 otherwise. The adv an tage of using more observ ations than the one bit of information provided b y a pac ket deliv ery is that it will help the GEIHS netw ork estimator more quic kly detect changes in the netw ork state. A more non-trivial extension of the GEIHS net work estimator w ould use additional observ ations pro vided b y pac kets from other flo ws (not from our controller) to help estimate the netw ork state, whic h could significantly decrease the time for the netw ork estimator to detect a change in the state of the netw ork. This is non-trivial b ecause the net w ork mo del would no w hav e to account for queuing at no des in the netw ork, which is inevitable with m ultiple flo ws. Note that the netw ork state probabilit y distribution, β k ( ˜ G ) in (8c) or β k − 1 | k − 1 ( ˜ G ) in (11e), does not need to conv erge to a probability distribution describing one topology realization to yield precise pac k et predictions f ν k + H − 1 k ( ˜ ν H − 1 0 ), where precise means there is one (or v ery few similar) high probability pack et delivery sequence(s) ˜ ν H − 1 0 . Several topology realizations ˜ G ma y result in the same pac k et deliv ery sequence. Also, note that the GEIHS netw ork estimator p erforms b etter when the links in the netw ork are more bursty . Long bursts of pac ket losses from burst y links result in p o or con trol system performance, which is when the net work estimator w ould help the most. 4 FPD Con troller In this section, we derive the FPD con troller using dynamic programming. Next, w e present tw o controllers for comparison with the FPD controller. These comparativ e controllers assume particular statistical mo dels (e.g., i.i.d. Bernoulli) for the pack et delivery sequence p df which may not describ e the actual p df, while the FPD controller allo ws for all pack et deliv ery sequence p dfs. W e deriv e the LQG cost of using these controllers. Finally , we prese n t the computational complexit y of the optimal con troller. 4.1 Deriv ation of the FPD Con troller W e first presen t the FPD con troller and then presen t its deriv ation. Theorem 4.1. F or a plant with state dynamics given by (1) , the optimal c ontr ol p olicy op er ating on the information set (5) which minimizes the c ost function (6) r esults in an optimal c ontr ol signal u k = − L k x k , wher e L k =  Q 2 + B T S k +1 ( ν k =1 , ν k − 1 0 ) B  − 1 B T S k +1 ( ν k =1 , ν k − 1 0 ) A (12) and S k : { 0 , 1 } k 7→ S ` + and s k : { 0 , 1 } k 7→ R + ar e the solutions to the c ost-to-go at time k , given by S k ( ν k − 1 0 ) = Q 1 + A T E  S k +1 ( ν k 0 ) | ν k − 1 0  A − P ( ν k = 1 | ν k − 1 0 )  A T S k +1 ( ν k =1 , ν k − 1 0 ) B  Q 2 + B T S k +1 ( ν k =1 , ν k − 1 0 ) B  − 1 B T S k +1 ( ν k =1 , ν k − 1 0 ) A  s k ( ν k − 1 0 ) = trace  E  S k +1 ( ν k 0 ) | ν k − 1 0  R w  + E  s k +1 ( ν k 0 ) | ν k − 1 0  . 10 Pr o of. The classical problem in ˚ Astr¨ om [18] is solved b y reformulating the original problem as a recursiv e min- imization of the Bellman equation deriv ed for every time instan t, beginning with N . At time n , w e hav e the minimization problem min u n ,...,u N − 1 E [ x T N Q 0 x N + N − 1 X i = n x T i Q 1 x i + ν i u T i Q 2 u i ] = E " min u n ,...,u N − 1 E [ x T N Q 0 x N + N − 1 X i = n x T i Q 1 x i + ν i u T i Q 2 u i |I C n ] # = E  min u n E [ x T n Q 1 x n + ν n u T n Q 2 u n + V n +1 |I C n ]  , where V n is the Bellman equation at time n . This is given b y V n = min u n E h x T n Q 1 x n + ν n u T n Q 2 u n + V n +1 |I C n i . T o solve the ab ov e nested minimization problem, we assume that the solution to the functional is of the form V n = x T n S n ( ν n − 1 0 ) x n + s n ( ν n − 1 0 ) , where S n and s n are functions of the past pac ket deliveries ν n − 1 0 that return a p ositiv e semidefinite matrix and a scalar, resp ectively . Ho wev er, b oth S n and s n are not functions of the applied con trol sequence u n − 1 0 . W e prov e this supp osition using induction. The initial condition at time N is trivially obtained as V N = x T N Q 0 x N , with S N = Q 0 and s N = 0. W e now assume that the functional at time n + 1 has a solution of the desired form, and attempt to derive this at time n . W e hav e V n = min u n E h x T n Q 1 x n + ν n u T n Q 2 u n + x T n +1 S n +1 ( ν n 0 ) x n +1 + s n +1 ( ν n 0 ) |I C n i = min u n E h x T n  Q 1 + A T S n +1 ( ν n 0 ) A  x n + ν n u T n  Q 2 + B T S n +1 ( ν n 0 ) B  u n + ν n x T n A T S n +1 ( ν n 0 ) B u n + ν n u T n B T S n +1 ( ν n 0 ) Ax n + w T n S n +1 ( ν n 0 ) w n + s n +1 ( ν n 0 ) |I C n i = min u n x T n  Q 1 + A T E  S n +1 ( ν n 0 ) | ν n − 1 0  A  x n + trace  E  S n +1 ( ν n 0 ) | ν n − 1 0  R w  + E [ s n +1 ( ν n 0 ) | ν n − 1 0 ] + P ( ν n = 1 | ν n − 1 0 )  u T n  Q 2 + B T S n +1 ( ν n =1 , ν n − 1 0 ) B  u n + x T n A T S n +1 ( ν n =1 , ν n − 1 0 ) B u n + u T n B T S n +1 ( ν n =1 , ν n − 1 0 ) Ax n  . (13) In the last equation ab o v e, the expectation of the terms preceded b y ν n require the conditional probability P ( ν n = 1 | ν n − 1 0 ) and an ev aluation of S n +1 with ν n = 1. The corresp onding terms with ν n = 0 v anish as they are multiplied b y ν n . The con trol input at sample time n which minimizes the ab ov e expression is found to b e u n = − L n x n , where the optimal control gain L n is given by (12), with k replaced by n . Substituting for u n in the functional V n , w e get a solution to the functional of the desired form, with S n and s n giv en b y S n ( ν n − 1 0 ) = Q 1 + A T E  S n +1 ( ν n 0 ) | ν n − 1 0  A − P ( ν n = 1 | ν n − 1 0 )  A T S n +1 ( ν n =1 , ν n − 1 0 ) B  Q 2 + B T S n +1 ( ν n =1 , ν n − 1 0 ) B  − 1 B T S n +1 ( ν n =1 , ν n − 1 0 ) A  (14a) s n ( ν n − 1 0 ) = trace  E  S n +1 ( ν n 0 ) | ν n − 1 0  R w  + E  s n +1 ( ν n 0 ) | ν n − 1 0  . (14b) Notice that S n and s n are functions of the v ariables ν n − 1 0 . When n = k , the current sample time, these v ariables are kno wn, and S n and s n are not random. But S n + i and s n + i , for v alues of i ∈ { 1 , . . . , N − n − 1 } , are functions 11 of the v ariables ν n + i − 1 0 , of whic h only the v ariables ν n + i − 1 n are random v ariables since they are unknown to the con troller at sample time n = k . Since the v alue of S n +1 is required at sample time n , we compute its conditional exp ectation as E  S n +1 ( ν n 0 ) | ν n − 1 0  = P ( ν n = 1 | ν n − 1 0 ) S n +1 ( ν n =1 , ν n − 1 0 ) + P ( ν n = 0 | ν n − 1 0 ) S n +1 ( ν n =0 , ν n − 1 0 ) . (14c) The ab ov e computation requires an ev aluation of S n + i ( ν n + i − 1 0 ) through a bac kward recursion for i ∈ { 1 , . . . , N − n − 1 } for all combinations of ν N − 2 n + i . More explicitly , the expression at any time n + i , for i ∈ { N − n − 1 , . . . , 1 } , is giv en b y E  S n + i ( ν n + i − 1 0 ) | ν n − 1 0  = Q 1 + A T E  S n + i +1 ( ν n + i 0 ) | ν n − 1 0  A − X ˜ ν i − 1 0 ∈{ 0 , 1 } i P  ν n + i = 1 , ν n + i − 1 n = ˜ ν i − 1 0 | ν n − 1 0  × A T S n + i +1 ( ν n + i =1 , ν n + i − 1 n = ˜ ν i − 1 0 , ν n − 1 0 ) B ×  Q 2 + B T S n + i +1 ( ν n + i =1 , ν n + i − 1 n = ˜ ν i − 1 0 , ν n − 1 0 ) B  − 1 × B T S n + i +1 ( ν n + i =1 , ν n + i − 1 n = ˜ ν i − 1 0 , ν n − 1 0 ) A E  s n + i ( ν n + i − 1 0 ) | ν n − 1 0  = trace  E  S n + i +1 ( ν n + i 0 ) | ν n − 1 0  R w  + E  s n + i +1 ( ν n + i 0 ) | ν n − 1 0  . Using the ab o v e expressions, w e obtain the net cost to b e J = trace S 0 R 0 + N − 1 X n =0 trace  E [ S n +1 ( ν n 0 ) ] R w  . (15) Notice that the control inputs u n are only applied to the plant and do not influence the netw ork or ν N − 1 0 . Thus, the arc hitecture separates in to a net w ork estimator and con troller, as sho wn in Figure 2. 4.2 Comparativ e con trollers In this section, w e compare the performance of the FPD con troller to t w o controllers that assume particular statistical mo dels for the pac k et deliv ery sequence p df, the I ID controller and the ON controller. IID Contr ol ler: The IID controller w as describ ed in Sc henato et al. [7] and assumes that the pack et deliveries are i.i.d. Bernoulli with pack et delivery probability equal to the a priori probabilit y of deliv ering a pac ket through the net w ork. 7 This is our first comparativ e con troller, where u k = − L IID k x k and the con trol gain L IID k is giv en b y L IID k =  Q 2 + B T S IID k +1 B  − 1 B T S IID k +1 A . Here, S IID k +1 is the solution to the Riccati equation for the control problem where the pack et deliv eries are assumed to b e i.i.d. Bernoulli. The bac kward recursion is initialized to S IID N = Q 0 and is giv en b y S IID k = Q 1 + A T S IID k +1 A − P ( ν k = 1) A T S IID k +1 B  Q 2 + B T S IID k +1 B  − 1 B T S IID k +1 A . ON Contr ol ler: The ON con troller assumes that the pack ets are alwa ys delivered, or that the netw ork is alwa ys online. This is our second comparative con troller, where u k = − L ON k x k and the con trol gain L ON k is giv en b y L ON k =  Q 2 + B T S ON k +1 B  − 1 B T S ON k +1 A . 7 Using the stationary probability of eac h link under the G-E link mo del to calculate the end-to-end probability of delivering a pac ket through the network. 12 Here, S ON k +1 is the solution to the Riccati equation for the classical con trol problem whic h assumes no pac ket losses on the actuation c hannel. The backw ard recursion is initialized to S ON N = Q 0 and is giv en b y S ON k = Q 1 + A T S ON k +1 A − A T S ON k +1 B  Q 2 + B T S ON k +1 B  − 1 B T S ON k +1 A . Comp ar ative Cost: The FPD controller is the most general form of a causal, optimal LQG controller that takes in to account the pack et deliv ery sequence p df. It do es not assume the pack et delivery sequence p df comes from a particular statistical mo del. Approximating the actual pack et delivery sequence p df with a p df describ ed by a particular statistical mo del, and then computing the optimal control policy , will result in a sub optimal controller. Ho w ever, it may be less computationally exp ensive to obtain the con trol gains for such a suboptimal controller. F or example, the IID controller and the ON con troller are suboptimal con trollers for netw orks like the one describ ed in Section 2.1, since they presume a statistical mo del that is mismatched to the pack et delivery sequence p df obtained from the net w ork mo del. Remark The a verage LQG cost of using a con troller with control gain L comp n is J = trace S sopt 0 R 0 + N − 1 X n =0 trace  E [ S sopt n +1 ( ν n 0 ) ] R w  , (16a) where S sopt n ( ν n − 1 0 ) = Q 1 + A T E  S sopt n +1 ( ν n 0 ) | ν n − 1 0  A + P ( ν n = 1 | ν n − 1 0 ) ×  L comp T n  Q 2 + B T S sopt n +1 ( ν n =1 , ν n − 1 0 ) B  L comp n − A T S sopt n +1 ( ν n =1 , ν n − 1 0 ) B L comp n − L comp T n B T S sopt n +1 ( ν n =1 , ν n − 1 0 ) A  , (16b) and E  S sopt n +1 ( ν n 0 ) | ν n − 1 0  is computed in a similar manner to (14c). The con trol gain L comp n can be the gain of a comparativ e con troller (e.g., L IID n or L ON n ) where the statistical mo del for the pac ket delivery sequence is mismatched to the actual mo del. This can b e seen from the pro of of Theorem 4.1, if we substitute for the control input with u sopt n = − L comp n x n in (13), instead of minimizing the expression to find the optimal u n . On simplifying, w e get the solution to the cost-to-go V n of the form x T n S sopt n ( ν n − 1 0 ) x n + s sopt n ( ν n − 1 0 ) , with S sopt n giv en b y (16b) and s sopt n giv en b y s sopt n ( ν n − 1 0 ) = trace  E  S sopt n +1 ( ν n 0 ) | ν n − 1 0  R w  + E  s sopt n +1 ( ν n 0 ) | ν n − 1 0  . 4.3 Algorithm to Compute Optimal Con trol Gain A t sample time k , we ha ve ν k − 1 0 . T o compute L k giv en in (12), w e need S k +1 ( ν k =1 , ν k − 1 0 ) , which can only b e obtained through a backw ard recursion from S N . This requires knowledge of ν N − 1 k , whic h are una v ailable at sample time k . Th us, we must ev aluate { S k +1 , . . . , S N } for every p ossible sequence of arriv als ν N − 1 k . This algorithm is describ ed b elo w. 1. Initialization: S N ( ν N − 1 k = ˜ ν N − k − 1 0 , ν k − 1 0 ) = Q 0 , ∀ ˜ ν N − k − 1 0 ∈ { 0 , 1 } N − k . 2. for n = N − 1 : − 1 : k + 1 a) Using (14c), compute E  S n +1 ( ν n 0 ) | ν k − 1 0 , ˜ ν n − k − 1 0  , ∀ ˜ ν n − k − 1 0 ∈ { 0 , 1 } n − k . b) Using (14a), compute S n ( ν n − 1 k = ˜ ν n − k − 1 0 , ν k − 1 0 ) , ∀ ˜ ν n − k − 1 0 ∈ { 0 , 1 } n − k . 3. Compute L k using S k +1 ( ν k =1 , ν k − 1 0 ) . F or k = 0, the v alues S 0 , E [ S 1 ( ν 0 ) ], and the other v alues obtained abov e can be used to ev aluate the cost function according to (15). 13 4.4 Computational Complexity of Optimal Con trol Gain The FPD con troller is optimal but computationally exp ensive, as it requires an enumeration of all possible pack et deliv ery sequences from the curren t sample time un til the end of the con trol horizon to calculate the optimal con trol gain (12) at ev ery sample time k . Prop ert y 2. The algorithm pr esente d in Se ction 4.3 for c omputing the optimal c ontr ol gain for the FPD c ontr ol ler takes O ( q 3 ( N − k )2 N − k ) op er ations at e ach sample time k , wher e q = max( `, m ) and ` and m ar e the dimensions of the state and c ontr ol ve ctors. Pr o of. The computational complexities b elow assume that multiplying an ` × m matrix with a m × n matrix tak es O ( `mn ), and that in v erting an ` × ` matrix tak es O ( ` 3 ). F or the computation of L k in (12), we need to run the algorithm presented in Section 4.3. The steps within the for-loop (Step 2) of the algorithm require matrix m ultiplications and inv ersions that tak e O ((2 ` 3 + 6 ` 2 m + 2 ` 2 + 2 `m 2 + m 3 + m 2 )2 N − k ) op erations, or O ( q 3 2 N − k ) op erations if we let q = max( `, m ). This m ust be rep eated N − k − 1 times in the for-loop, so Step 2 takes O ( q 3 ( N − k − 1)2 N − k ). Finally , Step 3 takes O (4 ` 2 m + `m 2 + m 3 + m 2 ) op erations for the matrix multiplications and in versions, whic h simplifies to O ( q 3 ). Combining these results and simplifying yields the computational complexity given in Prop ert y 2. F or the SIHS netw ork mo del, once the netw ork state estimates from the SIHS netw ork estimator conv erge, the conditional probabilities f ν k + H − 1 k will not c hange and the computations can b e reused. But, for a net work that ev olv es ov er time, like the GEIHS netw ork mo del, the computations cannot b e reused, and the computational cost remains high. 5 Examples and Sim ulations Using the system arc hitecture depicted in Figure 2, we will demonstrate the GEIHS netw ork estimator on a small mesh netw ork and use the pack et deliv ery predictions in our FPD con troller. Figure 6 depicts the routing topology and short rep eating schedule of the netw ork. P ac kets are generated at the source every 409 time slots, 8 and the pac ket delivery deadline is t 0 k − t k = 9 , ∀ k . The netw ork estimator assumes all links ha ve p u = 0 . 0135 and p d = 0 . 0015. The pac ket delivery predictions from the netw ork estimator are shown in Figure 7. Although the netw ork estimator pro vides f ν k + H − 1 k ( ˜ ν H − 1 0 ), at each sample time k we plot the a verage prediction E [ ν k + H − 1 k ]. In this example, all the links are up for k ∈ { 1 , . . . , 4 } and then link (3 , b ) fails from k = 5 on wards. After seeing a pac ket loss at k = 5, the netw ork estimator revises its pack et delivery predictions and now thinks there will b e a pack et loss at k = 7. The av erage prediction for the pack et delivery at a particular sample time tends to ward 1 or 0 as the net w ork estimator receiv es more information (in the form of pac ket deliveries) ab out the new state of the netw ork. The prediction for k = 7 (pack et generated at schedule time slot 3) at k = 5 is influenced by the pack et delivery at k = 5 (pack et generated at schedule time slot 1) b ecause hop-by-hop routing allows the pack ets to trav erse the same links under some realizations of the underlying routing top ology G . Mesh netw orks with many interlea ved paths allo w pack ets generated at different schedule time slots to pro vide information ab out each others’ deliveries, pro vided the links in the netw ork hav e some memory . As discussed in Section 3.4, since a pac ket delivery pro vides only one bit of information ab out the netw ork state, it may take sev eral pack et deliveries to get go o d predictions after the net w ork c hanges. 8 Effectively , the pack ets are generated every 9 + 4 K time slots, where K is a very large integer, so we can assume slow system dynamics with resp ect to time slots and ignore the dela y introduced by the netw ork. 14 Routing T op ology Sc hedule Figure 6: Example of a simple mesh netw ork for netw ork estimation. Figure 7: Pac ket deliv ery predictions when netw ork in Figure 6 has all links up and then link (3 , b ) fails. 15 Figure 8: Plot of the different control signals and state vectors when using the FPD controller, an I ID controller, and an ON con troller (see text for details). No w, consider a linear plan t with the follo wing parameters A =  0 1 . 5 1 . 5 0  , B =  5 0 0 0 . 2  , R w =  0 . 1 0 0 0 . 1  , R 0 =  10 0 0 10  Q 1 = Q 2 =  1 0 0 1  , Q n =  10 0 0 10  . The transfer matrix A flips and expands the components of the state at every sampling instan t. The input matrix B requires the second comp onent of the con trol input to b e larger in magnitude than the first comp onent to ha v e the same effect on the resp ective comp onent of the state. Also, the final state is w eighted more than the other states in the cost criterion. W e compare the three finite horizon LQG controllers discussed in Section 4, namely the FPD con troller, the I ID con troller, and the ON con troller with their costs (15) and (16a). The controllers are connected to the plant at sample times k = 9 , 10 , 11 through the netw ork example giv en in Figure 7. Figure 8 shows the con trol signals computed b y the differen t controllers and the plan t states when the con trol signals are applied following the actual pack et delivery sequence. F rom the predictions at k = 8 , 9 , 10 in Figure 7, we see that the FPD controller has b etter knowledge of the pack et delivery sequence than the other t w o con trollers. The FPD con troller uses this knowledge to compute an optimal control signal that outputs a large magnitude for the second component of u 10 , despite the high cost of this signal. The IID and ON con trollers believe the control pack et is likely to b e delivered at k = 11 and choose, instead, to output a smaller increase in the first comp onen t of u 11 , since this will ha ve the same effect on the final state if the con trol pack et at k = 11 is successfully deliv ered. The FPD con troller is b etter than the other controllers at driving the first comp onent of the state close to zero at the end of the control horizon, k = 12. Th us, the pack et delivery predictions from the netw ork estimator help the FPD controller significan tly low er its LQG cost, as shown in T able 1. The costs rep orted here are obtained from Mon te-Carlo simulations of the system, av eraged ov er 10,000 runs, but with the netw ork state set to the one describ ed ab o v e. 16 T able 1: Simulated LQG Costs (10,000 runs) for Example Described in Section 5 FPD Con troller I ID Con troller ON Con troller 681.68 1,008.2 1,158.9 6 Discussion on Net work Mo del Selection The ability of the netw ork estimator to accurately predict pack et deliveries is dep endent on the net work mo del. A natural ob jection to the GEIHS netw ork mo del is that it assumes links are indep endent and do es not capture the full behavior of a lossy and burst y wireless link through the G-E link mo del [15]. Why not use one of the more sophisticated link mo dels mentioned by Willig et al. [15]? Wh y not use a netw ork mo del that can capture correlation b etw een the links in the netw ork? A go o d netw ork mo del must be rich enough to capture the relev ant b eha vior of the actual net w ork, but not ha v e to o man y parameters that are difficult to obtain. In our problem setup, the relev ant b ehavior is the pac k et delivery sequence of the netw ork. As mentioned in Section 3.4, the net work state probabilit y distribution do es not need to iden tify the exact netw ork top ology realization to get precise pac ket delivery predictions. In this regard, the GEIHS net work mo del has to o man y states (2 E states) and may b e ov ermo deling the actual netw ork. Ho wev er, the more relev ant question is: Do es the GEIHS netw ork mo del yield accurate pack et delivery predictions, predictions that are close to the actual future pac k et delivery sequence? Do the simplifications from assuming link indep endence and using a G-E link model result in inaccurate pack et delivery predictions? These questions need further inv estigation, in volving real-world exp erimen ts. Our GEIHS net work mo del has as parameters the routing topology G , the sc hedule F ( T ) , the G-E link transition probabilities { p u l , p d l } l ∈E , the source a , the sink b , the pack et generation times t k , and the deadlines t 0 k . The most difficult parameters to obtain are the link transition probabilities, whic h m ust be estimated b y link estimators running on the no des and rela y ed to the GEIHS netw ork estimator. F urthermore, on a real netw ork these parameters will c hange o v er time, alb eit at a slow er time scale than the link state changes. The issue of how to obtain these parameters is not addressed in this pap er. Despite its limitations, the GEIHS netw ork mo del is a go o d basis for comparisons when other netw ork mo dels for our problem setup are prop osed in the future. It also raises several related researc h questions and issues. Are there classes of routing topologies where pack et delivery statistics are less sensitive to the parameters in our G-E link mo del p u l and p d l ? How do we build netw orks (e.g., select routing top ologies and schedules) that are “robust” to link mo deling error and provide go o d pack et deliv ery statistics (e.g., low pack et loss, low delay) for NCSs? The latter half of the question, building netw orks with go o d pack et delivery statistics, is partially addressed b y other w orks in the literature lik e Soldati et al. [19], which studies the problem of scheduling a netw ork to optimize reliabilit y giv en a routing top ology and pac k et deliv ery deadline. Another issue arises when we use a controller that reacts to estimates of the netw ork’s state. In our problem setup, if the net work estimator giv es wrong (inaccurate) pac ket delivery predictions, the FPD con troller can actually p erform worse than the ON controller. Ho w do we design FPD controllers that are robust to inaccurate pac ket deliv ery predictions? 7 Conclusions This paper proposes t wo net work estimators based on simple net w ork mo dels to characterize wireless mesh netw orks for NCSs. The goal is to obtain a b etter abstraction of the net work, and in terface to the netw ork, to presen t to the con troller and (future work) netw ork manager. T o get b etter p erformance in a NCS, the net w ork manager needs to c ontr ol and r e c onfigur e the net work to reduce outages and the con troller needs to r e act to or c omp ensate for the net w ork when there are una voidable outages. W e studied a sp ecific NCS arc hitecture where the actuation channel 17 w as ov er a lossy wireless mesh net w ork and a net work estimator provided pack et delivery predictions for a finite horizon, F uture-P ac ket-Deliv ery-optimized LQG controller. There are sev eral directions for extending the basic problem setup in this pap er, including those mentioned in Sections 3.3, 3.4, and 6. F or instance, placing the netw ork estimator(s) on the actuators in the general system arc hitecture depicted in Figure 1 is a more realistic setup but will introduce a lossy channel b etw een the netw ork estimator(s) and the controller(s). Also, one can study the use of pac ket delivery predictions in a receding horizon con troller rather than a finite horizon con troller. References [1] Wireless Industrial Net working Alliance, “WINA w ebsite,” http://www.wina.org, 2010. [2] In ternational So ciety of Automation, “ISA-SP100 wireless systems for automation website,” h ttp://www.isa. org/isa100, 2010. [3] I. Chlamtac, M. Con ti, and J. J. N. Liu, “Mobile ad hoc netw orking: Imp eratives and challenges,” A d Ho c Networks , v ol. 1, no. 1, pp. 13–64, 2003. [4] R. Bruno, M. Conti, and E. Gregori, “Mesh netw orks: commo dity m ultihop ad ho c net works,” IEEE Commu- nic ations Magazine , v ol. 43, no. 3, pp. 123–131, Mar. 2005. [5] J. P . Hespanha, P . Naghshtabrizi, and Y. Xu, “A survey of recent results in netw orked control systems,” Pr o c. of the IEEE , vol. 95, pp. 138–162, 2007. [6] C. Robinson and P . Kumar, “Control o v er netw orks of unreliable links: Con troller lo cation and p erformance b ounds,” in Pr o c. of the 5th International Symp osium on Mo deling and Optimization in Mobile, A d Ho c, and Wir eless Networks (WiOpt) , Apr. 2007, pp. 1–8. [7] L. Sc henato, B. Sinop oli, M. F ranceschetti, K. P o olla, and S. S. Sastry , “F oundations of control and estimation o v er lossy net w orks,” Pr o c. of the IEEE , v ol. 95, pp. 163–187, 2007. [8] H. Ishii, “ H ∞ con trol with limited comm unication and message losse s,” Systems & Contr ol L etters , v ol. 57, no. 4, pp. 322–331, 2008. [9] P . Seiler and R. Sengupta, “An H ∞ approac h to net work ed con trol,” IEEE T r ansactions on A utomatic Contr ol , v ol. 50, pp. 356–364, 2005. [10] N. Elia, “Remote stabilization ov er fading channels,” Systems & Contr ol L etters , vol. 54, no. 3, pp. 237–249, 2005. [11] R. Olfati-Saber, J. F ax, and R. Murray , “Consensus and co op eration in netw orked m ulti-agent systems,” Pr o c. of the IEEE , vol. 95, no. 1, pp. 215–233, Jan. 2007. [12] V. Gupta, A. Dana, J. Hespanha, R. Murray , and B. Hassibi, “Data transmission ov er netw orks for estimation and con trol,” IEEE T r ans. on Automatic Contr ol , vol. 54, no. 8, pp. 1807–1819, Aug. 2009. [13] D. V aragnolo, P . Chen, L. Sc henato, and S. Sastry , “Performance analysis of differen t routing proto cols in wireless sensor netw orks for real-time estimation,” in Pr o c. of the 47th IEEE Confer enc e on De cision and Contr ol , Dec. 2008. [14] K. S. J. Pister and L. Doherty , “TSMP: Time synchronized mesh proto col,” in Pr o c. of the IASTED Interna- tional Symp osium on Distribute d Sensor Networks (DSN) , Nov. 2008. 18 [15] A. Willig, M. Kubisc h, C. Ho ene, and A. W olisz, “Measuremen ts of a wireless link in an industrial en vironment using an ieee 802.11-compliant ph ysical lay er,” IEEE T r ans. on Industrial Ele ctr onics , vol. 49, no. 6, pp. 1265– 1282, Dec. 2002. [16] P . Smyth, D. Heck erman, and M. I. Jordan, “Probabilistic indep endence netw orks for hidden marko v proba- bilit y mo dels,” Neur al Comput. , vol. 9, no. 2, pp. 227–269, 1997. [17] R. Castro, M. Coates, G. Liang, R. No wak, and B. Y u, “Net work tomography: Recen t developmen ts,” Statis- tic al Scienc e , vol. 19, no. 3, pp. 499–517, Aug. 2004. [18] K. J. ˚ Astr¨ om, Intr o duction to Sto chastic Contr ol The ory . Academic Press, 1970, republished by Do v er Publications, 2006. [19] P . Soldati, H. Zhang, Z. Zou, and M. Johansson, “Optimal routing and sc heduling of deadline-constrained traffic ov er lossy netw orks,” in Pr o c. of the IEEE Glob al T ele c ommunic ations Confer enc e , Miami FL, USA, Dec. 2010. 19