Network Congestion Control with Markovian Multipath Routing

Mathematical Programming man uscript No. (will b e inserted b y the editor) Net w ork Congestion Con trol with Mark o vian Multipath Routing ? Rob erto Cominetti · Crist´ obal Guzm´ an Received: date / Accepted: date Abstract In this paper w e consider an in tegrated mo del for TCP/IP proto cols with m ultipath routing. The mo del combines a Netw ork Utility Maximization for rate con trol based on end-to-end queueing dela ys, with a Marko vian T raffic Equilibrium for routing based on total expected dela ys. W e prov e the existence of a unique equilibrium state which is c haracterized as the solution of an unconstrained strictly con vex program. A distributed algorithm for solving this optimization problem is prop osed, with a brief discussion of how it could b e implemen ted by adapting the current In ternet proto cols. Keyw ords Netw ork optimization · Congestion control · Multipath routing · Cross-la yer design 1 Introduction Routing and congestion control are t w o basic comp onents of pac k et-switc hed comm unication net w orks. While r outing is resp onsible for determining efficien t ? A preliminary short version of this paper was published in the Pro ceedings of the 5th International Conference on Network Games, Control and Optimization (NetGCOOP’2011, Paris). Partially supp orted by FONDECYT 1100046 and Instituto Milenio Sistemas Complejos de Ingenier ´ ıa. R. Cominetti Departamento de Ingenier ´ ıa Industrial, Universidad de Chile, Rep ´ ublica 701, Santiago, 8370439 Chile E-mail: rccc@dii.uchile.cl C. Guzm´ an School of Industrial and Systems Engineering Georgia Institute of T echnology , 765 F erst Driv e, NW, Atlan ta, GA 30332–0250, USA E-mail: cguzman@gatech.edu 2 Roberto Cominetti, Crist´ obal Guzm´ an paths along whic h the sources comm unicate to their corresponding receiv ers, c ongestion c ontr ol manages the transmission rate of each source in order to k eep net work congestion within reasonable limits. In current practice b oth mec hanisms b elong to separate design lay ers that op erate on different time- scales: the IP lay er (Internet Proto col) determines single-path routings which are up dated on a slow time-scale, while the TCP lay er (T ransmission Control Proto col) corresp onds to end-to-end users that perform rate congestion control at a faster pace for which routing can b e considered to b e fixed. Scalability considerations imp ose that these protocols must op erate in a decentralized manner. Roughly sp eaking, TCP controls the rate of a source b y managing a window size that b ounds the maximu m num b er of outstanding pack ets that hav e b een transmitted but not y et ackno wledged by the receiver. Once this window size is reached the source m ust w ait for an ackno wledgment b efore sending a new pac ket, so the rate is appro ximately one window of pack ets p er round-trip time. As the netw ork gets congested, the round-trip time increases and the transmission rate is automatically slow ed do wn. In addition, TCP d ynamically adjusts the window size of a source in resp onse to net work congestion. T o this end, links generate a scalar measure of their own congestion (e.g. pack et loss probabilit y , av erage queue length, queueing delay) and each source is fed back a c ongestion signal that reflects the aggregate congestion of the links along its route. This signal is used by the source to adjust its window size so that the higher the congestion the smaller the rate. The predominant TCP proto cols in use are T aho e and Reno which use p acket loss as congestion measure, and V egas which is based on queueing delay . W e refer to [21] for a description and comparison of curren t proto cols and their mo dels. The interaction of many sources p erforming a decentralized congestion con trol based on feedback signals that are sub ject to estimation errors and comm unication delays, gives rise to complex dynamics that are difficult to an- alyze. How ever, assuming that the dynamics stabilize on a steady state, the equilibrium can b e characterized as an optimal solution of a Netw ork Utilit y Maximization (NUM) problem (see [14, 20, 33]). Th us, the TCP mec hanism can b e viewed as a decentralized algorithm that seeks to optimize an aggre- gate utility function sub ject to net work constraints. The NUM approac h is also useful to compare different proto cols in regard to their fairness and efficiency . A second elemen t of pack et-switched netw orks is r outing . This function is p erformed by routers in a decentralized manner using routing tables that determine the next hop for eac h destination. The routing tables are up dated p erio dically by an async hronous distributed shortest path iteration that com- putes optimal paths according to some metric suc h as hop coun t, latency , dela y , load, reliabilit y , bandwidth, or a mixture of these. In current practice a single path that minimizes the num b er of hops is used for routing on each origin-destination pair. A very promising idea in traffic engineering is the use of multipath routing to increase the throughput by exploiting the av ailable transmission capacity on a set of alternativ e paths. This also impro ves the reliabilit y b ecause of the ability to redirect flow into alternate paths in case Netw ork Congestion Control with Marko vian Multipath Routing ? 3 of failures. While a few multipath techniques are av ailable in to day’s Inter- net ( e.g. MPLS tunnels [29] or MPTCP multihoming developed b y IETF [3]), sev eral other prop osals hav e b een tested through simulations. W e defer our discussion of the relev an t literature until section § 5. Another relev ant argu- men t in fav or of multipath routing is stability . Indeed, when route choice is based on metrics that are affected by congestion, such as queueing delay or link latencies, routing and rate control b ecome mutually inter-dependent and equilibrium can b e achiev ed only if b oth asp ects are considered jointly: rout- ing affects the rate con trol through the induced congestion signals, while rate con trol induces flows that determine in turn whic h routes are optimal. If rout- ing is restricted to a single path, congestion effects may lead to route flaps. A remedy for suc h unstable b ehavior is to allow flows to split ov er multiple paths in order to balance their loads. An appropriate to ol to capture these interac- tions b etw een rate control and routing is provided by W ardrop equilibrium. On the other hand, since congestion metrics are sub ject to estimation errors and random effects it is natural to mo del routing as a sto chastic equilibrium assignmen t. The goal of this pap er is to prop ose an analytical framework that pro vides a theoretical supp ort for cross-lay er designs for rate control under multipath routing. Our mo del com bines rate control mo deled by NUM with a routing strategy based on discrete choice distribution mo dels that lead to a Mark ovian T raffic Equilibrium (MTE). The latter is a decentralized sto chastic v ersion of W ardrop’s mo del. The com bination of the NUM and MTE mo dels leads to a system of e quations that corresp ond to the optimality conditions of an equiv- alen t Marko vian Netw ork Utility Maximization problem (MNUM), a strictly con vex unconstrained program of lo w dimension where the v ariables are the link congestion prices. This c haracterization allows us to establish the exis- tence and uniqueness of an equilibrium, and pro vides a basis for designing decen tralized proto cols for congestion control and multipath routing. The pap er is structured as follows. Section § 2 reviews the basic comp o- nen ts of our cross-lay er approach: we recall the NUM framework for mo deling the steady state of TCP proto cols and we discuss the concepts of W ardrop equilibrium and Marko vian routing. Section § 3 combines NUM and MTE, in- tro ducing the MNUM mo del for routing and rate control. In § 3.1 we reduce MNUM to a system of equations inv olving only the link congestion prices, and then in § 3.2 w e show that these equations admit a v ariational characteriza- tion (d-mnum) proving the existence of a unique equilibrium state. In § 4 we briefly discuss ho w the mo del migh t lead to a cross-la yer design of a distributed TCP/IP proto col. W e close the pap er with comparisons to previous work and some p ersp ectiv es on future research. 2 Notations and preliminaries The comm unication net w ork is modeled b y a directed graph G = ( N , A ), where the nodes i ∈ N represen t origins, destinations and in termediate routers, while 4 Roberto Cominetti, Crist´ obal Guzm´ an the arcs a ∈ A represent the netw ork links. Each link is characterized by a la- tency function λ a = s a ( w a ) = λ 0 a + ρ a ( w a ) where λ 0 a ≥ 0 represen ts a constan t propagation delay and ρ a ( w a ) is the exp ected queueing delay expressed as a con tinuous and strictly increasing function ρ a : [0 , c a ) → [0 , ∞ ) of the traffic w a on the link, with ρ a (0) = 0 and c a ∈ (0 , ∞ ] the maximal capacity . W e also consider a finite set of sources k ∈ K each one generating a flow rate x k ≥ 0 from an origin s k ∈ N to a destination d k ∈ N . 2.1 Rate con trol and utility maximization under single path routing Supp ose that each source k ∈ K routes its flo w along a fixed sequence of links ( a 1 , . . . , a j k ), so that the total traffic on a link a is w a = P k 3 a x k where the summation is ov er all the sources k ∈ K whose route contains that link. Consider the queueing delay p a = ρ a ( w a ) as a measure of link congestion and assume that eac h source k ∈ K adjusts its rate x k = f k ( q k ) as a function of the aggregate queueing q k = P a ∈ k p a on its route, where f k : (0 , ∞ ) → (0 , ∞ ) is con tinuous and strictly decreasing with f k ( q k ) → 0 as q k → ∞ . These equilibrium equations ma y b e written as f − 1 k ( x k ) = q k = X a ∈ k p a = X a ∈ k ρ a ( w a ) = X a ∈ k ρ a ( P s 3 a x s ) whic h correspond to the optimalit y conditions for the strictly conv ex program (num) min x ∈ R K X a ∈ A R a ( P s 3 a x s ) − X k ∈ K Z x k 0 f − 1 k ( z ) dz where R a ( · ) denotes a primitiv e of ρ a ( · ). Alternatively , the equations may b e stated in terms of the queueing dela ys as ρ − 1 a ( p a ) = w a = X k 3 a x k = X k 3 a f k ( q k ) = X k 3 a f k ( P b ∈ k p b ) whic h are the optimality conditions for the strictly conv ex dual program (d-num) min p ∈ R A X a ∈ A Z p a 0 ρ − 1 a ( y ) dy − X k ∈ K F k ( P b ∈ k p b ) where F k ( · ) is a primitiv e of f k ( · ). Example. Consider the mo del for TCP V egas prop osed in [20, 22]. F or each source k and time t , let W k t denote the size of the congestion window and T k t = D k + q k t the R TT expressed as the sum of the total propagation delay D k and the queueing delay q k t . A V egas source estimates D k as the minimum observ ed R TT, and tries to keep the difference betw een the exp e cte d r ate ˆ x k t = W k t /D k and the actual r ate x k t = W k t /T k t close to a given v alue α k > 0. T o this end, the congestion window is increased if ˆ x k t − x k t < α k , and decreased Netw ork Congestion Control with Marko vian Multipath Routing ? 5 when ˆ x k t − x k t > α k . At equilibrium we must hav e ˆ x k − x k = α k whic h yields the equilibrium rate functions x k = α k D k q k , f k ( q k ) . A simple model for queueing delay can be obtained by considering eac h link as an M/M/1 queue with service rate c a > 0 and an infinite buffer, whic h gives the exp ected queueing dela y p a = w a c a ( c a − w a ) , ρ a ( w a ) . The (num) formalism can handle other congestion measures b esides queueing dela y and has b een used to mo del the steady state of different TCP protocols, eac h one characterized by sp ecific maps f k and ρ a (see [7, 9, 14, 17, 21, 24]). 2.2 Routing and traffic equilibrium W e review next some equilibrium mo dels for traffic routing in congested net- w orks. In this setting the source flow rates x k are fixed but may b e routed along a set of alternative paths R k connecting the origin s k to the destination d k . The basic mo deling principle introduced b y W ardrop in [31] is that at equilibrium only paths that are optimal should b e used to route flow. In contrast with rate con trol which uses queueing dela y p a = ρ a ( w a ), route optimalit y will b e measured using the total dela ys λ a = λ 0 a + ρ a ( w a ) so that pac kets are routed along paths with smaller round trip times and not only small queuing delays. The rationale is that the earlier eac h pack et is deliv ered, the larger the rate. The proto col should automatically select the most efficient routes, dep ending on the congestion prev ailing on each link. A further adv an- tage of choosing the currently shortest path using total delay is to ensure that pac kets arrive in order to their destination, reducing the conflicts with the duplicate ack mec hanism for detecting pack et losses in TCP . Example. T o illustrate the p oin t, consider tw o parallel links with iden tical queuing capacit y but one of them with m uc h longer propagation delay . A routing based on queuing delay alone w ould yield a 50% traffic split, with ack dela ys dominated by the slo w link which unnecessarily limits the transmission rate (the fast link b eing under-utilized). Instead, a routing based on total dela y will use the fast link more intensiv ely until increased queuing makes the slow link comp etitive, achieving a higher throughput. Naturally , the fast link will ha ve a larger queue as compared to the slo w link, but not larger than in single- path routing as long as TCP is still controlling the amount of traffic using the V egas mechanism. Even tually , a very slow link will not b e used at all, which is again consisten t with supp orting higher rates. 6 Roberto Cominetti, Crist´ obal Guzm´ an 2.2.1 War dr op e quilibrium Supp ose that the flow x k is split into non-negative path-flows h r ≥ 0 so that x k = P r ∈ R k h r , and let w a = P r 3 a h r b e the induced total link-flows. Let H denote the set of suc h fe asible flows ( h, w ). An equilibrium [31] is c haracterized b y the fact that only optimal paths are used, namely , for eac h destination k ∈ K and each route r ∈ R k one has h r > 0 ⇒ c r = τ k (1) where c r = P a ∈ r λ a = P a ∈ r s a ( w a ) denotes the total delay of the route and τ k = min r ∈ R k c r is the minim um cost faced by source k . These equilibria w ere characterized in [4] as the optimal solutions of the con vex program (p-w) min ( h,w ) ∈ H X a ∈ A Z w a 0 s a ( z ) dz . Since the feasible set H is compact this problem has optimal solutions, while strict conv exity implies that the optimal w is unique. Alternatively , the equi- librium delays λ a = s a ( w a ) are the unique optimal solution of the strictly con vex unconstrained dual problem (d-w) min λ ∈ R A X a ∈ A Z λ a λ 0 a s − 1 a ( z ) dz − X k ∈ K x k τ k ( λ ) where λ 0 a = s a (0) and τ k ( λ ) , min r ∈ R k P a ∈ r λ a is the minimum total delay for source k ∈ K . Remark. The well known Braess’ Parado x indicates that there are situations in which forbidding flow on some links migh t lead to a mo dified equilibrium where all sources b enefit from smaller trav el times. This raises the relev ant question of which links should b e forbidden to optimize netw ork p erformance. Unfortunately , as sho wn in [27] this problem turns out to b e hard to solve even appro ximately and ev en for a single source. Namely , unless P=NP , there is no p olynomial approximation algorithm with approximation ratio less than n/ 2 where n the n umber of no des, while the optimal ratio n/ 2 is trivially attained b y forbidding no link. As a consequence, harmful links cannot be detected efficien tly . T o compensate, it is w orth men tioning that for sufficien tly high lev els of demand and congestion, Braess’ Parado x do es not o ccur (see [23]). 2.2.2 Markovian r outing and e quilibrium When link delays are sub ject to sto c hastic v ariability , the route delays ˜ c r b ecome random v ariables and the equilibrium conditions (1) are replaced by a sto chastic assignment of the form h r = x k P (˜ c r is optimal). F or instance, if Netw ork Congestion Control with Marko vian Multipath Routing ? 7 s k i j ˜ λ b ˜ λ a ˜ z k a d k τ k j Fig. 1 V ariables for dynamic programming equations the costs ˜ c r are i.i.d. Gumbel v ariables with exp ected v alue c r = E (˜ c r ), we get the Logit distribution rule common in the transp ortation literature h r = x k exp( − β c r ) P p ∈ R k exp( − β c p ) ( ∀ r ∈ R k ) whic h assigns flow to all the paths, fav oring those with smaller exp ected cost c r . The parameter β controls how concentrated is the repartition: for β ∼ 0 ev ery path receives an approximately equal share of the flow, while for β large the flow concentrates on paths with minimal cost. Unfortunately , given the exp onen tial num b er of end-to-end paths, such route-based distribution rules con trolled directly by sources do not seem amenable to design decentralized scalable routing proto cols. This b ecomes critical if the proto col is exp ected to b e resp onsiv e when facing route delays that v ary with traffic congestion. An alternative is to conceive routing as a sto chastic dynamic program- ming pro cess. Supp ose that each pack et exp eriences a random delay ˜ λ a when tra versing link a , and let ˜ τ k i b e a random v ariable that represents the total dela y from no de i to destination d k . Denote λ a = E ( ˜ λ a ) and τ k i = E ( ˜ τ k i ) their exp ected v alues. If a pack et at no de i is routed through the link a ∈ A + i w e ha ve ˜ τ k i = ˜ λ a + ˜ τ k j a , so that a shortest path routing should choose the link with smallest ˜ λ a + ˜ τ k j a . Unfortunately , while the link dela ys ˜ λ a for a ∈ A + i migh t b e observed at no de i , this is not the case for the ˜ τ k j a ’s which dep end on future delays that will b e exp erienced when trav ersing the downstream links. Supp ose instead that only the exp ected v alues τ k j a are known and a v ailable at no de i and that eac h pack et from source k ∈ K observ es the ˜ λ a ’s and is routed through the link a ∈ A + i that minimizes ˜ z k a = ˜ λ a + τ k j a to the next no de j a where the pro cess rep eats. Thus, denoting E k a , { ˜ z k a ≤ ˜ z k b ∀ b ∈ A + i } , the pac kets from source k ∈ K mo ve across the netw ork according to a Marko v c hain with transition probabilities P k ij = ( P ( E k a ) if ij = a 0 otherwise (2) for i 6 = d k , while the destination d k is an absorbing state. The exp ected flo ws corresp ond to the inv ariant measures of these Marko v chains, leading to a flo w 8 Roberto Cominetti, Crist´ obal Guzm´ an i } { A + i A  i X a 2 A  i v k a X a 2 A + i v k a x k y k i Fig. 2 Flow conserv ation diagram (here i = s k ) distribution rule in which the throughput flow y k i from source k that enters no de i , splits among the links a ∈ A + i according to (see Figure 2) v k a = y k i P ( E k a ) . (3) The throughputs y k = ( y k i ) i 6 = d k can b e computed from the stationary equa- tions y k = P ∞ j =0 [( ˆ P k ) 0 ] j δ k x k , where ˆ P k = ( P k ij ) i,j 6 = d k is the reduced transition matrix on the non-absorbing states, and δ k i = 1 for i = s k and δ k i = 0 other- wise. This may also b e written as y k = x k δ k + ( ˆ P k ) 0 y k whic h corresp onds to the standard flo w conserv ation equations y k i = x k δ k i + P a ∈ A − i v k a . (4) These equations can b e restated compactly using exp ected utility theory . Namely , let us write ˜ z k a = z k a +  k a as the sum of its exp ected v alue z k a = λ a + τ k j a plus a noise  k a with E (  k a ) = 0, and assume that the distribution of  k a do es not change with z k a (for a discussion of this assumption see § 6). Also, for simplicit y  k a is supp osed to hav e contin uous distribution so that the exp ected utility functions introduced next will b e differen tiable (distributions with point masses can b e treated as in [2]). Under these assumptions, the transition probabilities in (2) can b e expressed as P ( E k a ) = ∂ ϕ k i ∂ z k a ( z k ) where ϕ k i denote the exp ected utilit y functions ϕ k i ( z k ) = ( E ( min a ∈ A + i { z k a +  k a } ) if i 6 = d k 0 if i = d k (5) whic h allow us to rewrite the flow equations (3)-(4) as    v k a = y k i ∂ ϕ k i ∂ z k a ( z k ) ∀ a ∈ A + i y k i = x k δ k i + P a ∈ A − i v k a ∀ i 6 = d k . (6) On the other hand, assuming that the cost-to-go v ariables { ˜ τ k j a : a ∈ A + i } are indep endent from the local queueing times { ˜ λ a : a ∈ A + i } , we may compute Netw ork Congestion Control with Marko vian Multipath Routing ? 9 the exp ected v alue of ˜ τ k i b y conditioning on the even ts E k a as τ k i = E ( ˜ τ k i ) = P a ∈ A + i E ( ˜ λ a + ˜ τ k j a | E k a ) P ( E k a ) = P a ∈ A + i E ( ˜ λ a + τ k j a | E k a ) P ( E k a ) = E ( min a ∈ A + i { ˜ λ a + τ k j a } ) so that ( τ k i = ϕ k i ( z k ) ∀ i ∈ N z k a = λ a + τ k j a ∀ a ∈ A. (7) Under mild conditions it was pro ved in [2] that, given the λ a ’s, system (6)- (7) has a unique solution ( v , y , τ , z ). It was also shown that these equations, together with the equilibrium conditions λ a = s a ( w a ) w here w a = P k ∈ K v k a represen ts the total exp ected link load, hav e a unique solution ( λ, w , v , y , τ , z ) called a Markovian T r affic Equilibrium (MTE). This equilibrium is c haracter- ized by a pair of dual optimization problems analog to (p-w) and (d-w) . As a matter of fact, the dual problem has exactly the same form (d-mte) min λ ∈ R A X a ∈ A Z λ a λ 0 a s − 1 a ( z ) dz − X k ∈ K x k τ k ( λ ) where τ k ( λ ) , τ k s k ( λ ) with τ k i ( λ ) the solution of (7). The expected utilit y maps ϕ k i ( · ) con vey all the information required to describ e a Marko vian routing and may be considered as the primary mo deling ob jects. These maps are determined b y the random v ariables  k a whic h are ultimately tied to the arc random costs ˜ λ a . The class E of maps that can b e expressed in the form (5) admits an analytic characterization (see [2]): they are the C 1 maps ϕ : R n → R that are concav e, comp onent wise non-decreasing, and whic h satisfy in addition (a) ϕ ( x 1 + c, . . . , x n + c ) = ϕ ( x 1 , . . . , x n ) + c (b) ϕ ( x ) → x i when x j → ∞ for all j 6 = i (c) for x i fixed, ∂ ϕ ∂ x i ( x 1 , . . . , x n ) is a contin uous distribution function on the remaining v ariables. Note also that ϕ ( x ) ≤ min { x 1 , . . . , x n } . In what follows we assume that the mo del is sp ecified directly in terms of a family of maps ϕ k i ∈ E with ϕ k d k ≡ 0. Ho wev er, we note that these maps are not used explicitly by our distributed proto col in Section § 4. Remark. Since pack et mov ements are gov erned by a Marko v chain, cycling ma y o ccur and additional conditions are required to ensure that pac kets reac h the destination with probabilit y one. A simple case is when source k considers only the arcs in A + i that lead closer to destination d k ( e.g. τ k j a < τ k i a ), s o that 10 Roberto Cominetti, Crist´ obal Guzm´ an the corresp onding Marko v chain is supp orted ov er an acyclic graph ( N , A k ). T o deal with this case it suffices to redefine ϕ k i ( z k ) , E ( min a ∈ A k + i { z k a +  k a } ) so that P k ij = ∂ ϕ k i ∂ z k a ( z k ) = 0 for all a = ij 6∈ A k + i . 3 Rate control with Mark o vian routing W e pro ceed to develop a cross-lay er mo del that combines a NUM approach for rate control based on queueing dela ys, with a Marko vian multipath routing based on total delays. Each source k ∈ K is characterized b y an origin s k , a destination d k , and a contin uous decreasing rate function f k : (0 , ∞ ) → (0 , ∞ ) with f k ( q k ) → 0 as q k → ∞ , while every link a ∈ A has a contin uous increasing latency function s a : [0 , c a ) → [ λ 0 a , ∞ ) with λ 0 a = s a (0) ≥ 0. P ack ets are routed according to a Marko vian strategy characterized b y a family of maps ϕ k i ∈ E with ϕ k d k ≡ 0. Sources adjust their rates as a function x k = f k ( q k ) of the total queueing delay q k = τ k ( λ ) − τ 0 k , where τ k ( λ ) is the end-to-end exp ected delay defined in the previous section and τ 0 k is the minimal trav el time considering propagation dela ys only . Informally , the source rates x k induce flo ws v k a and total link loads w a . These loads determine link exp ected delays λ a = s a ( w a ) that yield end-to- end dela ys τ k ( λ ) for eac h source and corresp onding queueing delays q k . A t equilibrium, these queueing delays m ust induce the original rates x k = f k ( q k ). Definition 1 A p air ( w , x ) with w = ( w a ) a ∈ A and x = ( x k ) k ∈ K is c al le d a Mark ovian Netw ork Utility Maximization (MNUM) equilibrium if and only if w a = P k ∈ K v k a wher e ( v k , y k ) solve the flow c onservation c onstr aints (6) with ( τ k , z k ) satisfying (7) , to gether with the link delay r elations λ a = s a ( w a ) and the r ate e quilibrium c onditions x k = f k ( q k ) wher e q k = τ k ( λ ) − τ 0 k . 3.1 Reduced form ulation of MNUM In order to establish the existence and uniqueness of equilibria we b egin by reducing MNUM to an equiv alent set of equations that inv olves only the v ari- ables λ . T o this end w e need to extend the results in [2] for which we consider a fixed non-negativ e link dela y v ector ( λ a ) a ∈ A . W e first sho w that (7) uniquely defines z k and τ k as implicit functions of λ . This system can b e equiv alently stated solely in terms of the v ariables τ k as τ k i = ϕ k i  ( λ a + τ k j a ) a ∈ A  (8) so it suffices to pro ve that the latter uniquely defines τ k as a function of λ . Netw ork Congestion Control with Marko vian Multipath Routing ? 11 Prop osition 1 L et k ∈ K and denote ¯ τ k i the c ost of a shortest p ath fr om i to destination d k with link c osts λ a . Supp ose also that ˆ τ k ∈ R N is such that ˆ τ k i ≤ ϕ k i (( λ a + ˆ τ k j a ) a ∈ A ) ( ∀ i ∈ N ) . (9) Then ˆ τ k ≤ ¯ τ k and mor e over, starting fr om τ k, 0 = ¯ τ k , the iter ates c ompute d by τ k,n +1 i = ϕ k i (( λ a + τ k,n j a ) a ∈ A ) n = 0 , 1 , 2 . . . (10) ar e non-incr e asing and c onver ge to a solution τ k of (8) with τ k i ∈ [ ˆ τ k i , ¯ τ k i ] . Pr o of In order to prov e that ˆ τ k ≤ ¯ τ k let I = { i : ˆ τ k i ≤ ¯ τ k i } and supp ose b y con tradiction I 6 = N . Since ˆ τ k d k ≤ 0 = ¯ τ k d k w e ha ve d k ∈ I . Consider a shortest path from a no de i 0 6∈ I to d k , and let i 6∈ I b e the last no de b efore entering I and j ∈ I the next no de. Since ϕ k i ( z k ) ≤ min a ∈ A + i z k a w e get ˆ τ k i ≤ ϕ k i (( λ a + ˆ τ k j a ) a ∈ A ) ≤ λ ij + ˆ τ k j ≤ λ ij + ¯ τ k j = ¯ τ k i < ˆ τ k i . This con tradiction prov es that ˆ τ k ≤ ¯ τ k . Let us pro ve next the conv ergence of the iteration (10). W e note that τ k, 1 i = ϕ k i (( λ a + τ k, 0 j a ) a ∈ A ) ≤ min a ∈ A + i { λ a + ¯ τ j a } = ¯ τ i = τ k, 0 i from which it follows inductively that the sequence (10) is non-increasing: if τ k,n ≤ τ k,n − 1 then τ k,n +1 i = ϕ k i (( λ a + τ k,n j a ) a ∈ A ) ≤ ϕ k i (( λ a + τ k,n − 1 j a ) a ∈ A ) = τ k,n i . It remains to show that the sequence τ k,n is b ounded b elow by ˆ τ k . W e pro ve this by induction: the base case τ k, 0 = ¯ τ k ≥ ˆ τ k w as just prov ed ab ov e, while for the induction step it suffices to note that τ k,n ≥ ˆ τ k implies τ k,n +1 i = ϕ k i (( λ a + τ k,n j a ) a ∈ A ) ≥ ϕ k i (( λ a + ˆ τ k j a ) a ∈ A ) ≥ ˆ τ k i . By contin uity it follows that the limit of τ k,n satisfies (8) and τ k i ∈ [ ˆ τ k i , ¯ τ k i ].  Remark. The previous result gives a procedure to solve (8): compute the shortest path dela ys ¯ τ k and then iterate (10). Alternativ ely one may start from τ k, 0 = ˆ τ k in whic h case the iterates increase and are bounded from ab o ve by ¯ τ k , hence these iterates also con verge to a solution of (8). 12 Roberto Cominetti, Crist´ obal Guzm´ an Definition 1 W e denote P the set of all λ ∈ R A suc h that for each destination k ∈ K there exists ˆ τ k ∈ R N satisfying ˆ τ k i < ϕ k i (( λ a + ˆ τ k j a ) a ∈ A ) for all i 6 = d k . (11) Note that P is an op en conv ex domain, and for each λ ∈ P we hav e λ 0 ∈ P for all λ 0 ≥ λ . In the sequel we extend the results in [2] which were based on a muc h more stringent condition assuming that (11) holds with ˆ τ k = 0. The pro ofs differ substan tially so we present them b elow. Lemma 1 L et λ ∈ P and supp ose that ( τ k , z k ) solves (7) . L et ˆ Q k ( z k ) b e the matrix with entries ˆ Q k ia ( z k ) = ∂ ϕ k i ∂ z k a ( z k ) for i 6 = d k and a ∈ A . Then (a) F or e ach i 6 = d k ther e exists j ∈ N such that P k ij > 0 and ˆ τ k j − τ k j > ˆ τ k i − τ k i . (b) The matrix [ I − ˆ P k ( z k )] is invertible. (c) Equation (6) has a unique solution given by v k = ˆ Q k ( z k ) 0 y k ≥ 0 with y k = [ I − ˆ P k ( z k ) 0 ] − 1 δ k x k ≥ 0 . Pr o of (a) Since the ϕ k i ’s are conca ve and differentiable we ha ve ˆ τ k i < ϕ k i (( λ a + ˆ τ k j a ) a ∈ A ) ≤ ϕ k i ( z k ) + P a ∈ A ∂ ϕ k i ∂ z k a ( z k )( ˆ τ k j a − τ k j a ) = τ k i + P ij ∈ A + i P k ij ( ˆ τ k j − τ k j ) from whic h (a) follows directly . (b) Giv en i 6 = d k and using (a) inductiv ely we can find a finite sequence of no des i 0 , . . . , i m with i 0 = i, i m = d k and P i k i k +1 > 0. Th us, starting from i , the c hain has a p ositive probability of reaching the absorbing state d k in a finite num b er of steps. This implies that ˆ P k ( z k ) m is strictly submarko vian for m large enough, and therefore [ I − ˆ P k ( z k )] is in vertible. (c) The first equation of (6) can b e rewritten as v k = ˆ Q k ( z k ) 0 y k , whic h sub- stituted into the second equation yields y k = δ k x k + ˆ P k ( z k ) 0 y k , so that (b) implies y k = [ I − ˆ P k ( z k ) 0 ] − 1 δ k x k . The non-negativity of these quantities fol- lo ws from the fact that x k ≥ 0 while the matrices ˆ Q k ( z k ) and ˆ P k ( z k ) hav e non-negativ e entries with [ I − ˆ P k ( z k ) 0 ] − 1 = P ∞ m =0 [ ˆ P k ( z k ) 0 ] m .  The next result is the k ey to reduce the MNUM equations to a system in the v ariables λ . Prop osition 2 If λ ∈ P then, for e ach sour c e k ∈ K , the system (7) has a unique solution z k = z k ( λ ) > 0 and τ k = τ k ( λ ) > 0 . Mor e over, the functions λ → τ k i ( λ ) and λ → z k a ( λ ) ar e c onc ave, smo oth and c omp onent-wise non- de cr e asing. Netw ork Congestion Control with Marko vian Multipath Routing ? 13 Pr o of It suffices to show that (8) defines implicit maps λ 7→ τ k i ( λ ) ov er the domain P which are well defined, concav e, smo oth, and monotone. W e already pro ved the existence of a solution with τ k i ≥ ˆ τ k i . Let us pro ve its uniqueness. Uniqueness : Let k ∈ K and consider tw o solutions ( τ 1 , z 1 ) and ( τ 2 , z 2 ) for (7). Let α = max i ∈ N ( τ 2 i − τ 1 i ) and denote N ∗ the set of no des where the maximum is attained. F or every i ∈ N ∗ , the conca vity of ϕ k i ( · ) giv es τ 2 i = ϕ k i ( z 2 ) ≤ ϕ k i ( z 1 ) + P a ∈ A + i ∂ ϕ k i ∂ z k a ( z 1 )( z 2 a − z 1 a ) = τ 1 i + P a ∈ A + i ∂ ϕ k i ∂ z k a ( z 1 )( τ 2 j a − τ 1 j a ) and, since ( τ 2 j a − τ 1 j a ) ≤ α while the partial deriv atives add up to 1, we get α = τ 2 i − τ 1 i ≤ P a ∈ A + i ∂ ϕ k i ∂ z k a ( z 1 )( τ 2 j a − τ 1 j a ) ≤ α. It follows that for every a ∈ A + i suc h that ∂ ϕ k i ∂ z k a ( z 1 ) > 0 we necessarily ha ve j a ∈ N ∗ . Combining this fact with Lemma 1(a), we can find a finite sequence of no des in N ∗ starting at i and ending at d k . Hence d k ∈ N ∗ so that α = τ 2 d k − τ 1 d k = 0, which implies τ 2 ≤ τ 1 . Exchanging the roles of τ 1 and τ 2 w e get the con verse inequality so that τ 1 = τ 2 pro ving uniqueness. Conc avity : F or α ∈ (0 , 1) set λ α = αλ 1 + (1 − α ) λ 2 and τ α = ατ k ( λ 1 ) + (1 − α ) τ k ( λ 2 ). Denote z 1 a = λ 1 a + τ k j a ( λ 1 ), z 2 a = λ 2 a + τ k j a ( λ 2 ), and z α a = λ α a + τ α j a . Then z α = αz 1 + (1 − α ) z 2 and the conca vity of ϕ k i implies ϕ k i ( z α ) ≥ αϕ k i ( z 1 ) + (1 − α ) ϕ k i ( z 2 ) = α τ k i ( λ 1 ) + (1 − α ) τ k i ( λ 2 ) = τ α i . This prov es that ˆ τ k = τ α satisfies condition (9) for λ α , and then Prop osition 1 giv es τ k ( λ α ) ≥ τ α whic h yields precisely the concavit y inequality τ k i ( λ α ) ≥ ατ k i ( λ 1 ) + (1 − α ) τ k i ( λ 2 ) . Smo othness : This is a direct consequence of the implicit function theorem. Indeed, noting that τ k d k = 0 we may reduce (8) to a system in the v ariables ( τ k i ) i 6 = d k . The Jacobian of this reduced system is [ I − ˆ P k ] whic h is inv ertible b y Lemma 1(b), so the conclusion follows. Monotonicity : Let λ 1 ∈ P and take λ 2 ≥ λ 1 so that λ 2 ∈ P . The m onotonicit y of ϕ k i implies τ k i ( λ 1 ) = ϕ k i (( λ 1 a + τ k j a ( λ 1 )) a ∈ A ) ≤ ϕ k i (( λ 2 a + τ k j a ( λ 1 )) a ∈ A ) Hence ˆ τ k = τ k ( λ 1 ) satisfies (9) for λ 2 , and Prop osition 1 implies τ k ( λ 2 ) ≥ τ k ( λ 1 ).  14 Roberto Cominetti, Crist´ obal Guzm´ an The implicit maps τ k ( λ ) and z k ( λ ) defined b y (7), allow us to restate the MNUM equations solely in terms of the link delay vector λ . Indeed, let q k ( λ ) = τ k s k ( λ ) − τ 0 k and define ˜ x k ( λ ) = f k ( q k ( λ )). According to Lemma 1(c) the equations (6) hav e unique solutions v k = v k ( λ ) and y k = y k ( λ ). Denoting ˜ w a ( λ ) = P k ∈ K v k a ( λ ), the MNUM equations are equiv alent to the reduced system of equations (r-mnum) λ a = s a ( ˜ w a ( λ )) ∀ a ∈ A. 3.2 V ariational characterization W e show that the reduced system (r-mnum) corresp onds to the optimality conditions of an optimization problem which is a combination of the v ariational c haracterizations (d-num) and (d-mte) . Theorem 1 Assume that λ 0 ∈ P . Then ( x ∗ , w ∗ ) is an MNUM e quilibium iff x ∗ = ˜ x ( λ ∗ ) and w ∗ = ˜ w ( λ ∗ ) with λ ∗ an optimal solution of the strictly c onvex pr o gr am (d-mnum) min λ ∈P Φ ( λ ) , X a ∈ A Z λ a λ 0 a s − 1 a ( z ) dz − X k ∈ K F k ( q k ( λ )) wher e F k ( · ) denotes a primitive of f k ( · ) . Pr o of Since q k ( λ ) = τ k s k ( λ ) − τ 0 k is conca ve and the f k ’s are p ositive and decreasing, it follo ws that the map Φ 1 ( λ ) , − P k ∈ K F k ( q k ( λ )) is con vex. Also, since the s − 1 a ’s are increasing we obtain that Φ 0 ( λ ) , P a ∈ A R λ a λ 0 a s − 1 a ( z ) dz is strict y conv ex, and then so is Φ ( λ ) = Φ 0 ( λ ) + Φ 1 ( λ ). Hence, since P is op en and conv ex, an optimal solution for (d-mnum) is c haracterized by ∇ Φ ( λ ) = 0. No w ∂ Φ ∂ λ a = s − 1 a ( λ a ) − P k ∈ K f k ( q k ( λ )) ∂ q k ∂ λ a ( λ ) = s − 1 a ( λ a ) − P k ∈ K ˜ x k ( λ ) ∂ τ k s k ∂ λ a ( λ ) . An implicit differen tiation of (8) gives ∂ τ k ∂ λ = ˆ Q ( λ ) + ˆ P k ( λ ) ∂ τ k ∂ λ so that ∂ τ k ∂ λ a = [ I − ˆ P k ( λ )] − 1 ˆ Q ( λ ) e a , from whic h we get ˜ x k ( λ ) ∂ τ k s k ∂ λ a ( λ ) = ˜ x k ( λ ) e 0 s k [ I − ˆ P k ( λ )] − 1 ˆ Q ( λ ) e a = ([ I − ˆ P k ( λ ) 0 ] − 1 δ k ˜ x k ( λ )) 0 ˆ Q ( λ ) e a = y k ( λ ) 0 ˆ Q ( λ ) e a = v k ( λ ) 0 e a = v k a ( λ ) . Therefore ∂ Φ ∂ λ a = s − 1 a ( λ a ) − ˜ w a ( λ ) and the optimality condition ∇ Φ ( λ ) = 0 coincides with the (r-mnum) c haracterization of equilibria.  Netw ork Congestion Control with Marko vian Multipath Routing ? 15 This characterization allows us to prov e the existence and uniqueness of an MNUM equilibrium. Theorem 2 Pr oblem (d-mnum) is strictly c onvex and c o er cive, henc e it has a unique optimal solution and ther efor e ther e exists a unique MNUM e quilibrium. Pr o of W e already show ed that the ob jective function Φ ( λ ) is strictly con- v ex. Moreov er, since q k ( λ ) is comp onen twise non-decreasing while the term R λ a λ 0 a s − 1 a ( y ) dy is decreasing for λ a < λ 0 a , it follo ws that the minimum of (d- mnum) b elongs to the set { λ ≥ λ 0 } . Hence, in order to establish co ercivity , it suffices to show that the recession function satisfies Φ ∞ ( λ ) > 0 for all λ ≥ 0 with λ 6 = 0. Now, Φ ∞ 0 ( λ ) = lim t →∞ 1 t Φ 0 ( tλ ) = X a ∈ A lim t →∞ 1 t Z tλ a λ 0 a s − 1 a ( z ) dz = X a ∈ A lim t →∞ s − 1 a ( tλ a ) λ a = X a ∈ A c a λ a > 0 . In order to compute Φ ∞ 1 ( λ ) let us consider the conv ex maps U k ( r ) = − F k ( − r ) and h k ( λ ) = − q k ( λ ). It can b e shown (see [2]) that h ∞ k ( λ ) = − ¯ τ k s k with ¯ τ k s k the cost of a shortest path with link delays λ a . Since U ∞ k ( r ) = 0 for r ≤ 0, w e get ( U k ◦ h k ) ∞ ( λ ) = U ∞ k ( h ∞ k ( λ )) = 0 for all λ ≥ 0, and then Φ ∞ 1 ( λ ) = 0. As a consequence, for all directions λ ≥ 0 with λ 6 = 0 we get Φ ∞ ( λ ) > 0, so that (d-mnum) is inf-compact and therefore it has an optimal solution.  4 A distributed algorithm for MNUM This section briefly describ es how the MNUM framework can lead to a dis- tributed proto col for rate congestion control under Mark ovian m ultipath rout- ing. This proto col can b e interpreted as a distributed algorithm that solves the v ariational problem (d-mnum ). The algorithm is based on a Marko vian routing pro cess for pac k ets, with a slo w up date of the end-to-end exp ected de- la ys τ k i ’s. This pro cess is combined with a fast TCP adaptation of user’s rates b y estimating the end-to-end queueing delays to reac h the equilibrium rates ˜ x k ( λ ). A more detailed description and analysis of the distributed proto col will b e the sub ject of a forthcoming pap er [6]. 4.1 P ack et routing based on lo cal queues W e adapt the ideas of § 2.2.2 in order to define a routing p olicy based on lo cal information. T o do this, for each pac ket with destination k router i must find 16 Roberto Cominetti, Crist´ obal Guzm´ an the outgoing link a ∈ A + i that realizes the minim um of the v alues ˜ z k a = ˜ λ a + τ k j a , b y adding the propagation delay λ 0 a , plus the current queueing delay ˜ p a of the link, plus the estimate of the expected dela y τ k j a adv ertised in the routing table of the next hop j a . If destination k is not reac hable through j a w e take τ k j a as infinit y (or a very large v alue). This requires that each router perio dically advertises its routing table pro- viding an estimate of the total exp ected dela ys τ k i to all its kno wn destinations. These estimates may b e up dated on a slow time-scale by av eraging the ob- serv ed delays min a ∈ A + i { ˜ λ a + τ k j a } ov er a fixed num b er of pack ets or ov er all pac kets sent ov er a fixed time windo w. The observed av erage ˇ τ k i is used to up date the estimate of the exp ected dela y as τ k i ← (1 − α ) τ k i + α ˇ τ k i . 4.2 TCP proto col The TCP proto col for source rate con trol requires a mechanism by whic h ev ery source k can estimate q k ( λ ) = τ k s k ( λ ) − τ 0 k . Here we rely on the tw o time-scales assumption: sources control their rates at a muc h faster pace than routers, so they see link delays as constant. F or fixed exp ected delays λ , the exp ected forw ard time coincides with τ k s k ( λ ) so that, using standard proto cols for estimating the forw ard time, sources can get an unbiased estimation T k t of this total exp ected dela y . W e also need a mechanism to estimate τ 0 k , that is, the shortest distance considering propagation delay only and no queuing. Since the physical trans- mission sp eeds are roughly constant, these v alues are more stable and will only change when the netw ork top ology is mo dified by addition or remov al of a router or link. One option is to estimate τ 0 k b y the minim um forward time observ ed o ver all sent pack ets, just as in the single-path implementation of V egas. The hop e is that at least one pack et will b e routed along a shortest path and find no queues along its wa y . How ever, this estimator is known to b e biased if the netw ork is already congested and therefore it provides only an upp er b ound for τ 0 k . An alternative is to let each pack et accumulate the propagation delay along its route and feedback the total to the source in the corresp onding ack so that τ 0 k can be estimated as the minim um v alue observ ed. As a third option one could implement a RIP proto col, in parallel with the estimation of τ k i in § 4.1, to compute shortest paths using propagation delay as metric. This requires feedbac k from routers to sources and pro vides dis- tance estimates for aggregate destinations (autonomous systems) which differ from the end-to-end minimum times τ 0 k b y a small constant access time. This, ho wev er, do es not inv alidate the analysis in § 3. W e b elieve this option is an accurate and efficien tly implementable one. An alternative approach, prop osed in [26], is to replace the minimum τ 0 k and use instead the aver age end-to-end propagation delay as the reference Netw ork Congestion Control with Marko vian Multipath Routing ? 17 b aseR TT in the V egas proto col. This av erage propagation delay can b e esti- mated directly by sources using the moving av erage technique describ ed in [26] or, alternatively , by adapting the randomized ECN proto col in [1] for es- timating an additive measure for single-path routing. A generalization of this metho d to multipath routing is developed in [6]. While b oth approaches a v oid the computational ov erhead of a RIP proto col, we must note that the av er- age propagation delays dep end on the routes b eing currently used which are themselv es affected by the netw ork congestion so that this base R TT meas ure is not flo w-indep endent as required in the analysis of section § 3. Finally , the free-flow times T 0 k together with the unbiased estimators T k t of τ k s k for every pack et t arriving to destination, can b e used to adjust the rates b y a sto chastic approximation algorithm of the form x k t +1 ← (1 − δ ) x k t + δ f k ( Q k t ) (12) where Q k t = T k t − T 0 k . If we let sources adapt long enough so that Q k t ∼ q k and x k t ∼ f k ( q k ), we can then pro ceed to up date the router estimates of the end-to-end dela ys τ k i . 4.3 Complexit y and implementation considerations The prop osed schemes require little mo difications to current TCP/IP proto- cols. The total exp ected dela y up date for routers described in § 4.1 has similar computational, communication and memory requirements as distance-vector proto cols suc h as RIP . The main difference is that the hop-count metric is replaced by the exp ected delay for each destination known to the router. The routing tables and exp ected delays are up dated in a mo derated time-scale and adv ertised as usual to neigh b oring routers. F or other implementation consider- ations see [26], where the authors make a full description on how to generalize the hop-coun t distance by other congestion prices. Let us stress that our prop osed method for estimating the propagation dela y requires a parallel computation by RIP-type proto cols, which conv eys a memory ov erhead on routers to store an additional metric for each entry in the routing table. This might b e costly but p ermits an accurate estimation of queuing dela ys. The simpler alternativ e of accum ulating propagation delays on the pack et headers has also a storage ov erhead, although it can b e efficiently implemen ted in some situations [1]. Which of these approaches has the b est cost-b enefit tradeoff will b e the sub ject of another study [6]. Concerning TCP , the estimation of forward tra vel times and the ECN estimation are standard features of TCP/IP proto cols. A relev ant issue is that routing along paths with heterogeneous delays may induce pack et reordering so that the duplicate ACK feature of TCP must b e turned off, allowing for some buffering at the receiver to reorder the pack ets (see [26] for details). Ho wev er, since our routing strategy is based on minimizing end-to-end delay , in steady state pack ets should arrive approximately in order so that excessive buffering should not b e needed. An alternative to a void pac k et reordering is to 18 Roberto Cominetti, Crist´ obal Guzm´ an k eep individual TCP-connections single path and use a hashing technique to distribute these connections for load balancing (see e.g. [5, 28]). The do wnside of this tec hnique is its coarser gran ularit y which may cause instabilities in the load balancing and routing. 5 Comparison with related work Multipath routing can b e decomp osed in to tw o main tasks: computation of paths and traffic splitting. The splitting can b e controlled directly by sources or in a decentralized manner b y routers. Moreo v er, it can b e implemen ted either on a p er-pac ket basis (each pack et following a p ossibly different path) or a p er-flow basis where each TCP-connection is assigned a single path and load balancing is ac hieved b y distributing these connections among paths. Ac- cordingly , sev eral alternative approaches ha ve been prop osed in the literature. W e briefly compare MNUM with some of the previous works. F or more com- plete surveys and discussions of the challenges inv olved in m ultipath routing w e refer to [10, Go jmerac], [12, He and Rexford] and [18, Lee and Choi]. The seminal pap er [8, Gallager] introduced a distributed routing proto col that finds an optimal multi-commodity flo w minimizing av erage delays. The mo del considers flo w dependent latencies, but traffic demand and netw ork top ology are assumed to b e fixed. In a similar con text, the PEFT proto col in [32, Xu et al. ] develops a routing sc heme based on exp onen tial p enalties using link-prices sp ecifically tuned to repro duce an optimal multi-commodity flo w. Although PEFT op eration is distributed, flow optimization and link-price tuning require ce n tralized computation. F or large netw orks where the topology and the traffic change contin uously during op eration, this inv olves substantial pro cessing and communication ov erheads. In contrast, MNUM automatically adjusts the routing to v ariations in traffic and topology in a decentralized manner. Incidentally , we note that the MNUM routing also takes the form of an exp onen tial p enalty if link costs are distributed Gumbel, although MNUM deals directly with the actual randomness of the links without assuming any sp ecific a priori distribution. Multipath routing with elastic traffic was considered in [14, Kelly et al. ]. In this setting eac h source directly controls the flow rates to b e sent ov er a set of av ailable paths, using a TCP-like feedbac k mechanism based on congestion signals. The framework uses fluid-flow dynamics that conv erge to an optimal solution of a multipath version of NUM, which provide a template for design- ing pack et-level proto cols. A v ariant of this model including feedback dela ys is the basis for the overlay TCP scheme prop osed in [11, Han et al. ]. The effect of delays is also studied in [15, Kelly and V oice] pro viding sufficient conditions for dynamical stabilit y , while [16, Key et al. ] analyzes the asymptotic behavior as the num b er of connections increase. With a similar goal, [19, Lin and Shroff] consider a discrete iteration for solving m ultipath NUM using a v ariant of the pro ximal p oint algorithm that yields a decentralized algorithm with go o d con- v ergence prop erties. The implemen tation of source-controlled routing schemes Netw ork Congestion Control with Marko vian Multipath Routing ? 19 presupp oses that the netw ork supp orts multipath routing. One alternative for this is to use Lab el Switched Path tunnels using MPLS [29, Villamizar], how- ev er the pro cessing ov erhead of p er-flow routing do es not scale well with the n umber of connections. A different option is considered in ov erlay TCP b y es- tablishing a set of o verla y routers at some p eering p oin ts, with traffic b etw een these p oints controlled by standard single-path proto cols. While this fav ors incremen tal deploymen t, path div ersity is limited to the extent that traffic m ust b e routed through the predetermined p eers. In our approach an y neigh- b oring router can pla y the role of an o verla y no de, and no p er-flow routing is required since traffic splitting is controlled b y routers rather than sources. It is also worth noting that in contrast with the approaches that start from NUM and fluid-flow dynamics which then lead to a pack et level proto col, we pro ceed in the rev erse order from pac k et dynamics to its equilibrium describ ed b y MNUM. T raffic splitting controlled b y routers w as considered in [25, Paganini] and [26, Paganini and Mallada] by combining a routing scheme as in [8, Gallager] with a rate control as in [14, Kelly et al. ]. Flow splitting is decentralized at eac h router i by using split ratios ( α k a ) a ∈ A + i that con trol the fraction of pac kets from source k that are forwarded along eac h outgoing link a ∈ A + i . These ratios are dynamically adjusted so that the routing concentrates ov er the links that b elong to currently shortest paths. The framework uses a fluid-flow model from whic h a pack et-level proto col is derived. Our approac h is similar in the sense that flow splitting is also decided lo cally at routers based on current dela ys, so that congestion a w are paths are selected automatically . In b oth approaches all the paths are p otentially av ailable and only the currently optimal ones are used, although other path choice rules can also b e incorp orated by forbidding flo w on some links (see the remark at the end of § 2.2.2). A difference b etw een b oth approaches is that while [25, 26] is based on exp ected v alues of queuing dela y , our routing ev olves stochastically using the curren t state of lo cal queues and considering total delay including queueing plus propagation. One reason for this choice is to allow a finer per-pack et gran ularit y in load balancing, k eeping pack et reordering under control. In contrast, since [25, 26] uses paths with heterogenous total delays, load balancing is implemented at a coarser per- flo w granularit y by using hashing [28]. As a final remark, the proto col in [26] requires three time-scales to ensure conv ergence: a fast source rate adaptation, a medium time-scale for route price up dates, and a slow up date of splitting ratios. W e only use t wo time-scales: a slow one for estimating the delays and a fast one for rate con trol and routing. 6 Conclusions and future work W e prop osed a new cross-lay ering mo del for TCP/IP con trol under m ultipath routing. The motiv ation for our routing mechanism comes from using lo cal in- formation ab out queueing delays as well as the exp ected delays from the next hops to the destination, in order to exploit the a v ailable capacity by sending 20 Roberto Cominetti, Crist´ obal Guzm´ an pac kets through several alternative routes. T o achiev e this purp ose, we con- sidered a Mark o vian routing com bined with a V egas-lik e TCP protocol for rate con trol. The routing pro cess was characterized by studying the exp ected dynamic programming equations which lead to a Mark ovian T raffic Equilib- rium, together with a standard Netw ork Utility Maximization mo del for the TCP steady state. This led to a v ariational characterization of the equilibrium that allow ed us to prov e its existence and uniqueness, and which inspired a distributed proto col for attaining the equilibrium. There are sev eral unsolved issues. Firstly , further research is required to pro vide a theoretical support for the con vergence of these protocols. A detailed analysis should study the relation b etw een the pack et-lev el dynamics and our flo w-level model. Our equilibrium mo del relies on this assumption: namely , w e base our up dates on aggregated flow information as well as in the tw o time- scales conv ergence of equilibrium flows. A complete analysis should explain to whic h exten t the flo w mo del captures the pack et level dynamics, and how fast the equilibrium flows are attained b y sources. Interesting recent results along this line can b e found in [13, 30]. Another in teresting question is related to the model of randomness as- sumed. W e considered an additive structure ˜ z k a = z k a +  k a whic h presumes the same v ariability of delays regardless of the av erage flow levels observed. A more realistic mo del should consider higher v ariabilit y for higher exp ected dela ys, based either on a detailed analysis of the distribution of waiting times at queues, or at least using a simplified multiplicativ e randomness mo del of the form ˜ z k a = z k a (1 +  k a ). A final line of research has to do with simulating this protocol in a realistic en vironment. A fair comparison with single-path routing requires the presence of uncertain ty and dela ys in information transmission. Sim ulation may pro vide an idea on the effective increase in p erformance that one might exp ect from a Mark ovian multipath routing. References 1. Adler, M., Cai, J.Y., Shapiro, J., T owsley , D.: Estimation of congestion price using probabilistic pack et marking. T echnical Report pp. 1–36 (2002) 2. Baillon, J., Cominetti, R.: Marko vian traffic equilibrium. Mathematical Programming 111 , 33–56 (2008) 3. Barr´ e, S., Paasc h, C., Bonav enture, O.: Multipath tcp: from theory to practice. In: Proceedings of the 10th international IFIP TC 6 conference on Net working - Part I, pp. 444–457. Springer-V erlag, Berlin, Heidelberg (2011) 4. Beckman, M., McGuire, C., Winsten, C.: Studies in Economics of T ransp ortation. Y ale Universit y Press (1956) 5. Cao, Z., W ang, Z., Zegura, E.: P erformance of hashing-based schemes for internet load balancing. In: INF OCOM 2000. Nineteen th Annual Join t Conference of the IEEE Com- puter and Communications So cieties. Pro ceedings. IEEE, vol. 1, pp. 332–341 (2000). DOI 10.1109/INFCOM.2000.832203 6. Cominetti, R., Guzm´ an, C., Maureira, J.: Implemen tation of a distributed proto col for netw ork congestion control with marko vian multipath routing. F orthcoming (2014) 7. Dumas, V., Guillemin, F., Rob ert, P .: A mark ovian analysis of additiv e-increase multiplicativ e-decrease (aimd) algorithms. In: Adv ances in Applied Probability , pp. 85–111 (2002) Netw ork Congestion Control with Marko vian Multipath Routing ? 21 8. Gallager, R.: A minimum dela y routing algorithm using distributed computation. IEEE T ransactions on Communications 25 (1), 73–85 (1977) 9. Gibb ens, R., Kelly , F.: Resource pricing and the evolution of congestion control. Auto- matica 35 , 1969–1985 (1999) 10. Go jmerac, I.: Adaptiv e Multipath Routing for Internet T raffic Engineering. Ph.D. The- sis, T echnisc he Universitat Wien (2007) 11. Han, H., Shakkottai, S., Hollot, C.V., Srik ant, R., T owsley , D.: Overla y tcp for multi- path routing and congestion control. In: ENS-INRIA ARC-TCP W orkshop, Paris, F rance (2004) 12. He, J., Rexford, J.: T ow ards internet-wide multipath routing. Network, IEEE 22 (2), 16–21 (2008) 13. Kelly , F., Massouli´ e, L., W alton, N.: Resource p o oling in congested netw orks: prop or- tional fairness and pro duct form. Queueing Systems 63 (1-4), 165–194 (2009) 14. Kelly , F., Maulloo, A., T an, D.: Rate control for comm unication netw orks: shadow prices, proportional fairness and stabilit y . Journal of the Op erational Researc h So ciety 49 (3), 237–252 (1998) 15. Kelly , F.P ., V oice, T.: Stability of end-to-end algorithms for joint routing and rate control. Computer Comm unication Review 35 (2), 5–12 (2005) 16. Key , P .B., Massouli ´ e, L., T owsley , D.F.: Path selection and multipath congestion con trol. Communications of the ACM 54 (1), 109–116 (2011) 17. Kunniyur, S., Srik ant, R.: End-to-end congestion control schemes: utility functions, random losses and ecn marks. IEEE/ACM T ransactions on Netw orking 11 (5), 689–702 (2003) 18. Lee, G.M., Choi, J.S.: A survey of multipath routing for traffic engineering. Av ailable trhough http://www.slashdocs.com/ivmqst/a-survey-of-m ultipath-routing.html (2002) 19. Lin, X., Shroff, N.: Utility maximization for communication networks with multipath routing. IEEE T ransactions on Automatic Con trol 51 (5), 766–781 (2006) 20. Low, S.: A dualit y mo del of tcp and queue management algorithms. IEEE/ACM T rans- actions on Net working 11 (4), 525–536 (2003) 21. Low, S., Paganini, F., Doyle, J.: Internet congestion control. IEEE Control Systems Magazine 22 (1), 28–43 (2002) 22. Low, S., P eterson, L., W ang, L.: Understanding vegas: a duality mo del. Journal of the ACM 49 (2), 207–235 (2002) 23. Nagourney , A.: The negation of the braess paradox as demand increases: The wisdom of crowds in transp ortation net works. Europhysics Letters 91 (4), 48,002 (2010) 24. Padh y e, J., Firoiu, V., T owsley , D., Kurose, J.: Modeling tcp reno p erformance: a simple model and its empirical v alidation. IEEE/ACM T ransactions on Netw orking 8 (2), 133– 145 (2000) 25. Paganini, F.: Congestion control with adaptiv e m ultipath routing based on optimization. In: Information Sciences and Systems, pp. 333–338 (2006) 26. Paganini, F., Mallada, E.: A unified approach to congestion control and no de-based multipath routing. IEEE/ACM T ransactions on Net working 17 (5), 1413–1426 (2009) 27. Roughgarden, T.: On the severity of braess’s paradox: Designing netw orks for selfish users is hard. Journal of Computer and System Sciences 72 (5), 922–953 (2006) 28. Saib ene, J.P ., Lempert, R., Paganini, F.: An implemen tation of optimal dynamic load balancing based on multipath ip routing. In: G LOBECOM, pp. 1–5 (2010) 29. Villamizar, C.: Mpls optimized m ultipath (mpls-omp). Internet Draft, draft-ietf- mpls- omp-01, http://tools.ietf.org/html/draft-villamizar-mpls-omp-01 (1999) 30. W alton, N.S.: Proportional fairness and its relationship with multi-class queueing ne t- works. Annals of Applied Probabilit y 19 (6), 2301–2333 (2009) 31. W ardrop, J.G.: Some theoretical asp ects of road traffic research. Proceedings of the Institute of Civil Engineers, Part I I pp. 325–378 (1952) 32. Xu, D., Chiang, M., Rexford, J.: Link-state routing with hop-by-hop forwarding can achiev e optimal traffic engineering. IEEE/ACM T rans. Netw. 19 (6), 1717–1730 (2011) 33. Y a ¨ ıche, H., Mazumdar, R., Rosenberg, C.: A game theoretic framework for rate alloca- tion and charging of a v ailable bit rate (abr) connections in atm net works. In: Broadband Communications, pp. 222–233 (1998)