Asymptotics of the Invariant Measure in Mean Field Models with Jumps

Sto chastic Systems 2012,V ol. 2, No. 0, 1– 59 DOI: 10.12 14/12- SSY64 ASYMPTOTICS OF THE INV ARIANT MEASURE IN MEAN FIE L D MODELS WITH JUMPS ∗ By Vivek S. Borkar † ?? and Rajesh Sundaresan ‡ ?? Indian Institute of T e chnolo gy Bomb ay and Indian Institute of Scienc e W e consider the asymptotics of th e inv ariant measure for the process of the empirical spatial distribution of N coupled Marko v chai ns in the limit of a large num b er of chains. Eac h chain reﬂects the stochastic evol ution of one particle. The chai ns are coupled through the depen dence of the transition rates on this spatial distribu t ion of particles in the v arious states. Ou r model is a caricature for med ium access interactions in wireless local area netw orks. It is also applica- ble to the study of spread of epidemics in a netw ork. The limiting process sa tisﬁes a deterministic ordinary diﬀerential equation called the McKean-Vlasov equation. When this diﬀerential equation h as a unique globally asympt otically stable equilibrium, the spatial d istri- bution asymptotically concentrates on this equilibrium. More gener- ally , its limit p oints are su p p orted on a sub set of the ω -limit sets of the McKean-V laso v equation. Using a control-theoretic approach, w e examine the q uestion of large d eviations of th e in v arian t measure from th is limit. 1. Introduction. Spur r ed b y the seminal work of B ianc hi [ 5 ], there has b een a ﬂurry of act ivit y in the comm unication netw orks comm unit y on m ean ﬁeld models for carrier sense multiple access (CSMA) proto cols and their large time b ehavio r. Th e con tin uous-time mo del for the wireless lo cal area net w ork (WLAN) is as follo ws. Th ere are N particles (no d es) in the netw ork. A t eac h instant of time, a particle’s state is a p articular v alue tak en f rom the ﬁnite state space Z = { 0 , 1 , . . . , r − 1 } . A particle’s Received Jan uary 2012. ∗ A p art of th is p ap er without pro ofs was presented as an invited pap er at the 2011 Annual A llerton Conference on Comm unication, Con trol, and Computing, Allerton, IL, USA, Sep tember 2011. † W ork supp orted by a J. C. Bose F ello wship and by IFCP AR (Indo-F rench Centre for the Promotion of Ad v anced Researc h), Pro ject 4000-IT-1. ‡ W ork sup p orted by IFCP AR (Indo-F rench Centre for the Promotion of Adva nced Researc h), Pro ject 4000-IT-1. AMS 2000 subje ct classiﬁc ations: Primary 60K35, 60F10, 68M20, 90B18, 49J15, 34H05. Keywor ds and phr ases: Decoupling approximation, ﬂuid limit, inv arian t measure, McKean-Vlaso v equation, mean ﬁeld limit, small noise limit, stationary measure, sto chas- tic Liouville equation. 1 2 V. S. BORKAR AND R. SUNDARESAN state repr esen ts the num b er of failed attempts at transmission of the head- of-the-line p ac k et at that particle’s qu eue. When a particle is in state i , a successful transmission gets the pac k et out of th e sys tem, and the particle mo v es to state 0 to service the next p ac k et. A failed transmission m ov es the particle to state i + 1 (mo d r ). In the case when i w as initially r − 1, th at is, r − 1 unsuccessful transmission attempts w ere already made, another f ailed attempt results in the d iscarding of the pac k et. The p article then mo v es to state 0 w ith th e next pac k et readied for transmission. W e ma y interpret r as the maximum n umber of transmission attempts. Th e transition rate for a particle from state i to state j is go v erned by me an ﬁeld dynamics , that is, the trans ition rate is λ i,j ( µ N ( t )) wher e µ N ( t ) is the empirical distribu tion of the states of particles at time t . If X ( N ) n ( t ) is the state of the n th p article at time t , then one ma y wr ite µ N ( t ) as µ N ( t ) = 1 N N X n =1 δ { X ( N ) n ( t ) } . The particles in teract only through the dep endence of their transition r ates on the cu rrent empirical measure µ N ( t ). The tr ansitions allo w ed in the ab o v e m o del are fr om state i to either i + 1 (mo d r ) or 0. Let us sa y that E d enotes the set of allo wed transitions. In the ab o v e mo del, E = { ( i, i + 1) , i = 0 , 1 , . . . , r − 1 } ∪ { ( i, 0) , i = 0 , 1 , . . . , r − 1 } where the addition is taken mo dulo r . The p ro cess X ( N ) ( · ) = { X ( N ) n ( · ) , 1 ≤ n ≤ N } is clearly a Marko v pr o cess. But one diﬃcult y n eeds to b e su rmounte d in analyzing this system: the size of the state sp ace gro ws exp onentiall y in th e num b er of p articles. A step to w ards addressing this d iﬃcult y is to consider the evolutio n or ﬂow of the empirical measur e o v er time, whic h we shall call empiric al pr o c ess . This is a sto c hastic Liouville equation that liv es on a smaller state space. In the inﬁnite particle limit, this evo lution turns out to b e deterministic and is giv en by the McKean-Vlaso v equ ation, w hose large time b ehavio r is an ind icator of what one might exp ect of a ﬁnite bu t large p opu lation. In particular, if the deterministic evo lution, giv en by the McKean-Vlaso v equation, h as a un ique globally asymptoticall y stable equilibrium, then the states of a ﬁ n ite num b er of tagged particles are asymptotically in dep end en t and their joint la w is giv en b y the pro du ct of this equilibrium measur e. The idea in f act w as introd uced by Kac as a simp le mo del in kinetic theory [ 23 ] and was later studied b y McKean and others (see, e.g., [ 25 ]). See [ 32 ] for an extensiv e account and [ 22 ] for a tr eatment of p ro cesses with jumps. INV ARIA N T MEASU RE IN MEAN FIELD MODELS 3 Sev eral pap ers h a v e p ro vided rigorous analyses, along the ab o v e lin es, of Bianc hi’s heuristic f or studying WLANs. See, e.g., [ 9 , 27 , 8 , 33 , 3 , 26 ]. See [ 16 ] for an excellen t surve y , [ 31 , 1 , 2 ] for early pr ecursors, and [ 4 ] for an ap p lication of the s ame tec hnique in game theory . As remarked in [ 16 ], exp erimenta l evidence for CSMA p roto cols indicates that the m o del and the predictions made b y the analyses are sur prisingly accurate ev en for sm all p opulations. In deed, this is one of the main reasons for the m o del’s en ormous p opularity . Is there a justiﬁcation f or this concentrat ion phenomenon? The mean-ﬁeld analysis has b een successful w h en the McKean-Vlaso v equation has a un iqu e globally asymptotically stable equilibrium. This is indeed the case in th e simplest of WLAN settings with exp onen tial b ac k oﬀ parameters. But there are settings w ith m ultiple s table equilibria [ 33 ] or with a unique equilibrium that is not globally stable (see [ 3 ] for a malw are propagation mo del). In b oth cases, the dy n amics go v erned by the McKean- Vlaso v equation has multiple ω -limit sets. Signiﬁcant eﬀort has gone into iden tifying s u ﬃcien t conditions for a u nique equilibrium [ 26 ], and into iden- tifying furth er suﬃcient conditions for a uniqu e globally asymptotically sta- ble equilibrium [ 8 ]. If there are multiple ω -limit sets f or the McKean-Vlaso v dynamics, w hic h of these c haracterize the limiting b eha vior? As a step in the direction of un d erstanding these questions, we study the con tin uous-time mo del in this pap er w ith the follo wing goals. 1. Obtain a large deviation prin ciple o v er ﬁnite time d urations for the sequence of empirical m easures and empirical pr o cesses, uniformly in the initial condition. This is of course rough asymp totics for large N , but suggests exp onentia lly f ast conv ergence to the deterministic limit. 2. Obtain a large deviation prin ciple for the sequence of in v ariant mea- sures, with single or multi ple stable limit s ets. T his helps resolve wh ich of the sev eral ω -limit sets, w hen there are s everal, will b e selected in the large N limit. (Ou r work can b e straigh tforw ardly extended to study the n ature of transitions and exit times fr om the neighborho o d of one stable equ ilibr ium to the neighborh o o d of another, and help understand metastable b eha vior in such systems. W e do n ot p ursue these here.) 3. Pro vide a control theoretic framew ork to solv e the problem of in v arian t measure in order to exp ose its s trength and limitation. As we will see, we can go quite some distance using this deterministic approac h, but ev en tually need to stu d y the noisy system for resolution of some degeneracies. 4 V. S. BORKAR AND R. SUNDARESAN It m ust b e noted that the ab o v e con tin u ous-time mo d el do es not p erfectly capture all asp ects in a WLAN. In p articular, interac tions and c hanges of states o ccur in d iscrete-time units of slots in WLANs, and m ultiple n o des ma y transit in one slot. Mu ltiple transitions neve r o ccur, almost sur ely , in our con tin uous-time mo del. Nev ertheless, if the discrete-time mo del’s transition rates and the slot sizes are appropriately scaled do wn as N gro ws so that the transition r ates approac h constan ts, our con tin u ous time mo del provides accurate predictions of b ehavio r on the discrete-time mo del. Ou r mo del also has wider applicabilit y , one example b eing the study of sp read of epid emics in a n et w ork; see [ 15 , Sec. 2.4]. Large deviation principles o v er compact time durations for interacti ng diﬀusions an d interac ting jump Mark o v pro cesses h a v e b een we ll-studied b y s ev eral authors, e.g., [ 11 , 24 , 15 , 19 , 10 , 14 ]. T he works [ 15 , 10 , 14 ] establish large d eviation pr inciples for empirical measur es in path s pace o v er ﬁnite time d urations and characte rize the rate functions, while [ 19 ] considers in ﬁ nite time dur ations under the in ductiv e top ology . The wo rks [ 11 , 24 , 15 , 10 ], and [ 20 ] also study large deviation of the empirical pro cess from the McKean-Vlaso v limit, ov er ﬁnite time durations or inﬁnite time durations u nder the ind uctiv e top ology ([ 20 ]). Th e rate fu nction measures the d iﬃcult y of passage of the empir ical pro cess in the n eigh b orho o d of a deviating p ath. F or a ﬁxed N , wh en t → + ∞ , the stationary or in v ariant measur e is of in terest. When the limiting McKean-Vlaso v dynamics has a uniqu e globally asymptotically stable equilibrium , the inv arian t m easure con v erges we akly in the in ﬁnite particle limit to the p oin t mass at the equilibrium of the McKean-Vlaso v dynamics (see, e.g., [ 3 ]). When ther e are multiple equilib- ria, the in v arian t measur e concen trates on a subset of the ω -limit sets for the McKean-Vlaso v dyn amics ([ 21 , Ch. 6] and [ 4 ]). Large deviations from this limit h a v e b een well-studied (see [ 21 , C h . 6], [ 30 ]). As indicated earlier, large d eviation r esults for the inv arian t measure h elp r esolv e wh ic h of sev- eral p ossible ω -limit sets ma y b e selected in the limit of a large n umber of no des. They also help und erstand and p redict metastable b eha vior in suc h systems. I f the s ystem is trapp ed in an undesirable equilibrium, exit times from domains an d lik ely p aths can b e predicted. Our approac h for solving the large deviations of the inv ariant measur e exploits a control- theoretic view d escrib ed in [ 6 ], which consid ers a class of d iﬀu sions and generalizes results of [ 29 ] and [ 12 ] on small noise asym p - totics of inv arian t measures and exit pr ob ab ilities in diﬀus ions . See [ 28 ] for small noise asymptotics of exit probabilities in pr o cesses w ith j u mps. T he con trol-theoreti c approac h diﬀers from th ose of F reidlin and W entzel l [ 21 ] INV ARIA N T MEASU RE IN MEAN FIELD MODELS 5 and Shw artz and W eiss [ 30 ]. The latter rely on a stud y of an emb edded Mark o v chain of states at h itting times of neigh b orho o d of the s table limit sets (see, e.g., [ 21 , C h. 6.4]). Our control -theoretic approac h en ables the iden tiﬁcation of a un ique rate fun ction when there is a un ique globally asymptotically stable equilibrium for th e McKean-Vlaso v dynamics. When dealing with multiple ω -limit sets thou gh , our approac h ev en tually requires the study of the noisy system asso ciated with th e em b edded Mark o v c hain describ ed ab o v e, for resolution of certain b oundary conditions needed f or a full c haracterization of the rate function. W e now outline our m ain arguments and describ e the pap er’s organiza- tion. W e b egin with a formal description of the mo del and s tatement s of the main results on in v arian t measures in Section 2 . In Section 3.1 , we ﬁ rst es- tablish a large deviation principle for empirical measures of paths ov er ﬁn ite durations. F or pu re jum p pro cesses w ith interacti ons, this w as established b y [ 24 , 15 , 19 , 10 , 14 ], as in dicated earlier. While [ 24 ] considers a ﬁxed initial condition for all the particles, [ 10 , 19 ], and [ 14 ] consider ran d om, indep en- den t, and identica lly distr ibuted (also called c haotic) initial conditions for the particles. T o establish the large deviation principle for the in teracting system, they ﬁrst establish the large deviation principle for the in dep end en t and noninte racting system via S ano v’s theorem, then exploit the Girsano v transformation to describ e the probability measure for the inte racting case, and then apply the Laplace-V aradhan principle. In order to ev en tually p ass to the inv arian t measure, w e need to establish a stronger uniform large devi- ation principle when the initial conditions of the particles are suc h th at the initial empir ical measures con v er ge w eakly , bu t are otherwise arbitrary . (See the r emark f ollo wing [ 19 , T h . 4.1].) The limiting (initial) empirical measure deﬁnes the initial condition f or the limiting deterministic McKean-Vlaso v dynamics. This str on ger u n iform large deviation principle alluded to ab o v e is av ailable for diﬀusions with mean ﬁeld in teractions in [ 11 ] and for certain classes of ju m p pr o cesses in [ 15 ] where th e holding times alone are mediated b y the in teracti on and not the j ump p robabilities. Our r esult is a mild exten- sion facilitated by a generalizatio n of S ano v’s theorem giv en in [ 11 ]. While this p art of our r esult is n ot surpr ising, we could not ﬁ nd a ready reference in the literature, and s o w e state th e result in Section 3.1 and pr o vide a pro of in Section 5 b ased on the approac h in [ 24 ]. W e r eemphasize that this result is only for ﬁ nite d u rations, and is only a step to w ards addressing our next goal of asymptotics of the inv ariant measure. In S ection 3.2 , we apply the contract ion principle to obtain results on a large deviation fr om the McKean-Vlaso v limit. As in [ 24 ], we pro v e this 6 V. S. BORKAR AND R. SUNDARESAN under the ﬁner un iform norm top ology . In Section 3.3 , w e once again apply the con tractio n prin ciple to argue a large deviation p rinciple for term in al measure. W e then establish some crucial estimates on th e rate function for use in later sections. Finally in this section, we argue th at if th e initial measures satisfy a large deviation p rinciple, then s o do the joint initial and terminal measures. In Section 4 , we pro v e the large deviation principle for the inv arian t mea- sure. T o do this, w e ﬁrst establish a sub sequen tial large d eviation p rinciple, then argue that the rate function satisﬁes the dyn amic programming equ a- tion f or a particular control problem, and then ﬁ n ally show that th e rate function is u nique if the asso ciated McKean-Vlaso v dyn amics has a unique globally asymp totical ly stable equilibriu m. The arguments to establish the uniqueness of the rate fu nction parallel those of [ 6 ] w ith the s igniﬁcan t d if- ference that while the dynamic p rogramming equation was arrived at in [ 6 ] via the theory of viscosit y solutions, h ere we use the contrac tion prin ciple. While this provides a complete r esult in case of a single globally asymp toti- cally stable equilibrium, it lea v es some indeterminacy r egarding uniqueness of the so called ‘p oten tial fu n ction’ whose global minima th e in v ariant mea- sure concentrate s on in the large N limit. T o resolv e this, one has to fall bac k u p on the framework of F reidlin and W en tzell [ 21 , Ch.6] w ith minor mo diﬁcations. T hese mo diﬁ cations are detailed in the App endix. Pro ofs that are not cent ral to our con trol-theoret ic view p oint are r ele- gated to Sections 5 , 6 , 7 , and 8 . Th e App endix d etails the minor m o diﬁca- tions to the arguments of F reidlin and W ent zell [ 21 , Ch.6] for resolution of the rate function v alues at the stable limit p oin ts. 2. The mo del and main results. Consider N interact ing Mark o v c hains den oted by X ( N ) n ( t ) , 1 ≤ n ≤ N , t ≥ 0 , on a ﬁ nite state space Z = { 0 , . . . , r − 1 } , w ith dynamics as f ollo ws. X ( N ) n ( t ) denotes the state of the n th p article at time t . Let µ N ( t ) b e the emp irical measure of the particles at time t , that is, µ N ( t ) = 1 N N X n =1 δ { X ( N ) n ( t ) } ∈ M 1 ( Z ) , where M 1 ( Z ) is the set of probability vec tors o v er Z , endow ed with the top ology of wea k conv ergence on M 1 ( Z ). A particle n in state i transits to state j ∈ Z with a rate λ i,j ( µ N ( t )) that dep ends on the states of the other particles only through the empirical measure at that time. Thus the INV ARIA N T MEASU RE IN MEAN FIELD MODELS 7 pro cesses int eract only thr ou gh th e dep enden ce of their transition rates on the curr en t empirical measur e. Let E ⊂ Z × Z \ { ( i, i ) | i ∈ Z } b e the set of admissible jump s for eac h p article. Thus λ i,j ( · ) ≡ 0 wheneve r ( i, j ) / ∈ E and i 6 = j . W e write Z i = { j ∈ Z | ( i, j ) ∈ E } for the set of states to whic h a particle can jump from state i . W e m ake the follo w ing assumptions thr oughout the pap er. ( A1 ) The graph with v ertices Z and dir ected edges E is irr educible. ( A2 ) The mappings µ ∈ M 1 ( Z ) 7→ λ i,j ( µ ) ∈ [0 , + ∞ ) are Lipschitz. ( A3 ) The r ates of the admissib le jum ps are un iformly b ounded a w a y from zero, that is, th ere exists c > 0 such that, for all µ ∈ M 1 ( Z ) and all ( i, j ) ∈ E , w e ha v e λ i,j ( µ ) ≥ c . Since M 1 ( Z ) is compact and λ i,j ( · ) are con tin uous b y assumption ( A2 ), the rates are u niformly b ou n ded from ab o v e, that is, there is a C < ∞ such that for all µ ∈ M 1 ( Z ), and all ( i, j ) ∈ E , w e hav e λ i,j ( µ ) ≤ C . F or an y T ∈ (0 , + ∞ ), write X ( N ) n : [0 , T ] → Z for the pro cess of evol ution of particle n ov er time. This is an element of the set D ([0 , T ] , Z ) of all cadlag paths from [0 , T ] to Z equipp ed with the Sk oroho d top ology . W e set the p ath to b e left con tin uous at T . Let X N = ( X ( N ) n , 1 ≤ n ≤ N ) ∈ D ([0 , T ] , Z N ) denote the f ull d escription of paths of all N particles. Th e initial condition at time t = 0 is z N = ( z n , 1 ≤ n ≤ N ). T he pro cess X N , w ith its la w denoted P ( N ) z N , is a Mark o v p ro cess with cadlag p aths, state space Z N , and generator A ( N ) acting on b ound ed measurable functions Φ according to (2.1) A ( N ) Φ( a N ) = N X n =1 X j ∈Z a n  λ a n ,j  g N ( a N )   Φ( ♯ ( a N , n, j )) − Φ( a N )  where a N = ( a n , 1 ≤ n ≤ N ) ∈ Z N , ♯ ( a N , n, j ) is the elemen t of Z N that results from replacing th e n th comp onent of a N with j , and g N ( a N ) = 1 N P N n ′ =1 δ a n ′ is the emp irical measur e asso ciated with the conﬁguration a N . (The up p er b ound edness of the rates implies that the martingale pr ob- lem asso ciated with the generator A ( N ) , op erating on b ounded measur- able f unctions, and the in itial condition z N ∈ Z N admits a un iqu e solution P ( N ) z N ∈ M 1 ( D ([0 , T ] , Z N )); see for e.g., [ 17 , Prob lem 4.11.1 5, p. 263]). The empirical measure pro cess asso ciated with X N is give n b y µ N : t ∈ [0 , T ] 7→ µ N ( t ) = 1 N N X n =1 δ { X ( N ) n ( t ) } ∈ M ( N ) 1 ( Z ) , 8 V. S. BORKAR AND R. SUNDARESAN where M ( N ) 1 ( Z ) = { g N ( a N ) | a N ∈ Z N } ⊂ M 1 ( Z ). As a consequence of ( 2.1 ), the pr o cess µ N is itself a Marko v p ro cess w ith cadlag paths, ﬁ nite state s pace M ( N ) 1 ( Z ), and generator A ( N ) acting on b ounded measurable functions Ψ according to (2.2) A ( N ) Ψ( ξ ) = N X i ∈Z X j ∈Z i [ ξ ( i ) λ i,j ( ξ )]  Ψ( ξ − N − 1 δ i + N − 1 δ j ) − Ψ( ξ )  . W rite λ i,i ( ξ ) = − P j ′ 6 = i λ i,j ′ ( ξ ) and deﬁne (2.3) A ξ = ( λ i,j ( ξ )) ( i,j ) ∈Z ×Z to b e the r ate matrix o v er Z . I t is w ell kn own that the family ( µ N , N ≥ 1) satisﬁes the weak la w of large num b ers in the follo wing sense: if µ N (0) → ν w eakly as N → ∞ for some ν ∈ M 1 ( Z ), th en µ N → µ uniformly on compacts in probabilit y , wh ere µ solv es the M cKe an-Vlasov e qu ation (2.4) ˙ µ ( t ) = A ∗ µ ( t ) µ ( t ) with in itial condition µ (0) = ν . Here µ ( t ) is in terpreted as a column v ector and A ∗ µ ( t ) is the adj oint/transp ose of the m atrix A µ ( t ) . By assump tion ( A2 ), uniqueness of solutions holds for the nonlin ear ordinary d iﬀerential equation (ODE) ( 2.4 ). As a consequence of the assu m ption ( A1 ), for eac h N ≥ 1, ther e is a unique in v ariant measure f or the Mark o v pro cess X N , and hence there is a unique in v ariant measure ℘ ( N ) for the M ( N ) 1 ( Z )-v alued Mark o v pro cess µ N . Deﬁne τ : R → R + to b e (2.5) τ ( u ) = e u − u − 1 , and let τ ∗ : R → R + b e its Legendr e conjugate (2.6) τ ∗ ( u ) =    ( u + 1) log( u + 1) − u if u > − 1 1 if u = − 1 + ∞ if u < − 1 . The ﬁ rst main r esult is on the large N asymp totics of the sequence of in v ariant measures ( ℘ ( N ) , N ≥ 1). Theorem 2.1 . Assume ( A1 ) - ( A3 ) . L et the McK e an-Vlasov e quation ˙ µ ( t ) = A ∗ µ ( t ) µ ( t ) have a unique glob al ly asymptotic al ly stable e quilibrium ξ 0 . INV ARIA N T MEASU RE IN MEAN FIELD MODELS 9 Then the se qu enc e ( ℘ ( N ) , N ≥ 1) satisﬁes the lar g e deviation principle with sp e e d N and go o d r ate function s gi v en by (2.7) s ( ξ ) = inf ˆ µ Z [0 , + ∞ ) h X ( i,j ) ∈E ( ˆ µ ( t )( i )) λ i,j ( ˆ µ ( t )) τ ∗ ˆ l i,j ( t ) λ i,j ( ˆ µ ( t )) − 1 ! i dt wher e the inﬁmum is over al l ˆ µ that ar e solutions to the dynamic al system ˙ ˆ µ ( t ) = − ˆ L ( t ) ∗ ˆ µ ( t ) for some family of r ate matric es ˆ L ( · ) , with i ni tial c on- dition ˆ µ (0) = ξ , terminal c ondition lim t → + ∞ ˆ µ ( t ) = ξ 0 , and ˆ µ ( t ) ∈ M 1 ( Z ) for al l t ≥ 0 . Remarks . 1. As we sh all see later, the dynamics ˙ ˆ µ ( t ) = − ˆ L ( t ) ∗ ˆ µ ( t ) corresp onds to a time reve rsal when compared with the d irection of the McKean-Vlaso v d ynamics. The rate fun ction is giv en by the cost, asso ci- ated with a certain con trol prob lem, of the c heap est p ath (across all time) that transp orts the system state fr om the globally asymp toticall y stable equilibrium ξ 0 to ξ , in the forw ard-time dyn amics pro ceeding in the direc- tion of the McKean-Vlaso v dy n amics. In the time-rev ersed dynamics, this is the cost of a path with initial state ξ and terminal state ξ 0 . 2. By setting ˆ l i,j ( t ) ≡ λ i,j ( ξ 0 ), we see that if ˆ µ (0) = ξ 0 , then the sys tem state remains ˆ µ ( t ) ≡ ξ 0 , and the in tegral is 0. It follo ws that s ( ξ 0 ) = 0. 3. In reversed time, th e system state u nder the d ynamics ˙ ˆ µ ( t ) = − ˆ L ( t ) ∗ ˆ µ ( t ) ma y not in general lie in M 1 ( Z ). The minimization ho w ev er is o v er all paths that are constrained to lie in M 1 ( Z ). See additional remarks after Lemma 4.2 . 4. W e no w make some r emarks on wh y Theorem 2.1 and the s o on to follo w Theorem 2.2 are not sub s umed by the w orks of [ 30 ] and [ 21 ]. The transition rates for the Marko v p ro cess µ N ( · ) are ( µ N ( t )( i )) λ i,j ( µ N ( t )). While λ i,j ( µ N ( t )) is in deed b ounded a w a y from zero wh en ( i, j ) ∈ E , ( µ N ( t )( i )) λ i,j ( µ N ( t )) is not, and so the logarithm of these rates is not b ounded, a requirement in [ 30 ]. Next, the pro cess µ N tak es v alues only in M 1 ( Z ), and so when at the b ound ary M 1 ( Z ), it is constrained to mo v e only in those directions that k eep it within M 1 ( Z ). As a consequence, a ﬁn iteness condition [ 21 , p .146, I I] on the asso ciated Lagrangian fun ction d o es not hold. 5. Th eorem 2.1 pro vides a complete c haracterization of the rate function. Ho w ev er, numerical computation of the rate fu nction is a c hallenging prob- lem. One might p ossibly discretize time and the state space, and emplo y dynamic programming tec hniques to get an app ro ximation. This is an in- teresting line of work that is b eyo nd the scop e of th is pap er. S ee [ 4 ] and references therein for some r esults based on exit times. 10 V. S. BORKAR AND R. SUNDARESAN W e next state a generalizat ion of Theorem 2.1 when there may b e multiple ω -limit sets. Consider a time-v arying rate matrix L ( t ) asso ciated with the time-v arying rates ( l i,j ( t ) , ( i, j ) ∈ E ) and let L ( t ) ∗ b e its ad j oin t. W rite µ for the solution to the dyn amical system ˙ µ ( t ) = L ( t ) ∗ µ ( t ) with initial condition µ (0) = ν . Deﬁne (2.8) S [0 ,T ] ( µ | ν ) = Z [0 ,T ] h X ( i,j ) ∈E ( µ ( t )( i )) λ i,j ( µ ( t )) τ ∗  l i,j ( t ) λ i,j ( µ ( t )) − 1  i dt. As we shall see later, this is the cost of moving the system state along the tra jectory µ with in itial state µ (0) = ν . Let V ( ξ | ν ) b e the so-called quasip otent ial deﬁned by (2.9) V ( ξ | ν ) = inf { S [0 ,T ] ( µ | ν ) | µ (0) = ν, µ ( T ) = ξ , t ∈ [0 , T ] , T ≥ 0 } , that is, the inﬁmum cost of tra v ersal fr om ν to ξ o v er all ﬁnite time d urations. Let u s deﬁn e an equ iv alence relation on M 1 ( Z ) as follo ws. W e s a y ν ∼ ξ if V ( ξ | ν ) = V ( ν | ξ ) = 0. Using a later r esult (the th ir d part of Lemma 3.2 ), it is easy to sho w that the set of p oints that are equiv alen t to eac h other is closed and therefore compact (b eing a closed sub set of the compact set M 1 ( Z )). W e will generalize Theorem 2.1 under th e follo wing assumption on the dynamical system corresp ond ing to the McKean-Vlaso v equation ( 2.4 ): ( B ) There exist a ﬁ nite num b er of compact sets K 1 , K 2 , . . . , K l suc h that 1. ν 1 , ν 2 ∈ K i implies ν 1 ∼ ν 2 . 2. ν 1 ∈ K i , ν 2 / ∈ K i implies ν 1 ≁ ν 2 . 3. Ev ery ω -limit set of th e McKean-Vlaso v equation ˙ µ ( t ) = A ∗ µ ( t ) µ ( t ) is con tained in one of the K i . Under the hypothesis of Theorem 2.1 , assumption ( B ) holds w ith l = 1, K 1 = { ξ 0 } . W e sa w in a r emark follo wing Theorem 2.1 that s ( ξ 0 ) = 0. In the general case, we s h all see later in Lemma 4.6 that the rate function s is constant within eac h K i , and so let s i b e the v alue of s ( · ) o v er K i , for i = 1 , . . . , l . I n order to s p ecify the v alues of s 1 , . . . , s l , deﬁne ˜ V ( K i , K j ) = inf T > 0  S [0 ,T ] ( µ | µ (0)) | µ (0) ∈ K i , µ ( T ) ∈ K j , µ ( t ) / ∈ ∪ i ′ 6 = i,j K i ′ for t ∈ [0 , T ]  . (2.10) If the set is empty , the inﬁm um is tak en to b e + ∞ . Let us also d eﬁne V ( K i , K j ) = V ( ξ | ν ) | ν ∈ K i ,ξ ∈ K j . (2.11) INV ARIA N T MEASU RE IN MEAN FIELD MODELS 11 Again u sing Lemm a 4.6 , one can sho w that the ab o v e v alue is indep endent of ν ∈ K i and ξ ∈ K j . Also, as indicated by F reidlin and W entze ll ([ 21 , p.171]) and b y Lemma A.2 of th e App en d ix, one can easily argue that V ( K i , K j ) = ˜ V ( K i , K j ) ∧ min i 1 [ ˜ V ( K i , K i 1 ) + ˜ V ( K i 1 , K j )] ∧ min i 1 ,i 2 [ ˜ V ( K i , K i 1 ) + ˜ V ( K i 1 , K i 2 ) + ˜ V ( K i 2 , K j )] ∧ · · · ∧ min i 1 ,...,i l − 2 [ ˜ V ( K i , K i 1 ) + · · · + ˜ V ( K i l − 2 , K j )] . Consider the indices { 1 , 2 , . . . , l } for the compact sets K 1 , K 2 , . . . , K l . Let G { i } b e the set of all dir e cte d gr aphs on the vertex set { 1 , 2 , . . . , l } such that • there is n o out w ard edge from i ; • a vertex j 6 = i has exactly one outw ard edge; • there are no closed cycles in the graph. Deﬁne (2.12) W ( K i ′ ) = min G ∈ G { i ′ } X ( i,j ) ∈G V ( K i , K j ) , and ﬁn ally (2.13) s i ′ = W ( K i ′ ) − min i W ( K i ) . W e n o w state the generalization of Theorem 2.1 under assu m ption ( B ). Theorem 2.2 . Assume ( A1 ) - ( A3 ) and ( B ) . The se q u enc e ( ℘ ( N ) , N ≥ 1) satisﬁes the lar ge deviation principle with sp e e d N and go o d r ate function s given by (2.14) s ( ξ ) = in f l ′ inf ˆ µ   s l ′ + Z + ∞ 0 h X ( i,j ) ∈E ( ˆ µ ( t )( i )) λ i,j ( ˆ µ ( t )) τ ∗ ˆ l i,j ( t ) λ i,j ( ˆ µ ( t )) − 1 ! i dt   wher e the se c ond inﬁmum is over al l ˆ µ that ar e solutions to the dynamic al system ˙ ˆ µ ( t ) = − ˆ L ( t ) ∗ ˆ µ ( t ) for some family of r ate matric es ˆ L ( · ) , with initial c ondition ˆ µ (0) = ξ , terminal c ondition ˆ µ ( t ) → K l ′ as t → + ∞ , and ˆ µ ( t ) ∈ M 1 ( Z ) f or al l t ≥ 0 . 3. Large deviations ov er a ﬁnite time duration. In the p revious section, w e s tated th e main results of the pap er on the sequence of in v ariant measures. W e now b egin our journey to w ards the p ro ofs by studying large deviation principles o v er ﬁn ite time du rations. W e ﬁr st stud y the empiri- cal measure of paths o v er ﬁnite du rations, and then the empirical measure pro cess. 12 V. S. BORKAR AND R. SUNDARESAN 3.1. Empiric al me asur e. Recall that x N ∈ D ([0 , T ] , Z N ) denotes th e fu ll description of all the N particles. Let G N denote the mappin g that takes the fu ll description x N to the empirical measure G N : ( x n , 1 ≤ n ≤ N ) ∈ D ([0 , T ] , Z N ) 7→ 1 N N X n =1 δ x n ∈ M 1 ( D ([0 , T ] , Z )) . Giv en the r andom v ariable X N , the rand om empirical measure, denoted M N , is thus M N = G N ( X N ) ∈ M 1 ( D ([0 , T ] , Z )) . Clearly , the la w of M N dep end s on the initial condition z N only through its empirical m easure ν N = (1 / N ) P N n =1 δ z n . W rite P ( N ) ν N for the law of M N , the push forward of P ( N ) z N under the mapping G N , that is, P ( N ) ν N = P ( N ) z N ◦ G − 1 N . The M ( N ) 1 ( Z )-v alued cadlag empir ical pr o cess is µ N : t ∈ [0 , T ] 7→ µ N ( t ) = 1 N N X n =1 δ { X ( N ) n ( t ) } ∈ M ( N ) 1 ( Z ) , and the corresp ondin g mapp ing is denoted γ N : ( x n , 1 ≤ n ≤ N ) ∈ D ([0 , T ] , Z N ) 7→ µ N : [0 , T ] → M ( N ) 1 ( Z ) . Observe that µ N (0) = ν N , and that µ N ( t ) is the pro jection π t ( M N ) at time t , and we write µ N = π ( M N ). W e thus ha v e µ N = π ( M N ) = π ( G N ( X N )) = γ N ( X N ) . Consider n o w a hyp othetical tagged p article. When there is no interac- tion, w hen all transition rates for ( i, j ) ∈ E tran s itions are u nit y , and when all other transition rates are 0, we can deﬁ ne the evo lution of the tagged particle by the la w P z whic h is the un ique solution to the martin gale prob- lem in D ([0 , T ] , Z ) asso ciated w ith the generator A o op erating on b ou n ded measurable fu nctions Φ on Z according to A o Φ( i ) = X j ∈Z i 1 · (Φ( j ) − Φ( i )) and the in itial condition z . The existence of a solution and th e s olution’s uniqueness hold b ecause the transition rates are up p er b ound ed (see [ 17 , Problem 4.11.15 ]). F or an y ﬁxed µ ∈ D ([0 , T ] , M 1 ( Z )), let P z ( µ ) b e the INV ARIA N T MEASU RE IN MEAN FIELD MODELS 13 unique solution to the martingale p roblem in D ([0 , T ] , Z ) asso ciated with the (time-v arying) generator (3.1) A µ ( t ) Φ( i ) = X j ∈Z i λ i,j ( µ ( t )) · (Φ( j ) − Φ( i )) and the initial condition z . This is consisten t with ( 2.3 ) b ecause when w e view Φ and A ξ Φ as column vect ors, then the vect or A ξ Φ is th e resu lt of the rate matrix A ξ = ( λ i,j ( ξ )) ( i,j ∈Z ×Z ) righ t-m ultiplied by the vec tor Φ. Again, b y the upp er b ound edness of λ i,j ( · ) (from assu mptions ( A2 - A3 )), P z ( µ ) is unique, and the densit y of P z ( µ ) with resp ect to P z can b e wr itten as (see [ 24 , eqn. (2.4)]) (3.2) dP z ( µ ) dP z ( x ) = exp { h 1 ( x ; µ ) } where h 1 ( x ; µ ) = X 0 ≤ t ≤ T 1 { x t 6 = x t − } log λ x t − ,x t ( µ ( t − )) (3.3) − Z [0 ,T ] X j ∈Z x t ( λ x t ,j ( µ ( t )) − 1) dt. Consider the pro d uct d istribution P o, ( N ) z N = N N n =1 P z n where the N p ar- ticles ev olv e indep endently , with the n th particle’s initial condition b eing z n . Again with ν N = (1 / N ) P N n =1 δ z n , let P o, ( N ) ν N = P o, ( N ) z N ◦ G − 1 N . A simple application of Girsano v’s formula yields that (see [ 24 , eqn. (2.8)]) (3.4) dP ( N ) ν N dP o, ( N ) ν N ( Q ) = exp { N h ( Q ) } where h is related to h 1 ( · ; · ) as follo ws: for a Q ∈ M 1 ( D ([0 , T ] , Z )), (3.5) h ( Q ) = Z D ([0 ,T ] , Z ) h 1 ( x ; π ( Q )) Q ( dx ) . Let us deﬁne the spaces and top ologie s of interest. Similar to [ 24 ], consider the Poli sh space ( X , d ) wh ere X =  x ∈ D ([0 , T ] , Z ) | X 0 0 , ther e is a δ ∈ (0 , ε ) such that ρ 0 ( ν, ξ ) < δ implies S ε ( ξ | ν ) ≤ C 2 ε . Pr oof. The m ain idea is to sho w that the diﬃcult y of passage near the neigh b orh o o d of a constant v elo cit y straigh t lin e path is b ounded. See Section 6 . INV ARIA N T MEASU RE IN MEAN FIELD MODELS 19 W e n ext ha v e a u seful un iform con tin uit y r esult. Lemma 3.3 . The mapping ( ν, ξ ) 7→ S T ( ξ | ν ) is uniformly c ontinuous. Pr oof. See Section 7 . Th us far, the only condition we imp osed on the initial conditions were that ν N → ν w eakly . Let ℘ ( N ) 0 denote the law of th e initial emp irical measur e µ N (0) and let ℘ ( N ) 0 ,T denote the joint la w of ( µ N (0) , µ N ( T )). W e now consid er ( ℘ ( N ) 0 ,T , N ≥ 1). Theorem 3.4 . Supp ose that the se quenc e ( ℘ ( N ) 0 , N ≥ 1) satisﬁes the lar ge deviation principle with sp e e d N and go o d r ate function s : M 1 ( Z ) → [0 , + ∞ ] . Then the se quenc e of joint laws ( ℘ ( N ) 0 ,T , N ≥ 1) satisﬁes the lar ge deviation principle with sp e e d N and go o d r ate function S 0 ,T ( ν, ξ ) = s ( ν ) + S T ( ξ | ν ) . Pr oof. See Section 8 . Before w e close th is section, we state a useful result on the un iform cont i- n uit y of the quasip oten tial in ( 2.9 ). Th is is analogous to Lemma 3.3 , except that the time dur ations are ﬁnite b ut otherwise arbitrary . Lemma 3.4 . The mapping ( ν, ξ ) 7→ V ( ξ | ν ) i s uniformly c ontinuous. Pr oof. See the last part of Section 7 . 4. Inv ariant mea sure: A control theoretic approac h. Recall that b y assump tion ( A1 ), for eac h N ≥ 1, the ﬁ nite-state con tin uous-time Mark o v c hain µ N is irr educible, and hen ce has a unique inv ariant measure, wh ic h we denoted ℘ ( N ) . In this section, w e establish the large deviation principle for ( ℘ ( N ) , N ≥ 1) as stated in Theorems 2.1 and 2.2 . The outline of our con trol theoretic appr oac h is the follo wing. • In Lemma 4.1 , w e ﬁrst establish a subs equ en tial large deviation prin- ciple. W e shall also establish, via the con traction p r inciple, that the rate fun ction s satisﬁes a dyn amic programming equation (see ( 4.1 )). This equation n aturally suggests a con trol problem w ith an asso ciated runn in g cost. • There w ill b e multiple solutions to ( 4.1 ). But the rate f unction that w e are after will satisfy a fur th er condition. In Lemma 4.2 , w e shall 20 V. S. BORKAR AND R. SUNDARESAN sho w the existence of one single optimal path of inﬁn ite duration and shall extract a r ecursiv e equation for s from ( 4.1 ), a further condition that the rate function must satisfy . The heart of the control -theoretic approac h lies in this step. • W e then argue in Lemma 4.3 that th is optimal path m ust end up within a set that is p ositiv ely in v arian t to th e time-rev ers ed McKean-Vlaso v dynamics. • The ab ov e steps ﬁx the rate function at all p oints outside the ω -limit sets, assumin g the v alues at the ω -limit sets. • Subsection 4.1 then argues that in case of a unique globally asymptot- ically stable equilibriu m , the rate fun ction is zero at th e equilibrium. Subsection 4.2 falls bac k on the appr oac h of F reidlin and W en tzell [ 21 , Ch. 6] un der assum p tion ( B ) to ﬁx the v alues at the ω -limit sets. This ﬁxes the rate function uniquely for all sub sequent ial large deviation principles, and the main results f ollo w. W e b egin by establishing su bsequentia l large deviation prin ciples and the dynamic programming equation. Lemma 4.1 . F or any se quenc e of natur al numb ers going to + ∞ , ther e exists a subse quenc e ( N k , k ≥ 1) su ch that ( ℘ ( N k ) , k ≥ 1) satisﬁes the lar g e deviation principle with sp e e d N k and a go o d r ate function s that satisﬁes (4.1) s ( ξ ) = inf ν ∈M 1 ( Z ) [ s ( ν ) + S T ( ξ | ν )] for every T > 0 . F urthermor e, s ≥ 0 and ther e exists a ν ∗ ∈ M 1 ( Z ) su ch that s ( ν ∗ ) = 0 . Pr oof. Since the top ology on M 1 ( Z ) with metric ρ 0 has a count able base, and b ecause M 1 ( Z ) is compact, by [ 13 , Lem. 4.1.23], th er e is a su b- sequence N k → + ∞ of the giv en sequence su c h that ( ℘ ( N k ) , N k ≥ 1) satis- ﬁes the large d eviation prin ciple with sp eed N k and a go o d rate fun ction s : M 1 ( Z ) → [0 , + ∞ ]. W e n o w verify ( 4.1 ). Fix an arbitrary T > 0. By Theorem 3.4 , with ℘ ( N ) 0 = ℘ ( N ) , the in v arian t m easur e, the s equence of j oin t laws ( ℘ ( N k ) 0 ,T , k ≥ 1) satisﬁes the large deviation p rinciple along the su b sequence ( N k , k ≥ 1) with sp eed N k and go o d rate fun ction S 0 ,T ( ν, ξ ) = s ( ν ) + S T ( ξ | ν ). By the con tractio n principle, the sequence of terminal laws ( ℘ ( N k ) T , k ≥ 1) satisﬁes the large deviation principle along the s ubsequence w ith the go o d rate fun ction (4.2) inf ν ∈M 1 ( Z ) S 0 ,T ( ν, ξ ) = inf ν ∈M 1 ( Z ) [ s ( ν ) + S T ( ξ | ν )] . INV ARIA N T MEASU RE IN MEAN FIELD MODELS 21 But ℘ ( N ) is inv arian t to time shifts which yields ℘ ( N ) T = ℘ ( N ) 0 = ℘ ( N ) . T h e inﬁmum on the left-hand side of ( 4.2 ) m ust therefore ev aluate to s ( ξ ), whic h yields ( 4.1 ). Rate fu nctions are nonnegativ e and ha v e inﬁm um v alue of 0, that is, s ≥ 0 and inf ν ∈M 1 ( Z ) s ( ν ) = 0. S ince s is a go o d rate function, the inﬁ m um 0 is attained at some p oin t; call it ν ∗ . Th e pro of is no w complete. As ind icated earlier, there are multiple solutions to ( 4.1 ). Ind eed, s ( · ) ≡ 0 is one of them. In order to identify a furth er condition th at s m ust satisfy , we no w id en tify a con trol pr oblem asso ciated to ( 4.1 ). T o d o this, we shall n o w consider paths that are of time-du ration mT that end at ξ . Since the terminal condition is ﬁxed, it w ould b e conv enient to ﬁx the terminal time as 0 and lo ok at n egativ e times; in p articular, paths in th e time interv al [ − mT , 0] for m ≥ 1. But then, we ma y reverse time and consider the d ynamical system ˙ ˆ µ ( t ) = − ˆ L ( t ) ∗ ˆ µ ( t ) , ˆ µ (0) = ξ , t ∈ [0 , mT ] , m ≥ 1 . W e u se hats as in ˆ µ, ˆ l i,j , ˆ λ i,j , ˆ L to denote quantiti es where time ﬂo ws in the opp osite direction w ith r eference to the d irection under the McKean-Vlaso v dynamics. I n particular, ˆ µ ( t ) = µ ( mT − t ) ˆ l i,j ( t ) = l i,j ( mT − t ) , for all i, j ∈ Z ˆ L ( t ) = L ( mT − t ) . for t ∈ [0 , mT ]. Also, for uniform it y in notation, let ˆ λ i,j ( · ) = λ i,j ( · ) for all ( i, j ) pairs. O ne then views th e ˆ L ( t ) ab o v e as the con trol at time t w h en th e state is ˆ µ ( t ) with cost function at time t giv en by ˆ r ( ˆ µ ( t ) , ˆ L ( t )) = X ( i,j ) ∈E ( ˆ µ ( t )( i )) ˆ λ i,j ( ˆ µ ( t )) τ ∗ ˆ l i,j ( t ) ˆ λ i,j ( ˆ µ ( t )) − 1 ! . Observe that the cost fu nction is zero w hen ˆ l i,j ( t ) = ˆ λ i,j ( ˆ µ ( t )) f or almost ev ery t in the time duration of in terest. Th e total cost is (4.3) Z [0 ,mT ] h X ( i,j ) ∈E ( ˆ µ ( t )( i )) ˆ λ i,j ( ˆ µ ( t )) τ ∗ ˆ l i,j ( t ) ˆ λ i,j ( ˆ µ ( t )) − 1 ! i dt. whic h is simply S [0 ,mT ] ( µ | µ (0)) of ( 2.8 ) as can b e v eriﬁed by a change of v ariable in ( 4.3 ) that tak es t to mT − t . The follo wing lemma establishes the existence of one optimal path ˆ µ of inﬁnite du ration, starting at ξ . 22 V. S. BORKAR AND R. SUNDARESAN Lemma 4.2 . F or e ach ξ ∈ M 1 ( Z ) , ther e e xi sts a p ath ˆ µ : [0 , + ∞ ) → M 1 ( Z ) and a family of r ate matric es ( ˆ L ( t ) , t ∈ [0 , + ∞ )) such that ˆ µ ( · ) satisﬁes the O D E (4.4) ˙ ˆ µ ( t ) = − ˆ L ( t ) ∗ ˆ µ ( t ) , t ∈ [0 , + ∞ ) with initial c ondition ˆ µ (0) = ξ , and (4.5) s ( ξ ) = s ( ˆ µ ( mT )) + Z [0 ,mT ] ˆ r ( ˆ µ ( t ) , ˆ L ( t )) dt for al l m ≥ 1 . Remark. While the McKean-Vlaso v dynamics or the more general ˙ µ ( t ) = L ( t ) ∗ µ ( t ) with L ( t ) b eing a rate matrix ensures that µ ( t ) lies within M 1 ( Z ), this is not th e case for the dynamics given by ( 4.4 ). I n deed, at the b oundary of M 1 ( Z ), view ed as a sub set of R r , the v elo cit y for the dynamics in ( 4.4 ) p oints tow ards a d irection of immediate exit from M 1 ( Z ). The state sp ace for the dy n amics of ( 4.4 ) is therefore not r estricted to M 1 ( Z ). Ho w ev er, the lemma assures us that th e selected path ˆ µ stays within the compact su bset M 1 ( Z ) for all time. Pr oof. Our approac h to prov e th is is the follo wing. W e shall deﬁn e a top ology on a suitable subspace of p aths of inﬁnite duration, and then s ho w that we can restrict atten tion to a compact sub set. W e shall then argue that there exists a nested sequence of decreasing compact subsets, eac h of whic h is nonempty an d all of wh ose elemen ts satisfy the d esired prop erties. The in tersection will then b e nonempty to yield the desired path. Step 1 : W e shall no w r estrict attent ion to paths that lie ins id e M 1 ( Z ). F or a path ˆ µ : [0 , + ∞ ) → M 1 ( Z ), deﬁne its restrictions to [0 , mT ] b y ˆ µ 7→ ψ m ˆ µ ( · ) = ˆ µ ( m ) ( · ) : [0 , mT ] → M 1 ( Z ) whic h is the restriction of the path ˆ µ to [0 , mT ]. Consider the space of paths of inﬁ nite duration with metric ρ ∞ ( ˆ µ, ˆ η ) = ∞ X m =1 2 − m ( ρ mT ( ψ m ˆ µ, ψ m ˆ η ) ∧ 1) . Ob viously , ψ m is conti nuous for eac h m . W e shall also consid er rev ersed restrictions (d enoted without h ats) deﬁned b y (4.6) µ ( m ) ( t ) = ˆ µ ( m ) ( mT − t ) = ˆ µ ( mT − t ) , t ∈ [0 , mT ] . INV ARIA N T MEASU RE IN MEAN FIELD MODELS 23 Fix a B ∈ [0 , + ∞ ), and consider the set (4.7) Γ ∞ =  ˆ µ ( · ) : [0 , + ∞ ) → M 1 ( Z ) | sup m ≥ 1 S [0 ,mT ] ( µ ( m ) | µ ( m ) (0)) ≤ B  . Also, with η ( t ) = ˆ η ( mT − t ) for t ∈ [0 , mT ], deﬁ ne Γ m =  ˆ η : [0 , mT ] → M 1 ( Z ) | S [0 ,mT ] ( η | η (0)) ≤ B  . Γ ∞ is compact. T o see this, tak e an arbitrary inﬁn ite sequence ( ˆ µ n , n ≥ 1) ⊂ Γ ∞ . Since S [0 ,mT ] is a goo d r ate function, using Lemma 7.2 and Lemma 7.1 , it is easy to see that eac h Γ m is compact, an d so one can ﬁ nd an inﬁnite subset V 1 ⊂ N s u c h that ( ψ 1 ˆ µ n , n ∈ V 1 ) ⊂ Γ 1 con v erges. T ak e a further subsequence represen ted b y the inﬁnite subset V 2 ⊂ V 1 suc h that ( ψ 2 ˆ µ n , n ∈ V 2 ) ⊂ Γ 2 con v erges. Contin ue this pro cedur e and tak e the subsequence along the diagonal. Th is su bsequence con verges f or ev ery interv al [0 , mT ]. F or eac h t deﬁne ˆ µ ( t ) to b e the p oint-wise limit. Since for eac h m , w e ha v e ψ m ˆ µ ∈ Γ m , it follo ws that with µ ( m ) deﬁned as in ( 4.6 ), sup m ≥ 1 S [0 ,mT ] ( µ ( m ) | µ ( m ) (0)) ≤ B , and so ˆ µ ∈ Γ ∞ . Thus Γ ∞ is sequ en tially compact, and by virtue of its b eing a subset of a metric space, Γ ∞ is compact. Fix ξ ∈ M 1 ( Z ). Consider the time duration [0 , T ]. W e kno w from Lemma 4.1 that there is a ν ∗ with s ( ν ∗ ) = 0. Usin g this in ( 4.1 ) of Lemma 4.1 , we get s ( ξ ) ≤ S T ( ξ | ν ∗ ) ≤ C 1 ( T ) where the last inequ alit y is due to the second statement in Lemma 3.2 . T ake the constant B = C 1 ( T ); the corresp ondin g Γ ∞ is compact. Step 2 : Ob serv e that ( 4.1 ) can b e viewe d as a minimization ov er path space, with paths µ of d uration [0 , mT ] ending at ξ . Starting from any initial lo cation ν , the minimum v alue is upp er b ound ed b y B . Indeed, tra v erse the McKean-Vlaso v path with in itial condition ν for d uration ( m − 1) T . Th is con tributes zero to the cost. Then pro ceed to ξ in T units of time. T his costs at most B = C 1 ( T ). The minimum cost to go f r om ν to ξ in time [0 , mT ] is th us at most B . If we consider rev ersed and tran s lated time so that initial time is 0, the rev ersed paths ˆ µ b egin at ξ at time 0, ha v e cost at most B , and sta y in M 1 ( Z ) for the duration [0 , mT ]. Let Γ ∗ m = [ ν ∈M 1 ( Z ) n ˆ µ | µ ( t ) = ˆ µ ( mT − t ) , µ (0) = ν , µ ( mT ) = ξ , S [0 ,mT ] ( µ | ν ) = S mT ( ξ | ν ) o , 24 V. S. BORKAR AND R. SUNDARESAN that is, the collection of all minim um cost paths ˆ µ in [0 , mT ] fr om ξ to every lo cation in M 1 ( Z ); the m inim um cost is at m ost B . Clearly , Γ ∗ m ⊂ Γ m , and Γ ∗ m 6 = ∅ , m ≥ 1. Γ ∗ m is also compact. T his set b eing a s u bset of the compact set Γ m , it suﬃces to show that Γ ∗ m is closed. Let ˆ µ b e a p oin t of closure of Γ ∗ m . W e can then ﬁnd a sequence ( ˆ µ ( k ) , k ≥ 1) ⊂ Γ ∗ m suc h that lim k → + ∞ ˆ µ ( k ) = ˆ µ . Clearly , we must ha v e ˆ µ (0) = ξ . Let ν = ˆ µ ( mT ). By a simple application of lo w er s emicon tin uit y of S [0 ,mT ] ( ·| ν ), Lemma 7.2 , and Lemma 7.1 , we m ust ha v e S [0 ,mT ] ( µ | ν ) ≤ lim inf k → + ∞ S [0 ,mT ] ( µ ( k ) | µ ( k ) (0)) = lim inf k → + ∞ S mT ( ξ | µ ( k ) (0)) = S mT ( ξ | ν ) where the last inequalit y follo ws b ecause of the contin u it y of S mT in its argumen ts. But S mT ( ξ | ν ) is the least cost for paths th at tra v erse fr om ν to ξ in du ration [0 , mT ]. S o we must ha v e S [0 ,mT ] ( µ | ν ) = S mT ( ξ | ν ), whic h establishes that µ ∈ Γ ∗ m ; Γ ∗ m is therefore closed. Step 3 : Let us now ﬁnish the pro of of Lemm a 4.2 . Sin ce Γ ∗ m is nonempty and compact, the con tin uit y of ψ m implies that ψ − 1 m Γ ∗ m is nonempt y and closed. F u rther, b eing a closed su bset of the compact set Γ ∞ , ψ − 1 m Γ ∗ m is itself compact. A simple dynamic programming argument fu r ther sho ws that ψ − 1 m Γ ∗ m is a nested decreasing sequence of subsets. Their in tersection is nonempt y . T ak e a ˆ µ in the intersect ion. F o cusing on the duration [ mT , mT + T ], the path t ∈ [0 , T ] 7→ η ( m ) ( t ) = ˆ µ ( mT + T − t ) has S [0 ,T ] ( η ( m ) | η ( m ) (0)) ≤ B < + ∞ , and s o, by the last part of Th eorem 3.2 , we can ﬁnd rates L ( m ) ( t ) suc h that η ( m ) satisﬁes ˙ η ( m ) ( t ) = ( L ( m ) ( t )) ∗ η ( m ) ( t ), w ith in itial condition η ( m ) (0). Put these pieces of duration T together by deﬁnin g ˆ L ( mT + t ) = L ( m ) ( T − t ) f or t ∈ [0 , T ], and we get a rate matrix ˆ L ( · ) deﬁned on [0 , + ∞ ) suc h that ˆ µ is the solution to ( 4.4 ) with initial condition ˆ µ (0) = ξ . The last equalit y ( 4.5 ) follo ws b ecause ˆ µ attains the minimum in ( 4.1 ) for eac h duration [0 , mT ]. The next lemma sa ys that the optimal path ab ov e must end up in a sp eciﬁc in v ariant set for the dynamics giv en by the time-rev ersed McKean-Vlaso v equation. Lemma 4.3 . Consider the tr aje ctory ˆ µ given by L emma 4.2 . Its ω -limit set, which is the set of its limit p oints as t → + ∞ given by Ω = \ t> 0 { ˆ µ ( t ′ ) , t ′ ≥ t } , INV ARIA N T MEASU RE IN MEAN FIELD MODELS 25 is p ositively invariant for the ODE (4.8) ˙ ˆ µ ( t ) = − A ∗ ˆ µ ( t ) ˆ µ ( t ) , t ≥ 0 . Pr oof. T ak e the ˆ µ giv en by Lemma 4.2 . It remains within M 1 ( Z ), and satisﬁes the dynamics ˙ ˆ µ ( t ) = − ˆ L ( t ) ∗ ˆ µ ( t ) for t ∈ [0 , + ∞ ) with initial condi- tion ˆ µ (0) = ξ . F ur th ermore, ( 4.5 ) holds (for all m ≥ 1). Since the in tegrand in ( 4.5 ) is nonnegativ e, s ( ˆ µ ( mT )) must decrease as m increases. But s is b ound ed b et w een [0 , C 1 ( T )], and so there is an s ∗ suc h that s ( ˆ µ ( mT )) ↓ s ∗ as m ↑ + ∞ . Consider an y arbitrary subsequen ce of ( ˆ µ ( mT ) , m ≥ 1) and take a sub- sequen tial limit ξ ′ . O n this su bsequence, tak e a further su bsequenti al limit of ( ˆ µ ( mT + T ) , m ≥ 1) and call it ν . Call the su bsequence ( m k , k ≥ 1), and consider th e paths µ ( m k ) ( t ) = ˆ µ ( m k T + T − t ) , t ∈ [0 , T ] whic h are of d uration T and time reversals of fragmen ts of ˆ µ . W e thus ha v e the su b sequent ial conv ergence (4.9) ( µ ( m k ) (0) , µ m k ( T )) → ( ν, ξ ′ ) as k → + ∞ . T aking limits as k → + ∞ in ( 4.5 ) and using the fact that s ( µ ( m k ) (0)) as w ell as s ( µ ( m k ) ( T )) conv erge to s ∗ as k → + ∞ , the in tegral term, wh ic h is easily seen to b e S [0 ,T ] ( µ ( m ) | µ ( m ) (0)), satisﬁes lim sup k → + ∞ S [0 ,T ] ( µ ( m k ) | µ ( m k ) (0)) = 0 . This fact and the n onnegativit y of S T imply lim k → + ∞ S T ( µ ( m k ) ( T ) | µ ( m k ) (0)) = 0 . By the uniform cont in uit y of S T in b oth its argument s (Lemma 3.3 ) and by ( 4.9 ), we d educe that S T ( ξ ′ | ν ) = 0. But then the path that go es from ν to ξ ′ in time [0 , T ] and attains S T ( ξ ′ | ν ) = 0 is the McKean-Vlaso v path w hic h is the s olution to the dynamics ˙ µ ( t ) = A ∗ µ ( t ) µ ( t ) , t ∈ [0 , T ] with r ate matrix A µ ( t ) , initial condition µ (0) = ν , and ﬁnal condition µ ( T ) = ξ ′ . But this imp lies that µ ( t ) = µ ( T − t ) satisﬁes ( 4.4 ) with ˆ L ( t ) = A µ ( t ) and initial condition µ (0) = ξ ′ . It follo w s that Ω, the ω -limit set for ˆ µ , is con tained in the ω -limit set f or the dynamics ( 4.8 ) with in itial condition ˆ µ (0) = ξ . This concludes the pr o of. 26 V. S. BORKAR AND R. SUNDARESAN 4.1. Invariant me asur e: Unique glob al ly asymptotic al ly stable e quilibrium. In this subsection, w e consid er th e case wh en there is a unique globally asymptotically stable equilibrium ξ 0 . Lemma 4.4 . If the McKe an-Vlasov e quation ˙ µ ( t ) = A ∗ µ ( t ) µ ( t ) has a unique glob al ly asymptotic al ly stable e quilibrium ξ 0 , then s ( ξ 0 ) = 0 . Pr oof. Consider the McKean-Vlaso v d ynamics ˙ µ ( t ) = A ∗ µ ( t ) µ ( t ) with initial condition µ (0) = ν ∗ . By our assumption th at ξ 0 is the uniqu e globally asymptoticall y stable equilibrium , µ ( T ) → ξ 0 as T → + ∞ . By the second r emark follo wing Theorem 3.2 , the McKean-Vlaso v path has zero cost and so S T ( µ ( T ) | ν ∗ ) = 0 f or eac h T > 0. By ( 4.1 ), we then hav e s ( µ ( T )) ≤ s ( ν ∗ ) + S T ( µ ( T ) | ν ∗ ) = s ( ν ∗ ) . T ak e limits as T → + ∞ and use the low er semicont inuit y of s to get 0 ≤ s ( ξ 0 ) ≤ lim inf T → + ∞ s ( µ ( T )) ≤ s ( ν ∗ ) = 0 , whence s ( ξ 0 ) = 0. The follo wing lemma shows th at the r ate function for an y su bsequenti al large deviation principle is unique. Lemma 4.5 . If the McKe an-Vlasov e quation ˙ µ ( t ) = A ∗ µ ( t ) µ ( t ) has a unique glob al ly asymptotic al ly stable e quilibrium ξ 0 , then the solution to ( 4.1 ) and ( 4.5 ) is unique and i s given by (4.10) s ( ξ ) = inf ˆ µ Z [0 , + ∞ ) h X ( i,j ) ∈E ( ˆ µ ( t )( i )) ˆ λ i,j ( ˆ µ ( t )) τ ∗ ˆ l i,j ( t ) ˆ λ i,j ( ˆ µ ( t )) − 1 ! i dt wher e the inﬁmum is over al l ˆ µ that ar e solutions to the dynamic al system ˙ ˆ µ ( t ) = − ˆ L ( t ) ∗ ˆ µ ( t ) for some family of r ate matric es ˆ L ( · ) , with i ni tial c on- dition ˆ µ (0) = ξ , terminal c ondition lim t → + ∞ ˆ µ ( t ) = ξ 0 , and ˆ µ ( t ) ∈ M 1 ( Z ) for al l t ≥ 0 . Pr oof. The path giv en by Lemma 4.2 con v erges as shown in Lemma 4.3 to Ω, whic h is conta ined in an ω -limit set of th e ODE ( 4.8 ). S o Ω is connected and compact. Since the p ath ˆ µ ( · ) also sta ys completely within INV ARIA N T MEASU RE IN MEAN FIELD MODELS 27 M 1 ( Z ) (viewe d as a s ubset of R r ), Ω is a subset of M 1 ( Z ). F urthermore, Ω is inv arian t (b oth p ositiv ely and n egativ ely) to the d ynamics deﬁned by the ODE ( 4.8 ). Bu t then , by our assumption that ξ 0 is the un ique globally asymptotically stable equilibrium for the McKean-Vlaso v dynamics (4.11) ˙ µ ( t ) = A ∗ µ ( t ) µ ( t ) , w e must hav e Ω = { ξ 0 } b ecause this is the only su b set of M 1 ( Z ) that is in v ariant to b oth of the dyn amics in ( 4.8 ) and ( 4.11 ). Letting m → + ∞ in ( 4.5 ), it follo ws by low er semicon tin uit y of s that (4.12) s ( ξ ) ≥ s ( ξ 0 ) + Z [0 , + ∞ ) h X ( i,j ) ∈E ( ˆ µ ( t )( i )) ˆ λ i,j ( ˆ µ ( t )) τ ∗ ˆ l i,j ( t ) ˆ λ i,j ( ˆ µ ( t )) − 1 ! i dt. By Lemma 4.4 , s ( ξ 0 ) = 0, and thus s ( ξ ) ≥ Z [0 , + ∞ ) h X ( i,j ) ∈E ( ˆ µ ( t )( i )) ˆ λ i,j ( ˆ µ ( t )) τ ∗ ˆ l i,j ( t ) ˆ λ i,j ( ˆ µ ( t )) − 1 ! i dt for the sp ecial path ˆ µ . Th is establishes that s ( ξ ) is at least the r igh t-hand side of ( 4.10 ). By an application of ( 4.1 ) with ν = ξ 0 , it is obvious that s ( ξ ) is u pp er b ound ed b y the righ t-hand s ide of ( 4.10 ), w hence equ ality h olds in ( 4.10 ) and the uniqu eness of s ( ξ ) follo ws. W e are n o w ready to prov e T heorem 2.1 . Pr oof of Theorem 2.1 . T ak e any arb itrary sequen ce of natural n um- b ers gro wing to + ∞ . By Lemma 4.1 , there is a subsequence that satisﬁes the large deviation pr inciple w ith rate fu nction s s u c h that ( 4.1 ) holds. Giv en our assumption that ξ 0 is the unique globally asymptoticall y stable equilibrium for the McKean-Vlaso v equation, we ha v e s ≥ 0, and s ( ξ 0 ) = 0 by Lemma 4.4 . By Lemma 4.5 , s is un iquely sp eciﬁed b y ( 4.10 ), which is the same as ( 2.7 ). T h us ev ery sequ ence con tains a f u rther subs equence ( N k , k ≥ 1) su c h that ( ℘ ( N k ) , k ≥ 1) satisﬁes th e large d eviation p rinciple with sp eed N k and the same rate function s sp eciﬁed by ( 2.7 ). By [ 13 , Ex. 4.4.15(a)-(b), pp . 147-1 48], it follo ws that ( ℘ ( N ) , N ≥ 1) satisﬁes the large deviation principle with sp eed N and rate fun ction giv en by ( 2.7 ). 4.2. Invariant me asur e: The g ener al c ase. W e no w treat the general case under assumption ( B ). As noted earlier, under the hyp othesis of T heorem 2.1 , assumption ( B ) holds with l = 1, K 1 = { ξ 0 } , and moreov er s ( ξ 0 ) = 0 b y Lemma 4.4 . W e generalize Lemma 4.4 in Lemma 4.8 by ﬁrst proving the follo w ing lemma. The generalization of Lemm a 4.5 is L emm a 4.7 . 28 V. S. BORKAR AND R. SUNDARESAN Lemma 4.6 . Under assumption ( B ) , the r ate function s satisﬁes the fol lowing: • Ther e exists ξ 0 ∈ K i 0 for some i 0 = 1 , 2 , . . . , l that satisﬁes s ( ξ 0 ) = 0 . • Ther e exist nonne gative r e al numb e rs s 1 , s 2 , . . . , s l such that ξ ∈ K i implies s ( ξ ) = s i . Pr oof. The ﬁr s t s tatement immediately f ollo ws from the steps in th e pro of of Lemma 4.4 : ξ 0 is no w some elemen t in the ω -limit set for the McKean-Vlaso v d ynamics with in itial condition ν ∗ satisfying s ( ν ∗ ) = 0. F or th e second statemen t, let ν , ξ ∈ K i . Fix ε > 0. Since ν ∼ ξ , there exists T > 0 su c h that S T ( ξ | ν ) ≤ ε . Using ( 4.1 ), we get s ( ξ ) ≤ s ( ν ) + S T ( ξ | ν ) ≤ s ( ν ) + ε. Rev ersing the r ole of ξ and ν , we get s ( ν ) ≤ s ( ξ ) + ε , w hence | s ( ξ ) − s ( ν ) | ≤ ε . Since ε was arbitrary , we must hav e s ( ν ) = s ( ξ ). So all p oints in the compact set K i tak e the same v alue. T he second statemen t follo ws. W e now argu e that the function s that satisﬁes ( 4.1 ), ( 4.5 ), the cond ition s ≥ 0, and the condition min ν s ( ν ) = 0 is indeed u nique. W e do this in tw o steps via the follo wing lemmas. Lemma 4.7 . Assume that ( B ) holds. L et s 1 , s 2 , . . . , s l b e sp e ciﬁe d as the values on the c omp act sets K 1 , K 2 , . . . , K l . L e t s l ′ ≥ 0 for 1 ≤ l ′ ≤ l and let min { s 1 , s 2 , . . . , s l } = 0 . Then the solution to ( 4.1 ) and ( 4.5 ) is unique and is given by (4.13) s ( ξ ) = in f l ′ inf ˆ µ   s l ′ + Z + ∞ 0 h X ( i,j ) ∈E ( ˆ µ ( t )( i )) ˆ λ i,j ( ˆ µ ( t )) τ ∗ ˆ l i,j ( t ) ˆ λ i,j ( ˆ µ ( t )) − 1 ! i dt   wher e the se c ond inﬁmum is over al l ˆ µ that ar e solutions to the dynamic al system ˙ ˆ µ ( t ) = − ˆ L ( t ) ∗ ˆ µ ( t ) for some family of r ate matric es ˆ L ( · ) , with initial c ondition ˆ µ (0) = ξ , terminal c ondition ˆ µ ( t ) → K l ′ as t → + ∞ , and ˆ µ ( t ) ∈ M 1 ( Z ) f or al l t ≥ 0 . Pr oof. The same steps of the pr o of of Lemma 4.5 apply with th e fol- lo wing mo diﬁcations. Ω = { ξ 0 } gets replaced by Ω ⊂ K l ′ for some l ′ . Con- sequen tly , in the low er b ou n d in ( 4.12 ), s ( ξ 0 ) gets r eplaced by s l ′ . Recall that we deﬁned s 1 , s 2 , . . . , s l in ( 2.13 ). W e no w assert the f ollo win g. INV ARIA N T MEASU RE IN MEAN FIELD MODELS 29 Lemma 4.8 . The r ate f u nction s has values s 1 , s 2 , . . . , s l given by ( 2.13 ). Pr oof. Immediate fr om Theorem A.1 in App endix. Remark. In order to obtain these v alues, one h as to go b ey ond the ODE m etho d. One has to consider the empirical m easure Mark o v p ro cess sampled at hitting times of neigh b orho o ds of the compact sets. This is d one in F reidlin and W en tzell [ 21 , Ch. 6] for diﬀusions on a compact manifold, with V satisfying a Lipschitz prop ert y . Thanks to Corollary 3.1 , Lemma 3.2 , Lemma 3.3 , the same program can b e carried out with straigh tforw ard mo diﬁcations to accoun t f or the fact that we h a v e to h andle jum ps and the fact that the min imum cost function V ( ·|· ) satisﬁes only the uniform con tin uit y prop er ty . Th e app end ix pro vides the necessary ve riﬁcation. With this disambiguat ion for the v alues of s at the compact sets K i , we no w r eady to ﬁnish the pro of of Theorem 2.2 . Pr oof of Theorem 2.2 . Same as that of Theorem 2.1 , b ut with the use of Lemmas 4.7 and 4.8 . The f ollo w in g sections complete the p ro ofs of some assertions of Section 3 . 5. Pro of of Theorem 3.1 . T he pr o of is b ased on a generalizatio n of Sano v’s theorem due to Da wson and G¨ artner [ 11 ], the Girs an ov transform a- tion, and the Laplace-V aradhan principle. W e pro ceed thr ou gh a sequence of lemmas. 5.1. The noninter acting c ase. Consider ﬁrs t the nonint eracting case. Lemma 5.1 . Su pp ose that the initial c onditions ν N → ν we akly. Then the se quenc e ( P o, ( N ) ν N , N ≥ 1) satisﬁes the lar ge deviation principle in M 1 ,ϕ ( X ) , endowe d with the we ak* top olo gy σ ( M 1 ,ϕ ( X ) , C ϕ ( X )) , with sp e e d N and go o d r ate f unction J ( Q ) gi ven by ( 3.9 ). Pr oof. The family { P z , z ∈ Z } is clearly a subset of M 1 ,ϕ ( X ) since the transition rate fr om any state to any of the other (at most r ) states is upp er b ound ed b y 1. The family { P z , z ∈ Z } is also F eller conti nuous in the discrete top ology on Z . S ince ν N → ν , by Da wson and G¨ artner’s [ 11 , Th. 3.5] w hic h is a generalization of Sano v’s theorem, we ha v e that ( P o, ( N ) ν N , N ≥ 1) satisﬁes the large deviation principle in the w eak* top ology σ ( M 1 ,ϕ ( X ) , C ϕ ( X )) with sp eed N and go o d rate fu nction J ( Q ) giv en by ( 3.9 ). 30 V. S. BORKAR AND R. SUNDARESAN Our next lemma states that M 1 ,ϕ ( X ) cont ains all the probabilit y mea- sures of interest to u s. Lemma 5.2 . If J ( Q ) < + ∞ then (1) Q ∈ M 1 ,ϕ ( X ) and (2) Q ◦ π − 1 0 = ν . Pr oof. Observe th at || ϕ || ϕ ≤ 1, and the top ology on X is c hosen so that ϕ is con tin uous. Hence ϕ ∈ C ϕ ( X ). Recalling the exp ression for J ( Q ) in ( 3.9 ), we get that J ( Q ) < + ∞ imp lies (5.1) Z X ϕ dQ − X z ∈Z ν ( z ) log Z X e ϕ dP z < + ∞ . Since the tran s ition rates f or P z are upp er b ounded by 1, and there are at most r p ossibilities for jumps from any state, ϕ is sto chastic ally d om- inated by a Poisson random v ariable with parameter r T . Consequentl y 1 ≤ R X e ϕ dP z < + ∞ for eac h z ∈ Z , and it follo ws f rom ( 5.1 ) that R X ϕ dQ < + ∞ , and so Q ∈ M 1 ,ϕ ( X ). T o prov e the second conclusion, supp ose ν Q = Q ◦ π − 1 0 6 = ν . Consider b ound ed fu nctions f ( x ) = f 0 ( π 0 ( x )) that d ep end on x only through th e initial condition. Sin ce ν Q 6 = ν , there exists a function f of the ab ov e form that also satisﬁes P z f 0 ( z ) ν Q ( z ) − P z f 0 ( z ) ν ( z ) 6 = 0. By ﬂippin g the sign of f if necessary and by scaling, we m a y assume that P z f 0 ( z ) ν Q ( z ) − P z f 0 ( z ) ν ( z ) = a for an arb itrary a ∈ (0 , + ∞ ). T his f is b ounded con- tin uous and h ence is in C ϕ ( X ). A simple calculation yields Z X f dQ − X z ∈Z ν ( z ) log Z X e f dP z = X z f 0 ( z ) ν Q ( z ) − X z f 0 ( z ) ν ( z ) = a. Since a > 0 was arbitrary , J ( Q ) = + ∞ , and the s econd part is pro v ed by con trap osition. W e next get an alternativ e expression for J ( Q ) as a s u premum o v er the more familiar space C b ( X ) of b ounded con tin uous f u nctions on X . Lemma 5.3 . F or e ach Q ∈ M 1 ,ϕ ( X ) , J ( Q ) deﬁne d in ( 3.9 ) admits the alternative char acterization (5.2) J ( Q ) = sup f ∈ C b ( X ) " Z X f dQ − X z ∈Z ν ( z ) log Z X e f dP z # . Pr oof. In ( 5.2 ), the su p rem um is tak en o v er C b ( X ), the set of b oun ded con tin uous fu nctions on X , wh ile in ( 3.9 ) the su premum is o ver C ϕ ( X ). INV ARIA N T MEASU RE IN MEAN FIELD MODELS 31 Fix Q ∈ M 1 ,ϕ ( X ). S ince C b ( X ) ⊂ C ϕ ( X ), J ( Q ) d eﬁned b y ( 3.9 ) is at least the right-hand s id e of ( 5.2 ). F or the other direction, let f ∈ C ϕ ( X ), and consider the truncations f n of the fu nction f to [ − n, n ]. Clearly { f n } ⊂ C b ( X ), and f n → f a.e.-[ Q ] and a.e.-[ P z ]. Since w e also ha v e R X f dQ < + ∞ and R X e f dP z < + ∞ , b oth of whic h can b e easily chec ke d usin g || f || ϕ < + ∞ , an application of the Leb esgue dominated con v ergence theorem yields lim n → + ∞ " Z X f n dQ − X z ∈Z ν ( z ) log Z X e f n dP z # = Z X f dQ − X z ∈Z ν ( z ) log Z X e f dP z . Since f ∈ C ϕ ( X ) wa s arbitrary , J ( Q ) in ( 3.9 ) is at most the r igh t-hand side of ( 5.2 ). W e n o w c haracterize J ( Q ) further. W e b egin by getting a low er b ound. Lemma 5.4 . L et Q b e such that Q ◦ π − 1 0 = ν . De ﬁne P as i n ( 3.6 ). We then have H ( Q | P ) ≤ J ( Q ) . Pr oof. F or any f ∈ C b ( X ) and w ith P as deﬁned in ( 3.6 ), Jensen’s inequalit y yields Z X f dQ − X z ν ( z ) log Z X e f dP z ≥ Z X f dQ − log Z X e f dP . By Lemma 5.3 , J ( Q ) is the su premum of the left-hand side o v er all f ∈ C b ( X ). By the v ariational formula for relativ e en trop y ([ 13 , Lem. 6.2.13]), H ( Q | P ) is the supremum of the righ t-hand side o v er all f ∈ C b ( X ). T his establishes the inequalit y . W e next sh o w that J ( Q ) is upp er b ounded b y another relativ e entrop y . T o do th is, let u s in tro duce the P olish space ( ˆ X , ˆ d ) where ˆ X = Z × X and the metric ˆ d is ˆ d (( i, x ) , ( j, y )) = 1 { i 6 = j } + d ( x, y ) , where d is the metric on X . T he ﬁrst comp onen t of ˆ X shall denote the initial condition. F or t w o measures ˆ R 1 and ˆ R 2 on ˆ X , let H ( ˆ R 1 | ˆ R 2 ) b e the relativ e en trop y of ˆ R 1 with resp ect to ˆ R 2 . F or a ﬁxed ν ∈ M 1 ( Z ), let us n o w deﬁn e ˆ P as (5.3) d ˆ P ( z , x ) = ν ( z ) dP z ( x ) . 32 V. S. BORKAR AND R. SUNDARESAN The pu sh forw ard of ˆ P un der the pro jection mapping ( z , x ) ∈ ˆ X 7→ x ∈ X is clearly th e P deﬁned in ( 3.6 ). Let Q b e such that Q ◦ π − 1 0 = ν . W e then d eﬁne ˆ Q as (5.4) d ˆ Q ( z , x ) = dQ ( x ) 1 { π 0 ( x ) } ( z ) . Observing that ˆ X is P olish and that the p ush forward of ˆ Q under th e pro jec- tion ( z , x ) 7→ z is ν , it follo ws that there is a regular conditional probabilit y measure Q z satisfying (5.5) d ˆ Q ( z , x ) = ν ( z ) dQ z ( x ) . Putting ( 5.4 ) and ( 5.5 ) together and summ in g o v er z , we obtain that the second marginal of ˆ Q is (5.6) dQ ( x ) = X z dQ ( x ) 1 { π 0 ( x ) } ( z ) = X z ν ( z ) dQ z ( x ) . Since b oth ˆ Q and ˆ P h a v e the same ﬁ rst marginal ν , th e decomp osition result f or relativ e entrop y [ 13 , Th . D.8] giv es (5.7) H ( ˆ Q | ˆ P ) =    X z ∈Z ν ( z ) H ( Q z | P z ) if Q z ≪ P z a.e.-[ ν ] + ∞ otherwise . With these preliminaries, we are no w r eady to up p er b oun d J ( Q ). Lemma 5.5 . L et Q b e such that Q ◦ π − 1 0 = ν . With ˆ P and ˆ Q as ab ove, we have J ( Q ) ≤ H ( ˆ Q | ˆ P ) . Pr oof. F or f ∈ C b ( X ), by ( 5.6 ), w e ha v e R X f dQ = P z ν ( z ) R X f dQ z , and so Z X f dQ − X z ν ( z ) log Z X e f dP z = X z ν ( z )  Z X f dQ z − log Z X e f dP z  ≤ X z ν ( z ) sup f ∈ C b ( X )  Z X f dQ z − log Z X e f dP z  ! = H ( ˆ Q | ˆ P ) , INV ARIA N T MEASU RE IN MEAN FIELD MODELS 33 where the last equalit y follo ws from the observ ation that the term within paren thesis in the immediately pr eceding in equalit y is the v ariational r epre- sen tation for the relativ e en trop y H ( Q z | P z ), an d f rom ( 5.7 ). T ak e supr em um o v er all f ∈ C b ( X ) and u se Lemm a 5.3 to ded uce that J ( Q ) is u pp er b ounded b y H ( ˆ Q | ˆ P ). Lemma 5.6 . L et Q ∈ M 1 ,ϕ ( X ) . We then have J ( Q ) =  H ( Q | P ) , i f Q ◦ π − 1 0 = ν + ∞ otherwise. Pr oof. If Q ◦ π − 1 0 6 = ν , by Lemma 5.2 , w e ha v e J ( Q ) = + ∞ . So assume Q ◦ π − 1 0 = ν . By Lemmas 5.4 and 5.5 , w e get H ( Q | P ) ≤ J ( Q ) ≤ H ( ˆ Q | ˆ P ) where ˆ P and ˆ Q are deﬁned in ( 5.3 ) and ( 5.4 ), resp ectiv ely . The second marginal of ˆ P is P = P z ν ( z ) P z . F rom the deﬁn ition of ˆ P it is clear that d ˆ P ( z , x ) = dP ( x ) 1 { π 0 ( x ) } ( z ), so that the regular conditional pr ob- abilit y measures of b oth ˆ P and ˆ Q , given the second comp onen t x , are the same, that is, d ˆ Q ( z | x ) = d ˆ P ( z | x ) = 1 { π 0 ( x ) } ( z ) a.e.-[ Q ]. In particu- lar, H ( ˆ Q ( ·| x ) | ˆ P ( ·| x )) = 0 a.e.-[ Q ]. By the d ecomp osition r esult for relativ e en trop y [ 13 , Th . D.8], we get H ( ˆ Q | ˆ P ) = H ( Q | P ) + Z X H ( ˆ Q ( ·| x ) | ˆ P ( ·| x )) dQ ( x ) = H ( Q | P ) . The lemma follo ws. 5.2. Continuity of the function h ( Q ) . W e now pro ceed to address some preliminaries required for the int eracting case. Th e Radon-Nik od ym deriv a- tiv e ( 3.4 ) of P ( N ) ν N with resp ect to P o, ( N ) ν N is dP ( N ) ν N dP o, ( N ) ν N ( Q ) = exp { N h ( Q ) } . W e now study the con tin uit y p rop erty of h ( Q ). T o wa rds this, we ﬁrst es- tablish a regularit y prop ert y for all Q with J ( Q ) < + ∞ . W e then app eal to results of [ 24 ] to establish the conti nuit y of h ( Q ) when J ( Q ) < + ∞ . 34 V. S. BORKAR AND R. SUNDARESAN Lemma 5.7 . L et J ( Q ) < + ∞ and su pp ose that the r andom variable X is distribute d ac c or ding to Q . Then (5.8) sup t ∈ [0 ,T ] E " sup u ∈ [ t − α,t + α ] ∩ [0 ,T ] { 1 { X u 6 = X u − } } # → 0 as α ↓ 0 . Pr oof. A pro of for the case wh en Q ◦ π − 1 0 = δ z 0 for some ﬁxed z 0 can b e foun d in [ 24 , eqn. (2.14), p. 309-310]. Ou r argumen t b elo w is a simple mo diﬁcation and relies only on what we ha v e thus far established for J ( Q ). Let K = { x ∈ X : ∃ u ∈ [ t − α, t + α ] ∩ [0 , T ] satisfying x u 6 = x u − } . S ince J ( Q ) < + ∞ , it follo ws that Q ≪ P . W e ma y therefore write E " sup u ∈ [ t − α,t + α ] ∩ [0 ,T ] { 1 { X u 6 = X u − } } # = Q ( K ) = Z X  dQ dP  1 K dP ≤     dQ dP     τ ∗ ,P k 1 K k τ ,P (5.9) where || f || τ ∗ ,P is the O rlicz norm || f || τ ∗ ,P = inf  a > 0 : Z X τ ∗  | f ( x ) | a  dP ( x ) ≤ 1  (with resp ect to the fun ction τ ∗ and measure P ), || g || τ ,P is a sim ilarly deﬁned Orlicz norm w ith r esp ect to th e fu nction τ and measure P , and the inequalit y in ( 5.9 ) is the H¨ older inequalit y in Or licz spaces. See th e App endix in [ 24 ] for a su mmary of key results on Orlicz spaces. The lemma’s pro of will b e complete if we can sho w that   dQ dP   τ ∗ ,P is b ound ed, and k 1 K k τ ,P v anishes as α ↓ 0. W e p ro ceed to ju stify these claims. J ( Q ) < + ∞ implies   dQ dP   τ ∗ ,P < + ∞ . In deed, since lim u → + ∞ τ ∗ ( u ) u log u = 1 , c ho ose a large enou gh u 0 > 0 su ch that τ ∗ ( u ) ≤ 2 u log u for u ≥ u 0 . Also observ e that τ ∗ ( u ) is increasing in u and u log u ≥ − e − 1 for u ≥ 0. Thus with f = dQ/dP , Z X τ ∗ ( f ) dP ≤ τ ∗ ( u 0 ) + Z { x ∈X : f ( x ) ≥ u 0 } 2 f log f dP ≤ τ ∗ ( u 0 ) + 2 J ( Q ) + 2 e − 1 < + ∞ . (5.10) INV ARIA N T MEASU RE IN MEAN FIELD MODELS 35 Since τ ∗ is conv ex and τ ∗ (0) = 0, Jensen’s inequalit y yields τ ∗ ( f /a ) ≤ τ ∗ ( f ) /a f or a ≥ 1 . This fact in conjunction with ( 5.10 ) implies that || f || τ ∗ ,P < + ∞ . No w consider k 1 K k τ ,P . S ince τ (0) = 0, we get τ (( 1 K ) /a ) = τ (1 /a ) 1 K , and so Z X τ  1 K a  dP = τ (1 /a ) Z X 1 K dP = τ (1 /a ) P ( K ) . Under P , the transition rates are upp er b ound ed by 1. Moreo v er, there are at most r p ossible next states. Since K is the even t that there is a tr ansition in [ t − α, t + α ] ∩ [0 , T ], it f ollo ws that P ( K ) ≤ 2 αr . F rom its deﬁnition, the Orlicz norm is the sm allest p ositive a such that τ (1 /a ) P ( K ) ≤ 1, and so k 1 K k τ ,P = 1 τ − 1 (1 /P ( K )) ≤ 1 τ − 1 (1 / (2 αr )) . where the equ alit y in the ab ov e c hain holds b ecause τ ( u ) is increasing in u for u ≥ 0, and the inequalit y holds b ecause of the same prop ert y for τ − 1 ( u ). This last upp er b ou n d v anishes as α ↓ 0. The follo wing lemma implies Lemma 3.1 as a corollary . Lemma 5.8 . Consider M 1 ( D ([0 , T ] , Z )) endowe d with the top olo gy of we ak c onver g enc e and D ([0 , T ] , M 1 ( Z )) endowe d with the top olo gy induc e d by the metric ρ T deﬁne d i n ( 3.11 ). Then the mapping π : M 1 ( D ([0 , T ] , Z )) → D ([0 , T ] , M 1 ( Z )) is c ontinuous at e ach Q ∈ M 1 ( D ([0 , T ] , Z )) wher e J ( Q ) < + ∞ . Pr oof. Note th at, by Lemma 5.2 , J ( Q ) < + ∞ im p lies Q ∈ M 1 ,ϕ ( X ). The statemen t of the ab o v e Lemma is the same as [ 24 , Lem. 2.8]. The only d iﬀerence is that our repr esen tation for J ( Q ) diﬀers in order to handle nonc haotic initial conditions and allo ws Q ◦ π − 1 0 to b e any measure in M 1 ( Z ). The pro of of [ 24 , Lem. 2.8] holds ve rbatim if w e can establish ( 5.8 ) (whic h is the same as [ 24 , eqn. (2.14)]) for the more general case u nder consideration. This is d one in Lemma 5.7 . This is an appropriate lo cation to include the pro of of Lemma 3.1 , which is a corollary to the ab o v e Lemma. 36 V. S. BORKAR AND R. SUNDARESAN Pr oof of Lemma 3.1 . The ﬁ rst part is a corollary to Lemma 5.8 . In- deed, Lemma 5.8 sho ws th at π is con tin uous u nder th e coarser top ology of w eak con v ergence of probab ility measures metrized by d Sk o . Since the nat- ural embed ding of X in to D ([0 , T ] , Z ) (with top ology indu ced b y d Sk o ) is con tin uous, it immediately follo ws that π is a con tin uous mapp ing un der the ﬁn er top ology σ ( M 1 ,ϕ ( X ) , C ϕ ( X )). T o see the second part, ﬁx a Q suc h that J ( Q ) < + ∞ , a t ∈ [0 , T ], and consider a sequence Q N → Q . By the ﬁrst part, we ha v e π ( Q N ) → π ( Q ), whic h is the same as sa ying ρ T ( π ( Q N ) , π ( Q )) → 0. But then ρ 0 ( π t ( Q N ) , π t ( Q )) ≤ ρ T ( π ( Q N ) , π ( Q )) → 0 establishes the con tin uit y of π t . W e n o w come to the con tin uit y of the fu nction h ( Q ). Lemma 5.9 . Consid er the sp ac e M 1 ,ϕ ( X ) endowe d with the we ak* top ol- o gy σ ( M 1 ,ϕ ( X ) , C ϕ ( X )) . The function h : M 1 ,ϕ ( X ) → R deﬁne d in ( 3.5 ) is c ontinuous at every Q wher e J ( Q ) < + ∞ . Pr oof. The statemen t is the same as [ 24 , Lem. 2.9]. The same pro of applies. T h at pro of r equ ires con tin uit y of π , whic h is no w established in Lemma 5.8 u nder assump tions ( A1 )-( A3 ) . 5.3. The inter acting c ase. W e no w add ress the interact ing case. Pr oof of Theorem 3.1 . Recall the statemen t of T heorem 3.1 . W e are no w giv en th at the sequence of in itial empirical m easur es ν N → ν w eakly . By Lemma 5.1 , ( P o, ( N ) ν N , N ≥ 1) satisﬁes the large deviation principle in the top ological space M 1 ,ϕ ( X ) w ith rate f unction J ( Q ). By Lemma 5.2 and Lemma 5.9 , h is contin uou s on the set { Q ∈ M 1 ,ϕ ( X ) | J ( Q ) < + ∞} . F urthermore, b y [ 24 , Lem. 2.10], for every α > 0, w e h a v e lim sup N → + ∞ 1 N log Z M 1 ,ϕ ( X ) e N α | h | dP o, ( N ) ν N < + ∞ . Using the Lap lace-V aradhan prin ciple, see [ 24 , Prop . 2.5], w e can dr a w t w o conclusions. Th e ﬁrst conclusion is that (5.11) 1 N log Z M 1 ,ϕ ( X ) e N h dP o, ( N ) ν N → sup Q ′ ∈M 1 ,ϕ ( X ) [ h ( Q ′ ) − J ( Q ′ )] INV ARIA N T MEASU RE IN MEAN FIELD MODELS 37 as N → + ∞ . F rom ( 3.4 ), we ha v e e N h dP o, ( N ) ν N = dP ( N ) ν N , a pr obabilit y measur e. The left-hand side in ( 5.11 ) is therefore alw a ys 0, and so sup Q ′ ∈M 1 ,ϕ ( X ) [ h ( Q ′ ) − J ( Q ′ )] = 0. The second conclusion is that ( P ( N ) ν N , N ≥ 1) satisﬁes the large deviation pr inciple in the top ologic al sp ace M 1 ,ϕ ( X ) with go o d rate function I ( Q ) = J ( Q ) − h ( Q ) − inf Q ′ [ J ( Q ′ ) − h ( Q ′ )] = J ( Q ) − h ( Q ) where th e last equalit y holds b ecause the inﬁmum ab o v e is 0 by the ﬁ rst conclusion. This concludes the p ro of of the ﬁrst p art of Theorem 3.1 . W e now show ( 3.10 ). By assumption ( A1 )-( A3 ), it is easy to see that there exists a constan t K suc h that | h ( Q ) | ≤ K (1 + R X ϕ dQ ) so th at if Q ∈ M 1 ,ϕ ( X ) then | h ( Q ) | < + ∞ . By Lemma 5.6 , if either Q ◦ π − 1 0 6 = ν or Q is not absolutely con tin uous with resp ect to P , then J ( Q ) = + ∞ , and b y the ﬁn iteness of h ( Q ), we ha v e I ( Q ) = J ( Q ) − h ( Q ) = + ∞ . W e ma y therefore assume Q ◦ π − 1 0 = ν and Q ≪ P , w h ence, by Lemma 5.6 once again, J ( Q ) = H ( Q | P ) = R dQ log( dQ/dP ). It th er efore suﬃces to argue that Q ◦ π − 1 0 = ν and Q ≪ P ⇒ H ( Q | P ) − h ( Q ) = H ( Q | P ( π ( Q ))) . Let µ = π ( Q ) for con v enience. Observe that the densit y dP z ( µ ) dP z ( · ) = exp { h 1 ( · , µ ) } in ( 3.2 ) do es not dep end on z . It follo ws that the density of the mixture distribution P ( µ ) in ( 3.7 ) with resp ect to the mixture P in ( 3.6 ) is dP ( µ ) dP ( x ) = exp { h 1 ( x, µ ) } . Using this in ( 3.5 ), we get h ( Q ) = Z D ([0 ,T ] , Z ) dQ log dP ( µ ) dP , from which H ( Q | P ) − h ( Q ) = Z D ([0 ,T ] , Z ) dQ log dQ dP − Z D ([0 ,T ] , Z ) dQ log dP ( µ ) dP = Z D ([0 ,T ] , Z ) dQ log dQ dP ( µ ) = H ( Q | P ( µ )) follo w s. This concludes th e pr o of. 38 V. S. BORKAR AND R. SUNDARESAN 6. Pro of of Lemma 3.2 . W e addr ess the ﬁr st bullet. F or ease of exp o- sition, let u s for now allo w all p ossible transitions and ignore the constraint that only E transitions are allo w ed. Consider the constant v elo cit y path (6.1) µ ( t ) =  1 − t T  ν + t T ξ , t ∈ [0 , T ] for which ˙ µ ( t ) = ξ − ν T , t ∈ [0 , T ] . There is ﬂo w out of i if ξ ( i ) < ν ( i ), and ﬂo w into i otherwise. W e now construct a r ate matrix L ( t ) with en tries l i,j ( t ) that ensu re th e tra v ers al of this constant velocit y path. Since there is conserv ation of mass P z ξ ( z ) = P z ν ( z ), we can construct mass transp ort parameters { g i,j } su c h that for an i with ν ( i ) > ξ ( i ) and a j with ν ( j ) < ξ ( j ), the quan tit y g i,j is the fraction of the excess ν ( i ) − ξ ( i ) that go es from i to j . In particular, { g i,j } satisﬁes g i,j ∈ [0 , 1] for all i, j ∈ Z g i,j = 0 if ν ( i ) ≤ ξ ( i ) or ν ( j ) ≥ ξ ( j ) (6.2) X j : ν ( j ) <ξ ( j ) g i,j = 1 if ν ( i ) > ξ ( i ) (6.3) and ﬁn ally (6.4) X i : ν ( i ) >ξ ( i ) [ ν ( i ) − ξ ( i )] g i,j = ξ ( j ) − ν ( j ) if ν ( j ) < ξ ( j ) . Equation ( 6.3 ) sa ys mass is not destroy ed and ( 6.4 ) sa ys new mass is not created (all mass ente ring j must come from i ’s w ith excesses). Deﬁne the d iagonal elemen ts of the tr an s ition rate m atrix L ( t ) to b e (6.5) l j,j ( t ) = ( − ( ν ( j ) − ξ ( j )) T ( µ ( t )( j )) if j s atisﬁes ν ( j ) > ξ ( j ) 0 otherwise . No w deﬁn e the oﬀ-diagonal elemen ts of L ( t ) to b e (6.6) l j,i ( t ) =  0 if ν ( j ) ≤ ξ ( j ) , i ∈ Z − l j,j ( t ) g j,i if ν ( j ) > ξ ( j ) an d i 6 = j. W e next claim that ˙ µ ( t ) = L ( t ) ∗ µ ( t ). Indeed, for i su c h th at ν ( i ) ≥ ξ ( i ), w e hav e ( L ( t ) ∗ µ ( t )) ( i ) = X j ( µ ( t )( j )) l j,i ( t ) INV ARIA N T MEASU RE IN MEAN FIELD MODELS 39 = ( µ ( t )( i ) ) l i,i ( t ) + X j : j 6 = i,ν ( j ) ≤ ξ ( j ) ( µ ( t )( j )) l j,i ( t ) + X j : j 6 = i,ν ( j ) >ξ ( j ) ( µ ( t )( j )) l j,i ( t ) (a) = ( µ ( t )( i ) ) l i,i ( t ) + 0 + 0 (b) = ξ ( i ) − ν ( i ) T . In the ab o v e sequence of equalities, the second term in (a) v anished b ecause ν ( j ) ≤ ξ ( j ) imp lies l j,i ( t ) = 0 (see ( 6.6 )); the third term v anish ed b ecause, b y ( 6.2 ) and noticing that i is the second argument, ν ( i ) ≥ ξ ( i ) imp lies g j,i = 0 w h ic h in turn imp lies l j,i ( t ) = 0 again by ( 6.6 ). Lastly , (b) follo ws from ( 6.5 ). F or i suc h that ν ( i ) < ξ ( i ), w e h av e ( L ( t ) ∗ µ ( t )) ( i ) = X j ( µ ( t )( j )) l j,i ( t ) = ( µ ( t )( i ) ) l i,i ( t ) + X j : j 6 = i,ν ( j ) ≤ ξ ( j ) ( µ ( t )( j )) l j,i ( t ) + X j : j 6 = i,ν ( j ) >ξ ( j ) ( µ ( t )( j )) l j,i ( t ) (a) = 0 + 0 + X j : j 6 = i,ν ( j ) >ξ ( j ) ( µ ( t )( j )) ( − l j,j ( t ) g j,i ) (b) = 1 T X j : j 6 = i,ν ( j ) >ξ ( j ) ( ν ( j ) − ξ ( j )) g j,i (c) = ξ ( i ) − ν ( i ) T . In the ab ov e sequence of equalitie s, (a) follo ws from ( 6.5 ) and ( 6.6 ). Equation (b) follo ws from ( 6.5 ), and (c) follo ws fr om ( 6.4 ). Th e ab o v e argum ents establish ˙ µ ( t ) = L ( t ) ∗ µ ( t ). Let us now ev aluate the d iﬃcult y S [0 ,T ] ( µ | ν ) of passage near this constan t v elocity path µ . If we sho w th at th e in tegral in the righ t-hand side of ( 2.8 ) is ﬁnite, by Theorem 3.2 , S [0 ,T ] ( µ | ν ) equals this integ ral. Obser ve that l i,j ( t ) is not b ounded if one of µ (0)( i ) = ν ( i ) or µ ( T )( i ) = ξ ( i ) equals 0, and so w e do h av e some work to do. The right- hand side of ( 2.8 ) can b e expanded to b e Z [0 ,T ] h X i,j : j 6 = i  ( µ ( t )( i )) l i,j ( t ) log  l i,j ( t ) λ i,j ( µ ( t ))  (6.7) − ( µ ( t )( i )) l i,j ( t ) + ( µ ( t )( i )) λ i,j ( µ ( t )) i dt. 40 V. S. BORKAR AND R. SUNDARESAN F rom ( 6.6 ) and ( 6.5 ), we get ( µ ( t )( i )) l i,j ( t ) = T − 1 ( ν ( i ) − ξ ( i )) g i,j , and this is nonzero only if ν ( i ) > ξ ( i ) and ν ( j ) < ξ ( j ); see ( 6.2 ). F or con v enience, let us deﬁn e Υ = { ( i, j ) | j 6 = i, ν ( i ) > ξ ( i ) , ν ( j ) < ξ ( j ) } . By assump tions ( A2 )-( A3 ), | log λ i,j ( · ) | ≤ | log C | + | log c | . Using these ob- serv ations, ( 6.7 ) is u pp er b oun ded by Z [0 ,T ] h X ( i,j ) ∈ Υ  T − 1 ( ν ( i ) − ξ ( i )) g i,j log  ( ν ( i ) − ξ ( i )) g i,j T ( µ ( t )( i ))  (6.8) + T − 1 ( ν ( i ) − ξ ( i )) g i,j ( | log C | + | log c | + 1) + C i dt ≤ X ( i,j ) ∈ Υ ( ν ( i ) − ξ ( i )) g i,j | log(( ν ( i ) − ξ ( i )) g i,j ) | − X i : ν ( i ) >ξ ( i ) ( ν ( i ) − ξ ( i )) T − 1 Z [0 ,T ] log( µ ( t )( i )) dt + || ν − ξ || 1 | log T | + || ν − ξ || 1 ( | log C | + | log c | + 1) + C T r 2 , where in arriving at the last three terms we hav e rep eatedly used ( 6.3 ). Th e quan tit y || ν − ξ || 1 is the total v ariation distance b etw een ξ and ν . Let u s no w b ound the ﬁ rst t w o terms on the r igh t-hand side of ( 6.8 ). Observing that there is a constan t K suc h that s u p x ∈ [0 , 1] x | log x | ≤ K < + ∞ , the ﬁrst term on the righ t-hand side of ( 6.8 ) can b e u pp er b ounded as X ( i,j ) ∈ Υ ( ν ( i ) − ξ ( i )) g i,j | log(( ν ( i ) − ξ ( i )) g i,j ) | ≤ X i : ν ( i ) >ξ ( i ) ( ν ( i ) − ξ ( i )) | log ( ν ( i ) − ξ ( i )) | X j :( i,j ) ∈ Υ g i,j ! + X ( i,j ) ∈ Υ ( ν ( i ) − ξ ( i )) g i,j | log g i,j | ≤ X i : ν ( i ) >ξ ( i ) ( ν ( i ) − ξ ( i )) | log ( ν ( i ) − ξ ( i )) | + K || ν − ξ || 1 ≤ X i | ( | ν ( i ) − ξ ( i ) | log | ν ( i ) − ξ ( i ) | ) | + K || ν − ξ || 1 . (6.9) T o b ound the second term on the right- hand side of ( 6.8 ), us e ( 6.1 ) and emplo y the c hange of v ariable u = µ ( t )( i ) to get INV ARIA N T MEASU RE IN MEAN FIELD MODELS 41 − ( ν ( i ) − ξ ( i )) T − 1 Z [0 ,T ] log( µ ( t )( i )) dt = Z ξ ( i ) ν ( i ) log u du = [ u log u − u ] ξ ( i ) ν ( i ) ≤ | ξ ( i ) log ξ ( i ) − ν ( i ) log ν ( i ) | + | ν ( i ) − ξ ( i ) | . Summing this o v er all i , we see that the second term in ( 6.8 ) is up p er b ound ed b y (6.10) X i | ξ ( i ) log ξ ( i ) − ν ( i ) log ν ( i ) | + || ν − ξ || 1 . Since M 1 ( Z ) is compact, all terms in the upp er b oun ds ( 6.9 ) and ( 6.10 ) are b ounded. S ubstituting ( 6.9 ) and ( 6.10 ) on the right-hand side of ( 6.8 ), and noticing that T > 0, we see th at th e righ t-hand s id e of ( 6.8 ) is up p er b ound ed, and this up p er b ound serves as an up p er b ound on S [0 ,T ] ( µ | ν ), whic h we s ummarize as S [0 ,T ] ( µ | ν ) ≤ X i | ( | ν ( i ) − ξ ( i ) | log | ν ( i ) − ξ ( i ) | ) | + X i | ξ ( i ) log ξ ( i ) − ν ( i ) log ν ( i ) | + || ν − ξ || 1 ( | log T | ) + || ν − ξ || 1 ( | log C | + | log c | + K + 2) + C T r 2 (6.11) ≤ C 3 ( T ) for a suitable constan t C 3 ( T ) that is indep endent of ν and ξ . This concludes the pr o of of the ﬁrst bullet f or the case when all transitions are allo w ed. When only those transitions in the directed edge s et E can o ccur, sin ce the Mark o v chain is irreducible (by assump tion ( A1 )), there exists a ﬁnite sequence of inte rmediate p oin ts through whic h one can mov e fr om ν to ξ in m = m ( r, E ) < + ∞ steps: ν = ν (0) → ν (1) → · · · → ν ( m ) = ξ . Consider no w the p iecewise linear path that mo v es from ν to ξ throu gh the ab o v e sequence of p oin ts with v elo cities such th at eac h segmen t is co v ered in time T /m . Then S [0 ,T ] ( µ | ν ) ≤ C 1 ( T ) = mC 3 ( T /m ), and th e pro of of the ﬁrst bu llet is complete. S T ( ξ | ν ) ≤ C 1 ( T ) follo ws immediately from ( 3.15 ), whence the second bullet follo ws. T o see the third bullet, we us e ( 6.11 ). S ince M 1 ( Z ) is a sub set of a ﬁn ite dimensional space, the top ology of weak conv ergence is the same as the 42 V. S. BORKAR AND R. SUNDARESAN top ology indu ced b y the total v ariati on metric. In particular, if ρ 0 ( ν, ξ ) → 0 then ν ( i ) → ξ ( i ) f or ev ery i ∈ Z . As a consequence, for ev ery ε > 0 and with T = ε , we can c ho ose a δ > 0 suc h that eac h of the ﬁrst four terms in ( 6.11 ) is upp er b ounded by ε , and so ρ 0 ( ν, ξ ) < δ implies S ε ( ξ | ν ) ≤ 4 ε + C r 2 ε ≤ C 2 ε for some C 2 < + ∞ , and the pro of of the third bullet an d the Lemma is complete.  7. Pro ofs of Lemma 3.3 and Lemma 3.4 . W e b egin w ith t w o usefu l lemmas. Lemma 7.1 . L e t L ( t ) b e a matrix of r ates such that the solution µ : [0 , T ] → M 1 ( Z ) to the O DE ˙ µ ( t ) = L ( t ) ∗ µ ( t ) with µ (0) = ν has S [0 ,T ] ( µ | ν ) < + ∞ . Ther e exists a c onstant K < + ∞ such that Z [0 ,T ] h X ( i,j ) ∈E ( µ ( t )( i )) l i,j ( t ) i dt ≤ S [0 ,T ] ( µ | ν ) + K T . Pr oof. It is easy to ve rify th at τ ∗ ( u − 1) = u log u − u + 1 ≥ u − e + 1 for all u ≥ 0. By Theorem 3.2 , S [0 ,T ] ( µ | ν ) < + ∞ implies that its ev aluatio n is give n b y ( 2.8 ). Usin g these tw o facts, we get S [0 ,T ] ( µ | ν ) = Z [0 ,T ] h X ( i,j ) ∈E ( µ ( t )( i )) λ i,j ( µ ( t )) τ ∗  l i,j ( t ) λ i,j ( µ ( t )) − 1  i dt ≥ Z [0 ,T ] h X ( i,j ) ∈E ( µ ( t )( i )) λ i,j ( µ ( t ))  l i,j ( t ) λ i,j ( µ ( t )) − e + 1  i dt ≥ Z [0 ,T ] h X ( i,j ) ∈E ( µ ( t )( i )) l i,j ( t ) i dt − ( e − 1) C r T , and the lemma follo ws. The next lemma b ounds the in crease in the cost due to time scaling on a ﬁxed path b et w een tw o p oin ts. Lemma 7.2 . L e t L ( t ) b e a matrix of r ates such that the solution µ : [0 , T ] → M 1 ( Z ) to the O DE ˙ µ ( t ) = L ( t ) ∗ µ ( t ) with µ (0) = ν has S [0 ,T ] ( µ | ν ) < + ∞ and µ ( T ) = ξ . L et 0 < α < + ∞ b e a time sc aling. With T ′ = T /α , c onsider the p ath { ˜ µ ( t ) = µ ( αt ) | t ∈ [0 , T ′ ] } having ˜ µ (0) = ν and ˜ µ ( T ′ ) = ξ . Then ˙ ˜ µ ( t ) = ˜ L ( t ) ∗ ˜ µ ( t ) , t ∈ [0 , T ′ ] INV ARIA N T MEASU RE IN MEAN FIELD MODELS 43 wher e ˜ L ( t ) = αL ( αt ) . F urthermor e, the sc ale d p ath ˜ µ : [0 , T ′ ] → M 1 ( Z ) satisﬁes S [0 ,T ′ ] ( ˜ µ | ν ) ≤ S [0 ,T ] ( µ | ν ) + | log α | Z [0 ,T ] h X ( i,j ) ∈E ( µ ( t )( i )) l i,j ( t ) i dt (7.1) + | 1 − α | α C rT . Pr oof. Clearly ˜ µ (0) = µ (0) = ν and ˜ µ ( T ′ ) = µ ( αT ′ ) = µ ( T ) = ξ . Since ˙ µ ( t ) = L ( t ) ∗ µ ( t ), we also ha v e ˙ ˜ µ ( t ) = dµ ( αt ) dt = α ˙ µ ( αt ) = αL ( αt ) ∗ µ ( αt ) = αL ( αt ) ∗ ˜ µ ( t ) from whic h ˜ L ( t ) = αL ( αt ) is obvious. Its ( i, j )th entry ˜ l i,j ( t ) equals αl i,j ( αt ). The cost of ˜ µ : [0 , T ′ ] → M 1 ( Z ) is then S [0 ,T ′ ] ( ˜ µ | ν ) = Z [0 ,T ′ ] h X ( i,j ) ∈E ( ˜ µ ( t )( i )) λ i,j ( ˜ µ ( t )) τ ∗ ˜ l i,j ( t ) λ i,j ( ˜ µ ( t )) − 1 ! i dt = Z [0 ,T ′ ] h X ( i,j ) ∈E ( µ ( αt )( i )) λ i,j ( µ ( αt )) τ ∗  αl i,j ( αt ) λ i,j ( µ ( αt )) − 1  i dt = Z [0 ,T ′ ] h X ( i,j ) ∈E ( µ ( αt )( i )) λ i,j ( µ ( αt )) × α n τ ∗  l i,j ( αt ) λ i,j ( µ ( αt )) − 1  + l i,j ( αt ) λ i,j ( µ ( αt )) (log α ) + 1 − α α oi dt where we h a v e u s ed th e fact th at τ ∗ ( αu − 1) = α  τ ∗ ( u − 1) + u (log α ) + 1 − α α  , u ≥ 0 . Changing v ariables from αt to t and con tin uing, w e get S [0 ,T ′ ] ( ˜ µ | ν ) = Z [0 ,T ] h X ( i,j ) ∈E ( µ ( t )( i )) λ i,j ( µ ( t )) τ ∗  l i,j ( t ) λ i,j ( µ ( t )) − 1  i dt + (log α ) Z [0 ,T ] h X ( i,j ) ∈E ( µ ( t )( i )) l i,j ( t ) i dt + 1 − α α Z [0 ,T ] h X ( i,j ) ∈E ( µ ( t )( i )) λ i,j ( µ ( t )) i dt. Since the ﬁr s t term on the right-hand s ide ab o v e is S [0 ,T ] ( µ | ν ) and λ i,j ( · ) ≤ C , ( 7.1 ) follo ws. 44 V. S. BORKAR AND R. SUNDARESAN Pr oof of Lemma 3.3 . Fix T > 0. Fix an arbitrary ε suc h that 0 < ε < T / 4. Let δ > 0 b e as giv en by part 3 of Lemma 3.2 so th at ρ 0 ( ν, ξ ) < δ implies S ε ( ξ | ν ) ≤ C 2 ε . Let { ( ν i , ξ i ) , i = 1 , 2 } b e t w o p oin ts in Z × Z . By an abu se of notation, let ρ T giv en by ρ T (( ν 1 , ξ 1 ) , ( ν 2 , ξ 2 )) = max { ρ 0 ( ν 1 , ν 2 ) , ρ 0 ( ξ 1 , ξ 2 ) } . denote the metric on Z × Z . Let ρ T (( ν 1 , ξ 1 ) , ( ν 2 , ξ 2 )) < δ . W e n eed to sh o w that S T ( ξ 1 | ν 1 ) and S T ( ξ 2 | ν 2 ) are close to eac h other. Ob viously , ρ 0 ( ν 1 , ν 2 ) < δ and ρ 0 ( ξ 2 , ξ 1 ) < δ . Let µ denote the minim um cost path from ν 2 to ξ 2 in time T w ith cost S T ( ξ 2 | ν 2 ). Consider the path from ν 1 to ξ 1 as follo ws: • T ra v erse the p ath from ν 1 to ν 2 in time [0 , ε ], as giv en by part 3 of Lemma 3.2 . Th is tra v ersal costs at most C 2 ε . • Giv en the optimal [0 , T ]-path µ from ν 2 to ξ 2 , consider the sp ed-up path ˜ µ : [0 , T − 2 ε ] → M 1 ( Z ) giv en b y ˜ µ ( t ) = µ ( αt ) with α = T / ( T − 2 ε ). T ra v el from ν 2 to ξ 2 in the du ration [ ε, T − ε ] along the path ˜ µ . • T ra v erse the p ath from ξ 2 to ξ 1 in time [0 , ε ], again as given b y part 3 of Lemma 3.2 . This trav ersal’s cost is also at most C 2 ε . The minimum cost for tra v ersal fr om ν 1 to ξ 1 is at most the sum of these paths. Hence, b y Lemmas 7.2 and 7.1 , w e get S T ( ξ 1 | ν 1 ) ≤ 2 C 2 ε + S T ( ξ 2 | ν 2 ) +  log T T − 2 ε  ( S T ( ξ 2 | ν 2 ) + K T ) + 2 ε T C rT ≤ S T ( ξ 2 | ν 2 ) + 2 C 2 ε +  T T − 2 ε − 1  ( C 1 ( T ) + K T ) + 2 C r ε where we u sed log u ≤ u − 1 for u > 0. Observing that ε < T / 4 ⇒ T T − 2 ε − 1 = 2 ε T − 2 ε ≤ 4 ε T , w e dedu ce that S T ( ξ 1 | ν 1 ) ≤ S T ( ξ 2 | ν 2 ) + C 4 ( T ) ε where we ma y tak e C 4 ( T ) = 2 C 2 + 4 K + 4 C 1 ( T ) /T + 2 C r . Rev ersing the roles of ( ν 1 , ξ 1 ) and ( ν 2 , ξ 2 ), we d educe | S T ( ξ 1 | ν 1 ) − S T ( ξ 2 | ν 2 ) | ≤ C 4 ( T ) ε. This concludes the p ro of that ( ν , ξ ) 7→ S T ( ξ | ν ) is uniform ly con tin uous. INV ARIA N T MEASU RE IN MEAN FIELD MODELS 45 W e no w pro vide the pr o of of the r esult on uniform con tin uit y of th e anal- ogous quantit y V ( ξ | ν ). Pr oof of Lemma 3.4 . Fix ε > 0 and c ho ose δ as in the th ir d part of Lemma 3.2 . Let ( ν 1 , ξ 1 ) and ( ν 2 , ξ 2 ) b e suc h that the starting p oin ts are δ -close to eac h other and so are the ending p oin ts, that is, ρ 0 ( ν 1 , ν 2 ) < δ and ρ 0 ( ξ 1 , ξ 2 ) < δ . Consid er the follo wing p ath: • T ra v erse the p ath from ν 1 to ν 2 in time [0 , ε ], as giv en by part 3 of Lemma 3.2 . Th is tra v ersal costs at most C 2 ε . • T ra v erse by a path from ν 2 to ξ 2 in ﬁnite time b y a p ath with cost at most V ( ξ 2 | ν 2 ) + C 2 ε . • T ra v erse the p ath from ξ 2 to ξ 1 in time [0 , ε ], again as given b y part 3 of Lemma 3.2 . T his trav ersal’s cost is also at most C 2 ε . W e th en ha v e V ( ξ 1 | ν 1 ) ≤ C 2 ε + ( V ( ξ 2 | ν 2 ) + C 2 ε ) + C 2 ε = V ( ξ 2 | ν 2 ) + 3 C 2 ε. Rev ersing the roles of ( ν 1 , ξ 1 ) and ( ν 2 , ξ 2 ) and via a similar argumen t, we deduce that | V ( ξ 1 | ν 1 ) − V ( ξ 2 | ν 2 ) | ≤ 3 C 2 ε whic h shows that ( ν , ξ ) 7→ V ( ξ | ν ) is uniform ly con tin uous. 8. Pro of of Theorem 3.4 . Again, we pro ceed thr ough a sequ en ce of lemmas. Let ν N → ν weakly . By Theorem 3.2 , the sequence of la ws of the terminal measure ( p ( N ) ν N ,T , N ≥ 1) satisﬁes the large deviation principle w ith sp eed N and go o d r ate fu nction S T ( ξ | ν ). By V aradh an ’s lemma, for eve ry f ∈ C b ( M 1 ( Z )), w e h a v e (8.1) lim N → + ∞ 1 N log Z M 1 ( Z ) e N f dp ( N ) ν N ,T = sup ξ ∈M 1 ( Z ) [ f ( ξ ) − S T ( ξ | ν )] . Let us deﬁne (8.2) Λ( f | ν ) = sup ξ ∈M 1 ( Z ) [ f ( ξ ) − S T ( ξ | ν )] . Observe that th e r ate fun ction admits the c haracterizatio n (see, e.g., [ 13 , Th. 4.4.2]) (8.3) S T ( ξ | ν ) = sup f ∈ C b ( M 1 ( Z )) [ f ( ξ ) − Λ( f | ν )] . 46 V. S. BORKAR AND R. SUNDARESAN Lemma 8.1 . L et f ∈ C b ( M 1 ( Z )) . The mapping ν ∈ M 1 ( Z ) 7→ Λ( f | ν ) ∈ R is c ontinuous. Pr oof. Since f is cont in uous, by Lemma 3.3 , the mapping η : ( ν , ξ ) ∈ M 1 ( Z ) × M 1 ( Z ) 7→ f ( ξ ) − S T ( ξ | ν ) ∈ R is join tly con tin uous. As M 1 ( Z ) is compact, the supremum in the d eﬁ nition of ( 8.2 ) is attained. Let ν N → ν wea kly , and for eac h ν N , let ξ N denote a p oint where the supremum in the deﬁn ition of ( 8.2 ) is attained. In other words, Λ( f | ν N ) = η ( ν N , ξ N ) for eac h N . Th e sequence (( ν N , ξ N ) , N ≥ 1) has a conv ergent subsequence that con v erges to ( ν , ξ ), for s ome ξ . Reind ex so that we ma y tak e ( ν N , ξ N ) → ( ν , ξ ) as N → + ∞ . By the con tin uit y of η , Λ( f | ν N ) = η ( ν N , ξ N ) → η ( ν, ξ ) as N → + ∞ . The p ro of will b e complete if w e can show that Λ( f | ν ) = η ( ν, ξ ), that is, the supr em um in η ( ν , · ) is attained at ξ . T o see this, observe that for an y ξ ′ , w e h a v e η ( ν N , ξ ′ ) ≤ η ( ν N , ξ N ), an d so η ( ν, ξ ′ ) = lim N → + ∞ η ( ν N , ξ ′ ) ≤ lim su p N → + ∞ η ( ν N , ξ N ) = η ( ν , ξ ) . This completes the pro of of the lemma. Our next resu lt sho w th at the con v ergence in ( 8.1 ) is unif orm . This is where our uniform large deviation result f or nonc haotic in itial conditions comes in handy . Recall that M ( N ) 1 ( Z ) ⊂ M 1 ( Z ) is the subset of v alues tak en by th e in itial empirical m easure ν when there are N p articles. Lemma 8.2 . The c onver genc e i n ( 8.1 ) is uniform i n the fol lowing sense: for e ach f ∈ C b ( M 1 ( Z )) , we have (8.4) lim N → + ∞ sup ν ∈M ( N ) 1 ( Z )      1 N log Z M 1 ( Z ) e N f dp ( N ) ν,T − Λ( f | ν )      = 0 . Pr oof. W e w ill prov e this by con tradiction. S u pp ose the ab o v e limit is not zero. T hen there is an ε > 0 and an inﬁnite sub set V 1 ⊂ N su c h that sup ν ∈M ( N ) 1 ( Z )      1 N log Z M 1 ( Z ) e N f dp ( N ) ν,T − Λ( f | ν )      > ε, for ev ery N ∈ V 1 , INV ARIA N T MEASU RE IN MEAN FIELD MODELS 47 that is, the violations o ccur inﬁnitely often. S o w e can ﬁ nd a sequence ( ν N ) N ∈ V 1 suc h that      1 N log Z M 1 ( Z ) e N f dp ( N ) ν N ,T − Λ( f | ν N )      > ε, for every N ∈ V 1 . Extract a fur ther subsequ en ce, whic h is another inﬁnite su bset V 2 ⊂ V 1 , such that ( ν N ) N ∈ V 2 → ν for some ν . By Lemma 8.1 , (Λ( f | ν N )) N ∈ V 2 → Λ( f | ν ), and so (8.5)      1 N log Z M 1 ( Z ) e N f dp ( N ) ν N ,T − Λ( f | ν )      > ε 2 , for all suﬃ cien tly large N ∈ V 2 . Construct a new initial state sequence ( ν N ) N ≥ 1 that m atc h es with the ab o v e subsequence for N ∈ V 2 and such that ν N → ν . F or su c h a sequence, by Theorem 3.2 and V aradhan’s lemma, ( 8.1 ) holds. But ( 8.5 ) for all s u ﬃcien tly large N ∈ V 2 is a contradict ion to ( 8.1 ). Pr oof of Theorem 3.4 . Consid er the joint m easure ℘ ( N ) 0 ,T giv en by d℘ ( N ) 0 ,T ( ν, ξ ) = d℘ ( N ) 0 ( ν ) dp ( N ) ν,T ( ξ ) . W e sh all app ly F eng and Kur tz’s [ 18 , Prop. 3.25]. T o do this, we need to v erify thr ee conditions listed b elo w. • Exp onential tightness . The sequence ( ℘ ( N ) 0 ,T , N ≥ 1), w hic h comprises of p robabilit y measures on the compact pro du ct space, is trivially ex- p onentia lly tight. • Uniform c onver genc e of the L apla c e- V ar adhan functional in the i nitial c ondition . F or eac h N , the probabilit y m easur e p ( N ) ν,T is supp orted on the compact subs et M ( N ) 1 ( Z ). By Lemma 8.2 , for eac h f ∈ C b ( M 1 ( Z )), the con v ergence of the Laplace-V aradh an functional is uniform, as giv en in ( 8.4 ). • Continuity of L aplac e- V ar adhan functional in the initial c ondition . F or eac h f ∈ C b ( M 1 ( Z )), the fu nction ν 7→ Λ( f | ν ) is con tin uous, by Lemma 8.1 . Under the ab o v e conditions, F eng and Kurtz demonstrated in [ 18 , Prop. 3.25 and Rem. 3.26 ] that if the ﬁrs t marginal sequence ( ℘ ( N ) 0 , N ≥ 1) satisﬁes the large deviation principle with sp eed N and go o d rate function s , then so d o es the sequence of join t la ws ( ℘ ( N ) 0 ,T , N ≥ 1) with go o d rate f unction S 0 ,T ( ν, ξ ) = s ( ν ) + S T ( ξ | ν ). This concludes the p r o of. 48 V. S. BORKAR AND R. SUNDARESAN APPENDIX A: MUL TIPLE ω -LIMIT SET S In this app endix, w e v erify that the classical program of F reidlin-W ent zell [ 21 , Ch . 6] can b e extended to our setting. There are pr imarily t w o things to k eep in mind. First, for all ﬁnite N , we ha v e a jump pro cess on the simplex. Second, the q u an tit y V ( ·|· ) deﬁned in ( 2.9 ) is only un iformly con tin uous and not Lip sc hitz con tin uous. But this uniform contin uity suﬃces. Though the c hanges are minor, w e p r o vide the entire sequen ce of lemmas with mo diﬁ ed pro ofs f or completeness and ease of veriﬁcat ion. A.1. Auxiliary results. W e b egin with a sub set of the auxiliary results in [ 21 , Ch . 6] that w ere shown for diﬀusions on a compact manifold. Lemma A.1 . (F reidlin and W en tzell [ 21 , C h .6, Lemma 1.2]) F or any ε > 0 and any c omp act set K ⊂ M 1 ( Z ) , ther e exists a T 0 such that for any ν, ξ ∈ K ther e exists a function µ ( t ) , t ∈ [0 , T ] , µ (0) = ν , µ ( T ) = ξ , T ≤ T 0 with S [0 ,T ] ( µ | ν ) ≤ V ( ξ | ν ) + ε . Pr oof. Fix ε > 0. By part 3 of Lemma 3.2 , with the constant C 2 as in that Lemma, and ε 1 = ε/ (4 C 2 ), th ere is a δ 1 ∈ (0 , ε 1 ) suc h that tw o p oin ts within a distance δ 1 can b e connected b y a path of dur ation ε 1 and cost at most C 2 ε 1 = ε/ 4. By Lemma 3.4 , there is a δ 2 suc h that max { ρ 0 ( ξ 1 , ξ 2 ) , ρ 0 ( ν 1 , ν 2 ) } < δ 2 implies | V ( ξ 2 | ν 2 ) − V ( ξ 1 | ν 1 ) | ≤ ε/ 4 . Let δ = min { δ 1 , δ 2 } . Cho ose a ﬁ n ite δ -net { ν i } of p oin ts in K . Con n ect them with curv es µ i,j ( t ) , t ∈ [0 , T i,j ] , µ i,j (0) = ν i , µ i,j ( T i,j ) = ν j suc h that S [0 ,T i,j ] ( µ i,j | ν i ) ≤ V ( ν j | ν i ) + ε/ 4 . F or arbitrary ν , ξ ∈ K , let ν k and ν l b e the resp ectiv e closest p oin ts on the net. W e can now ﬁnd a path from ν to ν k , then to ν l along µ k ,l , and then to ξ , with ov erall cost at most ε/ 4 + ( V ( ν l | ν k ) + ε/ 4) + ε/ 4 = V ( ν l | ν k ) + 3 ε/ 4 ≤ V ( ξ | ν ) + ε where the last in equalit y follo ws from the unif orm contin u it y in Lemma 3.4 and the c hoice of δ . The du ration of the path is T k ,l + 2 ε 1 ≤ T 0 := (max i,j T i,j ) + 2 ε 1 . F or a set A ⊂ M 1 ( Z ), let [ A ] δ denote the (op en) δ -neigh b orho o d of A . Its closure w ill b e denoted [ A ] δ . F or the follo w ing lemma, recall the n otion of equiv alence b et w een t w o p oint s: ν ∼ ξ if V ( ξ | ν ) = V ( ν | ξ ) = 0 (see Section 4.2 ). INV ARIA N T MEASU RE IN MEAN FIELD MODELS 49 Lemma A.2 . (F reidlin and W en tzell [ 21 , Ch .6, Lemma 1.6]) L et al l p oints of a c omp act set K b e e quiv alent to e ach other, but not to any other p oint in M 1 ( Z ) . F or any ε > 0 , δ > 0 , ν, ξ ∈ K , ther e exists a T > 0 and a function µ ( t ) , 0 ≤ t ≤ T with µ (0) = ν, µ ( T ) = ξ , µ ( t ) ∈ [ K ] δ for al l t ∈ [0 , T ] , and S [0 ,T ] ( µ | ν ) < ε . Pr oof. Fix ε > 0 , δ > 0 , ν, ξ ∈ K . W e can ﬁn d a sequence ( T n , n ≥ 1) and paths µ ( n ) : [0 , T n ] → M 1 ( Z ) such that µ ( n ) (0) = ν, µ ( n ) ( T n ) = ξ for all n ≥ 1, and ε > S [0 ,T n ] ( µ ( n ) | ν ) → 0 as n → + ∞ . O bserve th at M 1 ( Z ) \ [ K ] δ is compact, and so if an inﬁn ite n um b er of µ ( n ) left [ K ] δ , there is a limit p oint z outside [ K ] δ . Using part 3 of Lemma 3.2 , and S [0 ,T n ] ( µ ( n ) | ν ) → 0 as n → + ∞ , it follo ws that V ( z | ν ) = V ( ξ | z ) = 0. T ogether with V ( ν | ξ ) = 0, w e conclude that V ( ν | z ) = 0 and so z ∼ ν . But then z is an equiv alen t p oin t outside K , whic h is a con tradiction. Hence µ ( n ) go es outside [ K ] δ for ﬁn itely man y n . T ake th e ﬁrst index larger than these. Th e corresp ondin g p ath remains completely within [ K ] δ , and meets all the other requirements. In this s ection, we shall use the notation τ A := inf { t ≥ 0 | µ N ( t ) / ∈ A } . The law for this exit time dep end s on N and ν through the la w p ( N ) ν for µ N . This dep end en ce will b e assumed as u ndersto o d and will b e sup pressed for brevit y . Lemma A.3 . (F reidlin and W ent zell [ 21 , Ch.6, Lemma 1.7]) . L et al l p oints of a c omp act set K b e e q u ivalent to e ach other and let K 6 = M 1 ( Z ) . F or a δ > 0 , let τ [ K ] δ := inf { t ≥ 0 | µ N ( t ) / ∈ [ K ] δ } . F or any ε > 0 , ther e exists a δ > 0 such that for al l suﬃciently lar ge N and al l ν ∈ [ K ] δ , we have E [ τ [ K ] δ ] < e + N ε wher e the exp e ctation is with r esp e ct to the me asur e p ( N ) ν . Pr oof. Fix ε > 0. Again b y part 3 of Lemma 3.2 , there is a δ 1 > 0 suc h that tw o p oin ts δ 1 -close ha v e a path connecting them of duration ε 1 = ε/ (4 C 2 ) and cost at most ε/ 4. C ho ose ξ outside K s uc h that ρ 0 ( ξ , K ) < δ 1 . Cho ose δ < ρ 0 ( ξ , K ) / 2; we thus ha v e 0 < δ < ρ 0 ( ξ , K ) / 2 < ρ 0 ( ξ , K ) < δ 1 . 50 V. S. BORKAR AND R. SUNDARESAN Consider a ﬁ nite δ -net of K . Lemma A.1 assures existence of paths that connect any pair of the net w ith cost at most ε/ 4. Let T ′ 0 denote the maxi- m um time among these paths, wh ere the maxim um is o v er pairs b elonging to the net, and let T 0 = T ′ 0 + 2 ε 1 . T ra v erse from an y ν ∈ [ K ] δ to its n earest p oint on th e net, then tra v erse from that p oint to th e p oin t on the net near- est to ξ , and thence to ξ . No w extend this path follo win g the McKean-Vlaso v dynamics so that the total du ration is n ow T 0 . T h is last app end age incurs no additional cost. Denote by µ the resu lting path of du ration T 0 . Clearly S [0 ,T 0 ] ( µ | ν ) ≤ 3 ε/ 4. No w, any tra jectory that is strictly δ -close to the tra j ectory µ exits [ K ] δ at least on ce in the in terv al [0 , T 0 ] b ecause f or some t ∈ [0 , T 0 ], we h av e µ ( t ) = ξ w h ic h is at a distance greater than 2 δ from K . W e then ha v e p ( N ) ν  τ [ K ] δ < T 0  ≥ p ( N ) ν { ρ T 0 ( µ N , µ ) < δ } ≥ e − 3 εN/ 4 , for all ν ∈ [ K ] δ , for all N ≥ some N 0 , where the last inequalit y holds by ( 3.14 ) in Corollary 3.1 . Consequ en tly p ( N ) ν  τ [ K ] δ ≥ T 0  ≤ 1 − e − 3 εN/ 4 , for all ν ∈ [ K ] δ , N ≥ N 0 . This un iform b oun d, the Mark o v p rop erty , and indu ction imp ly p ( N ) ν  τ [ K ] δ ≥ mT 0  ≤  1 − e − 3 εN/ 4  m , from which we obtain E [ τ [ K ] δ ] ≤ T 0 X m ≥ 0 (1 − e − 3 εN/ 4 ) m = T 0 e 3 εN/ 4 < e εN where the last inequalit y holds for all suﬃcientl y large N . T his concludes the pr o of. Lemma A.4 . (F reidlin and W entz ell [ 21 , Ch .6, Lemma 1.8]) . L et K b e an arbitr ary c omp act subset of M 1 ( Z ) and let G b e a neighb orho o d of K . F or any ε > 0 , ther e exists a δ > 0 such that for al l suﬃciently lar ge N and al l ν b elonging to g , with g = [ K ] δ and g = [ K ] δ , we have E " Z [0 ,τ G ] 1 g ( µ N ( t )) dt # > e − εN , wher e the exp e ctation is with r esp e ct to the me asur e p ( N ) ν . INV ARIA N T MEASU RE IN MEAN FIELD MODELS 51 Pr oof. Fix ε > 0. Ch o ose δ 1 small enough so that [ K ] δ 1 ⊂ G . Next, c ho ose δ 2 as in part 3 of Lemma 3.2 so th at with ε 1 = ε/ (2 C 2 ), any t w o δ 2 -close p oint s can b e connected by a path of du ration ε 1 and cost at m ost ε/ 2. No w let δ < min { δ 1 , δ 2 / 2 } and set g = [ K ] δ . Fix T > ε 1 . T ake any ν ∈ g . Connect it to the closest p oint on K (via a p ath of du ration ε 1 and cost ≤ ε/ 2) and then extend via the McKean- Vlaso v path with this initial condition for a fu rther dur ation of T − ε 1 . Call the ent ire path of d u ration T as µ . So long as µ is in side the δ / 3 n eigh- b orho o d of K , an y δ / 3 n eigh b orho o d of µ lies completely insid e g = [ K ] δ . By assumptions ( A2 )-( A3 ), the McKean-Vlaso v dyn amics has a b ounded v elocity ﬁeld. Consequently , th e part of µ that b egins at K and unt il either its exit from [ K ] δ/ 3 or time T , whic hev er o ccurs earlier, is of duration at least t 0 for some t 0 > 0, ind ep end ent of the starting p oin t. It follo ws that { ρ T ( µ N , µ ) ≤ δ / 3 } implies { τ g ≥ min { T , t 0 }} . F urthermore, τ G ≥ τ g and 1 g ( µ N ( t )) = 1 until the r an d om path exits g . Th us E " Z [0 ,τ G ] 1 g ( µ N ( t )) dt # ≥ E [ τ g ] ≥ E [ τ g · 1 { ρ T ( µ N , µ ) ≤ δ / 3 } ] ≥ m in { T , t 0 } · p ( N ) ν { ρ T ( µ N , µ ) ≤ δ / 3 } ≥ m in { T , t 0 } · e − N ε/ 2 (b y ( 3.14 )) ≥ e − N ε , where the last t w o inequalities hold for all suﬃ cien tly large N . Lemma A.5 . (F reidlin and W entz ell [ 21 , Ch .6, Lemma 1.9]) . L et K b e a c omp act subset of M 1 ( Z ) not c ontaining any ω -limit set entir ely. Ther e exist p ositive c onstants c and T 0 such that for al l suﬃcie ntly lar ge N , any T > T 0 , and any ν ∈ K , we have p ( N ) ν { τ K > T } ≤ e − N c ( T − T 0 ) . Pr oof. F or a suﬃ cien tly sm all δ , th e closed δ -neigh b orho o d of K , d e- noted [ K ] δ , do es not conta in any ω -limit set en tirely . Indeed, if this were not tru e, w e can ﬁnd a sequence of δ ↓ 0 suc h that eac h set in the nested decreasing sequence of sets [ K ] δ con tains an ω -limit set entirely . F or a δ , deﬁne Ω( δ ) to b e the closure of the u n ion of ω -limit sets con tained in [ K ] δ . Clearly , Ω( δ ) is a p ositiv ely inv ariant set, and the family indexed by δ is 52 V. S. BORKAR AND R. SUNDARESAN a nested decreasing sequence of nonempt y compact sets. Then ∩ δ Ω( δ ) is a nonempt y compact inv ariant set in K . F urther, it conta ins an ω -limit set en tirely , a con tradiction. F or ν ∈ [ K ] δ , denote b y τ ( ν ) the time for ﬁrs t exit of the solution to the McKean-Vlaso v equ ation w ith in itial condition ν from the set [ K ] δ . Since [ K ] δ do es not contai n any ω -limit set en tirely , τ ( ν ) < + ∞ for all ν ∈ [ K ] δ . The f unction τ ( ν ) is upp er semicon tin uous, and consequent ly , it attains its largest v alue max ν ∈ [ K ] δ τ ( ν ) = T 1 < + ∞ . Set T 0 = T 1 + 1 and consider all paths of d uration T 0 that tak e v alues only in [ K ] δ . It is easy to see that this set is closed. It follo ws that for eac h ν ∈ [ K ] δ , w e ha v e that S [0 ,T 0 ] ( ·| ν ) attains its minim um A ( ν ) on th is set. F urther, the mapping ν 7→ A ( ν ) is contin u ous, as can b e sh o wn by an easy application of L emmas 3.2 , 7.1 and 7.2 . Th us A := min ν ∈ [ K ] δ A ( ν ) is attained. This min imum is strictly p ositiv e since there are no tra jectories of the McKean-Vlaso v equation among the paths u nder consideration. Fix ε < A , a ν ∈ K , and consider the family of paths Φ ν ( A − ε/ 2) = { µ : [0 , T 0 ] → M 1 ( Z ) | S [0 ,T 0 ] ( µ | ν ) ≤ A − ε/ 2 } . An y p ath in this set exits [ K ] δ in the in terv al [0 , T 0 ]. With in itial state ν , the eve nt { τ K > T 0 } implies that the tra jectory r emains ent irely within K , and since an y path in Φ ν ( A − ε/ 2) exits [ K ] δ , we m ust ha v e ρ T 0 ( µ N , Φ ν ( A − ε/ 2)) ≥ δ. It follo ws that p ( N ) ν { τ K > T 0 } ≤ p ( N ) ν { ρ T 0 ( µ N , Φ ν ( A − ε/ 2)) ≥ δ } . By considering an y ν 1 ∈ K , and by us ing ( 3.13 ) of Corollary 3.1 , we ha v e sup ν 1 ∈ K p ( N ) ν 1 { τ K > T 0 } ≤ sup ν 1 ∈ K p ( N ) ν 1 { ρ T 0 ( µ N , Φ ν ( A − ε/ 2)) ≥ δ } ≤ exp {− N (inf { S [0 ,T ] ( µ | ν 1 ) | µ ∈ Φ ν 1 ( A − ε/ 2) } − ε/ 2) } for all suﬃcien tly large N ≤ e − N ( A − ε ) for all suﬃcien tly large N . F or our ﬁxed ν ∈ K , the Marko v prop ert y then implies p ( N ) ν { τ K > ( m + 1) T 0 } ≤ E [ 1 { τ K > mT 0 } · E [ 1 { τ K > T 0 } | µ ( N ) ( mT 0 )]] ≤ p ( N ) ν { τ K > mT 0 } ·  sup ν 1 ∈ K p ( N ) ν 1 { τ K > T 0 }  ≤ p ( N ) ν { τ K > mT 0 } · e − N ( A − ε ) . INV ARIA N T MEASU RE IN MEAN FIELD MODELS 53 By ind u ction, for a T > T 0 , w e h av e p ( N ) ν { τ K > T } ≤ p ( N ) ν  τ K >  T T 0  T 0  ≤ e − N ( A − ε ) j T T 0 k ≤ e − N ( A − ε )( T /T 0 − 1) = e − N c ( T − T 0 ) for c = ( A − ε ) /T 0 , and this completes the pro of. The ab ov e theorem has the follo wing immediate corollary . Corollar y A.1 . (F reidlin and W en tzell [ 21 , Ch .6, Corollary to Lemma 1.9]) . L et K b e a c omp act set not c ontaining any ω -limit set entir e ly. Ther e exists a p ositive inte ger N 0 and a p ositive c onstan t c such that for N ≥ N 0 and any ν ∈ K , we have E [ τ K ] ≤ T 0 + 1 / ( cN 0 ) wher e the exp e ctation is with r esp e ct to the me asur e p ( N ) ν . Recall the deﬁn ition of V giv en in ( 2.9 ), and the notion of equiv alence on M 1 ( Z ) giv en in Section 4.2 . Under condition ( B ) in Section 4.2 , w e ha v e equiv alen t s ets K 1 , . . . , K l to whic h all ω -limit sets con v erge. W e s hall no w deﬁne a discrete-time Mark o v c hain of states at hitting times of n eigh b or- ho o ds of these compact sets. In order to b oun d the tran s ition p robabilities of this c hain, recall the deﬁn itions of ˜ V ( K i , K j ) give n in ( 2.10 ) an d V ( K i , K j ) giv en in ( 2.11 ). Deﬁne the f ollo win g quan tities: • An r 0 suc h that 0 < r 0 < (1 / 2) min i,j ρ 0 ( K i , K j ), • An r 1 suc h that 0 < r 1 < r 0 , • The s et C as C := M 1 ( Z ) \ ∪ l i =1 [ K i ] r 0 , • The s et Γ i as Γ i := [ K i ] r 0 , • The s et g i as g i := [ K i ] r 1 , and ﬁnally , • The s et g as g := ∪ l i =1 g i . Let us no w deﬁne the follo win g s topping times: • τ 0 := 0, • The time for exit f rom the union of th e r 0 neigh b orho o ds of the com- pact sets K i ’s, that is, σ n := inf { t ≥ τ n | µ N ( t ) ∈ C } , • The time to re-en ter g , th at is, τ n := inf { t ≥ σ n | µ N ( t ) ∈ g } . 54 V. S. BORKAR AND R. SUNDARESAN Finally , we deﬁn e Z n := µ N ( τ n ). W e shall use the notation p ( N ) ( ν, g j ) for p ( N ) ν ( µ N ( τ n ) ∈ g j ) wh en µ N ( τ n − 1 ) = ν . Lemma A.6 . (F r eidlin and W en tzell [ 21 , Ch.6, L emm a 2.1]) . F or any ε > 0 , ther e is a smal l enough r 0 > 0 suc h that for any r 2 satisfying 0 < r 2 < r 0 , ther e is an r 1 satisfying 0 < r 1 < r 2 such that for al l suﬃcie ntly lar ge N , for al l ν ∈ [ K i ] r 2 , the one-step tr ansition pr ob abilities of Z n satisfy exp {− N ( ˜ V ( K i , K j ) + ε ) } ≤ p ( N ) ( ν, g j ) ≤ exp {− N ( ˜ V ( K i , K j ) − ε ) } . Pr oof. F or p airs with ˜ V ( K i , K j ) = + ∞ , th ere is n o smo oth curve fr om K i to K j without touc hing one of the other compact sets. It follo ws that f or an y arbitrary 0 < r 1 < r 2 < r 0 , for all suﬃ cien tly large N , there is no path in M ( N ) 1 ( Z ) from [ K i ] r 2 to g j without touching [ K i ′ ] r 0 for some i ′ 6 = i, j . Th us, for all suﬃ cien tly large N , w e ha v e p ( N ) ( ν, g j ) = 0. The v alidit y of the lemma is ob vious f or suc h pairs. F or all other pairs ˜ V ( K i , K j ) ≤ V 0 for some V 0 < + ∞ . Let us ﬁrs t argue the lo w er b ound . Fix ε > 0. Ch o ose δ as in part 3 of L emma 3.2 s o that with ε 1 = ε/ (10 C 2 ), an y t w o p oin ts δ -close can b e connected by a path of duration ε 1 and cost at most ε/ 10. Set r 0 = min { δ / 2 , (1 / 3) min i,j ρ 0 ( K i , K j ) } . Fix arbitrary r 2 satisfying 0 < r 2 < r 0 . F or eac h ( i, j ) w ith ˜ V ( K i , K j ) < + ∞ , choose paths µ i,j : [0 , T i,j ] → M 1 ( Z ) suc h that • µ i,j (0) ∈ K i , • µ i,j ( T i,j ) ∈ K j , • µ i,j ( t ) do es not touc h ∪ i ′ 6 = i,j K i ′ for t ∈ [0 , T i,j ], and • S [0 ,T ] ( µ i,j | µ i,j (0)) ≤ ˜ V ( K i , K j ) + 0 . 2 ε . No w c ho ose r 1 so that r 1 < m in n r 2 , r 0 2 , 1 2 min n ρ 0  µ i,j ( t ) , ∪ i ′ 6 = i,j K i ′  | t ∈ [0 , T i,j )] , 1 ≤ i, j ≤ l oo . Also c ho ose δ ′ < min { r 0 − r 2 , r 1 } so that r 2 + δ ′ < r 0 and hence r 0 + δ ′ < 2 r 0 < δ by the choice of δ . T ak e any ν ∈ [ K i ] r 2 . Fix a ﬁnite δ ′ -net of K i . If i 6 = j , consider the follo w ing path. • Connect ν to the n earest p oin t ν 1 ∈ K i with a path of d uration ε 1 and cost at most 0 . 1 ε . INV ARIA N T MEASU RE IN MEAN FIELD MODELS 55 • Connect ν 1 to th e nearest p oin t ν 2 on th e δ ′ -net of K i again with a path of du ration ε 1 and cost at most 0 . 1 ε . • Let ν 3 b e the p oin t on the δ ′ -net nearest to µ i,j (0). T ra v erse the path giv en by Lemma A.2 that connects ν 2 to ν 3 without lea ving the r 1 - neigh b orho o d of K i . Th anks to the ﬁn ite num b er of p oin ts on the δ ′ -net, this can b e d on e in b oun ded time. Moreov er, the cost is at most 0 . 1 ε . • Connect ν 3 to µ i,j (0) with path of du r ation ε 1 and cost at most 0 . 1 ε . • Then tra v erse th e path given b y µ i,j . If i = j , then s im p ly take ν to a p oint at a distance r 0 + δ ′ from K i and then to the n earest p oin t in K i . Note that r 0 + δ ′ < δ , an d so the dur ation of this path is 2 ε 1 and cost at most 0 . 2 ε . T he constru cted path is of b ounded time d uration, b ounded sa y b y T 0 . W e can th us extend all p aths to duration T 0 along the McKean-Vlaso v p ath, an app endage that incurs n o additional cost. Call the resulting p ath µ ( t ) , t ∈ [0 , T 0 ]. Clearly , S [0 ,T 0 ] ( µ | ν ) ≤ ˜ V ( K i , K j ) + 0 . 6 ε. If ρ T 0 ( µ N , µ ) < δ ′ , then the tra ject ory µ N b egins at a p oin t ν w ithin an r 0 -neigh b orho o d of K i , r eac h es the δ ′ -neigh b orho o d of K j and so h its g j = [ K j ] r 1 , is at most δ ′ distance a wa y fr om the tra jectory µ , and hence do es not hit the r 0 -neigh b orho o d of an y K i ′ , i ′ 6 = i, j ; then µ N ( τ n ) ∈ g j . In other words, { ρ T 0 ( µ N , µ ) < δ ′ } ⊂ { µ N ( τ n ) ∈ g j } , and so p ( N ) ( ν, g j ) ≥ p ( N ) ν { ρ T 0 ( µ N , µ ) < δ ′ } ≥ exp {− N ( S [0 ,T 0 ] ( µ | ν ) + 0 . 1 ε ) } ≥ exp {− N ( ˜ V ( K i , K j ) + ε ) } , where the second inequalit y holds f or all suﬃcien tly large N u n iformly ov er the initial condition, th an k s to ( 3.14 ) of Corollary 3.1 . This establishes the lo w er b ound. W e now pr o v e the upp er b ound. C onsider an y path µ of some duration T starting at ν in the r 1 -neigh b orho o d of K i , end ing at a p oin t say ξ in the δ ′ -neigh b orho o d of g j at time T , and not touc hing an y of the other compact sets K i ′ , i ′ 6 = i, j . By the c hoices of r 1 and δ ′ , there are s hort paths from a p oint ν ′ ∈ K i to ν and f r om ξ to a p oint ξ ′ ∈ K j , eac h of duration ε 1 and cost at most 0 . 1 ε . The path that trav erses from ν ′ to ν , and th en along µ to ξ , and thence to ξ ′ , has cost at most S [0 ,T ] ( µ | ν ) + 0 . 2 ε ≥ ˜ V ( K i , K j ), and so (A.1) S [0 ,T ] ( µ | ν ) ≥ ˜ V ( K i , K j ) − 0 . 2 ε. 56 V. S. BORKAR AND R. SUNDARESAN The same holds for an y path µ of some dur ation T starting at ν in the r 1 -neigh b orho o d of K i , touching the δ ′ -neigh b orho o d of g j at time in [0 , T ], but not touc hing any of the other compact sets K i ′ , i ′ 6 = i, j . By Lemma A.5 , with the set C in place of K , a set that do es not conta in an y ω -limit set entirely , and with T 1 = T 0 + V 0 /c where c, T 0 are as sp eciﬁed in that lemma, we obtain (A.2) p ( N ) ν { τ 1 > T 1 } ≤ sup ν ′ ∈ C p ( N ) ν ′ { τ C > T 1 } ≤ e − N V 0 for all suﬃcien tly large N . Consider a tra jectory µ N with µ N (0) = ν ∈ [ K i ] r 1 and µ N ( τ 1 ) ∈ g j . There are t w o p ossibilities: (1) τ 1 > T 1 , or (2) τ 1 ≤ T 1 in whic h case the tra jectory en ters g j in [0 , T 1 ]. In this second case, with Φ [0 ,T 1 ] ,ν ( v ) := { µ : [0 , T 1 ] → M 1 ( Z ) | S [0 ,T ] ( µ | ν ) ≤ v } , w e hav e (A.3) ρ T 1 ( µ N , Φ [0 ,T 1 ] ,ν ( ˜ V ( K i , K j ) − 0 . 3 ε )) ≥ δ ′ . T o see this, note the conditions δ ′ < r 1 , τ 1 ≤ T 1 , and µ N ( τ 1 ) ∈ g j . If µ is an y tra jectory satisfying ρ T 1 ( µ N , µ ) < δ ′ , then µ must hit the δ ′ -neigh b orho o d of g j without touc hing any of th e other compact sets K i ′ , i ′ 6 = i, j . F rom ( A.1 ), su btracting an extra 0 . 1 ε , w e get S [0 ,T 1 ] ( µ | ν ) > ˜ V ( K i , K j ) − 0 . 3 ε . By contraposition, u n der the n oted conditions, any µ with S [0 ,T 1 ] ( µ | ν ) ≤ ˜ V ( K i , K j ) − 0 . 3 ε must s atisfy ρ T 1 ( µ N , µ ) ≥ δ ′ , and hen ce ( A.3 ) follo ws. Putting the tw o cases toget her, we get p ( N ) ν { µ N ( τ 1 ) ∈ g j } ≤ p ( N ) ν { τ 1 > T 1 } + p ( N ) ν { ρ T 1 ( µ N , Φ [0 ,T 1 ] ,ν ( ˜ V ( K i , K j ) − 0 . 3 ε )) ≥ δ ′ } ( a ) ≤ e − N V 0 + exp {− N ( ˜ V ( K i , K j ) − 0 . 3 ε ) + N (0 . 1 ε ) } ( b ) ≤ exp {− N ( ˜ V ( K i , K j ) − ε ) } . In the ab o v e sequen ce of inequalities, (a) holds for all suﬃciently large N due to ( A.2 ), ( 3.13 ) of Corollary 3.1 , and the deﬁnition of Φ [0 ,T 1 ] ,ν . Inequalit y (b) also h olds for all suﬃcien tly large N b ecause ˜ V ( K i , K j ) ≤ V 0 . T h is pr o v es the up p er b oun d and completes the pro of. Recall the d eﬁnition G { i } in the paragraph preceding ( 2.12 ), the deﬁn ition of W ( K i ) in ( 2.12 ), and the deﬁnition of s i , i = 1 , . . . , l . W e are no w ready to state the main theorem of this app endix. INV ARIA N T MEASU RE IN MEAN FIELD MODELS 57 Theorem A.1 . (F reidlin and W en tzell [ 21 , Ch .6, Th eorem 4.1]) . Assume ( A1 ) - ( A3 ) and ( B ) hold. F or any ε > 0 , ther e is an arbitr arily smal l r 1 > 0 such that exp {− N ( s i + ε ) } ≤ ℘ ( N ) { [ K i ] r 1 } ≤ exp {− N ( s i − ε ) } wher e s i ar e deﬁne d i n ( 2.13 ). Pr oof. All the steps of the pro of of [ 21 , Ch.6, Th .4.1] hold, since the analogs of all the lemmas used in th at pro of ha v e no w b een veriﬁed to hold. Ac kno w ledgemen ts. W e thank an anon ymous reviewe r for suggestions that help ed improv e the presenta tion. References. [1] An antharam, V. (1991) “A mean ﬁeld limit for a lattice caricature of dynamic routing in circuit switched netw orks”, Annals of Appl. Pr ob. 1 , 481–503. MR1129770 [2] An antharam, V. and Be nchekroun, M . (1993) “A tec hnique for computing so- journ times in large netw orks of interacting queues”, Pr ob abil ity in the Engi ne ering and Informational Sci enc es 7 , 441–464. [3] B enaim, M. and Le Boudec J.-Y. (2008) “A class of mean ﬁeld in teraction models for comput er and comm unication systems”, Perform. Eval. 65 , 823–838. [4] B enaim, M. and We ibull, J. (2003) “D eterministic approximation of stochastic evol ution”, Ec onometric a 71 , 873-904. MR1983230 [5] B ianchi, G. (1998) “IEEE 802.11 - saturated throughpu t analysis”, IEEE Comm. L ett. 12 , 318–320. [6] B isw as, A. an d Borkar, V. S. (2009) “Small n oise asympt otics for in v arian t densi- ties for a class of d iﬀusions: a con trol theoretic view”, Journal of Mathematic al Anal- ysis and Applic ations 360 , 476– 484. Correction note: arXiv:1107. 2277v1 MR2561245 [7] B orkar, V. S. (1995) Pr ob abili ty The ory: An A dvanc e d Course , Springer V erlag, New Y ork. MR1367959 [8] B ordena ve , C. , McDonald, D. and Prouti ` ere, A. (2010) “A particle system in intera ction with a rapidly v arying environmen t: mean ﬁeld limits and applications”, Networks and Heter o gene ous Me di a , 5 (1). [9] B ordena ve , C. , McDonald, D. and Pro uti ` ere, A. (2005) “Ran d om multi-access algorithms, a mean ﬁ eld analysis”, Pr o c. 43th Al lerton c onfer enc e . [10] D ai Pra, P. and den Hollander, F. (1996), “McKean-Vlasov limit for interacting random processes in random media”, Journal of Statistic al Physics , 84 (3/4), 735– 772. MR1400186 [11] D a wson, D. A. and G ¨ ar tner, J. (1987) “Large d eviations from the McKean-V laso v limit for w eakly interacting diﬀusions”, Sto chastics , 20 , 247–30 8. MR0885876 [12] D a y, M. V. (1987) “Recent progress on the small p arameter ex it problem”, Sto chas- tics , 20 , 121–15 0. MR0877726 [13] Dembo, A. and Zeitouni , O. (2010) L ar ge Deviations T e chniques and Appli c ations , (2nd ed., corrected p rinting), Sp ringer-V erlag, Berlin Heidelb erg. MR2571413 [14] Del Moral, P. and Zaj ic, T. (2003) “Note on the Laplace-V aradhan Integral Lemma”, Bernoul li , 9 (1), F eb. 2003, 49–65. MR1963672 58 V. S. BORKAR AND R. SUNDARESAN [15] Djehiche, B. and Kaj , I. (1995) “The rate function for some measure-v alued jump processes”, The Annals of Pr ob ability , 23 (3), 1414–1438. MR1349178 [16] Duffy, K. R. (2010) “Mean ﬁeld Marko v mod els of wireless local area n etw orks”, Markov Pr o c esses and R elate d Fields 16 (2), 295–328. MR2666856 [17] Ethier, S. N. and Kur tz, T. G. ( 2005) M arkov Pr o c esses: Char acterization and Conver genc e (2nd ed.), John Wiley , New Y ork. MR0838085 [18] Feng, J. and Kur tz, T . G. (2006) L ar ge Deviations for Sto chastic Pr o c esses , Mathematical Surveys and Monogra phs, 131 . American Math. So c., Providence, RI . MR2260560 [19] Feng, S. (1994) “Large deviations for Mark ov p rocesses with mean ﬁeld in teraction and un b ounded jump s”, Pr ob abil ity Th. and R elate d Fiel ds 100 , 227– 252. MR1296430 [20] Feng, S. (1994) “Large deviations for empirical pro cess of mean-ﬁeld interacting particle system with unboun ded jumps”, The Annals of Pr ob ability , 22 (4), 2122– 2151. MR1331217 [21] Freidlin, M. I. and Wentze ll, A. D. ( 1998) R andom Perturb ations of Dynamic al Systems (2nd ed.), Springer V erlag, Berlin–Heidelberg. MR1652127 [22] Graham, C. (2000) “Chaoticit y on path space for a queu eing netw ork with selection of th e shortest qu eue among several”, J. A ppl . Pr ob ab. 37 (1), 198–211. MR1761670 [23] Kac , M. (1956) “F oundations of kin etic theory”, in ‘ Pr o c. of 3r d Berkeley Symp. on Math. Stat. and Pr ob. ’ (J. Neyman, ed.), Uni. of California Press, 171–197. MR0084985 [24] L ´ eonard, C. (1995) “Large deviations for long range intera cting particle systems with jumps”, Annales de l ’I. H. P., se ction B , 31 (2), 289–323 . MR1324810 [25] McKean, H. P. (1966) “A class of Marko v p ro cesses asso ciated with nonlinear parab olic equations”, Pr o c. Nat. A c ad. Sci. 56 , 1907–1911 . MR0221595 [26] Ramaiy an, V. , Kuma r, A. and Al tman, E. (2008) “Fixed p oint analysis of sin- gle cell IEEE 802.11e WLA Ns: uniquen ess and multistabilit y”, IEEE/ACM T r ans. Network. 16 , 1080–1 093. [27] Sharma, G . , Ganesh, A. J. and Key, P. (2009) “Perf ormance analysis of con tention based med ium access p rotocols”, I EEE T r ans. I nformation T h. 55 (4), 1665–1682. Also rep orted in Pr o c. IEEE Info c om (2006) . MR2582753 [28] Sheu, S-J. (198 5) “Stochastic con trol and exit probab ilities of jump processes”, SIAM J. Contr ol and Optimi zation 23 (2), 306–328, March 1985. MR0777462 [29] Sheu, S-J. (198 6) “Asymptotic behavior of the inv ariant density of a diﬀusion Mark o v process with small diﬀusion”, SIAM J. Math. Anal. 17 (2), 451–46 0. MR0826705 [30] Shw ar tz, A. and Weiss, A . (1995) L ar ge Deviations for Performanc e Analysis , Chapman & Hall. MR1335456 [31] Stol y ar, A. L. (1989) “Asymptotic b ehavior of the stationary distribution for a closed queueing system”, Pr oblemy Per e dachi Informatsii , 25 (4), 80–92. MR1040026 [32] Sznitman, A. S . (1991) “T opics in prop ogation of chaos”, in ‘ Ec ole d’´ et ´ e de Pr ob- abilit´ es de Saint Flour (1989) ’, Springer Lecture Notes in Mathematics No. 1464 , 166–251 MR1108185 [33] Vvedenska y a, N. D. and Sukhov Yu. M. ( 2007), “A multi-user m ultiple-access system: stability and metastabilit y”, Pr oblemy Per e dachi Informatsii 43 , 105–111. MR2360021 Dep a rt ment of Electrical Eng ineering Indian Institute of Technology Bomba y Pow ai, Mu mbai 400076, India E-mail: bork ar.vs@gmail. com Dep a rt ment of Electrical Communica tions Eng ineering Indian Institute of Science Bangalore 560 012, India E-mail: ra j eshs@ece.iisc.ernet.in

Asymptotics of the Invariant Measure in Mean Field Models with Jumps

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment