Sample average approximation of CVaR-based Wardrop equilibrium in routing under uncertain costs

Sample a v erage approximation of CV aR-based W ardrop equilibrium in routing under uncertain costs Ashish Cherukuri Abstract — This paper focuses on the class of routing games that h a ve un certain costs. Assuming that agents are risk- a verse and select paths with minimum conditional value-at- risk (CV aR) associated to them, we defin e the notion of CV aR- based W ardrop e quil ibrium (CWE). W e focus on computin g this equilibriu m under the condition that the distribution of the uncertainty is unkn own and a set of indepen dent and identically distributed samples is av ailable. T o this en d, we defin e the sample av erage app roxima tion scheme where CWE is estimated with solutions of a variational inequali ty p roblem in volving sample av erage approximations of the CV aR. W e establish two properties f or th is scheme. First, under continuity of costs and boundedn ess of uncertainty , we p ro ve asymptotic consistency , establishing almost sure conv ergence of approximate equilibria to CW E as the sample size grows. Second , un der the additional assumption of Lipschi tz cost, we pro ve exponenti al conv erge nce where th e probability o f the distance between an approximate solution and the CWE b eing smaller than any constant ap- proaches u nity exponentially fast. Simulation example v alidates our theore tical findin gs. I . I N T RO D U C T I O N Users o f a transpor tation network are of ten selfish, min- imizing th eir own cost functio n , such as travel time, when trav ersing throu gh the network. This p henom enon is popu- larly mo deled as a ( d eterministic) nonato mic routing g ame where the numbe r of users is assumed to be large, each controllin g in finitesimal amo u nt of flow in the n etwork. Therefo re, an ind i vidu al u ser do es no t affect the co st incu rred on a path by u n ilaterally chang ing its rou te choice . At an e quilibrium o f th is game , termed W ardr op equilibrium (WE) [1], p aths with no nzero flo w h av e the least cost among all the alternati ves. I n real-life, the cost a ssociated to a path is u ncertain, affected by unplanned events such as, acciden ts, weather fluctuation s, and co nstruction work. Different user s might min im ize different objectives und er this uncer tainty , e.g., expected cost, specific quan tile of the cost, or a r isk measure. These disparate behaviors lead to different notio ns of equilibriu m . Computing these eq uilibria an d an alyzing their pr operties help p r edict conge stion patterns. Motivated by this, o ur goal h ere is to estimate the equ ilibria, using samples of th e un certainty , when agents are risk-averse and seek paths that have minimum conditio nal value-at-risk. Literatur e r e view: In a routing setup, experimental stud- ies [2], [ 3] validate th e fact that agen ts arrive at som e eq u ilib- rium path cho ice after repeated in teraction with each other . T raditiona lly , equilibriu m flow was p redicted in a deter min- istic setup un der the fo rmalism of WE [4]. Su ch prediction s were used for d esigning to lls [5] an d p lanning for f uture The author is with the Engineering and T echnology Institut e Groningen, Uni versit y of Groning en. Email: a.k.cherukuri@rug. nl. transportatio n infrastructur e [6]. When co sts are un certain, no single traf fic flow works as a WE for all rea liza tio ns of the unce rtainty . Therefore, to estimate e q uilibrium flow , some works solve a stoch a stic no nlinear complementarity problem [7], either in an expec ta tio n b asis [8], [9] or in a ro bust manner [10], [ 11]. I n [12], eq u ilibrium flow is hy p othesized to be the minimizer of the regret experien ced by user s. These app roaches mig ht be too conservati ve or migh t n ot account fo r the risk-sensitive beh avior o f agen ts. Among the works that consider risk, [13] and [1 4] con sider the cost to be the weighted sum of the mean and the variance of the uncertain cost. Furth e r, for this risk criter ia, [ 1 5] intr oduces the n otion of pr ic e of r isk aversion and [16] determ in es tighter bound s f or it. In the transportatio n literature, the CV aR-based equilibrium is also known as th e mean excess traffic equilibriu m , see e.g., [17], [18] and r e ferences the rein. While th ese works have explored num erous algor ithms for computin g the eq uilibrium , th ey lack theor etical perfo rmance guaran tee s for sample-based so lutions. This paper attem p ts to bridg e this gap. On the tec h nical side, our paper relates to the bo dy of work on sample av erage app roximatio n , see [19, Chapter 5] for a detailed overview . In particular, we bo rrow ideas from studies o n sample av erage app roximatio n of stoch a stic variational ineq ualities [ 20], gen e ralized eq uations [2 1], and mathematical progra ms with equ ilibrium constrain ts [2 2]. Setup and contributions: Our starting point is the def- inition of the nonatom ic routin g g ame where age nts aim to minimize the condition al value-at-risk (CV aR) of the uncertain co st associated with each path. W e assume th a t the demand is fixed and determ inistic, the c o st fu nctions ar e continuo us, and the support o f the un certainty is b ounded . At an equilib r ium of this gam e, ter med CV aR-based W ar d rop equilibriu m (CWE), paths with no nzero flow have lowest CV aR. Given a certain n umber of indep e ndent and identically distributed sam ples of the uncertainty , we define sample av erage appro ximation of the CV aR b y r eplacing the expec- tation operator with its samp le average. Subseq uently , we formu late a variational inequ ality (VI ) problem using these approx imate costs. Our a im then is to stud y the statistical proper ties of the solutions o f this approximate VI p roblem as the n umber of samp les grow . In particu lar our contributions are twofold: (i) W e show that as the samp le size grows, the set of solutions o f th e appro ximate VI prob lem conv erge almost sur ely , in a set-valued sense, to the set of CWE. (ii) Unde r the additio nal assum p tion that the costs are Lipschitz con tinuous, we establish th e expon ential co n - vergence of the ap proxim ate solution s to the set o f CWE. That is, g iv en any con stant, the p robab ility th at the distance of an appro ximate solution from the set of CWE is less th an that constant ap proache s u nity exponentially with num ber o f samp les. W e provide a simple simulation example illustra ting th ese guaran tee s. I I . N O TA T I O N A N D P R E L I M I N A R I E S Let R , R ≥ 0 , R > 0 , and N den ote th e set of real, r eal nonnegative, real positiv e, and natural nu mbers, respecti vely . Let k · k denote the 2 -norm. W e use [ N ] := { 1 , . . . , N } for N ∈ N . For x ∈ R , we let [ x ] + = max( x, 0) , [ x ] − = min( x, 0) , and ⌈ x ⌉ be the smallest integer grea te r than or equal to x . Th e cardinality of a set is denoted by || . The distance o f a po int x ∈ R m to a set ⊂ R m is den oted as dist( x, ) := inf y ∈ k x − y k . The deviation of a set A ⊂ R m from is D ( A , ) := sup y ∈A dist( y , ) . 1) V a riational ineq uality: Given a map F : R n → R n and a clo sed set H ⊂ R n , the variatio nal ineq u ality (VI) problem , denoted VI( F, H ) , in volves find ing h ∗ ∈ H su c h that ( h − h ∗ ) ⊤ F ( h ∗ ) ≥ 0 for all h ∈ H . Such a point is called a solutio n o f th e VI pro blem. The set o f solutions of VI( F, H ) are d enoted b y SOL( F, H ) . 2) Graph th eory: A dir ected graph is a pair G = ( V , E ) , where V is a finite set called the vertex set o r node set , E ⊆ V × V is called the edge set , wh e r e ( i, j ) ∈ E if th ere is a d irected edge fro m vertex i to j . A path is an ordered sequence of uniqu e vertices such th at two subseque n t vertices form an edge. A sour ce is a vertex with no incoming edge and a sink is a vertex with n o o utgoing ed ge. 3) Uniform c on ver gence: A seque n ce o f func tions { f N : X → Y } ∞ N =1 , wh ere X and Y are Euc lidean spaces, is said to conver ge unifo rmly on a set X ⊂ X to f : X → Y if for any ǫ > 0 , there exists N ǫ ∈ N such th at sup x ∈ X k f N ( x ) − f ( x ) k ≤ ǫ, for all N ≥ N ǫ . Similar definitio n applies for conver genc e in prob ability . That is, consider a ran dom sequence o f fun ction { f ω N : X → Y } ∞ N =1 defined on a p robability space (Ω , F , P ) . The sequence is said to conver ge un iformly to f : X → Y o n X almost surely (shorthand, a.s.) if f ω N → f unifo rmly on X for almost all ω ∈ Ω . 4) Risk measures: Next we revie w notion s on value- at-risk and cond itional value-at-risk following [19]. Given a real-valued r andom variable Z with pro bability distribu- tion P , we deno te the cumulative d istribution fu nction by H Z ( ζ ) := P ( Z ≤ ζ ) . Th e left-side α -qua n tile of Z is defined as H − 1 Z ( α ) := inf { ζ | H Z ( ζ ) ≥ α } . Given a proba bility lev el α ∈ (0 , 1) , the value- at-risk of Z at le vel α , denoted V a R α [ Z ] , is the left-side (1 − α ) -quan tile of Z . Formally , V a R α [ Z ] := H − 1 Z (1 − α ) = inf { ζ | P ( Z ≤ ζ ) ≥ 1 − α } = inf { ζ | P ( Z > ζ ) ≤ α } . The c ondition a l va lu e-at-risk (CV aR ) , also referred to as the average value-at-risk in [19], of Z a t level α , den o ted CV aR α [ Z ] , is th e expectation o f Z when it takes values bigger than V aR α [ Z ] . Th at is, CV aR α [ Z ] := E [ Z ≥ V aR α [ Z ]] . (1) One can show that, equ ivalently , CV aR α [ Z ] = inf t ∈ R n t + 1 α E [ Z − t ] + o . (2) The parameter α c haracterizes th e risk-averseness. When α is close to unity , the decision-m aker is risk- neutral, whereas, α close to th e o r igin imp lies high risk-averseness. Th e minimum in (2) is attained at a point in the interval [ t m , t M ] , where t m := inf { ζ | H Z ( ζ ) ≥ 1 − α } , and t M := sup { ζ | H Z ( ζ ) ≤ 1 − α } . I I I . R O U T I N G G A M E W I T H U N C E RTA I N C O S T S Consider a network repr esented u sing a directed gr aph G = ( V , E ) , wh ere V an d E ⊆ V × V stand for the set of nod es and edges, r espectiv ely . Her e V and E , for instance, model th e sets o f in tersections and streets in a city when G is a tr affic network. The sets of origin and destinatio n no des ar e the sets of sources and sinks in the network , and are deno te d by O and D , respectively . Th e set of origin- destination (OD) pairs is W ⊆ O × D . Le t P w denote the set of av ailable paths for the OD pair w ∈ W and let P = ∪ w ∈ W P w be the set of all paths, see Section II for relev ant de finitions. W e assume that n umerou s agents traverse throug h the network in a no ncoop e rativ e man ner . Th is f ramework is modeled as a nonatom ic r outing gam e where each individual agent’ s actio n has infinitesimal imp act on the aggr egate traffic flo w . As a consequen ce, flow is modeled as a continu ous variable. W e assume that each agent is associated with an OD pair w ∈ W and is allo wed to select any path p ∈ P w . The r oute choices of all a g ents g iv e rise to the agg regate traffic wh ich is mod eled as a flow vec to r h ∈ R |P | ≥ 0 with h p being the flow on a path p ∈ P . Th e flow b e tween each OD pair mu st satisfy the tra vel demand. W e denote the demand for the OD pair w ∈ W by d w ∈ R ≥ 0 and the set of feasible flows b y H := n h ∈ R |P | ≥ 0    X p ∈P w h p = d w for all w ∈ W o . Agents wh o choo se path p ∈ P experience a nonn egati ve uncertain co st den oted as C p : R |P | ≥ 0 × R m → R ≥ 0 , ( h, u ) 7→ C p ( h, u ) , where u ∈ R m models the uncertainty . Th at is, the cost on a g iv en p ath depends on the flo w on all paths and also on a rand om variable. T o be more p r ecise about the uncertainty , let (Ω , F , P ) be a prob ability space and u b e a random vector m a pping into ( R m , B σ ( R m )) , where B σ ( R m ) is the Borel σ -algeb ra on R m . L et P and U ⊂ R m be the distribution and sup port of u , respectively . W e a ssum e that U is comp act. For the cost fun ction, we assume that fo r every p ∈ P and u ∈ U , the fu nction h 7→ C p ( h, u ) is contin u ous. In add itio n, for every p ∈ P and h ∈ H , the fu nction u 7→ C p ( h, u ) is measurab le with respe ct to B σ ( R m ) and for a fixed h ∈ H , either E P [ C p ( h, u )] + or E P [ C p ( h, u )] − is finite. Here, E P [ · ] denotes the expectation under P an d [ · ] + and [ · ] − denote the positi ve and negati ve p arts, respectively . Additional assumption s o n the cost functions will be made wherever necessary . Collecting the above described elem ents, a r outing game with uncertain costs is define d b y th e tuple ( G , W , P , C, d, U , P ) . Note that since the cost is u ncertain, one needs to assign an ap propr iate o bjective for ag e nts which in turn defines a n otion o f equilibrium . In this work, we assume that ag ents are risk-averse and look fo r path s with least conditional value-at-risk (cf. Section II). W e assume that all ag ents have the same risk- av ersion chare cterized b y the par ameter α ∈ (0 , 1) . T his assumption eases n otational burden and our r esults do h old for th e ge n eral case with heteroge n eous risk- av ersion. Using the fo rm (2), th e CV aR associated to path p as a f u nction of the flow is CV aR α [ C p ( h, u )] = inf t ∈ R n t + 1 α E P [ C p ( h, u ) − t ] + o . (3) The notion of equilibrium th en is that of W ardr op [1], where the cost associated to a path is its CV aR . Definition III.1. (Condition al v alue-at-r isk b ased W ardr op equilibriu m (CWE)): A flow vector h ∗ ∈ R |P | ≥ 0 is called a CV aR -based W ardr op equilibrium (CWE) fo r the ro uting game with un certain costs ( G , W , P , C, d, U , P ) if: ( i) h ∗ satisfies the d emand for all OD pairs and (ii) fo r any OD pair w , a path p ∈ P w has non zero flow if the CV aR of path p is m inimum amon g all paths in P w . Formally , h ∗ is a CWE if h ∗ ∈ H and h ∗ p > 0 for p ∈ P w only if CV aR α [ C p ( h ∗ , u )] ≤ CV aR α [ C q ( h ∗ , u )] , ∀ q ∈ P w . (4) W e denote th e set of CWE by S CWE ⊂ H . • One can verify that the set S CWE is equ iv alent to the set o f solutions to the variational inequality (VI) p r oblem VI( F , H ) (see Section II for relevant no tions) [23], where F p ( h ) := CV aR α [ C p ( h, u )] , for all p ∈ P . Note that the set H is compa c t a nd con vex. Further, the map h 7→ F ( h ) is co ntinuou s since C p , p ∈ P are so [ 2 4, Theorem 2]. Theref o re, the set of solutions SOL( F, H ) is n onempty and compact [25, Corollar y 2.2 .5]. Consequently , the set S CWE is nonemp ty and compact. The set of CWE pre d ict flow patterns when costs ar e un - certain an d agen ts behave in a risk-averse w ay , in particu lar , they minimize CV aR . T o compu te this set, one requires to know th e prob ability distribution P of the un certainty alo ng with the cost function als and the fixed deman d . I n real- life, P is unkn own and the decision- maker h a s only access to samples o f the uncer tainty . Th e objective of this paper is to p rovide a method to ap p roximate the set of CWE using av ailable samples. T o this end , we define the sample av erage ba sed (deter ministic) appro ximate VI pro blem that acts as a surrog ate to the VI p roblem defining the CWE. W e will then study statistical pro perties, tha t is, co nsistency and exponential convergence, of th e solutio ns of this app roximate VI pro blem. Note that solving the deter m inistic approximate VI p r oblem efficiently is a valid research q uestion o n its own and is not con sidered in the scope of this pap er . I V . S A M P L E A V E R A G E A P P ROX I M AT I O N O F C W E The app roach in the sample av erage fr amew ork is to replace the expectatio n o perator in any prob lem with the av erage over the obtained samp les [19]. This is one of the main Monte Carlo me th ods for p roblems with expectatio ns; see [26] f or a detailed survey of o ther sample-based tech - niques. In our setup, for each pa th of the network, we will r eplace the expectation opera tor in the definition of the CV aR of each path (3) with the sample average. The thus formed set of function s result into a VI pr oblem that approx imates VI ( F, H ) . Let b U N := { b u 1 , b u 2 , . . . , b u N } be the set of N ∈ N inde - penden t an d identically distributed samples o f th e uncerta in ty u drawn f rom P . Then, th e samp le a verage approxima tio n o f the CV aR associated to path p ∈ P is \ CV aR N α [ C p ( h, u )] := inf t ∈ R n t + 1 N α N X i =1 [ C p ( h, b u i ) − t ] + o . (5) The above expression is also known as the emp irical estimate of the CV aR , o r empiric a l CV aR in short. Note tha t the operator \ CV aR N α is random as it dep e nds on the realizatio n b U N of the unc ertainty . Different set o f samples form different set of functionals. T o emp hasize this d ependen cy on the uncertainty , we represent with b · N entities that are random , depend ent on the obta in ed samples. Using (5) as the ap- proxim a te cost, define the (sample-dep endent) approximate variational inequality prob lem as VI( b F N , H ) , where b F N p ( h ) := \ CV aR N α [ C p ( h, u )] , for all p ∈ P . W e den ote the set of solu tions of VI( b F N , H ) by b S N CWE ⊂ H . This serves as a remin der that it approx imates S CWE . The notion of appr o ximation is mad e p recise n ext. Definition IV .1 . (Asymptotic co nsistency an d exponen tial conv ergence): The set b S N CWE is an asympto tica lly consistent approx imation of S CWE , or in short, b S N CWE is asymp totically consistent, if a ny sequence of solution s { b h N ∈ b S N CWE } ∞ N =1 has almost surely (a. s.) all accum ulation poin ts in S CWE . The set b S N CWE is said to converge exponentially to S CWE if f o r a ny ǫ > 0 , the r e exist positive constants c ǫ and δ ǫ such th at fo r any sequ ence { b h N ∈ b S N CWE } ∞ N =1 , the following h olds P N  dist( b h N , S CWE ) ≤ ǫ  ≥ 1 − c ǫ e − δ ǫ N (6) for all N ∈ N . • The asymptotic consistency of b S N CWE is eq uiv alent to saying D ( b S N CWE , S CWE ) → 0 a.s. as N → ∞ . The expression (6) giv es a precise rate fo r this convergence. I n our work, all conv ergence results are for N → ∞ and so we d rop restating this fact for convenience’ s sake. I n th e following sections, we will e stablish the asympto tic consistency a nd the exponential conv ergence o f b S N CWE under suitable assumptions. Remark IV .2. (Existing sample a verage ap proxim a tio ns to CV aR and stoch astic VI ) : The works [ 27] and [28] study stochastic optim ization p r oblems w h ere CV aR is either b eing minimized or used to de fine the constrain ts. Both e mploy the sample a verage appro ximation a s proposed in ( 5) and study asympto tic consistency and exponential conv ergence of the Karush-Kuhn- T ucker (KKT) points. Since CV aR is used to define a VI pr oblem in our case, the an a lysis does not follow directly fr om these existing results. M oreover , the expo nential boun ds derived here are explicit, without in volving am biguou s co nstants, than the gener al large de- viation b ound s provided in [27] an d [28]. In ano ther data- based appro ach [29], the CV aR is p erceived as the expected shortfall ( 1) and d esirable statistical gua rantees are obtained for the optimizers of its samp le average. • A. Asymptotic con sistency of b S N CWE W e begin with stating the bo und on the optimizers of the problem de fining the CV a R (3) and the empirical CV aR (5). This restricts our attention to compact domain s for variables ( h, t, u ) , a pro p erty useful in showing con sistency . Denote for each p ∈ P , f unctions ψ p ( h, t ) := t + 1 α E P [ C p ( h, u ) − t ] + , (7a) b ψ N p ( h, t ) := t + 1 N α N X i =1 [ C p ( h, b u i ) − t ] + . (7b) The map b ψ N p is th e samp le average of ψ p . Giv en o u r assumption that the exp e cted value of the cost C p is bou nded for any h ∈ H , o ne can deduce by stron g law of large number s [30] that for any fixed ( h, t ) ∈ H × R , a lm ost surely , b ψ N p ( h, t ) → ψ p ( h, t ) . W e howev er req uire uni- form conv ergence of these maps to conclude consistency , which will be established in T heorem IV .6 below . Observe that, by d e fin ition, CV aR α [ C p ( h, u )] = inf t ∈ R ψ p ( h, t ) and \ CV aR N α [ C p ( h, u )] = inf t ∈ R b ψ N p ( h, t ) . The f ollowing result giv es explicit bo unds o n th e op timizers of these prob lems. Lemma IV .3. (Bounds on optimizers of problems definin g (empirical) CV aR ): F or an y h ∈ H and p ∈ P , the optimizers of th e pr oblems in (3) a nd (5) exist and belo ng to the compac t set T = [ m, M ] , wher e m := min { C p ( h, u ) | h ∈ H , u ∈ U , p ∈ P } , M := ma x { C p ( h, u ) | h ∈ H , u ∈ U , p ∈ P } . Furthermore , th e set of functio ns φ p ( h, t, u ) := t + 1 α [ C p ( h, u ) − t ] + , (8) for p ∈ P , satisfy for a ll ( h, t, u ) ∈ H × T × U , φ p ( h, t, u ) ∈ h m, m + M − m α i . (9) Pr oof. From [19, Chapter 6] , optimizers of these prob lems exist and they lie in the closed interval defined by the left- and the right-side (1 − α ) -qu antile (cf. Section II) of the respective ran d om variables. Since this interval belong s to the set of values the functio ns take, we conclud e that the optimizers belon g to T . T o co nclude (9), n ote that φ p ( h, t, u ) = t + 1 α [ C p ( h, u ) − t ] + ≤ t + 1 α [ M − t ] + = t + 1 α ( M − t ) = (1 − 1 α ) t + 1 α M ≤ (1 − 1 α ) m + 1 α M . Here, the first ineq u ality follows from the bou nd on C p , the first equ a lity is because t ∈ [ m, M ] , a nd the second inequality is due to the fact that α < 1 . Similarly , for the lower boun d, φ p ( h, t, u ) ≥ t + 1 α [ m − t ] + = t ≥ m. This comp le tes the pro o f. W e make a note here that optimizers of prob lems defining the CV a R in (3) and (5) exist and are bou nded for more general cases, e ven when the support of the uncertainty is u nbou nded, see e.g. , [19, Chapter 6]. Nevertheless, the above result provides an explicit b ound which is used later in deriving precise expo n ential conv ergence gu arantees. As a co nsequen c e o f L e mma IV .3, on e can show unifor m conv ergence of b ψ N p to ψ p . Ou r next step is to an alyze the sensiti vity o f F as one perturbs th e under lying map ψ . In combinatio n with the u niform convergence of b ψ N p , this result leads to the unif orm conver gence of b F N to F . Lemma IV .4. (Sensitivity o f F with respe c t to ψ ): F or an y ǫ > 0 , if sup p ∈P , ( h,t ) ∈H×T | b ψ N p ( h, t ) − ψ p ( h, t ) | ≤ ǫ , wher e T is d efined in Lemma IV .3, then sup h ∈H k b F N ( h ) − F ( h ) k ≤ p |P | ǫ. Pr oof. The first step is to sh ow the sensitivity of the map CV aR α [ C p ( · , u )] with respect to ψ p . T o this en d, fix p ∈ P and h ∈ H , and let b t N p ( h ) ∈ ar gmin t ∈ R b ψ N p ( h, t ) a nd t p ( h ) ∈ ar gmin t ∈ R ψ p ( h, t ) . These optimizers exist due to Lemma IV .3. W e now have ψ p  h, t p ( h )  − ǫ ≤ ψ p  h, b t N p ( h )  − ǫ ≤ b ψ N p  h, b t N p ( h )  . The first inequality is due to optimality and the secon d inequality holds b y assumption. Similarly , o ne can sho w that b ψ N p  h, b t N p ( h )  − ǫ ≤ ψ p  h, t p ( h )  . The above two sets of inequa lities along with the fact that \ CV aR N α [ C p ( h, u )] = b ψ N p  h, b t N p ( h )  and CV aR α [ C p ( h, u )] = ψ p  h, t p ( h )  lead to the conclusion sup h ∈H    \ CV aR N α [ C p ( h, u )] − CV aR α [ C p ( h, u )]    ≤ ǫ. (10) Finally , the conclu sio n follows from the inequality k b F N ( h ) − F ( h ) k ≤ p |P | sup p ∈P | b F N p ( h ) − F p ( h ) | . The final prelim inary result states prox imity of b S N CWE to S CWE giv en that the d ifference betwe en b F N and F is b ounde d. The proo f is a con sequence of [20, Lem ma 2.1] that studies sensiti vity of g eneralized equ ations and th e ir solution sets. Lemma IV .5. (Sensitivity of S CWE with respect to F ): F or any ǫ > 0 , th er e exists δ ( ǫ ) > 0 such that D ( b S N CWE , S CWE ) ≤ ǫ whenever sup h ∈H k b F N ( h ) − F ( h ) k ≤ δ ( ǫ ) . Next is the main result of this sectio n, establishing th e asymptotic co nsistency of b S N CWE . Th e proo f puts to use the preliminar y lemmas o n sensitivity presented ab ove along with the unifo r m convergence of b ψ N p to ψ p . Theorem IV .6 . (Asymptotic co nsistency of b S N CWE ): W e have D ( b S N CWE , S CWE ) → 0 almo st su r ely . Pr oof. Consider first the a.s. u niform con vergence b ψ N p → ψ p over the comp act set H × T . No te that ψ p ( h, t ) = E P [ φ p ( h, t, u )] where φ p is given in (8) and so , b ψ N p is the samp le average of ψ p . For any fixed u ∈ U , the map φ p ( · , · , u ) is continuou s and for any ( h, t ) ∈ H × T , due to Lem ma IV .3, the map φ p ( h, t, · ) is dominate d b y the integrable fun ction (a constant in this case) m + M − m α . Hence, by the un iform law of large n umbers r e sult [19, Theorem 7.48 ], we co nclude that b ψ N p → ψ p unifor m ly a.s. o n H × T . Usin g this fact in the sensiti vity result of Lemma IV .4 implies that b F N → F unifor mly a.s. on the set H . Finally , we arrive at the conclusion u sin g Lemma IV .5. B. Expo nential conver gence o f b S N CWE In this section , our strategy will be to u se the conc entration inequality for the e mpirical CV aR gi ven in [31] and deri ve the unifor m expone ntial con vergence o f b F N to F . Later, we will use Lemm a IV .5 to infer expo nential conver gence of b S N CWE . Note that the inequality giv en in [31] requires compact support of th e ra n dom variable and it is tight whe n it com es to the depe n dency on the risk parameter α . For unboun ded support, one can use deviation in e qualities f rom [ 3 2]. For a fixed p ∈ P and h ∈ H , the deviation betwe e n the CV aR an d its empir ic a l counterpa r t can be bound ed using the results in [31] as P N     \ CV aR N α [ C p ( h, u )] − CV aR α [ C p ( h, u )]    ≥ ǫ  ≤ 6 exp  − αǫ 2 11( M − m ) 2 N  . (11) In the above bound, the denominato r in the expone n t uses the fact that for any path and flow vector, the cost seen as a random variable is su pported on the compact set [ m, M ] . Similar to the narra ti ve of th e p r evious section, while th e above inequality holds p ointwise, what we need is uniform exponential bo u nd f o r proxim ity of F to b F N . In th e seq u el, we will derive such a bou n d un der the follo wing condition . Assumption IV .7. (Lipschitz con tin uity o f C p ): There exists a constant L > 0 such that | C p ( h, u ) − C p ( h ′ , u ) | ≤ L k h − h ′ k , (12) for all h, h ′ ∈ H , u ∈ U , an d p ∈ P . • Under th e above Lipsch itz con dition on the cost functions, one can show the fo llowing. Lemma IV .8. (Lipschitz continuity of (empirical) CV aR ): Under Assump tion IV .7, for any path p ∈ P , the function s h 7→ \ CV aR N α [ C p ( h, u )] an d h 7→ CV aR α [ C p ( h, u )] ar e Lipschitz over the set H with con stant L α . Pr oof. W e w ill show the pr operty for the fun ction h 7→ \ CV aR N α [ C p ( h, u )] . The reasoning for h 7→ CV aR α [ C p ( h, u )] follows analo gously . Conside r any h, h ′ ∈ H . Recall fr om (7 ) tha t    \ CV aR N α [ C p ( h, u )] − \ CV aR N α [ C p ( h ′ , u )]    =    inf t ∈ R b ψ N p ( h, t ) − inf t ∈ R b ψ N p ( h ′ , t )    . (13) Assumption IV .7 yields Lipschitz proper ty f or the m ap b ψ N p . T o establish this, fix any p ∈ P and t ∈ R and no tice th at    b ψ N p ( h, t ) − b ψ N p ( h ′ , t )    =    t + 1 N α N X i =1 [ C p ( h, b u i ) − t ] + −  t + 1 N α N X i =1 [ C p ( h ′ , b u i ) − t ] +     ≤ 1 N α N X i =1    [ C p ( h, b u i ) − t ] + − [ C p ( h ′ , b u i ) − t ] +    ≤ 1 N α N X i =1    C p ( h, b u i ) − C p ( h ′ , b u i )    ≤ L α k h − h ′ k . Above, the first relation is a consequen ce o f the trian gle inequality , the second inequality fo llows from the fact that the map [ · ] + is Lipschitz with constant as un ity , and the last in equality u ses L ipschitz p roper ty o f the costs. Now let ¯ t, ¯ t ′ ∈ R b e such th at b ψ N p ( h, ¯ t ) = inf t ∈ R b ψ N p ( h, t ) and b ψ N p ( h ′ , ¯ t ′ ) = inf t ∈ R b ψ N p ( h ′ , t ) . Ex isten ce of such an optimizer follows from the discussion in [1 9, Chapte r 6]. Now note the following sequ ence of ine q ualities that can be inferred f r om the optimality condition and the Lipschitz proper ty o f b ψ N p shown ab ove, inf t ∈ R b ψ N p ( h, t ) = b ψ N p ( h, ¯ t ) ≤ b ψ N p ( h, ¯ t ′ ) ≤ b ψ N p ( h ′ , ¯ t ′ ) + L α k h − h ′ k = inf t ∈ R b ψ N p ( h ′ , t ) + L α k h − h ′ k . (14) One can exchange h with h ′ in the above reasoning and obtain inf t ∈ R b ψ N p ( h ′ , t ) ≤ inf t ∈ R b ψ N p ( h, t ) + L α k h − h ′ k . (15) Inequa lities ( 14) and (15) imply that    inf t ∈ R b ψ N p ( h, t ) − inf t ∈ R b ψ N p ( h ′ , t )    ≤ L α k h − h ′ k . The proof conclu des by using this fact in (13). Next we state exp o nential co n vergence of b F N . The p roof is largely inspired from the steps g iv en in [22, T heorem 5.1] and is a stand ard argument in these set of results. W e note that the obtaine d b o und is very crude and in practice, the achieved pe r forman ce is m u ch better . Proposition IV .9. (Uniform expon e n tial conv ergence of b F N to F ): Under Assumptio n IV .7, for a ny ǫ > 0 , the follo win g inequality holds for all N ∈ N , P N  sup h ∈H k b F N ( h ) − F ( h ) k > ǫ  ≤ γ ( ǫ ) e − β ( ǫ ) N , wher e γ ( ǫ ) := 3 |P |⌈|P | / 2 ⌉ ! π |P | / 2  12 L diam( H ) ǫα  |P | (16a) β ( ǫ ) := αǫ 2 44 |P | ( M − m ) 2 (16b) Her e, dia m( H ) = sup h,h ′ ∈H k h − h ′ k is the diame te r of H . Pr oof. The idea of m oving from the po intwise exponential bound (11) to a u niform bou nd is to impo se the poin twise bound jointly on a finite nu mber of points and use the Lipschitz property (L emma IV .8) to bound the deviation of the rest o f the set from this finite set. Making precise the mathematical details, no te th a t on e can cover th e set H with K := ⌈|P | / 2 ⌉ ! 2 π |P | / 2  12 L diam ( H ) ǫα  |P | number of points, lab eled C := { ˜ h 1 , . . . , ˜ h K } , su c h that fo r any h ∈ H , there exists a p oint ˜ h i ( h ) ∈ C with L α k h − ˜ h i ( h ) k ≤ ǫ 4 . (17) The existence of such a set of po in ts is discussed fur ther in Remark IV .10 be low and it relates to the cov ering numb e rs of sets. Co mbining the Lipsch itz boun d given in Lemma IV .8 and the inequality (17), we g et fo r all p ∈ P and h ∈ H ,    \ CV aR N α [ C p ( h, u )] − \ CV aR N α [ C p ( ˜ h i ( h ) , u )]    ≤ ǫ 4 , (18a)    CV aR α [ C p ( h, u )] − CV aR α [ C p ( ˜ h i ( h ) , u )]    ≤ ǫ 4 . (18b ) The ab ove in e qualities con trol th e deviation of \ CV aR N α [ C p ( · , u )] an d CV aR α [ C p ( · , u )] over the set H from th e values these function s take on the set C . The next step entails bound ing the deviation o f the CV aR and the emp irical CV aR on the set C . Em ploying (11) and the union bound , we have P N  sup p ∈P ,h ∈C    \ CV aR N α [ C p ( h, u )] − CV aR α [ C p ( h, u )]    ≥ ǫ 2  ≤ X p ∈P X h ∈C P N     \ CV aR N α [ C p ( h, u )] − CV aR α [ C p ( h, u )]    ≥ ǫ 2  ≤ 6 |P | K exp  − αǫ 2 44( M − m ) 2 N  . (19) The next set of inequalities characterize the difference be- tween the CV a R and the empirical CV aR over the set H using the Lip schitz prop erty (18). Fix p ∈ P and let h ∈ H . Note th at using (18), | \ CV aR N α [ C p ( h, u )] − CV aR α [ C p ( h, u )] | ≤ | \ CV aR N α [ C p ( h, u )] − \ CV aR N α [ C p ( ˜ h i ( h ) , u )] | + | \ CV aR N α [ C p ( ˜ h i ( h ) , u )] − CV aR α [ C p ( ˜ h i ( h ) , u )] | + | CV aR α [ C p ( ˜ h i ( h ) , u )] − CV aR α [ C p ( h, u )] | ≤ ǫ 2 + | \ CV aR N α [ C p ( ˜ h i ( h ) , u )] − CV aR α [ C p ( ˜ h i ( h ) , u )] | . Next, the deviation between the CV aR and its empirical counterp art is bound ed using (1 9) and th e above ch aracteri- zation as P N  sup p ∈P ,h ∈H    \ CV aR N α [ C p ( h, u )] − CV aR α [ C p ( h, u )]    ≥ ǫ  ≤ P N  sup p ∈P ,h ∈C    \ CV aR N α [ C p ( h, t )] − CV aR α [ C p ( h, u )]    ≥ ǫ 2  ≤ 6 |P | K exp  − αǫ 2 44( M − m ) 2 N  (20) The final step is to co nnect the above inequ ality to the dif- ference between b F N and F . Fro m the proof of L emma IV .4, one can deduce that if sup h ∈H k b F N ( h ) − F ( h ) k > ǫ , then sup p ∈P ,h ∈H    \ CV aR N α [ C p ( h, u )] − CV aR α [ C p ( h, u )]    > ǫ p |P | . Therefo re, using (20) we obtain P N ( sup h ∈H k b F N ( h ) − F ( h ) k > ǫ ) ≤ P N  sup p ∈P ,h ∈H    \ CV aR N α [ C p ( h, t )] − CV a R α [ C p ( h, u )]    > ǫ p |P |  ≤ 6 |P | K exp  − αǫ 2 44 |P | ( M − m ) 2 N  . This conclu des the proo f. Remark IV .10. (A suitable cover for the set H ): Here we compute the number of points K , that denotes the cardinality of some set { ˜ h 1 , . . . , ˜ h K } ⊂ H , required to cover the set H accordin g to the cond itions in the p roof of Propo sition IV .9. In particular, for all h ∈ H , there exists a point ˜ h i ( h ) , i ( h ) ∈ [ K ] such tha t L α k h − ˜ h i ( h ) k ≤ ǫ 4 . That is, k h − ˜ h i ( h ) k ≤ ǫα 4 L . Fro m [33], this is possible with  3 ( ǫα/ 4 L )  |P | vol( H ) vol( B ) number of points, wh e r e v ol( H ) is the volume of the set H and vol( B ) is the volume of the u nit norm ball in R |P | . Since vol( H ) ≤ diam( H ) |P | and vol ( B ) ≥ 2 π |P | / 2 ⌈|P | / 2 ⌉ ! , we get the desired up per estimate o n K . • The main result is given below . Th e proof with mino r modification s is as given in [20, Theo rem 2.1 ]. I t f ollows from th e unifo rm expon ential conv ergence of b F N p . Theorem IV .11. (Expo n ential conv ergence of b S N CWE to S CWE ): Let A ssumption IV .7 hold. Then , for any sequence { b h N ∈ b S N CWE } ∞ N =1 , ǫ > 0 , and N ∈ N , the following inequality holds P N  dist( b h N , S CWE ) ≤ ǫ  ≥ 1 − γ ( δ ( ǫ )) e − β ( δ ( ǫ )) N , wher e γ and β a r e given in (16) and δ : R > 0 → R > 0 is a map such that the pair ( ǫ, δ ( ǫ )) satisfies th e condition of Lemma I V .5. Pr oof. Consider any ǫ > 0 . By Lemma IV .5, if sup h ∈H k b F N ( h ) − F ( h ) k ≤ δ ( ǫ ) , then dist( b h N , S CWE ) ≤ ǫ . From Propo sition IV .9, for any δ ( ǫ ) > 0 , there exist γ ( δ ( ǫ )) and β ( δ ( ǫ )) , gi ven in (1 6a) and (16 b), respecti vely , such that P N  sup h ∈H k b F N ( h ) − F ( h ) k > δ ( ǫ )  ≤ γ ( δ ( ǫ )) e − β ( δ ( ǫ )) N for all N . Th e proo f follows by using the above facts and the following set of ineq ualities P N (dist( b h N , S CWE ) ≤ ǫ ) ≥ P N  sup h ∈H k b F N ( h ) − F ( h ) k ≤ δ ( ǫ )  = 1 − P N  sup h ∈H k b F N ( h ) − F ( h ) k > δ ( ǫ )  . Remark IV .12. (Sample guaran tees for appr oximating S CWE with b S N CWE ): Theorem IV .11 implies that if o ne wants dist( b S N CWE , S CWE ) ≤ ǫ w ith co nfidence 1 − ζ , wh ere ζ ∈ (0 , 1) is a small positive number, then on e would req uire at m o st N ( ζ , ǫ ) = 1 β ( δ ( ǫ )) log  γ ( δ ( ǫ )) ζ  = 44 |P | ( M − m ) 2 αδ ( ǫ ) 2  log  3 |P |⌈|P | / 2 ⌉ ! π |P | / 2 ζ  + |P | log  12 L diam ( H ) δ ( ǫ ) α  number o f samples of th e un certainty . Due to the expone n tial rate, a good f eature of th is sample g uarantee is that N depend s on the accur acy ζ logarithm ically . That is, one can obtain h igh confiden ce bou n ds with fewer samples. Howe ver , the sample size g rows poorly with many other para m eters, especially , the accu racy of the estimate ǫ an d th e number of paths. Furth er , no te that to obtain an accurate sample guaran tee , one needs to estimate δ ( · ) w h ich d epends o n the regularity of the cost f unctions. I mproving the sample complexity f o r specific cost functions such as, piece wise affine, is part of o ur futu re work. • V . S I M U L A T I O N Here we illustrate th e method of sample average ap proxi- mation fo r th e com putation of the CWE through an e xamp le. W e consider a simple n etwork with two nod es V = { A, B } and fiv e edg es. The set o f OD-pairs is { ( A, B ) , ( B , A ) } . Three edges { 1 , 2 , 3 } go from A to B and two { 4 , 5 } go from B to A . T he set of ed ges form the available p aths. The network a nd cost fun ctions ar e adapted from [11, Sectio n 6.3]. The demand is 260 from A to B , and is 170 fro m B to A . The vector o f c o st function s is given by C ( h ; u ) =       40 h 1 + 20 h 4 + 1000 + 3000 u 1 60 h 2 + 20 h 5 + 950 80 h 3 + 3000 8 h 1 + 80 h 4 + 1000 + 40 00 u 2 4 h 2 + 100 h 5 + 130 0       . The unc e rtainty u = ( u 1 , u 2 ) appe a rs in an affine manne r in the cost associated to edges { 1 , 4 } . The supp ort and distribution of both random variables is [0 , 1] and un iform, respectively , and th ey are ind ependen t of each other . W e set α = 0 . 2 . This defin es completely th e routing game with uncertain costs. Since the uncertain ty is additive in the costs, one can compu te the CV aR of co sts as    CV aR α [ C 1 ( h, u )] . . . CV aR α [ C 5 ( h, u )]    =       40 h 1 + 20 h 4 60 h 2 + 20 h 5 80 h 3 8 h 1 + 80 h 4 4 h 2 + 100 h 5       +       1000 + 3 0 00 CV aR α [ u 1 ] 950 3000 1000 + 4 0 00 CV aR α [ u 2 ] 1300       . The ob tained cost fu nctions are affine in the flows and so, th e CWE is the solution of a linear com plementar- ity pr oblem (LCP) [11 ]. Solving th e LCP , which in th is case is a con vex optimization problem with q uadratic cost and affine constra int, y ields the uniqu e CWE as h ∗ = (89 . 52 , 98 . 39 , 72 . 09 , 74 . 32 , 95 . 68) . For the samp le average a p proxim ation, we con sider three scena r ios with d ifferent numb e r of samples, N ∈ { 50 , 500 , 5 000 } . For each of th ese scenar ios, we consider 500 run s. Each run collects N n u mber of i.i.d samples of the uncertain ty u , co nstructs the empirical CV aR costs, and com putes the ap proxim ation of the CWE b h N . Figu re 1 illustrates ou r results. It plots the cum ulativ e distribution function o f the ra n dom variable k b h N − h ∗ k as estimated using the 50 0 runs. Note that the complete distribution moves to the left with increa sin g number of samples. Th is confirms our theor etical findings that as N increases, the app roximate solution b h N approa c h es the CWE almost surely . V I . C O N C L U S I O N S W e hav e considered a no n atomic ro uting g ame with uncer- tain costs and defined th e W ar drop equilibrium where agents opt fo r paths with least c o nditiona l value-at-risk. Giv en i.i.d samples of th e uncer tainty , we h av e in vestigated the statistical pro perties of the sample average appro ximation of the CV aR -based W ardr o p equilib rium. I n p articular, we have established the asympto tic co nsistency and the expo nential conv ergence o f the approximatio n sche me under su itable regularity cond itions on the cost functio n s. Future work will in volve exploring mo n otonicity of the determ inistic VI prob- lem formed using samp le averages and designin g efficient algorithm s for solving it. W e also wish to investigate o ther 0 2 4 6 8 10 0 0.2 0.4 0.6 0.8 1 N=50 N=500 N=5000 Fig. 1: Plot demonstrate s the con vergenc e of the approximate solution b h N to the CV aR-based W ardrop equilibri um h ∗ for the two-nod e fi ve-edge exa mple, see Sect ion V for details. Each line corresponds to a dif fere nt sample size and depicts the cumulat iv e distributi on of k b h N − h ∗ k as obtaine d using 500 runs. The lines mo ve tow ards the origin as the number of samples increase depicti ng the con ve rgence of b h N to h ∗ . data-driven app roaches, such as stochastic ap proxim ation routines, for com puting the equilibrium. Fin a lly , we p lan to characterize the price of risk-aversion and the benefit, if any , of having he te r ogeneo us risk-averseness of agen ts. R E F E R E N C E S [1] J. G. W ard rop, “Some theoret ical aspects of road tra ffic research , ” Pr oceeding s of the Insti tution of Civil Engineer s , vol. 1, no. 3, pp. 325– 362, 1952. [2] B. Monnot, F . Benita, and G. Piliouras, “Routing games in the wild: Effic iency , equili bration and regret, ” in International Confere nce on W eb and Internet Economics , pp. 340–353, 2017. [3] K. Lam, W . Krichene, and A. Bayen , “On learning how players learn: E stimation of learning dynamics in the routing game , ” in 2016 ACM /IEEE 7th International Confer ence on Cyber-Physical Systems (ICCPS) , pp. 1–10, April 2016. [4] J. R. Correa and N. E. Stier-Moses, “W ardro p equi libria, ” Encyclope- dia of Operations Resear ch and Manag ement Science . W ile y , 2011. [5] P . N. Brown and J. R. Marden, “Studies on rob ust social influence mechanisms, ” IEEE Contr ol Sy stems , pp. 98–115, 2017. [6] M. Florian, “Untangling traffic congest ion: Application of netw ork equili brium models in transportati on planning , ” OR/MS T oda y , pp. 52– 57, 199 9. [7] U. V . Shanbhag, “Stochastic varia tional inequality problems: Applica- tions, analysis, and algorithms, ” TUTORIALS in Operat ions Researc h , pp. 71– 107, 2013. [8] C. Zhang, X. Chen, and A. Sumalee, “Rob ust wardrop’ s user equi- librium assignment under stochast ic demand and supply: exp ected residual m inimiza tion approa ch, ” T r ansportation Researc h P art B: Methodo logica l , vol. 45, no. 3, pp. 534–552, 2011. [9] X. Chen, R. J.-B. W ets, and Y . Zhang, “Stochastic v ariation al in- equali ties: residual minimizati on smoothing sample averag e approxi- mations, ” SIAM Journal on Optimizati on , vo l. 22, no. 2, pp. 649–673, 2012. [10] F . Ordóñez and N. E. Stier -Moses, “W ar drop equilibria with risk- av erse users, ” T ransport ation Science , v ol. 44, no. 1, pp. 63–86, 2010. [11] Y . Xie and U. V . Shanbhag, “On robust solutions to uncert ain lin- ear complementarit y probl ems and their v ariants, ” SIAM Journal on Optimizati on , vol. 26, no. 4, pp. 2120–2159, 2016. [12] J. Dietrich, A. R. Hota, and A. Cherukuri , “Data-dri ve n re gret m in- imizati on in routing games under uncertai nty , ” in E ur opean Contr ol Confer ence , (Naples, Ital y), pp. 1702–1707 , June 2019. [13] E. Nikol ov a and N. E. Stier -Moses, “ A m ean-risk model for the traf fic assignment problem with stoc hastic trav el times, ” Operations Resear ch , v ol. 62, no. 2, pp. 366–382, 2014. [14] A. A. Prakash, R. Seshadri, and K. K. Sriniv asan , “ A consistent reliab ility-ba sed user -equilib rium problem with risk-av erse users and endogeno us tra vel time correlat ions: Formul ation and solution algo- rithm, ” T r ansportation Researc h P art B , vol. 114, pp. 171–198, 2018. [15] E. N ikolov a and N. E. Stier-Mose s, “The burden of risk av ersion in mean-risk selfish routing, ” in ACM Confere nce on E conomics and Computati on , pp. 489–506, 2015. [16] T . Lianeas, E. Nikolo va , and N. E . Stier-Mose s, “Risk-av erse selfish routing, ” Mathematics of Oper ations Resear ch , 2018. T o appear . [17] A. Chen and Z. Zhou, “The α -reliable mean-exce ss traffic equilib rium model with stochasti c trave l times, ” T ranspo rtation Researc h P art B , vol. 44, pp. 493–5 13, 2010. [18] X. Xu, A. Che n, L. Cheng, and C. Y ang, “ A link-ba sed mean-exc ess traf fic equilibrium m odel under uncer tainty , ” T ra nsportation Researc h P art B , vol . 95, pp. 53–75, 2017. [19] A. Shapiro, D. Dentche va , and A. Ruszczy ´ nski, Lectur es on stochast ic pr ogr amming . Philade lphia, P A: SIAM, 2014. [20] H. Xu, “Sample average approximation method s for a class of stochasti c varia tional inequali ty problems, ” Asia-P ac ific Journal of Operat ional Researc h , vol . 27, no. 1, pp. 103– 119, 2010. [21] H. Xu, “Uniform exponent ial con ver gence of sample a verage random functio ns under general sampling with application s in stochastic programming, ” Jou rnal of Mathematical Analysi s and Appli cations , vol. 368, pp. 692– 710, 2010. [22] A. Shapiro and H. Xu, “Stoc hastic mathema tical progra ms wit h equi - librium constraints, modelli ng and sample avera ge approximati on, ” Optimizati on , vol. 57, no. 3, pp. 395–418, 2008. [23] M. J. S m ith, “The existen ce, uniq ueness and stability of traf fic equili bria, ” T ransport ation Resear ch P art B: Methodolo gical , vol. 13, no. 4, pp. 295–304, 1979. [24] R. T . Rockafella r , “Coherent approaches to risk in optimizatio n under uncerta inty , ” in OR T ools and Applications: Glimpses of Future T echnol ogies , pp. 38–61, INFORMS, 2007. [25] F . Facchine i and J.-S. Pang, F inite -dimensional variatio nal inequali ties and compl ementarity pro blems, vol.1 . Springer-V erlag, 2003. [26] T . H. de Mell o and G. Bayraksan, “Mont e carlo s ampling-based methods for stochastic optimizatio n, ” Surve ys in Operations Researc h and Mana geme nt Science , vol. 19, no. 1, pp. 56–85 , 2014. [27] F . W . Meng , J. Sun, and M. Goh, “Stochast ic optimizatio n problems with cvar risk measure and thei r sample ave rage approximatio n, ” J ournal of O ptimization Theory and Application s , vol. 146, no. 2, pp. 399 –418, 2010. [28] H. Sun, H. Xu, and Y . W ang, “ As ym ptotic analysis of sample av erage approximat ion for s tochastic optimiz ation problems with joi nt chanc e constrai nts via condit ional value at risk and differe nce of con ve x functio ns, ” J ournal of Optimization Theory and Applicat ions , vol. 161, no. 1, pp. 257–284, 2014. [29] F . A. Ra mponi and M. C. Campi, “Expe cted shortf all: Heurist ics and certific ates, ” Eur opean Journal of Operat ional Researc h , vol. 267, pp. 100 3–1013, 2018. [30] R. Durrett, Pr obabil ity: theory and examp les . Ne w Y ork, US A : Cambridge Uni versi ty Press, 2010. [31] Y . W ang and F . Gao, “De viation inequaliti es for an estimato r of the conditi onal value -at-risk, ” Operat ions Researc h Letters , vol. 38, no. 3, pp. 236 –239, 2010. [32] R. K. Kolla, L. A. Prashanth, S. P . Bhat, and K. Jagannathan, “Conce ntration bounds for empirical conditional value -at-risk: The unbounded case, ” Operati ons Researc h Letters , vol. 47, no. 1, pp. 16– 20, 201 9. [33] Y . W u, “Lecture notes on informationa l-theoret ic meth- ods in h igh-dimensiona l statisti cs, ” 2016. A vaila ble at: http://www.stat.yale.edu /~yw562/teachin g/598/lec14.pdf .