A New Generalization of Chebyshev Inequality for Random Vectors

A New Generalizatio n of Cheb yshev Inequalit y for Random V ectors ∗ Xinjia Chen June 2007 Abstract In this article, we der ive a new gener alization of Cheb yshev inequality for random vec- tors. W e demonstrate tha t the new g eneralizatio n is muc h less conserv ative than the classical generaliza tion. 1 Classical Generalization of Cheb yshev inequalit y The Chebyshev inequality d iscloses the fund amen tal relationship b et we en the mean an d v ariance of a random v ariable. Extensiv e researc h works ha ve b een dev oted to its generalizations for random vect ors . F or example, v arious generalizatio ns can b e foun d in Marshall and Olkin (1960), Go dwin (195 5), Mallo ws (1956) and the references therein. A natur al generalizatio n of Chebyshev inequalit y is as follo ws. F o r a rand om vect or X ∈ R n with cu m u lativ e distribution F ( . ), Pr {|| X − E [ X ] | | ≥ ε } ≤ V ar( X ) ε 2 ∀ ε > 0 (1) where || . || denotes the Euclidean norm of a v ector and V ar( X ) def = Z V ∈ R n || V − E [ X ] || 2 dF ( V ) This classical generalization can b e found in a num b er of te xtb o oks of probabilit y theory a nd statistics (se e, e.g., pp . 446-451 of Laha and R oh atgi (197 9)). ∗ The author is with Department of Electrical and Computer Engineeri n g, Louisi ana State Univ ersity , Baton Rouge, LA 70803; Email: c henxin jia@gmail. com. 1 2 New G eneralization of Cheb yshev inequalit y The classical generalization (1) p erfectly assem bles its counte r part for s calar random v ariables. Ho we ver, it m ay be too conserv ativ e. T o add ress the conserv atism, w e deriv e a n ew multiv ariate Chebyshev inequalit y as foll o ws. Theorem 1 F or any r andom ve ctor X ∈ R n with c ovarianc e matr ix Σ , Pr n ( X − E [ X ]) ⊤ Σ − 1 ( X − E [ X ]) ≥ ε o ≤ n ε , ∀ ε > 0 (2) wher e the sup erscript “ ⊤ ” denotes the tr ansp ose of a matrix. Pro of . Let D ε =  V ∈ R n : ( V − E [ X ]) ⊤ Σ − 1 ( V − E [ X ]) ≥ ε  . By the defi n ition of D ε , we ha ve 1 ε ( V − E [ X ]) ⊤ Σ − 1 ( V − E [ X ]) ≥ 1 , ∀ V ∈ D ε . Hence, Pr { X ∈ D ε } ≤ 1 ε Z V ∈ D ε ( V − E [ X ]) ⊤ Σ − 1 ( V − E [ X ]) dF ( V ) ≤ 1 ε Z V ∈ R n ( V − E [ X ]) ⊤ Σ − 1 ( V − E [ X ]) dF ( V ) . F o r i = 1 , · · · , n , let u i denote the i -th elemen t of V − E [ X ]. F or i = 1 , · · · , n and j = 1 , · · · , n , let σ ij denote the elemen t o f Σ in the i -th row and j -th column . Similarly , let ρ ij denote the elemen t of Σ − 1 in th e i -th ro w and j -th column. Then, ( V − E [ X ]) ⊤ Σ − 1 ( V − E [ X ]) = n X i =1 u i n X k =1 ρ ik u k ! = n X i =1 n X k =1 ρ ik u i u k . It follo w s that Z V ∈ R n ( V − E [ X ]) ⊤ Σ − 1 ( V − E [ X ]) dF ( V ) = Z V ∈ R n n X i =1 n X k =1 ρ ik u i u k ! dF ( V ) = n X i =1 n X k =1 ρ ik  Z V ∈ R n u i u k dF ( V )  . By the definition of the co v ariance matrix Σ and its symmetry , w e ha v e Z V ∈ R n u i u k dF ( V ) = σ ik = σ k i 2 for i = 1 , · · · , n and k = 1 , · · · , n . Hence, Z V ∈ R n ( V − E [ X ]) ⊤ Σ − 1 ( V − E [ X ]) dF ( V ) = n X i =1 n X k =1 ρ ik σ k i = tr(Σ − 1 Σ) = n where tr( . ) denotes the trace of a matrix. Th erefore, Pr { X ∈ D ε } ≥ 1 ε Z V ∈ R n  Σ − 1 ( V − E [ X ])( V − E [ X ]) ⊤  dF ( V ) = n ε . The proof is thus completed. ✷ Remark 1 The or em 1 indic ates a fundamental r elationship b etwe en the me an and c ovarianc e of a r andom ve ctor and describ es how a r andom ve ctor deviates fr om its exp e ctation. Sp e cial ly, for n = 1 , we have Σ = V ar( X ) and by The or em 1 , for any ǫ > 0 , Pr n ( X − E [ X ]) ⊤ Σ − 1 ( X − E [ X ]) > ǫ o = Pr n || X − E [ X ] || > p ǫ V a r ( X ) o ≤ 1 ǫ , fr om which we de duc e Pr {|| X − E [ X ] | | > ε } ≤ V ar( X ) ε 2 by letting ε = p ǫ V a r ( X ) . This shows that The or em 1 includes the wel l-k nown Chebyshev in- e qu ality as a sp e cial c ase. Remark 2 A ctual ly, we had establishe d The or em 1 in [1, pp . 8–9] in 1997 . The applic ations of this r esult to c ontr ol engine ering c an b e found in [1, 2]. R e c ently, The or em 1 has b e en extende d to r andom elements taking values in a sep ar ate Hilb ert sp ac e by R ao [7] and to r andom elements taking values in a sep ar ate Banach sp ac e by Zho u and Hu [8]. 3 Comparison with Classical Generalization In this s ection, we sh all sh ow that the inequalit y in Theorem 1 can b e muc h less conserv ative than th e cla ss ical ge neralized Cheb ysh ev in equ alit y (1). 3 Let δ ∈ (0 , 1). Based on in equalit y (1), s p here B δ def =  V ∈ R n : || V − E [ X ] | | 2 ≤ tr(Σ) δ  is the smallest set th at can b e constructed to ensure Pr { X ∈ B δ } > 1 − δ . O n the other hand, b y applying T h eorem 1 w e can co n struct an ellipsoid E δ def = n V ∈ R n : ( V − E [ X ]) ⊤ Σ − 1 ( V − E [ X ]) ≤ n δ o , whic h guaran tees Pr { X ∈ E δ } > 1 − δ . F o r a co m p arison of the conserv ativ eness of generalized Chebyshev in equalities (1) and (2), it is natural to consider the ratio v ol( B δ ) v ol( E δ ) where vol( . ) is a v olume function such that vol( S ) = R v ∈ S dv for any S ⊂ R n . Inte r estingly , we hav e Theorem 2 F or any r andom ve ctor X ∈ R n , v ol( B δ ) v ol( E δ ) =  q tr(Σ) n  n p det(Σ) > 1 wher e det(Σ) is the determinant of Σ . Pro of . By the definitions of v ariance a nd co v ariance, we ha ve V ar( X ) = tr(Σ). I t follo ws that v ol( B δ ) = K r tr(Σ) δ ! n where K > 0 is a constan t. App lying a lin ear transform u = Σ − 1 2 ( v − E [ X ]) to the int egration v ol( E δ ) = R v ∈ E δ dv , w e ha v e v ol( E δ ) = det(Σ 1 2 ) Z || u || 2 ≤ n δ du = p det(Σ) K  r n δ  n and th us v ol( B δ ) v ol( E δ ) =  q tr(Σ) n  n p det(Σ) . T o show v ol( B δ ) v ol( E δ ) > 1, it is equ iv alen t to show tr(Σ) n ≥ [det(Σ)] 1 n . Recall that the geo metric a v erage is no less than the arithmetic a verag e, tr(Σ) n = P n i =1 σ ii n ≥ n Y i =1 σ ii ! 1 n , (3) 4 where σ ii , i = 1 , · · · , n are the diagonal comp onen ts of Σ. Note that the co v ariance matrix Σ is p ositiv e definite, hence by Hadamard’s inequalit y , det(Σ) ≤ n Y i =1 σ ii . (4) It follo w s from (3) and (4) that tr(Σ) n ≥ [det (Σ )] 1 n . The pro of is th us completed. ✷ As an illustrativ e example, consider a t w o-dimensional r andom vec tor X = " y y + z # where y and z are ind ep endent Guassian ran d om v ariables with zero means and v ariances σ 2 , k σ 2 resp ectiv ely . S tr aigh tforwa rd computation giv es Σ = " σ 2 σ 2 σ 2 ( k + 1) σ 2 # and v ol( B δ ) v ol( E δ ) = k + 2 2 √ k ≥ √ 2 . Ob viously , as k increases fr om 2 to ∞ or decreases fr om 2 to 0, the r atio of v olumes increases monotonically an d tends to ∞ . In the follo win g Figure 1, ell ipsoid E δ and sphere B δ are constructed for σ = 1 , k = 25 and δ = 0 . 1. Moreo ver, 10 00 i.i.d. samples of X are generate d to sh o w the co v er age of the ellipsoid and s phere. It can b e seen that most samples are includ ed in the ellipsoid. Th is ind icates that Theorem 1 is m u c h less co nserv ativ e th an the classical generalized Chebyshev inequalit y in describing ho w a random v ector deviate s from its exp ectation. References [1] X. C hen, On the Pr ob abilistic Ch ar acterization of M o del Unc ertainty and R obustness , pp. 8–9, Master thesis, Louisiana State Univ ersit y , 1997. [2] X. C h en and K . Zhou, “On the Pr obabilistic Characterization of Mo del Uncertain ty and Robustness,” Pr o c e e ding of the 36 -th CDC , pp. 3616–362 1, San Diego , Decem b er 19 97. [3] H. J. Go d win, “On Generaliza tions of Cheb ysh ev Inequalit y ,” Journal of the Americ an Sta- tistic al Asso ciation , pp. 923–945 , V o l. 50, No. 271, 1955. [4] R. G. Laha and V. K. Rohatgi, Pr ob ability The ory , pp. 446–451, J ohn Wiley and Sons, 1979. 5 −20 −15 −10 −5 0 5 10 15 20 −25 −20 −15 −10 −5 0 5 10 15 20 25 Sphere Ellipsoid Figure 1: Comparison of Generalized Cheb yshev Inequalities 6 [5] C. L. Mallo ws, “Generalizations of Ch eb ys hev I nequalities,” Journa l of the R oyal Statistic al So ciety, Series B , pp. 139– 171, V ol. 18, No. 2 , 19 56. [6] A. W. Marsh all and I. Olkin, “ Multiv ariate Ch eb y s hev I n equalities,” The Anna ls of Mathe- matic al Sta tistic s , pp. 1001–10 14, V ol. 31, No. 4, 1960. [7] B. L. S. P . Rao, “Ch eb ysh ev’s in equalit y for Hilb ert-sap ce-v alued r andom element s ,” Statis- tics and Pr ob ability L etters , V ol. 80, p p . 1039–1042 , 2010. [8] L. Zhou and Z. C. Hu, “Chebyshev’s inequalit y for Banac h -sap ce-v alued rand om elements,” arXiv:1106 .0955 v1 [math.PR], June 2011. 7