Uniform Approximation and Bracketing Properties of VC classes

Uniform Appro ximatio n and Brac k eting Prop erti es of V C classes T errence M. Adams ∗ and Andrew B. Nob el † July 2010 Abstract W e show that the sets in a family with finite VC dimension can b e unifor mly a p- proximated within a given error by a finite partition. Immediate coro lla ries include the fact that VC classes hav e finite bracketing num be rs, s atisfy uniform laws of averages under strong depe ndence , and e x hibit unifo r m mixing. Our results are based on rec e nt work c o ncerning unifor m laws of av e r ages for V C clas ses under erg o dic sampling. ∗ T errence Adams is wi t h the Departmen t of Defense, 9800 Sa v age Rd. Suite 6513, Ft. Meade, MD 2 0755 † Andrew Nob el is with the Department of Statistics an d O p erations Researc h, U niversit y of North Car- olina, Chap el Hill, NC 27599-3260. Emai l: nobel@email.unc.edu 1 1 In tro d uction Let X b e a complete separable m etric s p ace with Borel sigma fi eld S , and let C ⊆ S b e a family of measurable sets. F or eac h finite set D ⊆ X , let { C ∩ D : C ∈ C } b e the collection of subsets of D induced b y the mem b ers of C . The family C is said to b e a V apnik-Chervonenkis (V C) class if th er e is a fi nite integ er k such that |{ C ∩ D : C ∈ C }| < 2 k for every D ⊆ X with | D | = k . (1) Here and in what follo w s | · | d enotes cardinalit y . The smallest k f or which (1) holds is kno wn as the V C-dimension of C . Classes of sets having fin ite VC-dimension pla y a cen tral role in the theory of mac hin e learning and empirical pr o cesses ( c.f. [7 , 9, 4, 5]). 1.1 Principal Result Let µ b e a prob ab ility measure on ( X , S ), and let π b e a fin ite, measurable partition of X . F or ev ery set C ∈ C , the π -b oundary of C , denoted ∂ ( C : π ), is the union of all th e cells in π that intersect b oth C and its complemen t with p ositiv e prob ab ility . F ormally , ∂ ( C : π ) = ∪ { A ∈ π : µ ( A ∩ C ) > 0 and µ ( A ∩ C ) > 0 } . Note that ∂ ( C : π ) dep ends on µ ; this dep end ence is supp ressed in our notation. Of interest here is the existence of a fix ed fi nite partition π s u c h that the measure of the b ound ary ∂ ( C : π ) is small for eve r y s et C in C . In general, the existence of a uniformly appr o ximating partition d ep ends on the family C and the measur e µ . Our main result s h o ws th at VC classes p ossess this uniform approximati on pr op ert y , regardless of the measure µ . Theorem 1. L et µ b e a pr ob ability me asur e on ( X , S ) . If C is a VC-class, then for every ǫ > 0 ther e exists a finite me asur able p artition π of X such that sup C ∈C µ ( ∂ ( C : π )) < ǫ. (2) Sev eral corollaries of Theorem 1 are discussed in the next section. The pr o of of Th eorem 1 is p resen ted in Section 3. 2 Corollaries of Theorem 1 Here w e p resen t sev eral immed iate corollaries of Theorem 1 th at ma y b e of indep enden t in terest. 2 2.1 Brac keting of VC Classes Let µ b e a p r obabilit y measure on ( X , S ). F or eac h pair of s ets A, B ∈ S , the brac ket [ A, B ] consists of all those sets C ⊆ X such that A ⊆ C ⊆ B . I f A is not a su bset of B , then [ A, B ] is emp t y . The br ac k et [ A, B ] is s aid to b e an ǫ -brac ke t if µ ( B \ A ) ≤ ǫ . The brac keting n u m b er N [ ] ( ǫ, C , µ ) of a family C ⊆ S is the least num b er of ǫ -brac ke ts needed to co ver C . Note that the sets defining the minimal br ac k ets need not b e elemen ts of C . Corollary 1. L et µ b e any pr ob ability me asur e on ( X , S ) . If C is a c ountable VC-class, then N [ ] ( ǫ, C , µ ) is finite for every ǫ > 0 . Remark: Using routine argument s , the assump tion th at C is countable can b e replaced b y the wea ker assu mption that there exists a count able sub-family C 0 ⊆ C suc h that the indicator function of every set in C is the p oint wise limit of the indicator functions of sets in C 0 . Pro of: Fix a probabilit y measure µ and ǫ > 0. Let π = { A 1 , . . . , A m } b e a finite measurable partition of X suc h that (2) holds, and assume with ou t loss of generalit y that eac h s et A j has p ositiv e µ -measure. Let A j b e an elemen t of π . F or eac h C ∈ C , remo ve p oin ts in C from A j if µ ( A j ∩ C ) = 0, and remov e p oin ts in C c from A j if µ ( A j ∩ C c ) = 0. Denot e the resu lting set by B j . Clearly B j ⊆ A j and, as C is count ab le, µ ( A j \ B j ) = 0. The definition of B j ensures that for eac h C ∈ C exact ly one of the follo w in g relations holds : B j ⊆ C , B j ⊆ C c , or µ ( B j ∩ C ) · µ ( B j ∩ C c ) > 0. Let B 0 = X \ ∪ m j =1 B j , and define the partition π ′ = { B 0 , B 1 , . . . , B m } . Giv en C ∈ C let C l = ∪{ B ∈ π ′ : B ⊆ C } and C u = ∪{ B ∈ π ′ : B ∩ C 6 = ∅} . A str aigh tforwa r d argum en t sh ows that C l ⊆ C ⊆ C u , and that µ ( C u \ C l ) = µ ( ∂ ( C : π ′ )) = µ ( ∂ ( C : π )) < ǫ . It follo ws that Θ = { [ C l , C u ] : C ∈ C } is a collecti on of ǫ -brac kets cov ering C . Th e cardinalit y of Θ is at most 2 2 | π | . 2.2 Uniform La ws of Large N um b ers Let X 1 , X 2 , . . . b e a stationary ergo d ic pro cess taking v alues in ( X , S ) with X i ∼ µ . Th e er- go d ic theorem ensur es that, for ev ery C ∈ S , the sample a verage s n − 1 P n i =1 I C ( X i ) conv erge with prob ab ility on e to µ ( C ). F or V C classes and i.i.d. sequences { X i } this conv ergence is kno wn to b e uniform ov er C [10]. Using Corollary 1 it is easy to sho w that this un if orm con ve r gence extends to ergo dic p ro cesses as we ll. Theorem 2. If C is a c ountable VC-class of sets and X 1 , X 2 , . . . ∈ X is a stationary er go d ic 3 pr o c ess with X i ∼ µ , then sup C ∈C      1 n n X i =1 I C ( X i ) − µ ( C )      → 0 with pr ob ability one as n tends to i nfinity. Pro of: This follo ws easily from Corollary 1 and the Blum DeHardt la w of large num b ers ( c.f. [9]), w hic h establishes that families with fin ite brack eting num b ers hav e the Gliv enko Can telli prop erty . The uniform strong la w in Theorem 2 was established in [1] using argument s s im ilar to those forTh eorem 1. Analogous uniform strong la ws for V C ma jor and V C graph classes are giv en in [1], wh ile [2] con tains uniform strong la ws f or classes of fu n ctions ha ving fi nite gap (fat shattering) dimension. S ee th ese pap ers for a discussion of earlier and related wo r k. 2.3 Uniform Mixing Conditions in E rgo dic Theory Let T b e an ergo dic µ -measure preserving transformation of ( X , S ). T is said to b e strongly mixing if f or eac h pair A , B of measur ab le sets, lim n →∞ µ ( A ∩ T − n B ) = µ ( A ) µ ( B ) . T heorem 1 can b e applied to show that strong mixing o ccurs un iformly o ver a counta b le VC class. Prop osition 1. If C ⊆ S is a c ountable VC-class of me asur a b le sets, and T is a str ongly mixing tr ansfor mation, then lim n →∞ sup A,B ∈C   µ ( A ∩ T − n B ) − µ ( A ) µ ( B )   = 0 . Pro of: Giv en ǫ > 0, let π b e a finite p artition such that sup C ∈C µ ( ∂ ( C : π )) < ǫ . C h o ose a natural num b er N suc h that f or n ≥ N and eac h pair D 1 , D 2 ∈ π , | µ ( D 1 ∩ T − n D 2 ) − µ ( D 1 ) µ ( D 2 ) | < ǫ µ ( D 1 ) µ ( D 2 ) . F or every measurab le set A let A = ∪{ D ∈ π : µ ( D ∩ A ) > 0 } and A = ∪{ D ∈ π : D ⊂ A } b e, resp ectiv ely , upp er and low er app ro ximations of A derived f rom the cells of π . Note that if A, B are measur able sets satisfying A = A and B = B , th en | µ ( A ∩ T − n B ) − µ ( A ) µ ( B ) | = | X D ⊆ A X D ′ ⊆ B µ ( D ∩ T − n D ′ ) − X D ⊆ A X D ′ ⊆ B µ ( D ) µ ( D ′ ) | ≤ X D ⊆ A X D ′ ⊆ B | µ ( D ∩ T − n D ′ ) − µ ( D ) µ ( D ′ ) | < X D ⊆ A X D ′ ⊆ B ǫ µ ( D ) µ ( D ′ ) ≤ ǫµ ( A ) µ ( B ) ≤ ǫ. 4 Supp ose no w that A, B are sets in C . Then for n ≥ N , | µ ( A ∩ T − n B ) − µ ( A ) µ ( B ) | = | µ ( A ∩ T − n B ) ± µ ( A ∩ T − n B ) ± µ ( A ∩ T − n B ) ± µ ( A ) µ ( B ) ± µ ( A ) µ ( B ) − µ ( A ) µ ( B ) | ≤ 2 µ ( B \ B ) + 2 µ ( A \ A ) + | µ ( A ∩ T − n B ) − µ ( A ) µ ( B ) | < 5 ǫ, where the fir st inequalit y follo w s from the triangle inequalit y , and the second f ollo ws from the previous t wo d isp la ys. As A, B ∈ C and ǫ > 0 w ere arbitrary , T heorem 1 follo ws. A similar argument can b e used to sho w that an y weak m ixing transf ormation satisfies uniform con ve r gence o ver coun table VC classes. A measure preservin g transformation T is w eak mixing if give n measur able sets A and B , lim n →∞ 1 n n − 1 X i =0 | µ ( A ∩ T − i B ) − µ ( A ) µ ( B ) | = 0 . Prop osition 2. If C is a c ountable VC-class of me asur a ble sets and T is a we akly mixing tr ansfo rmation, then lim n →∞ sup A,B ∈C 1 n n − 1 X i =0 | µ ( A ∩ T − i B ) − µ ( A ) µ ( B ) | = 0 . 3 Pro of of Theorem 1 The pro of of Th eorem 1 follo w s argum en ts u sed in [1] to establish u niform laws of large n u m b ers f or V C classes under ergo dic samp ling, and w e mak e use of sev eral auxiliary resu lts from that pap er in wh at follo ws. 3.1 Joins and the VC dimension Definition: The join of k sets A 1 , . . . , A k ⊆ [0 , 1], denoted J = W k i =1 A i , is the partition consisting of all non-empty intersect ions ˜ A 1 ∩ · · · ∩ ˜ A k where ˜ A i ∈ { A i , A c i } for i = 1 , . . . , k . Note th at J is a fin ite p artition of [0 , 1]. The join of A 1 , . . . , A k is said to b e full if it has (maximal) card inalit y 2 k . Th e next Lemma (see [6, 1]) mak es an elemen tary connection b et ween f u ll joins and the V C d imension. Lemma 1. L et C b e any c ol le ction of subsets of X . If for some k ≥ 1 ther e exists a c ol le ction C 0 ⊆ C of 2 k sets having a fu l l join, then VC-dim ( C ) ≥ k . 5 The pro of giv en h ere establishes that the approximat in g partition π is measur able σ ( C ). A simple counterexample sho w s that it is n ot sufficient for the ele m ents of π to belong to S ∞ n =1 σ ( C 1 , C 2 , . . . , C n ). T o see this, let X = [0 , 1] and let λ be Leb esgue measure. Let a 1 , a 2 , . . . > 0 b e a sequence of n umb ers suc h that s = P ∞ n =1 a n < 1. Let s n = P n i =1 a i for n ≥ 1 and let s 0 = 0. Define C n = [ s n − 1 , s n ) for n ≥ 1. Clearly , the VC- dimension of the class { C 1 , C 2 , . . . } equals 1, since its constituen t sets are disj oin t. Define J n = C 1 ∨ C 2 ∨ . . . ∨ C n . Then A n = [ s n , 1] is a single elemen t in J n with measur e 1 − s n > 1 − s > 0. Moreo v er, b oth A n ∩ C n +1 and A n ∩ C ′ n +1 ha ve p ositiv e measur e, so that µ ( ∂ ( C n +1 : J n )) > 1 − s for n ≥ 1. 3.2 Reduction t o t he Unit In terv al Fix a p robabilit y measur e µ on ( X , S ) and let C ⊆ S h a ve fin ite V C dimension. It follo w s from standard results on the L p -co v ering num b ers of V C classes ( c. f . Theorem 2.6.4 of [9]) that there exists a counta b le sub-family C 0 of C suc h that inf C ′ ∈C 0 µ ( C ′ △ C ) = 0 for eac h C ∈ C . An elementa ry argumen t th en sh o ws that, for every fin ite partition π , sup C ∈C µ ( ∂ ( C : π )) = sup C ∈C 0 µ ( ∂ ( C : π )) , and w e may therefore assume that C is counta b le. Let X 0 = { x : µ ( { x } ) > 0 } b e the s et of atoms of µ and let µ 0 ( A ) = µ ( A ∩ X 0 ) b e the atomic comp onen t of µ . As X 0 is coun table, it is easy to see that inf π ∈ Π sup C ∈C µ 0 ( ∂ ( C : π )) = 0 , and w e ma y therefore assu me th at µ is non-atomic. F ollo wing the p ro of in [1], w e mak e tw o f u rther r eductions. Let λ ( · ) b e L eb esgue mea- sure on the u nit interv a l [0 , 1] equipp ed with its Borel su bsets B . Using the existence of a measure-preserving isomorph ism b et w een ( X , S , µ ) and ([0 , 1] , B , λ ) ( c.f. [8]) a straigh tfor- w ard argument ensures th at we lose no generalit y in assuming that X = [0 , 1], µ = λ , and that C ⊆ B is a count able family with fi nite V C d imension. Using an additional isomor- phism describ ed in Lemma 6 of [1] we ma y fu rther assume that eac h elemen t of C is a finite union of in terv als. Based on the reductions ab ov e, Theorem 1 is a corollary of the follo wing result. 6 Theorem 3. L et C ⊆ B b e a c ountable VC class, e ach of whose elements is a finite union of intervals. F or every ǫ > 0 ther e exists a finite p artition of [0 , 1] such that sup C ∈C λ ( ∂ ( C : π )) < ǫ. Remark: The p ro of of Theorem 3 follo ws the pr o of of Prop osition 3 f r om [1 ]. Beginning with th e assumption that the conclusion of the theorem is false, we construct, in a step-wise fashion, a sequence of “splitting sets” R 1 , R 2 , . . . ⊆ [0 , 1] from the sets in C . A t the k th stage the splitting set R k is obtained from a sequen tial p ro cedure that make s use of the splitting sets R 1 , . . . , R k − 1 pro du ced at pr evious stages. T he splitting sets are then u s ed to iden tify fi nite, but arbitrarily large, collect ions of s ets in C h a ving fu ll join. The existence of these collections implies that C has in finite V C dimension by Lemma 1. Pro of of Theorem 3: Sup p ose to the cont r ary that there exists an η > 0 such that sup C ∈C λ ( ∂ ( C : π )) > η for every π ∈ Π . (3) F or n ≥ 1 let D n = { [ k 2 − n , ( k + 1) 2 − n ] : 0 ≤ k ≤ 2 n − 1 } b e the set of closed d y adic in terv als of order n . Stage 1. Let C 1 (1) b e any set in C . S upp ose that sets C 1 (1) , . . . , C 1 ( n ) ∈ C ha ve already b een selected, an d let J 1 ( n ) = D n ∨ C 1 (1) ∨ · · · ∨ C 1 ( n ). It follo ws from (3 ) that there is a set C 1 ( n + 1) ∈ C su ch that G 1 ( n ) = ∂ ( C 1 ( n + 1) : J 1 ( n )) has measure greater than η . Let J 1 ( n + 1) = D n +1 ∨ C 1 ∨ · · · ∨ C n +1 and contin ue in the same fash ion. Th e sets { G 1 ( n ) } are naturally asso ciated with a tight family of sub-pr obabilit y measures { λ n ( · ) = λ ( · ∩ G 1 ( n )) } . There is th erefore a su bsequence { λ n 1 ( r ) } that conv erges we akly to a sub -probabilit y ν 1 on ([0 , 1] , B ). It is easy to see that ν 1 is absolutely con tinuous w ith r esp ect to λ and that ν 1 ([0 , 1]) ≥ lim sup r →∞ λ n r ([0 , 1]) ≥ η . The Radon-Nik o dym d eriv ativ e dν 1 /dλ is well defin ed , and is b ounded ab o ve by 1. Define the splitting set R 1 = { x : ( dν 1 /dλ )( x ) > η / 2 } . F rom the pr evious r emarks it follo ws that η ≤ ν 1 ([0 , 1]) = Z 1 0 dν 1 dλ dλ ≤ Z R 1 1 dλ + Z R c 1 η / 2 dλ ≤ λ ( R 1 ) + η / 2 , (4) and therefore λ ( R 1 ) ≥ η / 2. Subsequen t stages. In order to construct the sp litting set R k at s tage k , let C k (1) b e an y elemen t of C , and sup p ose that C k (2) , . . . , C k ( n ) hav e already b een selected. Define the 7 join J k ( n ) = D n ∨ k − 1 _ j =1 R j ∨ n _ i =1 C k ( i ) . (5) By (3) there exists a set C k ( n + 1) ∈ C su ch that G k ( n ) = ∂ ( C k ( n + 1 : J k ( n )) has measure greater than η . This p ro cess con tinues as in stage 1. As b efore, there is a sequence of in tegers n k (1) < n k (2) < · · · suc h that the measures λ ( B ∩ G k ( n k ( r ))) con verge we akly to a sub-prob ab ility measure ν k on ([0 , 1] , B ) that is absolutely con tin u ous with resp ect to λ ( · ). Define R k = { x : ( dν k /dλ )( x ) > δ } . Construction of F ull Joins. Fix a n integ er L ≥ 2. As the measures of the s ets R k are b oun ded aw a y from zero, there exist p ositiv e inte gers k 1 < k 2 < . . . < k L suc h that λ ( T L j =1 R k j ) > 0. Su p p ose without loss of generalit y that k j = j , and defin e the intersect ions Q r = L − r \ j =1 R j for r = 0 , 1 , . . . , L − 1. Note that Q 0 ⊆ Q 1 ⊆ · · · ⊆ Q L − 1 . W e sh o w that th ere exist sets D 1 , D 2 , . . . , D L − 1 ∈ C s uc h that, for l = 1 , . . . , L − 1, (i) the join K l = D 1 ∨ D 2 ∨ · · · ∨ D l has card inalit y | K l | = 2 l , and (ii) B o ∩ Q l is n on-empt y f or eac h B ∈ K l , where B o denotes the interior of B . W e pro ceed by indu ction, b eginning w ith the case l = 1. Let x 1 b e a Leb esgue p oint of Q 0 , and let ǫ = η / 2 ( η + 2). Th en th er e exists α 1 > 0 such that the inte r v al I 1 △ = ( x 1 − α 1 , x 1 + α 1 ) satisfies λ ( I 1 ∩ Q 0 ) ≥ (1 − ǫ ) λ ( I 1 ) = 2 α 1 (1 − ǫ ) . (6) It follo ws f rom th e last displa y and the definition of R L ⊇ Q 0 that ν L ( I 1 ∩ R L ) = Z I 1 ∩ R L dν L dλ dλ > eta 2 λ ( I 1 ∩ R L ) ≥ α 1 (1 − ǫ ) η . (7) Let { n L ( r ) : r ≥ 1 } b e the subsequen ce used to define the su b-probabilit y ν L . As I 1 is an op en set, the p ortman teau theorem and (7) imp ly that lim inf r →∞ λ ( I 1 ∩ G L ( n L ( r ))) ≥ ν L ( I 1 ) ≥ ν L ( I 1 ∩ R L ) > α 1 (1 − ǫ ) η . Cho ose r sufficient ly large s o that λ ( I 1 ∩ G L ( n L ( r ))) > α 1 (1 − ǫ ) η an d 2 − n L ( r ) < η α 1 / 8. W e requir e the f ollo wing lemma from [1]. 8 Lemma 2. Ther e exists a c el l A of J L ( n L ( r )) such that A ⊆ ∂ ( C L ( n L ( r ) + 1) : J L ( n L ( r )) , A ⊆ I 1 and λ ( A ∩ Q 1 ) > 0 . Mor e o v er, A is c ontaine d in Q 1 . Let D 1 = C L ( n L ( r ) + 1) ∈ C , and let A b e the set identified in Lemma 2 . By definition of the b oundary , λ ( A ∩ D 1 ) > 0 and λ ( A ∩ D c 1 ) > 0 and ther efore λ ( Q 1 ∩ D 1 ) > 0 and λ ( Q 1 ∩ D c 1 ) > 0 as wel l. As the Leb esgue measure of the b oundary D 1 \ D o 1 of D 1 is zero, assertion (ii) ab ov e follo ws. Supp ose now that we ha ve identi fi ed sets D 1 , . . . , D l ∈ C , with l ≤ L − 2, such that (i) and (ii) hold. Let the j oin K l = { B j : 1 ≤ j ≤ 2 l } , and for eac h j let x j ∈ B o j ∩ Q l . Select α l +1 > 0 such that for eac h j the in terv al I j △ = ( x j − α l +1 , x j + α l +1 ) is conta in ed in B o j and satisfies λ ( I j ∩ Q l ) ≥ (1 − ǫ ) λ ( I j ) = 2 α l +1 (1 − ǫ ) . T o s im p lify n otation, let κ = L − l . Let { n κ ( r ) : r ≥ 1 } b e the subsequen ce u sed to define the sub-pr obabilit y ν κ . F or eac h in terv al I j , lim inf r →∞ λ ( I j ∩ G κ ( n κ ( r ))) ≥ ν κ ( I j ) ≥ ν κ ( I j ∩ R κ ) > α l +1 (1 − ǫ ) η , where the last inequalit y foll o ws from the previous display , and the fact that Q l ⊆ R κ . Cho ose r suffi cien tly large so that λ ( I j ∩ G κ ( n κ ( r ))) > α l +1 (1 − ǫ ) η for eac h j , and 2 − n κ ( r ) < η α l +1 / 8. By applying the Lemma 2 to eac h int er v al I j , one ma y establish the existence of sets A j ∈ ∂ ( C κ ( n κ ( r ) + 1) : J κ ( n κ ( r )) such that A j ⊆ I j ⊆ B o j , λ ( A j ∩ Q l +1 ) > 0, and A j ⊆ Q l +1 . Let D l +1 = C κ ( n κ ( r ) + 1) ∈ C . Argumen ts like those for the case l = 1 ab o v e sh o w that for eac h j the in tersections A j ∩ D o l +1 and A j ∩ ( D c l +1 ) o are non-empt y , and the indu ctiv e step is complete. Give n an y tw o dya d ic in terv als, they are disj oin t, intersect at one p oint, or one con tains the other. Th erefore, among the sets D 1 , . . . , D L − 1 , at most one can b e a dy adic interv al ; the r emainder are conta in ed in C . Ac kno wledgemen ts The authors would lik e to thank Ramon v an Hand el for p oin ting out an o ve r sigh t in the pro of of Corollary 1. The work pr esen ted in this pap er was supp orted in part by NSF gran t DMS-0907 177. 9 References [1] Ad a ms, T.M. and N obel, A.B. (2010) Uniform con ve r gence of V apnik-Chervo nenkis classes u nder ergo dic sampling. Annals of Pr ob ability 38:4 1345–13 67. [2] Ad a ms, T.M. and No bel, A.B. (2010) T he gap dimens ion and un iform la ws of large n u m b ers for ergo dic pro cesses. arXiv:1007. 2964 v 1 [math.PR]. Su bmitted for pub lication. [3] Billingsley, P . (1995). Pr ob ability a nd Me asur e , 3rd e d ., Wiley , New Y ork. MR13247 86 (95k:60001 ) [4] Devr oye, L. and Gy ¨ orfi, L. and Lugosi, G. (1996) A Pr ob abilistic The ory of Pattern R e c o gnition Springer. [5] Dudley, R.M. (1999) Uniform Centr al Limit The or ems Cam bridge Univ. Press, New Y ork. MR172071 2 (2000k:60 050) [6] Ma tousek, J. (2002) Lectures on Discrete Geometry . Gr aduate T exts in M athematics 212 S pringer, New Y ork. MR189929 9 (2003f:5 2011) [7] Pollard, D. (1984) Con ver genc e of Sto chastic Pr o c esses Spr inger, New Y ork. MR07629 84 (86i:6007 4) [8] R oyden, H.L. (1 988) R e al Analysis , 3rd ed. Macmillan Publishin g Compan y , New Y ork. MR101311 7 (90g:000 04) [9] v an der V aar t, A.W . and We llner, J.A. (1996) We ak Conver genc e and Empiric al Pr o c esses . Spr inger, New Y ork. MR1385671 (97g:6 0035) [10] V a pnik, V.N. and Cher vonenkis, A .Y a. (197 1) On the un iform con v ergence of relativ e frequencies of eve nts to their probabilities. The ory Pr ob ab. Appl. 16 264–280. MR06278 61 (83d:60031) 10