Explicit Formula for Constructing Binomial Confidence Interval with Guaranteed Coverage Probability

EXPLICIT F ORMULA F OR CONSTRU CTING BINOMIAL CONFIDENCE INTER V AL WITH GUARANTEED CO VERAG E PR OBABILITY XINJIA CHEN, KEMIN ZHOU AND JORGE L. ARA VENA Abstract. In this pap er, we deri ve an explicit f or mula for constructing the confidence interv al of binomial parameter wi th guaranteed co verage probabil- ity . The formula o ve rcomes the limitation of normal approximation whic h is asymptotic in nature and thus i nevitably i nt ro duce unknown errors in ap- plications. Moreov er, the formula is very tigh t in comparison with classic Clopper- Pea rson’s approac h from the p ersp ectiv e of interv al width. Based on the rigorous formula, we also obtain approximate formulas with excellen t perf ormance of co verag e pr obabilit y . 1. Classic Confidence Inter v als The construction of c onfidence interv al of binomial par ameter is frequen tly en- countered in comm unications and many other areas of science and eng ineering. Clopp er and P ears o n [3] has provided a rigor ous appr oach for constructing confi- dence in terv al. How ever, the computational complexit y in volv ed with this approach is very high. The standard technique is to use normal approximation which is not accurate for rare events, esp ecia lly in the con text of s tudying the bit error rate of communication systems, blo cking probability of communication netw orks and prob- ability o f insta bility o f uncer tain dynamic sys tems . Moreover, it has b een rece n tly prov en by Brown, Ca i and DasGupta [1, 2] that the s tandard norma l approximation approach is per sistently p o or. The co verage pro ba bility of the confidence interv al can b e sig nificantly b elow the sp ecified confidence level even for very large sample sizes. Since in many s itua tions, it is desirable to quickly construct a confidence int erv al with guaranteed cov erage probability , our go al is to derive a simple and rigoro us for mula for confidence interv al c onstruction. Let the probabilit y space b e denoted as (Ω , F, P ) where Ω , F , P are the sample space, the algebra of even ts and the pr obability measure resp ectively . Let X b e a Bernoulli random v ariable with distribution P r { X = 1 } = P X , Pr { X = 0 } = 1 − P X where P X ∈ (0 , 1). L e t the sa mple size N and confidence parameter δ ∈ (0 , 1) be fixed. W e refer an observ a tion with v alue 1 as a success ful tr ial. Let K denote the num ber of successful tria ls during the N i.i.d. sampling exp eriments. Let k = K ( ω ) wher e ω is a sample p oint in the sa mple s pace Ω. Date : June 2006. Key wor ds and phr ases. Confidence Inte rv al, Probability , Statistics, Normal Approximation. This research was supp orted in part by grants fr om NASA (NCC5-573) and LEQSF (NASA /LEQSF(2001-04) -01). 1 2 XINJIA CHEN, KEMIN ZHOU AN D JORGE L. ARA VENA 1.1. Clopp er-P earson Confide nce Limits. The classic Clopp er-Pearson low er confidence limit L N ,k ,δ and uppe r confidence limit U N ,k ,δ are given resp ectively by L N ,k ,δ def =  0 if k = 0 p if k > 0 and U N ,k ,δ def =  1 if k = N p if k < N where p ∈ (0 , 1) is the solution of the following equation (1.1) k − 1 X j =0  N j  p j (1 − p ) N − j = 1 − δ 2 and p ∈ (0 , 1) is the solution o f the following equation (1.2) k X j =0  N j  p j (1 − p ) N − j = δ 2 . The pr obabilistic implication of the confidence limits can be illustrated as follows: Define random v ariable L : Ω → [0 , 1] b y L ( ω ) = L N ,K ( ω ) ,δ ∀ ω ∈ Ω and random v a riable U : Ω → [0 , 1] by U ( ω ) = U N ,K ( ω ) ,δ ∀ ω ∈ Ω. Then Pr { L ≤ P X ≤ U } > 1 − δ. The exact v alue of Pr { L ≤ P X ≤ U } is referred as the c ov er age pr ob ability . Ac- cordingly , we refer Pr { P X < L or P X > U } a s the err or pr ob ability . 1.2. Normal Appro ximation. It is easy to s ee that the equatio ns (1.1) a nd (1.2) are very hard to solve a nd thus the co nfidence limits are very difficult to determine using Clopp er-Pearso n’s approach. F or lar ge sample size, it is computationally prohibitive. T o get a round the difficult y , norma l approximation has b een widely used to develop simple appr oximate formulas (see, for example, [1, 2 , 5, 6] and the references therein ). The basis of the normal approximation is the Central Limit Theorem, i.e., lim N →∞ Pr      K N − P X   q P X (1 − P X ) N < z    = 2 Φ( z ) − 1 where z > 0 a nd Φ( . ) is the normal distribution function. Let Z δ 2 be the critical v a lue such that Φ( Z δ 2 ) = 1 − δ 2 . It follows tha t lim N →∞ Pr ( K N − Z δ 2 r P X (1 − P X ) N < P X < K N + Z δ 2 r P X (1 − P X ) N ) = 1 − δ, i.e., lim N →∞ Pr      K N + Z 2 δ 2 2 N − Z δ 2 s K N (1 − K N ) N + Z 2 δ 2 4 N 2 1+ Z 2 δ 2 N < P X < K N + Z 2 δ 2 2 N + Z δ 2 s K N (1 − K N ) N + Z 2 δ 2 4 N 2 1+ Z 2 δ 2 N      = 1 − δ . Since Z 2 δ 2 N ≈ 0 for sufficiently large sample s iz e N , the low er and upp er confidence limits ca n b e estimated resp ectively a s e L ≈ k N − Z δ 2 s k N (1 − k N ) N BINOMIAL CONFIDENCE INTER V AL 3 and e U ≈ k N + Z δ 2 s k N (1 − k N ) N . The cr itical problem with the no r mal approximation is that it is of asymptotic nature. It is not cle a r how lar ge the sa mple size is sufficient for the appr oximation error to b e neg ligible. Suc h an a symptotic appro ach is not go o d enough for many practical applica tions inv olving r are even ts. 2. Rigor ous F ormula It is desira ble to have a simple formula which is rigorous and v ery tight for the confidence interv a l constr uction. W e now pr op ose the following simple for mula for constructing the confidence limits. Theorem 1. D efine (2.1) L ( k ) def = k N + 3 4 1 − 2 k N − q 1 + 4 θ k (1 − k N ) 1 + θN , k = 0 , 1 , · · · , N and (2.2) U ( k ) def = k N + 3 4 1 − 2 k N + q 1 + 4 θ k (1 − k N ) 1 + θN , k = 0 , 1 , · · · , N with θ = 9 8 ln 2 δ . Then Pr {L ( K ) < P X < U ( K ) } > 1 − δ . Mor e over, L ( k ) < L N ,k ,δ < U N ,k ,δ < U ( k ) . R emark 1 . L ( k ) and U ( k ) are tight b ounds for the classic Clopp er-Pearso n confi- dence limits L N ,k ,δ and U N ,k ,δ (See Figures 1-1 2). A bisection search can be per - formed based on such bo unds for computing the classic Clopp er-Pearson c o nfidence limits. T o show Theorem 1, w e nee d some preliminar y results. The following Le mma 1 is due to Ma ssart [7]. Lemma 1. P r  K N ≥ P X + ǫ  ≤ ex p  − N ǫ 2 2( P X + ǫ 3 ) (1 − P X − ǫ 3 )  for al l ǫ ∈ (0 , 1 − P X ) . Of co urse, the ab ove upp er b ound holds trivially for ǫ ≥ 1 − P X . Th us, Lemma 1 is actually true for any ǫ > 0. Lemma 2. P r  K N ≤ P X − ǫ  ≤ ex p  − N ǫ 2 2( P X − ǫ 3 ) (1 − P X + ǫ 3 )  for al l ǫ > 0 . Pro of. Define Y = 1 − X . Then P Y = 1 − P X . A t the sa me time when we are conducting N i.i.d. exp eriments for X , we a re also co nducting N i.i.d. ex per iments for Y . Let the num b e r of successful tr ials of the exper imen ts for Y b e denoted as K Y . Obviously , K Y = N − K . Applying Lemma 1 to Y , we hav e Pr  K Y N ≥ P Y + ǫ  ≤ ex p  − N ǫ 2 2( P Y + ǫ 3 ) (1 − P Y − ǫ 3 )  . It follows that Pr  N − K N ≥ 1 − P X + ǫ  ≤ exp  − N ǫ 2 2(1 − P X + ǫ 3 ) [1 − (1 − P X ) − ǫ 3 ]  . 4 XINJIA CHEN, KEMIN ZHOU AN D JORGE L. ARA VENA The pro o f is th us completed b y observing that P r  N − K N ≥ 1 − P X + ǫ  = P r  K N ≤ P X − ǫ  . ✷ The fo llowing lemma can be found in [4]. Lemma 3. P k j =0  N j  x j (1 − x ) N − j de cr e ases monotonic al ly with r esp e ct t o x ∈ (0 , 1) for k = 0 , 1 , · · · , N . Lemma 4. P k j =0  N j  x j (1 − x ) N − j ≤ exp  − N ( x − k N ) 2 2 ( 2 3 x + k 3 N ) (1 − 2 3 x − k 3 N )  ∀ x ∈ ( k N , 1 ) for k = 0 , 1 , · · · , N . Pro of. Conside r binomial ra ndo m v ariable X with parameter P X > k N . Let K be the num ber of succ e s sful tr ials dur ing N i.i.d. sampling exp eriments. Then k X j =0  N j  P j X (1 − P X ) N − j = P r { K ≤ k } . Note that Pr { K ≤ k } = P r  K N ≤ P X −  P X − k N  . A pplying Lemma 2 with ǫ = P X − k N > 0 , we hav e k X j =0  N j  P j X (1 − P X ) N − j ≤ exp − N ( P X − k N ) 2 2( P X − P X − k N 3 ) (1 − P X + P X − k N 3 ) ! = exp − N ( P X − k N ) 2 2 ( 2 3 P X + k 3 N ) (1 − 2 3 P X − k 3 N ) ! . Since the argument holds for ar bitrary bino mial random v ariable X with P X > k N , the pr o of of the lemma is th us completed.  Lemma 5. P k − 1 j =0  N j  x j (1 − x ) N − j ≥ 1 − exp  − N ( x − k N ) 2 2 ( 2 3 x + k 3 N ) (1 − 2 3 x − k 3 N )  ∀ x ∈ (0 , k N ) for k = 1 , · · · , N . Pro of. Conside r binomial ra ndo m v ariable X with parameter P X < k N . Let K be the num ber of succ e s sful tr ials dur ing N i.i.d. sampling exp eriments. Then k − 1 X j =0  N j  P j X (1 − P X ) N − j = P r { K < k } = Pr  K N < P X + ( k N − P X )  . Applying Lemma 1 with ǫ = k N − P X > 0, w e hav e that k − 1 X j =0  N j  P j X (1 − P X ) N − j ≥ 1 − ex p − N ( k N − P X ) 2 2( P X + k N − P X 3 ) (1 − P X − k N − P X 3 ) ! = 1 − ex p − N ( P X − k N ) 2 2 ( 2 3 P X + k 3 N ) (1 − 2 3 P X − k 3 N ) ! . Since the argument holds for ar bitrary bino mial random v ariable X with P X < k N , the pr o of of the lemma is th us completed.  Lemma 6. L et 0 ≤ k ≤ N . Then L N ,k ,δ < U N ,k ,δ . BINOMIAL CONFIDENCE INTER V AL 5 Pro of. Obviously , the lemma is true for k= 0 , N . W e consider the ca se that 1 ≤ k ≤ N − 1. Le t S ( N , k , x ) = P k j =0  N j  x j (1 − x ) N − j for x ∈ (0 , 1). Notice that S ( N , k, p ) = S ( N , k − 1 , p ) +  N k  p k (1 − p ) N − k = δ 2 . Thu s S ( N , k − 1 , p ) − S ( N , k − 1 , p ) = 1 − δ 2 −  δ 2 −  N k  p k (1 − p ) N − k  . Notice that δ ∈ (0 , 1) and that p ∈ (0 , 1), we hav e that S ( N , k − 1 , p ) − S ( N , k − 1 , p ) = 1 − δ +  N k  p k (1 − p ) N − k > 0 . By Lemma 3, S ( N , k − 1 , x ) decr e ases mono tonically with resp ect to x , we hav e p < p and complete the pr o of o f the lemma.  W e are no w in the position to prov e Theorem 1 . It can b e easily v erified that U N ,k ,δ ≤ U ( k ) for k = 0 , N . W e need to show that U N ,k ,δ ≤ U ( k ) for 0 < k < N . Straightforw ard computation shows that U ( k ) is the only ro ot of eq uation exp − N ( x − k N ) 2 2 ( 2 3 x + k 3 N ) (1 − 2 3 x − k 3 N ) ! = δ 2 with resp ect to x ∈ ( k N , ∞ ). There are tw o cases: U ( k ) ≥ 1 and U ( k ) < 1. If U ( k ) ≥ 1 then U N ,k ,δ ≤ U ( k ) is triv ially true. W e only need to consider the case that k N < U ( k ) < 1. In this case, it follows fro m Lemma 4 that k X j =0  N j  [ U ( k )] j (1 − U ( k )) N − j ≤ ex p − N ( U ( k ) − k N ) 2 2 ( 2 3 U ( k ) + k 3 N ) (1 − 2 3 U ( k ) − k 3 N ) ! = δ 2 . Recall that k X j =0  N j  U j N ,k ,δ (1 − U N ,k ,δ ) N − j = δ 2 , we have k X j =0  N j  U j N ,k ,δ (1 − U N ,k ,δ ) N − j ≥ k X j =0  N j  [ U ( k )] j (1 − U ( k )) N − j . Therefore, by Lemma 3, we have that U N ,k ,δ ≤ U ( k ) for 0 < k < N . Th us, we have shown that U N ,k ,δ ≤ q for all k . Similarly , by Lemma 5 and Lemma 3 , we can show that L N ,k ,δ ≥ L ( k ). By Lemma 6, we have L ( k ) < L N ,k ,δ < U N ,k ,δ < U ( k ). Finally , the pr o of of Theo- rem 1 is completed by inv oking the probabilis tic implica tion of the Clopp er-Pearso n confidence interv a l. 6 XINJIA CHEN, KEMIN ZHOU AN D JORGE L. ARA VENA 3. Numerical Experiments and Empirical F ormulas In co mparison with the Clopper-Pearso n’s approach, our approa ch is very tight from the p ersp ective of interv al width (see, for example, Figur e s 1 -12). Moreov er, there is no co mparison o n the computationa l co mplex it y . Our formula is simple enough for hand calculation. Our num erica l results are in agreement with the discov ery made by Brown, Cai a nd DasGupta [1, 2]. It can b e seen from Fig ures 21-27 that the co verage probability o f c onfidence interv als obtained by the sta ndard normal appr oximation can b e substantially low er than the specifie d confidence level 1 − δ (This is tr ue even when the co ndition for applying the rule of th um b, i.e., N P X (1 − P X ) > 5, is satisfied). Moreover, the situation is w orse for smaller confidence parameter δ . See, for example, Fig ures 25-27, if one wishes to mak e an inference with an er ror frequency less than one out o f 1000 , using the normal a pproximation can lea d to a frequency of er ror higher than 10 0 out of 1 000. In lig ht of the excess ively high er ror rate of inference ca used by the nor mal approximation, the r igoro us formula may b e a better c hoice. The rigo r ous form ula guarantees the e r ror probability below the sp ecify level δ . It s hould b e noted that th e rigorous formula is conser v ativ e (with actual er ror probability aro und 10% to 20 % of the requir ement). It should b e noted that by tuning the para meter θ in the rigo rous formula, o ne can o btained s imple formulas which meet the s pec ified confidence levels. F or exam- ple, to construct confidence int erv al with confidence parameter δ = 0 . 0 5 , 0 . 01 , 0 . 001, we can simply co mpute L ( k ) and U ( k ) defined in Theorem 1 with θ = 1 2 , 1 3 , 1 5 resp ectively (The v alues o f θ presented her e are no t optimal. Better cov erag e p er - formance can be ac hieved by a fine tuning of θ ). More s pec ifically , Pr  K N + 3 4 1 − 2 K N − √ 1+2 K (1 − K N ) 1+ N 2 < P X < K N + 3 4 1 − 2 K N + √ 1+2 K (1 − K N ) 1+ N 2  ≈ 0 . 95 ; Pr  K N + 3 4 1 − 2 K N − √ 1+ 4 K 3 (1 − K N ) 1+ N 3 < P X < K N + 3 4 1 − 2 K N + √ 1+ 4 K 3 (1 − K N ) 1+ N 3  ≈ 0 . 99 ; Pr  K N + 3 4 1 − 2 K N − √ 1+ 4 K 5 (1 − K N ) 1+ N 5 < P X < K N + 3 4 1 − 2 K N + √ 1+ 4 K 5 (1 − K N ) 1+ N 5  ≈ 0 . 99 9 . Confidence limits computed by these formulas for different N and δ ar e depic ted by Figures 13-20. It is interesting to note that, in most situations, the confidence limits computed by our empirical for mulas almost coincide with the corr esp onding limits derived by Clopp er-Pearso n metho d. The numerical in vestigation of the cov erage probability of differen t co nfidence in terv als is sho wn in Fig ures 21 -27. It can b e seen that the empirical formulas have excellent coverage p erfor mance. References [1] Brown, L. D. Cai, T. DasGupta, A. (2001). In terv al estimation for a binomial prop ortion. Statistical Science 16:101-133. [2] Brown, L. D. Cai, T. D asGupta, A . (2002). Interv al estimation f or a binomial propor tion and asymptotic expansions. The Annals of Statistics 30:160-201. [3] Clopper C. J. Pearson E. S. (1934). The use of confidence or fiducial limits i llustrated in the case of the binomial . Bi ometrik a 26:404-413. [4] Clunies-Ross, C. W. (1958). In terv al estimation f or the parameter of a binomial distribution. Biometrik a 45:275-279. BINOMIAL CONFIDENCE INTER V AL 7 0 1 2 3 4 5 6 7 8 9 10 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 1. Confidence Interv al ( N = 1 0 , δ = 0 . 05.) 0 1 2 3 4 5 6 7 8 9 10 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 2. Confidence Interv al ( N = 1 0 , δ = 0 . 01.) 8 XINJIA CHEN, KEMIN ZHOU AN D JORGE L. ARA VENA 0 5 10 15 20 25 30 35 40 45 50 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 3. Confidence Interv al ( N = 5 0 , δ = 0 . 05.) 0 5 10 15 20 25 30 35 40 45 50 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 4. Confidence Interv al ( N = 5 0 , δ = 0 . 01.) BINOMIAL CONFIDENCE INTER V AL 9 0 10 20 30 40 50 60 70 80 90 100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 5. Co nfidence Interv al ( N = 1 00 , δ = 0 . 05 .) 0 10 20 30 40 50 60 70 80 90 100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 6. Co nfidence Interv al ( N = 1 00 , δ = 0 . 01 .) 10 XINJIA CHEN, KEMIN ZHOU AN D JORGE L. ARA VENA 0 50 100 150 200 250 300 350 400 450 500 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 7. Co nfidence Interv al ( N = 5 00 , δ = 0 . 05 .) 0 50 100 150 200 250 300 350 400 450 500 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 8. Co nfidence Interv al ( N = 5 00 , δ = 0 . 01 .) BINOMIAL CONFIDENCE INTER V AL 11 0 100 200 300 400 500 600 700 800 900 1000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 9. Co nfidence Interv al ( N = 1 000 , δ = 0 . 0 5 .) 0 100 200 300 400 500 600 700 800 900 1000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 10. Co nfidence Interv a l ( N = 1000 , δ = 0 . 01.) 12 XINJIA CHEN, KEMIN ZHOU AN D JORGE L. ARA VENA 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 11. Co nfidence Interv a l ( N = 5000 , δ = 0 . 05.) 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by formula (3) Upper limit by formula (4) Figure 12. Co nfidence Interv a l ( N = 5000 , δ = 0 . 01.) BINOMIAL CONFIDENCE INTER V AL 13 0 5 10 15 20 25 30 35 40 45 50 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by empirical formula Upper limit by empirical formula Figure 13. Confidence Interv al ( N = 50 , δ = 0 . 05 .) 0 5 10 15 20 25 30 35 40 45 50 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by empirical formula Upper limit by empirical formula Figure 14. Confidence Interv al ( N = 50 , δ = 0 . 01 .) 14 XINJIA CHEN, KEMIN ZHOU AN D JORGE L. ARA VENA 0 10 20 30 40 50 60 70 80 90 100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by empirical formula Upper limit by empirical formula Figure 15. Co nfidence Interv a l ( N = 100 , δ = 0 . 0 5 .) 0 10 20 30 40 50 60 70 80 90 100 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by empirical formula Upper limit by empirical formula Figure 16. Co nfidence Interv a l ( N = 100 , δ = 0 . 0 1 .) BINOMIAL CONFIDENCE INTER V AL 15 0 50 100 150 200 250 300 350 400 450 500 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by empirical formula Upper limit by empirical formula Figure 17. Co nfidence Interv a l ( N = 500 , δ = 0 . 0 5 .) 0 50 100 150 200 250 300 350 400 450 500 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by empirical formula Upper limit by empirical formula Figure 18. Co nfidence Interv a l ( N = 500 , δ = 0 . 0 1 .) 16 XINJIA CHEN, KEMIN ZHOU AN D JORGE L. ARA VENA 0 100 200 300 400 500 600 700 800 900 1000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by empirical formula Upper limit by empirical formula Figure 19. Co nfidence Interv a l ( N = 1000 , δ = 0 . 05.) 0 100 200 300 400 500 600 700 800 900 1000 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Number of Successful Trials Confidence Limits Lower limit (Clopper−Pearson) Upper limit (Clopper−Pearson) Lower limit by empirical formula Upper limit by empirical formula Figure 20. Co nfidence Interv a l ( N = 1000 , δ = 0 . 01.) BINOMIAL CONFIDENCE INTER V AL 17 50 100 150 200 250 300 350 400 450 500 10 −3 10 −2 10 −1 10 0 Sample Size N 1 − Coverage Probability A B C Figure 21. Err or P robability ( P X = 0 . 5 , δ = 0 . 05 . A – Nor mal, B – Empirica l, C – Rigorous ) 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 10 −3 10 −2 10 −1 10 0 Sample Size N 1 − Coverage Probability A B C Figure 22. E rror Pr obability ( P X = 0 . 01 , δ = 0 . 05. A – Normal, B – Empirica l, C – Rigorous) 18 XINJIA CHEN, KEMIN ZHOU AN D JORGE L. ARA VENA 50 100 150 200 250 300 350 400 450 500 10 −4 10 −3 10 −2 10 −1 10 0 Sample Size N 1 − Coverage Probability A B C Figure 23. Error Pr obability ( P X = 0 . 5 , δ = 10 − 2 . A – Normal, B – Empirica l, C – Rigorous) 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 10 −4 10 −3 10 −2 10 −1 Sample Size N 1 − Coverage Probability A B C Figure 24. Error P robability ( P X = 0 . 01 , δ = 10 − 2 . A – Nor- mal, B – Empirica l, C – Rigor ous) BINOMIAL CONFIDENCE INTER V AL 19 100 200 300 400 500 600 700 800 900 1000 10 −5 10 −4 10 −3 10 −2 10 −1 10 0 Sample Size N 1 − Coverage Probability A B C Figure 25. Error Pr obability ( P X = 0 . 5 , δ = 10 − 3 . A – Normal, B – Empirica l, C – Rigorous) 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 10 −6 10 −5 10 −4 10 −3 10 −2 10 −1 Sample Size N 1 − Coverage Probability A B C Figure 26. Erro r Probability ( P X = 10 − 2 , δ = 10 − 3 . A – Normal, B – Empirical, C – Rigoro us) 20 XINJIA CHEN, KEMIN ZHOU AN D JORGE L. ARA VENA 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 x 10 6 10 −5 10 −4 10 −3 10 −2 10 −1 Sample Size N 1 − Coverage Probability A B C Figure 27. Erro r Probability ( P X = 10 − 5 , δ = 10 − 3 . A – Normal, B – Empirical, C – Rigoro us) [5] Hald, A. (1952). Statistical The ory with Engine ering Applic ations , pp. 697-700, John Wiley and Sons. [6] John, N. Kotz, L. S. Kemp, A. W. (1992 ) Univariate Discr ete D istributions , 2rd ed., pp. 124-130, Wiley . [7] Massart, P . (1990). The tigh t constan t in the Dv oretzky-Kiefer-W olfowitz inequalit y . The Annals of Probability 18:1269-1283. Dep ar tm ent of Elect rical and Computer E ngineering , Louis iana St a te Un iversity, Ba ton Rouge, LA 70803 E-mail addr ess : chan@ece.lsu .edu, kemin@ece.lsu. edu, aravena@ece .lsu.edu