Scatter and regularity imply Benfords law... and more

Scatter and regularit y imply Benford’s La w... and more N. Gauvrit ∗ L ab or atoir e Andr ´ e R evuz, Un iversity Paris VII , F r anc e J.-P . D elaha y e L ab or atoir e d’Informa t ique F ondamentale, USTL, Lil le, F ra nc e Abstract A random v aria ble (r.v.) X is s aid to follo w Benford’s la w if log ( X ) is uniform mo d 1. Man y exp erimen ta l data sets prov e to follow an appro ximate v ersion of it, a nd so do man y mathematical series and con tin uous random v ariables. This phenomenon receiv ed some in terest, and sev eral explanatio ns ha ve b een put forw ard. Most of them fo cus on specific dat a , dep ending on strong assumptions, often link ed with the log function. Some authors hin ted - implicitly - that the tw o most imp ortant c harac- teristics of a random v ariable when it comes to Benford are regularit y and scatter. In a first part, we pro v e tw o theorems, making up a forma l v ersion of this in tuition: s cattered and regular r .v.’s do appro ximately follow Benford’s law. The pro ofs only need simple mathematical to ols, making the analysis easy . Previous explanations th us b ecome corollaries of a more general and simpler one. These results suggest that Benford’s law do es not dep end on prop erties link ed with the log function. W e th us prop ose and test a g eneral v ersion of the Benford’s law. The success of these tests may b e view ed as an a p osteriori v alidation o f the analysis form ulated in the first part. Key wor ds: Benford’s law, scatter, digit analysis JEL co de C16 2010 MSC: 00A06, 47N30 ∗ Corresp o nding author Email addr esses: adems @free. fr (N. Gauvr it), jpd @lifl. fr (J.-P . Delahaye) Pr eprint submitte d to Mathematic al So cial Scienc es Novemb er 5, 2018 1. In tro duction First noticed b y New com b (1881), and again later by Benford (1938), the so-called Benford’s law states that a sequence of ” r andom” n umbers should b e suc h that their logarithms are uniform mo d 1. As a consequence , the first non-zero digit of a sequence of ”random” n um b ers is d with probability log  1 + 1 d  , an unexp ectedly non- uniform probability law. log here stands for the base 10 logarithm, but an easy generalisation follow s: a random v a riable (r.v.) conforms to b ase b Benford’s la w if its base b logarithm log b ( X ) is uniform mo d 1. Lolb ert (2008) r ecently prov ed that no r.v. follo ws base b Benford’s law fo r all b. Man y exp erimen tal data roughly conform to Benford’s law (most of whic h no more than r oughly). Ho w ev er, the v ast ma jority of real da ta sets t ha t hav e b een tested do not fit this law a t all. F or instance, Scott and F asli (2001) rep orted that only 1 2 .6% of 23 0 real data sets passed the test for Benford’s la w. In his seminal pap er, Benford (1938) tested 20 data sets including lak es areas, length of rive rs, p opulat io ns, etc., half of whic h did not confo r m to Benford’s law . The same is true of mathematical sequences or con tinuous r.v.’s. F or ex- ample, binomial arrays  n k  , with n ≥ 1 , k ∈ { 0 , ..., n } , tend tow ard Benford’s la w (Diaconis , 1977), whereas simple sequences suc h as (10 n ) n ∈ N ob viously don’t. In spite of all this, Benford’s law is a ctually used in the so-called ”digital analysis” to detect anomalies in pricing (Sehit y et al., 2005) or fra uds, for instance in accoun ting rep orts (Dra ke and Nigrini, 2000) or campaign finance (Cho and Gaines , 2007). F ak ed dat a indeed usually depart from Benford’s la w more than real ones (Hill, 1988). Ho wev er, Hales et al. (2008) advise caution, arguing that real dat a do not alw ay s fit the law . Man y explanations hav e b een put forw ar d to elucidate the app earance of Benford’s la w on natural or mathematical data. Some authors fo cus on pa r - ticular random v ariables (Engel and Leuen b erger , 2003), sequence (Jo lissaint, 2005), real data (Burk e and Kincanon , 1991), or orbits of dynamical systems (Berger et al., 2004). As a rule, other explanations assume special prop erties of the data. Hill (1995 b) or Pinkham (1961 ) sho ws that scale in v a riance im- plies Benford’s la w. Base in v ariance is an o ther sufficien t condition (Hill, 1995a). Mixtures of uniform distributions (Jan vresse and Delarue, 2004) 2 also conform to Benford’s law, and so do the limits of some random pro- cesses (Sh ¨ urger, 2008). Multiplicativ e pro cesses ha v e b een men tioned as w ell (Pietronero et a l., 2001). Eac h of these explanations accoun ts for some ap- p earances of data fitting Benford’s law, but lac ks generalit y . While lo oking for a truly general explanation, some authors noticed that data sets are more like ly to fit Benfor d’s la w if they w ere scattered enough. More precisely , a sequence should ”co ve r sev eral orders of magnitude”, a s Raimi (1976) expres sed it. Of course, scatter alo ne is no sufficien t condition. The sequence 0.9, 9, 90, 900... indeed co v ers sev eral orders of magnitude, but is far from conforming to Benford’s law. The con tinuous random v aria bles that a r e known to fit Benford’s law usually presen t some ” regularit y”: ex- p onen tial densities, normal densities, or lognormal densities are of this kind. In v ar ia nce assumptions (base-inv ariance or scale-in v ariance) lead to ”regu- lar” densities and so do cen tral limit-lik e theorem assumptions of mixture. Some tec hnical explanations ma y b e view ed as a mathematical expression of the idea that a random v ariable X is more lik ely to confor m to Benford’s la w if it is regular and scattered enough. (Mardia and Jupp, 2000, Exam- ple 4.1.) link ed Benford’s la w to P oincar ´ e’s theorem in circular statistics, and Smith (2007) expresse d it in terms of F ourier tra nsforms and signal pro cessing. Ho wev er, a non exp ert reader would hardly not ice the smo oth- and-scattered implications of these deve lo pments. Though scatter has b een explicitly mentioned and regularity allusiv ely ev ok ed, the idea t ha t scatter and regularit y (in a sense that will b e made clear further) may actually b e a s ufficient explanation for Benford’s phenomenon related to contin uo us r.v.’s ha v e nev er b een fo rmalized in a simple wa y , to our kno wledge, except in a recen t article by F ewster (2009). In this pap er, F ews ter h yp othesizes that ” any distribution [...] that is r e as onably sm o oth and c overs sever al or ders o f magni tude is almost guar ante e d to ob ey Ben for d’s law.” He then defines a smo othing pro cedure fo r a r.v. X based on [ π 2 ( x )] ′′ , π b eing the pro babilit y densit y function (henceforth p.d.f. ) of log ( X ) , and illustrates with a few elo quen t examples that under smo o thness a nd scatter constrain ts, a r.v. cannot depart m uch from Benford’s law. Ho w eve r, no theorem is given that w ould formalise this idea. In the first part of this pap er, w e prov e a theorem from whic h it follows that scatter and regularity can b e mo delled in suc h a w a y that they , alone, imply r ough compliance to Benford’s la w (again: real da t a usually do not p erfectly fit Benfo r d’s law, irrespectiv e of the sample size). It is not surprising that man y data sets or ra ndom v ariables samples ar e 3 scattered and regular hence our explanation o f Benford’s phenomena corrob- orates a widespread in tuition. The pro of o f this theorem is straigh tfor ward and requires only basic ma t hematical to ols. F urthermore, as we shall see, sev eral of the existing explanations can b e understo o d a s corollaries of ours. Our explanation encompasses more sp ecific ones, and is fa r simpler to un- derstand and to pro ve . Scatter and regularit y do not presuppo se an y lo g-related prop erties (suc h as the prop erty of log-normality , scale-in v ariance, or m ultiplicative prop er- ties). F or this reason, if w e are right, Benford’s law should also admit ot her v ersions. W e set that a r.v. X is u -Benford for a f unction u if u ( X ) is uni- form mo d 1. The classical Benford’s la w is th us a sp ecial case of u -Benford’s la w, with u = log. W e test real data sets and mathematical sequences for ” u - Benfordness” with v arious u , and test a second theorem ec hoing the first one. Most data conform to u - Benfo r d’s law for differen t u , whic h is an argumen t in fav our of our explanation. 2. Scatter and regularit y: a k ey t o Benford The basic idea at the r o ot of t heorem 1 ( b elo w) is t wofold. First, w e hypothesize that a con tinuous r.v. X with densit y f is almost uniform mo d 1 as so on as it is scattered and regular. More precisely , an y f that is non-decreasing o n ] − ∞ , a ], and then non-increasing on [ a, + ∞ [ (for regularit y) and suc h that its maxim um m = sup( f ) is ” small” ( f or scatter) should corresp ond to a r.v. X approaching uniformit y mo d 1 . Figure 1 il- lustrates this idea. 4 Figure 1 — Illustration of the idea tha t a regular p.d.f. is almost b ound to giv e rise to uniformit y mo d 1. The strip es — restrictions of the densit y on [ n, n + 1] — o f the p.d.f. of a r.v. X are stac k ed to form the p.d.f. of X mo d 1 . The slop es partly comp ensate, so that the resulting p.d.f. is almo st uniform. If the initia l p.d.f. is linear on ev ery [ n, n + 1] , the comp ensation is p erfect. Second, note t hat if X is scattered a nd regular enough, so should b e log ( X ). These tw o ideas ar e fo rmalized and prov ed in theorem 1. Henceforth, for any real n umber x, ⌊ x ⌋ will denote the greatest in teger not exce eding x, and { x } = x − ⌊ x ⌋ . Any positive x can b e written as a pro duct x = 1 0 ⌊ log x ⌋ . 10 { log x } , and the Benford’s law may b e rephrased as the uniformit y o f the random v aria ble { log ( X ) } . Theorem 1. L et X b e a c on tinuous p ositive r andom variable with p.d.f. f such that I d.f : x 7− → xf ( x ) c onfo rm s to the two fol lowing c onditions : ∃ a > 0 such that (1) max( I d.f ) = m = a.f ( a ) and (2) I d.f is nonde c r e asing on ]0 , a ] , a nd nonincr e a s i ng on [ a, + ∞ [ . Then, f o r any z ∈ ]0 , 1] , | P ( { lo g X } < z ) − z | < 2 ln (10) m. In p articular, ( X n ) b eing a s e quenc e of c ontinuous r.v.’s with p.d.f. f n sat- isfying these c onditions and such that m n = max ( I d.f n ) − → 0 , { log ( X n ) } c onv e r ges towar d uniformity on [0 , 1[ in law. 5 Proo f. W e first prov e t ha t for an y contin uous r.v. Y with densit y g such that g is nondecreasing on ] − ∞ , b ] , and then nonincreasing o n [ b, + ∞ [ , the follo wing holds: ∀ z ∈ ]0 , 1] , | P ( { Y } < z ) − z | < 2 M where M = g ( b ) = sup ( g ) . W e may supp o se without loss of generality that b ∈ [0 , 1[ . Let z ∈ ]0 , 1[ (the case z = 0 is obv io us). Put I n,z = [ n, n + z [ . F or an y in teger n ≤ − 1 , 1 z Z I n,z g ( t ) dt ≤ Z n +1 n g ( t ) dt. Th us 1 z X n ≤− 1 Z I n,z g ( t ) dt ≤ Z 0 −∞ g ( t ) dt. F or an y inte ger n ≥ 2 , 1 z Z I n,z g ( t ) dt ≤ Z n + z n − 1+ z g ( t ) dt, so 1 z X n ≥ 2 Z I ,z g ( t ) dt ≤ Z + ∞ 1+ z g ( t ) dt. Moreo v er, R I 0 ,z g ≤ z M a nd R I ,z g ≤ z M . Hence, 1 z X n ∈ Z Z I n,z g ≤ Z ∞ −∞ g + 2 M . W e prov e in the same fashion t hat 1 z X n ∈ Z Z I n,z g ≥ Z ∞ −∞ g − 2 M . Since P Z R I n,z g = P ( { Y } < z ) , z < 1 and R ∞ −∞ g = 1 , the result is pro v ed. No w, applying this to Y = log ( X ) pro v es theorem 1 . Remark 1. The conv ergence theorem is still v alid if w e accept f to hav e a finite num b er of mono ton y c hanges, provide d this num b er do es not exceed a previously fixed k . The pro o f is straightforw a rd. Remark 2. The assumptions made on I d.f ma y b e seen as a measure of scatter and regula rit y f o r X , adjusted for our purp ose. 6 3. Examples 3.1. T yp e I Par eto A con tin uous r.v. X is type I P ar eto with parameters α and x 0 ( α, x 0 ∈ R ∗ + ) iff it admits a density function f x 0 ,α ( x ) = αx α 0 x α +1 I [ x 0 , + ∞ [ Besides its classic al use in income and w ealth mo delling, type I Pareto v ariables a rise in hyd ro logy and astronom y (Paolella, 2006, page 252 ) . The function I d.f = g : x 7− → αx α 0 x α I [ x 0 , ∞ [ is decreasing. Its ma ximum is sup ( I d.f ) = I d.f ( x 0 ) = α. Therefore, X is nearly Benford-lik e, in the exten t that | P ( { lo g X } < z ) − z | < 2 ln(10) α. 3.2. T yp e I I Par e to A r.v. X is type I I P areto with para meter b > 0 iff it a dmits a densit y function defined b y f b ( x ) = b (1 + x ) b +1 I [0 , + ∞ [ It arises in a so-called mixtur e mo del , with mixing comp onents b eing gamma distributed r .v.’s sequences. The function I d.f b = g b : x 7− → bx (1+ x ) b +1 I [0 , + ∞ [ is C ∞ ( R + ) , with deriv ative g ′ b ( x ) = b (1 − bx ) ( x + 1) b +2 , whic h is p ositiv e whenev er x < 1 b , then negat ive. F rom this result we deriv e sup g b = g b  1 b  = 1  1 + 1 b  1+ b =  b 1 + b  b +1 , since ln "  b 1 + b  b +1 # = ( b + 1) [ln b − ln ( b + 1)] , whic h tends tow ard −∞ when b tends t ow ard 0, sup g b − → b − → 0 0 . Theorem 1 applies. It fo llo ws that X confo r m to w ard Benford’s la w when b − → 0 . 7 3.3. L o gnorma l distributions A r.v. X is log normal iff log ( X ) ∼ N ( µ, σ 2 ). Lognormal distributions ha v e b een related to Benford ( ? ). It is easy to prov e that whenev er σ − → ∞ , X tends to w ard Benford’s law. Although the pro o f may use differen t to ols, a straigh tfo rw ard w a y to do it is t heorem 1. One classical explanation of Benford’s law is that man y data sets are ac- tually built through m ultiplicative pro cesses (Pietronero et al., 20 01). Th us, data may b e see n a s a pro duct of man y small effects . This ma y be mo delled b y a r.v. X that ma y b e written as X = Y i Y i , Y i b eing a sequence of random v ariables. Using the log transfor ma t io n, this leads to log ( X ) = P log ( Y i ) . The multiplic ative c entr al - l i m it the or em therefore pro ve s that, under usual assumptions, X is b ound to b e almost lognormal, with log ( X ) ∼ N ( µ, σ 2 ) , and σ − → ∞ , thu s ro ughly conforming to Benford, as an application of theorem 1. 4. Generalizing Benford If we are righ t to think that Benford’s law is to b e understo o d as a conse- quence of mere scatter and regularity , instead of sp ecial characteristic s link ed with m ultiplicative, scale-in v ar ia nce, or whatev er log- related prop erties, we should b e able to state, prov e, a nd c hec k on real data sets, a generalized v ersion of the Benford’s la w w ere some function u replaces the log . Indeed, our basic idea is that X b eing scattered and regular enough im- plies log ( X ) t o b e scattered and regular as well, so that log ( X ) should b e almost uniform mo d 1. The same should be true of any u ( X ) , u being a function preserving scatter and regularity . Actually , some u should ev en b e b etter shots than log , since log reduces scatter o n [1 , + ∞ [ . First, let us set out a generalized v ersion of theorem 1, the pro of of whic h is closely similar to tha t of theorem 1. Theorem 2. L et X b e a r.v. taking values in a r e a l interval I , with p.d.f. f . L et u b e a C 1 incr e asing function I − → R , such that f u ′ : x 7− → f ( x ) u ′ ( x ) c onfo rms to the fol lowing: ∃ a > 0 such that (1) max  f u ′  = m = f u ′ ( a ) and 8 (2) f u ′ is non- de cr e asing on ]0 , a ] , and non- incr e asing o n [ a, + ∞ [ ∩ I . Then, for al l z ∈ [0 , 1[ , | P ( { u ( X ) } < z ) − z | < 2 m. In p articular, if ( X n ) is a se quenc e of such r.v.’s with p.d.f. f n and max ( f n /u ′ ) = m n , and lim + ∞ ( m n ) = 0 , then { u ( X n ) } c onver ges i n law towar d U ([0 , 1[) when n − → ∞ . A r.v. X suc h that { u ( X ) } ∼ U ([0 , 1[) will b e said u -Benford henceforth. 4.1. Se quenc e Although our t wo theorems o nly apply to con tinuous r.v.’s, the underly- ing in tuition that log - Benford’s law is only a sp ecial case (hav ing, how ever, a sp ecial interes t thanks to its implication in terms of leading-dig its interpre- tation) of a more general law do es also apply to sequence. In this section, w e exp erimentally t est u - Benfor dness for a f ew sequences ( v n ) and a four functions u. W e will use six mathematical sequences. Three of them, namely ( π n ) n ∈ N , prime nu mbers ( p n ), and ( √ n ) n ∈ N are known not to follow Benfor d. The three others, ( n n ) n ∈ N , ( n !) n ∈ N and ( e n ) n ∈ N conform to Benfor d. As for u, we will fo cus on four cases: x 7− → log [log ( x ) ] x 7− → log ( x ) x 7− → √ x x 7− → π x 2 The first o ne increases v ery slow ly , so w e ma y exp ect that it will not w ork p erfectly . The second leads to the classical Benford’s law. The π co efficien t of the last u allows us to use in teger n umbers, for whic h { x 2 } is nil. The result of the exp erimen t is giv en in T able 1. v n log ◦ log ( v n ) log ( v n ) √ v n π v 2 n √ n ( N = 10 000 ) 68 . 90 ( . 000 ) 45 . 90 ( . 000) 4 . 94 ( . 000) 0 . 02 ( . 000) π n ( N = 10 000) 44 . 08 ( . 000 ) 26 . 05 ( . 000) 0 . 19 (1 . 000 ) 0 . 80 ( . 544) p n ( N = 10 000 ) 53 . 92 ( . 000 ) 22 . 01 (0 . 000) 0 . 44 (0 , 990) 0 . 69 ( . 719) e n ( N = 1 000) 6 . 91 (0 . 000 ) 0 . 76 (1 . 000) 0 . 63 ( . 8 1 5) 0 . 79 ( . 56 0) n ! ( N = 1 00 0) ( ∗ ) 7 . 39 ( . 000) 0 . 5 8 ( . 8 87) 0 . 61 ( . 844) 0 . 90 ( . 387) n n ( N = 1 000) ( ∗ ) 7 . 45 ( . 000) 0 . 80 ( . 543) 16 . 32 ( . 000) 0 . 74 ( . 646 ) 9 T a ble 1 — Results of the K olmogorov-Smirno v tests applied on { u ( v n ) } , with four differen t functions u ( columns) and six sequences (lines). Eac h sequence is tested through its first N terms (from n = 1 to n = N ) , with an exception for log ◦ log ( n n ) and log ◦ log ( n !) , f o r whic h n = 1 is not considered. Eac h cell displa ys the Ko lmogorov - Smirnov z and the corresp onding p v alue. The sequences ha v e b een arranged according to the sp eed with whic h it con v erges to + ∞ ( and so a re t he functions u ). None of the six sequences is log ◦ log -Benford (but a faster div ergen t sequence suc h as  10 e n  w ould do). Only the last three are log-Benford. These are the sequenc es go ing to ∞ faster than a n y p olynomial. Only o ne sequence ( n n ) do es not satisfy √ . - Benfordness. Ho we ver, this can b e understo o d as a pathological case, since √ n n is integer whenev er n is eve n, or is a perfect square. Doing the same Kolmogorov-Smirno v test with o dd n umbers not b eing p erfect squares giv es z = 0 , 45 and p = 0 , 987 , showing no discrepancy with √ . -Benfordness for ( n n ) . All six sequences are π . 2 -Benford. Putting aside the case of √ n n , what T able 1 rev eals is that the conv er- gence speed of u ( v n ) completely determines the u -Benfordness of ( v n ) . More precisely , it seem s that ( v n ) is u -Benford whenev er u ( v n ) increases as fast as √ n, and is not u -Benford whenev er u ( v n ) increase as slo wly as ln ( n ) . Of course, this rule-of-thum b is not to b e tak en as a theorem. Obvious ly enough, one can actually decide to increase or decreas e conv ergence sp eed of u ( v n ) without c hanging { u ( v n ) } , adding or substracting ad ho c integer n um b ers. Nev ertheless, this observ ation suggests that w e giv e a closer lo ok at se- quence f ( n ) , where f is an increasing and concav e real function conv erging to ward ∞ , a nd lo ok for a condition for ( { f ( n ) } ) n to conv erge to unifor- mit y . An in tuitive idea is tha t ( { f ( n ) } ) n will depart f rom uniformit y if it do es not increase fast enough: w e may define brack ets of integers — namely [ f − 1 ( n ) , f − 1 ( n + 1) − 1[ ∩ N , within whic h ⌊ f ( n ) ⌋ is constan t, and of course { f ( n ) } increasing. If these brac ke ts are ” to o lar g e”, the relativ e heigh t o f the last considered brac k et is so imp ortan t that it ov ercomes the first terms of the sequence f (0) , ..., f ( n ) mo d 1. In that case, there is no limit to the prob- abilit y distribution of ( { f ( n ) } ) . The we ight of the brac kets should therefore b e small relativ e to f − 1 ( n ) , whic h ma y b e written as f − 1 ( n ) − f − 1 ( n + 1) f − 1 ( n ) − → ∞ 0 . 10 Pro vided that f is regular, this leads to ( f − 1 ) ′ ( x ) f − 1 ( x ) − → ∞ 0 , or  ln  f − 1 ( x )  ′ − → ∞ 0 . F unctions f : x 7− → x α , α > 0 satisfy this condition. Any n α should then sho w a uniform limit probabilit y la w, except for pathological cases ( α ∈ Q ) . T a king α = 1 π giv es (with N = 100 0 ) , a Kolmo g oro v-Smirnov z = 1 , 331 , and a p -v alue 0.058, whic h means there is no significan t discrepancy from uniformit y . On the other hand, the log function whic h do es not confo rm to this condition is suc h that { lo g ( n ) } is not uniform, confirming once again our rule-of-thum b conjecture. 4.2. R e al data W e test three data sets for u -Benfordness using a Kolmogorov-Smirno v test for uniformity . First dat a set is the op ening v alue of the D o w Jones, the first da y of eac h mon th from O cto b er 1928 to Nov ember 2007. The second and t hir d are country a reas expressed in millions of square-km 2 and the p opulations of the differen t coun tries, as estimated in 2008, expressed in millions of inhabitants. The t wo last seque nces are provided by the CIA 1 . T a ble 2 displa ys the results. log ◦ log ( v n ) log ( v n ) √ v n π v 2 n Do w Jones ( N = 950) 5 . 90 ( . 000) 5 . 20 ( . 0 00) 0 . 75 ( . 635) 0 . 44 ( . 99 2) Area pa ys ( N = 256) 1 . 94 ( . 001) 0 . 51 ( . 9 59) 0 . 89 ( . 404) 1 , 88 ( . 002) P opulations ( N = 242) 3 . 39 ( . 000 ) 0 . 79 ( . 568) 0 . 83 ( . 494 ) 0 . 42 ( . 9 94) T a ble 2 — Results of the K olmogorov-Smirno v tests applied on { u ( v n ) } . This table confirms our analysis: classical Benfordness is actually less often b o rne o ut than √ . -Benfordness on these data. The la st column show s that our previous conjectured rule has exceptions: div ergence sp eed is no t an absolute criterion by itself. F or country areas, the fast growin g u : x 7− → π x 2 giv es a discrepancy from uniformity , whereas the slo w-growing log do es not. Ho w ev er, allo wing for exceptions, it is still a go o d rule-of-thum b. 1 ht tp:// www.cia.gov/librar y/publications/the- world-factbo ok/do cs/rankorderguide.h tml 11 4.3. Continuous r.v .’s Our theorems apply on con tin uous r.v.’s. W e no w fo cus on three examples of suc h r.v.’s, with the same u as ab o ve (except for log ◦ log , which is not defined ev erywhere on R ∗ + ): the uniform densit y on ]0 , k ] ( k > 0) , exp onential densit y , a nd absolute v alue of a normal distribution. 4.3.1. Uniform r.v.’s It is a kno wn fact that a unifor m distribution X k on ]0 , k ] ( k > 0 ) do es not approac h classical Benfordness, eve n as a limit. On ev ery bra c k et [10 j − 1 , 10 j − 1[ , the leading digit is uniform. Therefore, taking k = 10 j − 1 leads to a uniform (and not lo g arithmic) distribution for leading digits, whatev er j migh t b e. The densit y g k of √ X k is g k ( x ) = 2 x k , x ∈ i 0 , √ k i and g k ( x ) = 0 otherwise. It is an increasing f unction on ] − ∞ , √ k ], decreasing on [ √ k , + ∞ [ with maximum 2 √ k − → 0 when k − → ∞ . Theorem 2 applies, sho wing t ha t X k tends tow ard √ . -Benfordness in la w. No w, theorem 3 b elow prov es t hat X k tends tow ar d u -Benfordness, when u ( x ) = π x 2 . Theorem 3. If X fol lows a uniform density on ]0 , k ] , { π X 2 } c o n ver ges in law towar d uniformity on [0 , 1[ when k − → ∞ . Proo f. Let X ∼ U (]0 , k ]) . The p.d.f. g of Y = π X 2 is g ( x ) = 1 2 a √ x , x ∈ ]0 , a 2 ] where a = k √ π . The c.d.f. G of Y is then G ( x ) = √ x a , x ∈ ]0 , a 2 ] . Let now δ ∈ ]0 , 1[ . Call P δ the probability that { Y } < δ . ⌊ a 2 − δ ⌋ X j =0 G ( j + δ ) − G ( j ) ≤ P δ ≤ ⌊ a 2 − δ ⌋ +1 X j =0 G ( j + δ ) − G ( j ) 12 1 a ⌊ a 2 − δ ⌋ X j =0 p j + δ − p j ≤ P δ ≤ 1 a ⌊ a 2 − δ ⌋ +1 X j =0 p j + δ − p j The square-ro ot function b eing conca ve, p j + δ − p j ≥ δ 2 √ j + δ and, for a n y j > 0 , p j + δ − p j ≤ δ 2 √ j . Hence, δ 2 a ⌊ a 2 − δ ⌋ X j =0 1 √ j + δ ≤ P δ ≤ 1 a    √ δ + ⌊ a 2 − δ ⌋ +1 X 1 δ 2 √ j    δ 2 a ⌊ a 2 − δ ⌋ X j =0 1 √ j + δ ≤ P δ ≤ √ δ a + δ 2 a ⌊ a 2 − δ ⌋ +1 X 1 1 √ j x 7− → 1 √ x b eing decreasing, ⌊ a 2 − δ ⌋ X j =0 1 √ j + δ ≥ ⌊ a 2 − δ ⌋ +1+ δ Z δ 1 √ t dt ≥ 2 h p ⌊ a 2 − δ ⌋ + 1 + δ − √ δ i and ⌊ a 2 − δ ⌋ +1 X 1 1 √ j ≤ Z ⌊ a 2 − δ ⌋ +1 0 1 √ t dt ≤ 2 h p ⌊ a 2 − δ ⌋ + 1 i . So, δ a h p ⌊ a 2 − δ ⌋ + 1 + δ − √ δ i ≤ P δ ≤ δ a h p ⌊ a 2 − δ ⌋ + 1 i . As a consequence , for an y fixed δ , lim a − →∞ ( P δ ) = δ , a nd { π X 2 } con v erges in law to uniformit y on [0 , 1[. 13 4.3.2. Exp onential r.v.’s Let X λ b e an exp onential r.v. with p.d.f. f λ ( x ) = λ exp ( − λx ) ( x ≥ 0 , λ > 0) . Engel and Leuen b erger [200 3] demonstrated that X λ tends to ward the Ben- ford’s law when λ − → 0 . The p.d.f. of √ X λ is x 7− → 2 λx exp ( − λx 2 ) , whic h increase s on  0 , 1 2 λ  and then decreases. Its maxim um is exp  − 1 4 λ  . Theorem 2 th us applies, sho wing that X λ is √ . -Benford as a limit when λ − → 0 . Finally , theorem 4 b elo w demonstrates that X λ tends tow ard u -Benfordness for u ( x ) = π x 2 as w ell. Theorem 4. If X ∼ E X P ( λ ) ( with p.d.f. f : x 7− → λ exp ( − λx )) , then Y = π X 2 c onv e r ges towar d uniformity mo d 1 when λ − → 0 . Proo f. Let X b e suc h a r.v. Y = π X 2 has densit y g with g ( x ) = µ 2 √ x exp  − µ √ x  , x ≥ 0 where µ = λ √ π . The Y c.d.f. G is th us, fo r all x ≥ 0 G ( x ) = 1 − e − µ √ x . Let P δ denote the probability that { Y } < δ , for δ ∈ ]0 , 1[ . P δ = ∞ X j =0 h e − µ √ j − e − µ √ j + δ i x 7− → exp ( − µ √ x ) b eing con v ex, δ µ 2 √ j + δ e − µ √ j + δ ≤ e − µ √ j − e − µ √ j + δ for any j ≥ 0 , and e − µ √ j − e − µ √ j + δ ≤ δ µ 2 √ j e − µ √ j for any j > 0 . Th us δ ∞ X j =0 µ 2 √ j + δ e − µ √ j + δ ≤ P δ ≤ 1 − e − µ √ δ + δ ∞ X j =1 µ 2 √ j e − µ √ j . 14 x 7− → 1 √ x exp ( − µ √ x ) b eing decreasing, δ ∞ X j =0 µ 2 √ j + δ e − µ √ j + δ ≥ δ Z ∞ √ δ µ 2 √ t e − µ √ t dt ≥ δ h − e − µ √ t i ∞ √ δ = δ e − µ √ δ , and 1 − e − µ √ δ + δ ∞ X j =1 µ 2 √ j e − µ √ j ≤ 1 − e − µ √ δ + δ Z ∞ 0 µ 2 √ t e − µ √ t dt ≤ 1 − e − µ √ δ + δ The t w o expressions tend tow ard δ when µ − → 0 , so that P δ − → δ. The pro of is complete. 4.3.3. Absolute value of a normal di s tribution T o test the absolute v alue of a normal distribution X with mean 0 and v ariance 10 8 , w e pick ed a sample of 2000 v alues a nd used the same pro cedure as for real data. It app ear s, a s sho wn in T able 3, that X significan tly departs from u -Benfordness with u = log and u = π . 2 , but not with u = √ .. log ( X ) √ X π X 2 U ([0 , k [) k − → ∞ NO YES YES E X P ( λ ) λ − → 0 YES YES YES |N ( 0 , 10 8 ) | 14 . 49 ( . 000) 0 . 647 ( . 797) 28 . 726 ( . 000) T a ble 3 — The ta ble display s if uniform distributions, exp onential distributions, and absolute v alue of a normal distribution, a r e u -Benford for differen t functions u, or not. The last line sho ws the results (and p -v alues) of the Kolmogorov-Smirno v tests applied to a 2 000-sample. It could b e read as ”NO; YES; NO”. As w e already noticed, the b est shot when one is lo oking for Benford seems to b e the square-ro o t rather than log . 15 5. Discussion Random v aria bles exactly conforming the Benford’s classical la w are rar e, although many do roughly approach the la w. Indeed, man y explanations hav e b een prop osed for this approx imate law to hold so often. These explanations in v olve complex characteris tics, sometimes directly related to log arithms, sometimes through m ultiplicativ e pr o p erties. Our idea — f o rmalized in theorem 1 — is mor e simple a nd general. The fact that real data often are r egula r and scattered is intuitiv e. What we pro v ed is an idea which has b een recen tly expressed b y F ewster [2009]: scatter and regularit y are actually sufficient condition to Benfordness. This fact thus pro vides a new explanation of Benford’s la w. Ot her expla- nations, of course, ar e acceptable as well. But it may b e a rgued that some of the most p opular explanations a re in fact corollaries of our theorem. As w e ha ve seen when studying P areto ty p e I I densit y , mixtures o f distributions ma y lead to regular and scattered densit y , to whic h theorem 1 applies. Th us, w e ma y argue that a mixture o f densities is nearly Benford b e c ause it is nec- essarily scattered and regular. In the same fashion, multiplic atio ns of effects lead to Benford-like densities, but also ( as the m ultiplicative central-limit theorem states) to regular and scattered densities. Apart from the fact that our explanation is simpler and (arguably) more general, a go o d argumen t in its f a v or is tha t Benfordness may b e generalized — unlik e log-related explanations. Scale in v ariance or m ultiplicativ e prop er- ties are log-related. But as we ha v e seen, Benfordness is not dep endan t on log, a nd can easily b e generalized. Actually , it seems that square ro ot is a b etter candidate than log. The historical imp o r tance o f log -Benfordness is of course due t o the implications in terms of leading digits whic h b ears no equiv alence with square-ro o t. References Benford, F., 1 9 38. The law of anomalous num b ers. Pro ceedings of the Amer- ican Philosophical So ciet y 78, 5 51–572. Berger, A., Bunimo vich, L ., Hill, T., 2004. One-dimensional dynamical sys- tems and b enford’s law. T ransactions of the American Mathematical So- ciet y 357, 197– 2 19. 16 Burk e, J., Kincanon, E., 1 991. Benford’s law and ph ysical constan ts: The distribution o f initial digit s. American Jo urnal of Ph ysics 5 9 , 952. Cho, W. K. T., Gaines, B. J., 2007. Breaking the (b enford) law: Statistical fraud detection in campaig n finances. The American Statistician 61, 218– 223. Diaconis, P ., 1977. The distribution of leading digits and unifo rm distribution mo d 1 . The Annals o f Probabilit y 5, 72 –81. Drak e, P . D ., Nigrini, M. J., 2000. Computer a ssisted analytical pro cedures using b enford’s law . Journal of Accoun ting Education 18, 127–1 4 6. Engel, H.- A., Leuen b erger, C., 2 003. Benford’s law f or expo nen tial random v ariables. Statistics and Probabilit y Letters 63 , 361– 365. F ews ter, R., 2009. A simple explanation of b enford’s law. The American Statistician 63, 26–32 . Hales, D. N., Sridharan, V., Radhakrishnan, A., Chakrav orty , S. S., Siha, S., 2008 . T esting the accuracy of employ ee-rep ort ed data : An inexp ensiv e alternativ e appro ac h to traditional metho ds. Europ ean Journa l of Op era- tional Research 189, 583– 5 93. Hill, T., 1988. Random-num b er guessing and the first-digit phenomenon. Psyc holo gical Rep o rts 62, 96 7–971. Hill, T., 1995a. Base-in v ariance implies b enford’s law. Pro ceedings of the American Mathematical So ciet y 123, 887–895. Hill, T., 19 95b. A statistical deriv ation of the significant-digit law. Statistical Science 10 , 354–363. Jan vresse, E., Delarue, T., 2004. F rom uniform distributions to b enford’s la w. Journal of Applied Probability 41, 1203– 1210. Jolissain t, P ., 2005. L o i de b enford, relations de r ´ ecurrence et suites ´ e quidistribu ´ ees. Elemen te der Mathematik 60, 10–18 . Lolb ert, T., 2008 . On the non- existence of a general b enford’s law. Mathe- matical So cial Sciences 55, 103–1 06. 17 Mardia, K. V., Jupp, P . E., 2000 . Directional statistics. Chic hester: Wiley . New com b, S., 1881. Note on the frequency of use of the different digits in natural num b ers. American Journal of Mathematics 4, 39–40. P aolella, M. S., 2006. F undamen tal probabilit y: A computational a ppro ac h. Chic hester: Wiley . Pietronero, L., T osatti, E., T osatti, V., V espignani, A., 2001. Explaining the unev en distribution o f n umbers in nat ure: The laws of b enford and zipf. Ph ysica A 29 3 , 297–304. Pinkham, R. S., 1961. On the distribution of first significan t digits. Annals of mathematical statistics 32 , 1223–12 3 0. Raimi, R . A., 1976. The first digit pro blem. The American Mathematical Mon thly 83, 521– 538. Scott, P . D., F asli, M., 2001. Benford’s law: An empirical in ves - tigation and a nov el explanation. Ph.D. thesis, CSM tec hnical re- p ort 349, Departmen t of Computer Science, U nivers ity of Es sex, h ttp://citeseer.ist.psu.edu/709593.h tml. Sehit y , T., Ho elzl, E., Kirc hler, E., 2005. Price dev elopmen ts after a nominal sho c k: Benford’s law and psyc holog ical pr icing after the euro in tro duction. In ternational Journal o f Researc h in Marke ting 2 2, 471–48 0 . Sh ¨ urger, K., 2008. Extensions o f blac k-sc holes pro cesses a nd b enford’s law. Sto c hastic Pro cesses and their Applications 118 , 1219–124 3. Smith, S. W., 2007. The scien tist and engineer’s guide to digital signal pro- cessing. http://www.ds pguide.com/. 18