Probability and Statistics: Essays in Honor of David A. Freedman

Institute of Mathematical Statistics COLLECTIONS V olume 2 Probabilit y and Statis tics: Essa ys in Ho no r of David A. F reedman Deb ora h Nolan a nd T erry Sp eed, Editors Institute of Mathematical Statistics Beachw o o d, Ohio, USA Institute of Mathematical Statistics Col le ctions Series Editor: An thon y D avison The pro duction of the Institute of Mathematic al Statistics Col le ctions is managed b y the IMS Office: Rong Che n, T reasurer and Elyse Gustafson, Executiv e Director. Library of Congress Con t r o l Num b er: 2007 939122 In ternational Standard Bo o k Num b er (10) 0-940600- 74-9 In ternational Standard Bo o k Num b er (13) 978-0-940 600-74- 4 In ternational Standard Serial Num b er 1939 -4039 Cop yrigh t c  2008 Institute of M athematical Statistics All righ ts reserv ed Prin ted in the United States of America Con ten ts Preface Deb or ah Nolan and T erry Sp ee d . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi PR OBABILITY Probability theory and its mo dels Paul Humph r eys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Dutc h b o ok in simple multiv ariate norma l predict i on: Another lo ok Morris L. Eaton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 A t ransient Mark ov c hain wi th finitel y many cutpo ints Nicholas James, R ussel l Lyons and Y uval Per es . . . . . . . . . . . . . . . . . . . . . . 24 Moments of conv ex distr i bution functions a nd co m pletely alt er nating sequences Alexand er Gne din and Jim Pitman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Brownian mo tion on disconnected sets, basic hypergeometri c functions, and some continue d fractions of Ramanujan Shankar Bhamidi, Steven N. Evans, R on Pele d and Pet er R alph . . . . . . . . . . . . 42 ST A TISTICS Characteri stics of ha nd and ma ch ine-assig ned scores to col lege st udents’ answers to o pen-ended tasks Stephen P. Klein . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 Alternative formulas f o r synthetic dual sy st em estima tion in t he 2000 census L awr ence Br own and Z hanyun Zhao . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 On t he histor y and use of some standard stati stical mo dels E. L. Lehm ann . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Counting the homeless in Los Angeles Co un ty Richa r d Berk , Brian Krie gler and Donald Ylvisaker . . . . . . . . . . . . . . . . . . . 127 Statisti cal a djustment for a measur e of healthy lifestyle doesn’t yield the truth ab out hor mone thera py Diana B. Petitti and Wansu Chen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 Multiple tests of a sso ciation with biolog i cal annotation metadat a Sandrine Dudoit, S¨ und¨ uz Kele¸ s and Mark J. van der L aan . . . . . . . . . . . . . . . 153 Multiv ariate data analysis: T he F rench way Susan Holmes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 The future of census cov erage surveys Kenneth Wachter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 Three months journeying of a Haw aiia n monk seal David R. Bril linger, Br ent S. Stewart and Charles L. Littnan . . . . . . . . . . . . . . 246 Pro jection pursuit for discret e data Persi Diac onis and Julia Salzman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 DNA Probabil ities in Pe ople v. Princ e : When a re racia l and ethnic stat istics relev a n t? David H. Kaye . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 iii iv Contents T esting ear thquake predict ions Br ad Luen and Philip B. Stark . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 Curse-of-dimensionali ty r evi sited: Collapse of the particle filter in very large scale systems Thomas Bengtsson, Peter Bickel and Bo Li . . . . . . . . . . . . . . . . . . . . . . . . 316 Higher or der influence functions a nd minimax estima tion of no nl inear functionals James R obins, L ingling Li, Eric Tchetgen and A ad van der V aart . . . . . . . . . . . 335 Preface This volume is our tr ibute to David A. F reedman, whom we reg ard as one o f the great statisticians of our time. He r e ceived his B.Sc. deg ree from McGill Univ ersity and his Ph.D. from Princeto n, and joined the Department of Statistics o f the Uni- versit y of California, Ber keley , in 1962, where, apart from s abbaticals, he has been ever since. In a career of ov er 45 years, David has made many fine contributions to proba- bilit y and statistical theory , and to the application of statistics. His ea rly r esearch was on Markov chains and mar tingales, and tw o to pics with which he has had a lifelong fas cination: exchangeability and De Finetti’s theorem, and the cons istency of Bayes estimates. His asymptotic theory for the b o o tstrap was also highly influ- ent ial. David was elected to the America n Academy of Arts and Scie nc e s in 1991, and in 20 03 he received the J ohn J. Carty Aw ard for the Adv ancement of Science from the U.S. Nationa l Academy of Sciences. In addition to his purely aca de mic rese a rch, Da vid has extensive exp er ience as a consultant, including w or king for the Carnegie Commission, the City of San F r ancisco, and the F eder al Reser ve, as well as several Departments of the U.S. Gov ernment–Energy , T rea sury , Justice, and Commerc e . He has testified as an ex- per t witness o n statistics in a nu mber o f law case s , including Piva v. Xer ox (em- ploymen t discrimination), Garza v. County of L os A ngeles (voting rights), and New Y ork v. Dep artment of Commer c e (census a djustmen t). Lastly , he is an exce ptionally g o o d writer and tea cher, and his many bo o ks a nd review articles a re a rguably his most impor tant contribution to our sub ject. His widely used elemen tar y text Statistics , written with R. Pisani and R. Pur ves, now in its 4th edition, is rightly regarded as a classic introducto ry exp ositio n, while his second text Statist ic al Mo dels (2005) is set to b eco me just as success ful in its field. The r oles of theoretica l resea rcher, consulta nt, and exp ositor are no t disjoint asp ects o f David’s per sonality , but fully integrated ones. F or over 20 years now, he has b een wr iting extensively on statistical mo deling . He has cont ributed to theory , and prepar ed illuminating expo sitions and given p enetra ting critiques of o ld a nd new models and metho ds in a wide range o f c ontexts. The result is a quite remar k - able b o dy of resea r ch o n the theor y and applica tion of statistics, pa rticularly to the decennial U.S. census, the so cial sciences (esp ecially econometric s , politica l science and the law), and epidemiology . These themes are reflected in this volume of pa p e r s by friends and co lleagues of Da vid’s. W e’d lik e to thank him for his w onderful b o dy of w or k, and to wis h him well for the future. Debo rah No lan T er ry Sp eed v Con tribu tors to this v olume Bengtsson, T., Bel l L abs Berk, R., Un iversity of Califo rnia, L os Angeles Bhamidi, S., U niversity of California , Berkeley Bick el, P ., University of California, Berkeley Brillinger, D. R., Un iversity of Califo rnia, Berkeley Brown, L., Un iversity of Pennsylvania Chen, W., Kaiser Permanente Southern California Diaconis, P ., St anfor d University Dudoit, S., University of California, Berkeley Eaton, M. L., Un iversity of Minnesota, Minne ap olis Ev ans, S. N., University of California, Berkeley Gnedin, A., Ri jksuniversiteit Utr e cht Holmes, S., Stanfor d University Humphreys, P ., University of V ir ginia James, N., Berkeley, California Kay e, D. H., Ari zona State U niversity Kele¸ s, S., Un iversity of Wisc ons in, Madison Klein, S. P ., GANSK & Asso ciates Kriegler , B., University of California, L os Angeles Lehmann, E. L., University of California, Berkeley Li, B., Tsinghua University Li, L., H arvar d Scho ol of Public He alth Littnan, C. L., Pacific Islands Fisheries Scienc e Center Luen, B., U niversity of California , Berkeley Lyons, R., Indiana University Peled, R., U niversity of California , Berkeley Peres, Y., Un iversity of Califo rnia, Berkeley, and Micr osoft Rese ar ch Petitti, D. B ., University of Southern California Pitman, J., Un iversity of Califo rnia, Berkeley Ralph, P ., University of California, Berkeley Robins, J., Harvar d Scho ol of Public He alth Salzman, J ., Stanfor d University Stark, P . B., University of California, Berkeley Stew ar t, B. S., Hubbs-Se aWorld Re se ar ch Instit u te Tchetgen, E., Harvar d Scho ol of Public He alth vi Contributors to this volume vii v a n der Laa n, M. J., University of Califo rnia, Berkeley v a n der V aart, A., V rije Universiteit W acht er , K., University of California, Berkeley Ylvisaker, D., University of Calif ornia, L os Angeles Zhao, Z., Mathematic a Policy Rese ar ch, Inc.