The Early Statistical Years: 1947--1967 A Conversation with Howard Raiffa

Statistic al Scienc e 2008, V ol. 23, No. 1, 136– 149 DOI: 10.1214 /0883423 07000000104 c  Institute of Mathematical Statisti cs , 2008 The Ea rly Statistical Y ea rs: 1947– 1967 A Conversation with Ho w a rd Raiffa Stephen E. Fienb erg Abstr act. Ho ward Raiffa earned his bac helor’s degree in mathematics, his master’s degree in statistics and h is Ph.D. in mathematics at the Univ ersit y of Mic higan. Since 1957, Raiffa has b een a member of the facult y at Harv ard Un iversit y , where he is no w the F rank P . Ramsey Chair in Managerial Economics (Emeritus) in the Graduate Sc ho ol of Business Administration and the Kenn ed y S c ho ol of Go vernmen t. A pioneer in the creation of the field kn own as decision analysis, his re- searc h in terests sp an statistical decision theory , game theory , b ehavio ral decision theory , risk analysis and n egotiat ion analysis. Raiffa has su- p ervised more than 90 do ctoral dissertations and written 11 b o oks. His new b o ok is Ne gotiation Analysis: The Scienc e and Art of Col lab or ative De cision M aking. Another b o ok, Smart Choic es , co-authored with his former do ctoral students John Hammond and Ralph Keeney , w as the CPR (formerly kno wn as the Center for Public Resources) Institute for Dispute Resolution Bo ok of the Y ear in 1998. Raiffa help ed to create the Internatio nal Institute for Applied Systems Analysis and he later b ecame its first Director, s erving in that capacit y fr om 1972 to 1975. His many h onors and aw ards include the Distinguish ed Contribution Aw ard from the So ciet y of Risk Analysis; the F rank P . Ramsey Medal for ou tstand ing con tributions to the field of decision analysis from the Op erations Research So ciet y of America; and the Melamed Prize from the Un iv ersit y of Chicago Bu s iness Sc ho ol for The Art and Scienc e of Ne gotiation . He earned a Gold Medal fr om th e In ternational Asso cia- tion for Conflict Manag ement and a Lifetime Ac hiev ement Aw ard from the CPR Institute for Dispute Resolution. He h olds honorary d o ctor’s degrees from Carnegie Mellon Universit y , the Universit y of Mic higan, North w estern Universit y , Ben Gurion Univ ersity of t he Negev and Ha r- v ard Univ ersit y . Th e latter was a wa rded in 2002. Stephen E. Fienb er g is Mauric e F alk University Pr ofessor of S tatistics and So cial Scienc e, Dep artment of Statistics and Machine L e arning Dep artment, Carne gie Mel lon University, Pittsbur gh, Pennsylvania 15213 -3890, USA (e-mail: Fienb er g@s tat.cmu.e du ). This is a n electronic reprint of the origina l article published b y the Institute of Mathematica l Statistics in Statistic al Scienc e , 200 8, V o l. 2 3, No. 1, 136–1 49 . This reprint differs fr o m the original in pag ination and t yp ogr aphic detail. This con v ersation to ok place as p art of an in- formal seminar in the Department of Statistics at Carnegie Mellon Univ ersit y on Ap ril 3, 2000, pre- ceding b y a d a y a semin ar at wh ic h Ho ward Raiffa receiv ed the 1999 Dic ks on Prize in Science fr om the Univ ersit y . Others presen t and participating in th e discussion included William Eddy , Rob Kass, Ja y Kadane and R aiffa’s wife of 55 years, Estelle. The topic of Ho w ard’s pr esen tation at the Dic kson cere- mon y was: “The Analytical Ro ots of a Decision S ci- en tist.” F or the Department of Statistics he elab o- rated on the y ears 1947–1 967. 1 2 S. E. FIENBERG Fig. 1. Howar d R aiffa fol lowing the interview. April, 2000. Fien b erg: In 1964 I arriv ed as a graduate s tudent at Harv ard an d in my first class on statistical in- ference, a f acult y member, wh ose name I will not men tion, b egan teac h ing inference from a Neyman– P earson p ersp ectiv e, that is, hyp othesis testing and confidence in terv als. In fact, w e stud ied Eric h Lehmann’s b o ok on hyp othesis testing. It w as clear to me that this w asn’t the w ay I wa nte d to think ab out statisti cs. I ask ed aroun d the department ab out what alternativ es were a v ailable to me and some- one said: “On Monday afterno ons th ey ha ve a semi- nar at the b usiness school across the Charles Riv er.” So I just sho w ed up one Mond a y afterno on. It was one of the most wo nderf ul exp eriences of m y gradu- ate career. Although I ha v e no memory of who was sp eaking that first afternoon, I recall that it w as the most animated and heated discussion I had engaged in at an y p oint in m y career up to that p oint. It turned out th at the seminar and the heated discus- sion we re rep licated ev ery Monday afterno on. One of the leaders of t hat M onday afterno on seminar was Ho ward Raiffa. Raiffa: That w as called the Decision Un d er Un- certain ty seminar, the DUU seminar, and it w as one of the exciting parts of m y life as w ell. And it w en t on f or four y ears, from 1961 to 1964. I r an that seminar with m y colle ague Rob ert S c hlai- fer. Bet wee n the w eekly sessions of the seminar, a few of u s exc hanged a flo o d of m emos comment- ing on what was discussed and w hat should ha ve b een discussed. Although there w as a demand for air time, we neve r sc h eduled starting a n ew topic unt il we felt that w e had completed the train of re- searc h thinking on th e table. Half-bake d ideas w ere giv en p riorit y . Fien b erg: I w as lucky to b e able to attend for a couple of them! I brought along with me a couple of slices of statistical history this afterno on: Applie d Statistic al De c ision The ory (Raiffa and Sc hlaifer, 1961 ) and Intr o duction to Sta tistic al De cision The ory (Pratt, Raiffa and Sc hlaifer, 1995 ). T hey come from the same p er io d as the DUU seminar and both b o oks had a tremendous impact on the Ba y esian reviv al of the 1960s. I’m hoping w e will ha ve a c hance to talk ab out them in a f ew minutes, but let’s go bac k a lit- tle further in time and start with how you b ecame a statistic ian. Raiffa: Let me start at a decision no de. I wa s a First Lieutenan t in the Air F orce in c h arge of a radar-blind-landing system at T ac hik a wa Airfield near T oky o—ground-con trolled appr oac h it was called—and I w as due for a routine disc harge after completing 44 mon ths in the armed services. Should I co ntin ue in Japan acting in th e same extraordinar- ily exciting capacit y as a civilian, earn ing o o d les of money , or should I return to the States to complete m y b ac h elor’s degree, and if so, where, and to stud y what? I w as 22. I thought of b ecoming an engineer building u p on my practical exp erience with radar. I learned from an army bud dy of a fi eld I had nev er heard of: actuarial mathematics. Merit counted in the actuarial profession since bud ding actuaries had to pass nine comp etitiv e exams. The place to go to prepare for the first three of them w as Univ ersit y of Mic h igan. Fien b erg: Ho ward, I though t yo u w ent to Cit y College. Raiffa: Before going into the arm y , I w as more in terested in athletics than sc h olarship. I w as the captain of m y h igh sc ho ol bask etball team in New Y ork Cit y—a high s c ho ol of 14,000 stud en ts—at a CONVERSA TION WITH HOW ARD R AIFF A 3 time wh en bask etball w as the rage in NYC. I w as a medio cre B s tudent in m y freshman and sophomore y ears at Cit y College of New Y ork (CCNY), the col- lege of choi ce for p o or and middle-income students in New Y ork—I w as on the p o or side. F our y ears later, when I r eturned to college at the Universit y of Mic higan, I b ecame a sup erb stud ent. Th at sur- prised me. I also was a married man determined to b ecome emplo yable after getting m y bac helors de- gree. F or a while I thought the students at CC NY w ere just brigh ter than those in Mic higan, but on reflection I d idn’t like that explanation. Something c h anged within me. I w as sho oting for straigh t A’s and getting th em. Fien b erg: W e could statistically inv estigate this c h ange if w e had the righ t sort o f data. But Ho w ard, did you ever take those actuarial exams and ho w come you sta y ed on and got yo ur master’s d egree in statistics? Raiffa: I got m y b ac h elor’s in Actuarial Math and passed the first three exams and w as on m y wa y to b ecoming an actuary preparing for the fourth exam when I d ecided th at I really w ant ed to stud y some- thing m ore cerebral—something m ore theoretical . Fien b erg: Wh y statistics? Raiffa: Th e second actuarial exam was on proba- bilit y theory and w e actuarial studen ts took a co urse that deligh ted me. It u sed Whit wo rth’s Choic e and Chanc e , full of in triguing pu zzlers on combinatio ns and p ermutat ions. Brain teasers galo re. The b o ok w as short on theory and long on pr ob lems and I lo ved it, and I was goo d at it. I j ust couldn’t put it do wn. I was go o d enough that my p rofessors enco uraged me to go on and get a m aster’s d egree. So I to ok a master’s degree in s tatistics. F rankly , I w as also dis- app ointe d in that as w ell b ecause, at that time, th e statistics program in the mathematics departmen t w as s hort on theory and dep th and long on compu- tational manipulations. I r emem b er ha ving to i nv ert an 8 × 8 matrix. Peo ple are laughing b ecause now y ou en ter an 8 × 8 matrix in your computer and p o of, ou t comes the answer. But in those da ys w e had a mec hanical Marc h an t calculato r and I hated it. Ev en though I won a prize for sp eed, it did me in. Fien b erg: S o ho w come yo u sta yed on for a d o c- torate? Raiffa: Along with the courses I to ok in statistics, I also t o ok for cu ltu r al curiosit y a course in the foun- dations of m athematics by a Professor Cop eland who later b ecame one of m y men tors. Cop eland taught the course using th e R. L. Mo ore p edagogic al style . Are you familiar with this approac h , Stev e? Fien b erg: Y es, I am. Bu t please tell u s a little ab out the R. L. Mo ore p edagogy anyw ay . I under- stand that Jimmy S a v age also b ecame enamored with study in g mathematics after b eing exp osed to the Mo ore appr oac h. Raiffa: I nev er knew this un til I a ttended a memo- rial service for Jimmy . W ell, I was simila rly affected. I w as studyin g for a master’s degree in s tatistics; going for a do ctorate wa s not an alternativ e in my p ersonal decision space—it just w asn’t, but it should ha v e b een if I had only kno wn more ab out making smart choice s. Cop eland’s fir st assignmen t w as w eird: “Here are some seemingly unrelated mathematical curiosities. Think ab out them. T ry to make some conjectures ab out them. T ry to prov e y our conjectures. T r y to disco v er something of inte rest to tal k ab out.” I dr ew a blank and I came to class with n othing to con- tribute. S o did tw ent y other stud ents. A t the b eginning of class on that first da y the instructor asked, “Does anyb o dy hav e an y con tri- butions to make ?” W e sat and sat and sat and ten min utes we nt by and he said, “Class dismissed.” He added, “The same assignmen t tomorrow.” The fol- lo w in g da y , h e started the class: “An y b o d y ha ve an y- thing to sa y?” Finally , someone raised their h and and asked a question. T he course w as pure R. L. Mo ore. No b o oks were used, absolutely no b o oks. It w as tab o o to lo ok at the literature b ecause y ou migh t fin d hints. Y ou should act as if y ou were a mathematician in the 17th cent ury tr ying to pro ve something new. No matter that w e w ere “disco ver- ing” well- known results; it was new to us. W e stu- den ts did not study mat hematics; w e did mathemat- ics. T h e R. L. Mo ore metho d of teac hing turned m e on. I knew then that I wan ted to b ecome a math- ematician b ecause it was so m uch fun and, to m y surpr ise, I found out that I w as prett y go o d at it. So I b ecame a student of pur e m athematics and I was deliriously h ap py . Thanks to my wife, w ho b y th at time b ecame an elementa ry sc ho ol teac her and could supp ort me in a mann er that I grew ac- customed to, w e wen t from ab ject p ov ert y to solid ric hes. In the y ear I stud ied statistics, I don’t think I h eard the w ord Bay es. As a w a y of inference it w as nonexis- ten t; inference was all strictly d one from the Neyman– P earson p ersp ectiv e. And the v ersion of Neyman– P earson statistics I was exp osed to w asn’t very th e- 4 S. E. FIENBERG oretical as well . Th ey talke d ab out tests of signifi- cance bu t they really didn’t talk ab out the p o w er of the tests. Eddy: Who were your pr ofessors? Raiffa: Paul Dwyer and Cecil C. Craig. Fien b erg: And th ey w ere all stalw arts of the In- stitute of Mathematica l Statistics in its first couple of decades. So y ou weren’t turned on b y the kind of statistics they did? Raiffa: They were goo d mathematicians and goo d statisticia ns; but to them, statistics mean t some- thing quite restrictiv e. Dwye r w as computational and Craig did m ultiv ariate samp ling theory . Th ey w ere ve ry goo d at w hat they did, bu t there was n ot a decision b one in their b o dies. Fien b erg: So you are no w studying pure mathe- matics. Ho w did yo ur int erest in game theory start? Raiffa: F or ten hours a w eek I w orked as a researc h assistan t on an Office of Na v al Researc h (ONR)- sp onsored researc h program administered join tly by the Mathematics Departmen t and the Sc ho ol of En - gineering. My role w as to attend national meetings and listen to app lied problems that p eople w ere talk- ing ab out that related to our O NR p r o ject and to try to form ulate inte resting mathematical prob lems for m y more mathematical-orien ted co lleagues to w ork on. I b ecame qu ite adept at taking ill-formed situa- tions and translating them int o mathematical prob- lems that o ther p eople could work on, in cluding m y- self. Because submarine warfare wa s a hot topic, I read v on Neum ann’s and Morgenstern’s b o ok on Game Theory—or at least p arts of it. Jerry T hompson, a fello w stud en t, and I develo p ed a pap er on ho w to solv e t w o-p erson zero-sum games and we fou n d out to our deligh t that this s ame algo rithm could solv e linear programming pr ob lems. A t that time w e didn’t know an ythin g ab out the simplex metho d, so for m a yb e t w o w eeks w e had the b est algorithm for solving linear p rogramming problems. In ciden tally , Jerry has sp ent his distinguish ed academic career here at Carn egie Mellon Univ ersity . A t that time there w as n o w ell-dev elop ed theory of the simplest tw o-p erson, n on-zero-sum ga mes— there still isn’t. I (and probably hundreds of others unknown to me) in vestiga ted the man y qualitativ ely differen t bi-matrix games having tw o strategies for eac h p la yer. I , naturally , b ecame intrigued with a game n o w kno w n as the Prisoner’s Dilemma game. I kno w y ou are familiar with this game. The p oin t is that eac h pla y er has a dominant strategy so the op- timal thing for eac h to do is to choose this strategy . But the rub is that if eac h p lay er pla ys wisely , they eac h get miserable p a y offs. T he parado x is that t w o wise pla y ers d o w orse than t wo dumb p la y ers. Still, in a s in gle-shot situation, eac h pla y er sh ould c h o ose wisely . Rational individu al c hoice leads to grou p in- efficiencies. T he anomaly lies in th e structure of the game. When the game is rep eated for a pre-sp ecified n umb er of iterations, equilibr ium analysis sp ecifies that ea c h play er should use his or h er d ominating strategy at eac h trial with the result that eac h do es miserably trial after trial. The game presents a s o- cial p athology . If the pla y ers had pre-pla y comm u- nication and could mak e bind ing agreemen ts, they w ould agree to co op erate at eac h trial by taking the my opically dominated strategy . In the fi nitely rep eated game, doub le crossin g at eac h trial (i.e., c h o osing the non-co op erativ e strategy at eac h trial) is the b est r etort if the other guy acts that w a y . But double crossing at eac h trial is not the b est retort against someone who is n ot pla ying the double cross strategy at every trial. In th e lab oratory , most ana- lytically inclined sub jects start by coop erating b ut switc h to a b elligeren t stance tow ard the end of the n umb er of trials. But there is un certain t y w here the switc h will take p lace. In Pa rt A of a rep ort I wrote in 1950 on the t w o-p erson non-zero-sum game for the ONR pro ject, I considered the tw o-p erson Prisoner’s Dilemma game rep eated a fixed n umber of times (sa y 20). I as- signed a sub je ctive pr ob ability distribution o v er the w a y I though t others would pla y in order to figure out the b est w a y I sh ould p la y . In my naiv ete, w ith- out any theory or anything lik e that, I d id wh at I no w r ecognize as a prescriptiv e analysis for one part y , making u se of a descriptiv e mo deling pro- cess for the p ossib le decisions of the other parties. The d escriptiv e mo deling pro cess in v olv ed assessing judgmenta l probability d istributions. I slipp ed in to b eing a sub jectivist w ithout realizing how radical I was b ehaving. That was the natural thing to do. No big deal. P art B of the rep ort dealt with complex non-zero- sum games where there’s no solution. What w ould I do if t wo budd ies of min e came ov er t o me and said, “Lo ok, Ho w ard, we ca n’t solv e this game; there’s n o solution. Y ou resolv e it for u s. What’s fair?” Essen- tially I sough t an arb itration r ule th at would pro- p ose a compromise solution for an y non-zero-sum CONVERSA TION WITH HOW ARD R AIFF A 5 game. A t that time I w as familiar with th e sem- inal wo rk of K enneth Arr o w. Arrow sough t a so- cial welfare fun ction that w ould combine ind ividual preferences to arriv e at a so cial or group preference. He examined a set of v ery plausible constraint s on this so cial w elfare function only to pro v e that these requirement s w er e incompatible. No social w elfare function exists that satisfies p rop erties X, Y and Z. I adopted th e Arro w appr oac h: in m y state of co nfu - sion, I prop osed a set of reasonable desiderata for an arbitration sc heme to satisfy and then I inv estigated their j oin t implications. I remember distinctly ho w I started m y r esearch on arbitration rules. I attended a lecture b y a lab or arbitrator by the name of William Hab er. During his talk ab out arb itration, I exp erienced an “aha” inspiration; I jum p ed out of m y c hair w hile the lec- ture w as going on, I w ent b ac k to my stud y and wrote vigorously for hou r s without a br eak on how I would arb itrate n on-zero-sum games. That consti- tuted P art B of m y ONR rep ort on non-zero-sum games. I used the Kakutani Fixed Poi nt T heorem to s ho w the existence of equilibria strategies. That rep ort w as pub lished informally in the Engi- neering Departmen t. I t w as not p eer review ed; it w as simply a n informal report. A t the t ime I w as prepar- ing to tak e my oral qualifying exam and searc hing for a thesis topic in linear, normed spaces, Banac h spaces. This w as in April of 1950 . F or th e oral quali- fying exa ms in the Mat h Department, the candidate first had to write a r ep ort on what h e or she would lik e to b e examined on; then, d ep ending up on the rep ort, the examiners structur ed the oral exam—its breadth and depth. In my written prop osal I ex- amined ho w all s orts of mathematical ideas found their w a y into the theory of sto c h astic pro cesses. And then a surp rising thing happ ened. My wife, Estelle, receiv ed a telephone call from the v ery famous algebraist, Ric hard Brauer, wh o wa s the chairman of my oral examining committee. He informed her that, on the basis of my written re- p ort, the committee d ecided to excuse m e fr om m y oral exam. And then he said, “By th e wa y , the com- mittee would lik e to talk to me ab out my thesis.” I came in the next day all excited ab out the f act that I didn’t need to take m y oral exam and was told that the committee thought it appropriate that I u se m y recen tly completed En gineering Rep ort as m y do ctoral dissertation. I w as stunn ed. So I en d ed up not havi ng to tak e an oral exam, n ot ha ving to write a thesis, and I wa s through b efore I thought I s tarted. Fien b erg : So that wa s the end of y our graduate education! Did yo u start immediately lo oking for a job? Raiffa: Th at was in April; it w as to o late to go on to the job marke t and I didn’t kno w what to do. The Departmen ts of Mathematics and Psyc hology initi- ated an in terdisciplinary semin ar on Mathematics in the S o cial Sciences and as a p ost-do c I was hired to b e the rapp orteur of the seminar. I was c harged to record what w as said d uring the meetings and “what should ha ve b een said.” I had a ball! F or that one y ear I steep ed m yself mostly in p syc hological measuremen t theory , working with C lyde Co ombs from psyc hology and Larry Klein from economics. That inv olv emen t constituted an imp ortan t part of m y analytica l ro ots. During m y p ost-do c ye ar I also ga ve a series of seminar talks to the statistical facult y and do ctoral student s on Ab raham W ald’s newly p ublished b o ok on Statistic al De c ision The ory. I w as invited to d o so b ecause the b o ok wa s v ery mathematical and made extensiv e use of game theory in existence pro ofs. W ald’s b o ok w as full of Ba y esian decision rules. Not as a wa y of making decisions but as a w a y of elimi- nating noncon tend ers—inadmissible r ules. W ald never used sub jectiv e probabilities or judgmen ts to c h o ose a decision rule. Ba ye sian analysis w as just a math- ematical tec hn ique for fi nding out complete classes of admiss ib le decision rules. The follo win g yea r I w as ready to go on the job mark et an d I had sev eral offers fr om mathemat- ics departments, b ut there were also t wo statistics ones: one from Columbia Universit y’s Department of Mathematica l S tatistics, th e other one wa s w orking with George Sh annon at Bell Labs. The Bell Labs job paid a lot more th an Colum bia. But Columbia present ed a unique opp ortunity for me. A y ear ear- lier, Abraham W ald, who w as the star of the s tatis- tics departmen t at Colum b ia, was killed in an air- plane acciden t o v er India and his colleagues Jac k W olfo witz a nd Jac k Keifer left for C ornell when W ald died. The dep artment w as d ecimated but still there remained T ed And erson, Henr y Sc heffe, Ho ward Lev- ene and Herb ert Solomon. But they needed someone desp erately to teac h W ald’s stuff and to su p ervise his man y do ctoral student s. I was su p p osed to fill that bill b ecause I knew W ald’s b o ok. I really w as not prepared. W ald’s do ctoral stu- den ts knew more statistics than I and th ere w ere 6 S. E. FIENBERG Fig. 2. R aiffa with his wi fe Estel le and Carne gi e Mel lon Pr esident Jar e d L. Cohon at Dickson Prize Cer emony in April , 2000. times when I h adn’t the fain test idea w hat th ey w ere talking ab out. I had to learn and giv e lec- tures on adv anced topics in statistics. I did what I called “ju st-in-time” teac h ing. I used a b o ok by Blac kwell and Girshic k. A t that time it w as a v ail- able only in a pre-print ed form and it h adn’t y et b een published. Bu t it w as a w onderf ul b o ok. Blac k- w ell and Girshick were more in clined to wa rd th e Ba yesia n viewp oin t than W ald, b u t not really . (A t least n ot then, although Blac kwe ll later b ecame an arden t su pp orter of the Ba yesian p ersp ectiv e.) The b o ok carefully skirted issues of measurabilit y by con- fining itself to the denumerable case; it did n othing con tin uous. I brac ke ted their presen tation by exam- ining more closely the fi nite case, sp en ding a lot of time on n = 2 , and then going abstract b y consider- ing the most adv anced measure-theoretic v ersion of the id eas. I pro du ced copious n otes for the course. I w as a nerv ous wrec k b ecause my statistical col- leagues all audited that course. I also taugh t the first course in statistics. I taugh t Neyman–P earson theory , tests of hyp otheses, confidence inte rv als, un- biased estimation and pr eac hed ab ou t the d an gers of optional stoppin g that I n o longer b eliev e to b e true. Gradually I b ecame disillusioned. I just d idn’t b e- liev e th at the basic concepts I was teac hing were cen- tral to what the field should be ab out. S o I repeated what I knew ho w to d o b est. I w ent bac k to the Ar- ro w approac h and in an axiomatic st yle examined what p rimitiv e things that I b elieve d in and explored their joint imp lications. A t that time I shou ld ha ve kno wn ab out some of the work of Jimm y Sa v age, b ut I didn’t; I didn’t eve n kn o w who Jimm y Sa v age w as. But in 1954 I came across a pap er by Herman Cher- noff justifying the Laplace solution to these prob- lems. Essen tially in a state of ignorance it argued for using a uniform p rior distribution ov er states. I had m y reserv ations ab out the fu ll analysis, but Chernoff u sed an axiom he attributed to Herman Rubin, calle d the Sur e Thing Principle. I em braced it wholeheartedly . The implications w ere dev astat- ing. It argued for making inferences and decisio ns based on the lik eliho o d function. It also ruled out the Neyman–P earson theory that preac h ed that y ou can’t infer what y ou should do based on an observ ed sample outcome until y ou thin k what yo u w ould do for all p oten tial sample outcomes that could ha v e o c- curred bu t didn ’t. “Nonsense,” sa ys the su re thin g principle. Out w en t tests and unbiased estimates and confidence in terv als and lots of stuff on optional stopping. Not muc h left. No w that w e had destro ye d so muc h , it was time to r eally seek an alternativ e to NP theory . CONVERSA TION WITH HOW ARD R AIFF A 7 Consider the standard problem inv olving an u n - kno wn p opulation p rop ortion p . T hat’s th e usual binomial mo del. The s ample p r o duces nine F ’s and one S in that order and the lik eliho o d function is (1 − p ) 9 p 1 . No w compare that with the stopping rule that samples until the fi rst success app ears; and s up- p ose that the fi rst S o ccurs on the tenth trial. The lik eliho o d f unction would b e the same whether y ou had the first stopping rule or the second stopping rule, and therefore the inferen ce should b e the same. But using Neyman–P earson tests of hyp othesis, the answ ers are differen t—y ou ha v e to wo rry ab out not only wh at h app ened , but what could ha ve h app ened according to the sampling plan. I pushed th e axiomat ics and con vinced myself that it made sense to assign a pr ior probability distri- bution ov er the states of th e pr oblem and maxi- mize exp ected utilit y . I t was f or me akin to a reli- gious con version—from b eing a Neyman–P earsonian to b eing a Ba yesian. I b ecame a closet Ba y esian. I did n ’t come out of the closet b ecause my asso- ciates, whom I admired, w ere vociferously opp osed to Bay esianism. Th ey though t it w as a step back- w ard. They’d sa y , “Look, Ho w ard, what a re y ou try- ing to do? Are you t rying to in tro du ce squish y judg- men tal, p syc hological stuff int o somethin g whic h w e think is science?” Jimm y Sa v age, I think, h ad the b est retort to this. He said: “Y es, I w ould rather build an ed ifi ce on the shifting sands of sub jectiv e probabilities than b uilding on a voi d.” The biggest difference b et w een me and my col- leagues at Columbia w as the kind of problems that w e w ork ed on. Th ey were b asically driv en b y the problem of inf er en ce. They paid lip service to de- cision problems by consid ering whether one sh ould reject a n ull h yp othesis, but really basically wh at they were in terested in w as problems of statistical sampling and inference going from ob s erv ations to parameters. I came from a bac kgroun d in game the- ory and op erational research, so for me a proto- t ypical pr oblem w as: ho w muc h s hould yo u sto c k of a p ro du ct when demand wa s u nkno wn. F or me, that unk n o wn demand was the p opulation v alue and that p opulation v alue had a probabilit y distrib u tion. The whole p oin t of doing sampling was to get b et- ter information, to get more informed p robabilit y distributions. Decisions w ere tied to real eco nomic problems, not p h on y ones like should yo u accept a n ull h yp othesis. That w as really th e divid e, b ecause m y Colum bia colle agues, whom I grea tly admired, Henry Sc heff ´ e, T ed Ander s on, Herb Robbins and others weren’t in terested in my class of problems. T o me they were pr imarily inference p eople. Fien b erg: I notice from yo ur r esume that in 1957 y ou pu blished the highly acclaimed b o ok on Games and De ci sions with Duncan Luce (Lu ce and Raiffa, 1957 ). Y ou must h a v e w ritten this du ring the same p erio d of time y ou w ere primarily learning and teac h- ing and establishing y our p hilosophical ro ots in statis- tics. Could y ou tell us ab out y our game theory ac- tivities during th is p erio d? Raiffa: A group of us started th e interdisciplinary Beha vioral Mo d els Pro j ect at Colum bia and we hired a mathemat ical psycholo gist, Duncan Luce, to su - p ervise the pro ject. Du n can was not affiliated with an y department at th e time. T h e prin cipals were P aul Lazersfeld from psycholog y , Ern est Nagel f rom philosophy , Bill Vic ke ry fr om economics, and I sup- p ose I sh ould include myself from statistics. As th e junior mem b er of that steering committee, I acted as Chairman of the pro ject an d Duncan and I carried the burden of pu s hing th e pro ject alo ng. W e had ex- ternal financial supp ort and ran seminars and hired pre-do cs. W e prop osed pu blishing a series of s h ort 50-page monographs on the topics that featured the use of mathematics in the so cial sciences—topics suc h as learning theory , psycholog ical measur ement theory , game theory , informatics and cyb ernetics— but w e couldn’t get an y authors to submit manu- scripts. S o Dun can and I decided to wr ite a 50-page do cument on games and d ecisions, on game theory really . W e ev ent ually ga ve up writing that fift y-page do cument and wrote instead a 500-page b o ok on games and decisions. It to ok us t wo years to write. In 1953 –1954, Duncan wa s at the Center for Ad - v anced Study in the Behavi oral Sciences at Stanford and I was at Columbia. T he follo wing year I wen t to that institute and he w as back at Colum bia. W e w ere together face to face for fiv e da ys in these tw o y ears, in an era w ell b efore e-mail, and we w r ote the Games and De cisi ons b o ok. It included a lot of material from my un publish ed “non -d issertation.” Fien b erg: That w as a landmark b o ok in man y senses and it’s still in prin t as a Do v er pap erb ac k almost a half a cen tury later. What happ ened next in y our career? Raiffa: Th e b o ok still sells a few thousand copies a yea r. O.K., it is no w ’57 and out of the blue I got t w o offers: one from the Un iv ersit y of California, Berk eley—not in statistics—and one from Harv ard Univ ersit y—a join t a pp oin tment with the newly c re- ated Statistics D epartment and the Business Sc ho ol. 8 S. E. FIENBERG The newly form ed Statistics Departmen t was led b y F red Mosteller, a wonderful m an and statistician, and it also in clud ed Bill Co c hran, another great statisticia n. S o I was v ery , very flattered, except I was wo rried. Ho w w ould they receiv e my new con- v ersion to Ba yesia nism? I talk ed to F red Mosteller ab out that and he w as luk ewarm, but he w as tol- eran t. And Co chran said: “W el l, you’ll grow u p.” A t that time I literally was at Columbia f or fi v e y ears and I nev er k n ew that Columbia had a busi- ness school all this time. I really didn’t kno w an y- thing ab ou t b usiness and the only reason I decided to go to Harv ard w as b ecause of the Statistics De- partmen t. T hey we re willing to doub le m y Col umbia salary . Columbia, b elatedly , agreed to matc h it, and promote me, bu t w e decided to go to Harv ard. It w as a close call in making th at decision. My wife, Estelle, and I stew ed ab out the Harv ard offer b ecause there w ere m an y conflicting ob jectiv es w e had to balance. W e did a sort of formal anal- ysis of this decision problem. Our analysis inv olv ed ten ob jectiv es that w e scored a nd w eigh ted. My wife is n ot mathematically inclined at all, bu t for th is case she joined me in making all the assessmen ts. It turned out that we a greed on p ractically everything. Harv ard was the clear winner. Of cour se, th ere were some dimensions wh ere Columbia w as b etter, so it w asn’t a dominating solution, but the formalizatio n help ed us really decide that it wa sn’t a close call at all. W e then follo wed some advice that was giv en to us by Pat t y Lazersfeld. She advised t hat in decisions of this kind, don’t ev er mak e y our choi ce without testing it. Y ou tell y our friends that y ou’re going to Harv ard, y ou tell yo ur family , but you don’t tell the adm inistration. Th en b efore yo u officially com- mit y ourself, yo u s ee ho w y ou sleep for a w eek. And that’s wh at we did. W e slept we ll, we felt cont ent , and w e end ed u p at Harv ard. Fien b erg: But when I arriv ed at Harv ard in 1964 y ou were in essence full-time at the b usiness sc ho ol. Ho w d id this sh ift o ccur? Raiffa: Sur prisingly to me, m y a cademic life didn’t rev olv e around the s tatistics departmen t; it rev olv ed around a place called the Business Sc ho ol. At the B-Sc h o ol I w ork ed closely with Rob ert O. Schlai fer. He’s prob ab ly the p erson who influ enced me more in my life than an yb o dy else. He w as trained as a classical historian and classical Greek sc holar. Dur- ing the w ar he w orked for the underwate r laboratory writing prose for tec hn ical rep orts. He ended up at the end of the w ar writing a tome on the engineering and eco nomics of a viation engines. By some in vol ve - men t, by some fluke, he receiv ed an app ointme nt at the Business Sc ho ol but he had no sp ecialt y . Th e single p rofessor at the B-Sc h o ol who taugh t a prim- itiv e course in statistics r etired at that time and Rob ert was ask ed to teac h that cour se. Thus his- tory wa s made. T r ained as a classical historian, he knew nothin g ab out s tatistics, s o he read the “clas- sics”: R. A. Fisher, Neyman and P earson—not W ald and not Sa v age—and h e concluded that standard statistica l p edagogy d id n ot add ress th e main prob- lem of a b usinessman: h ow to m ak e decisions un - der uncertaint y . Not kn o wing an ything ab out the sub j ective /ob jectiv e philosophical d ivide, he threw a wa y the b o oks and in v en ted Ba y esian decision the- ory from scr atch. Since h e had little mathematics, most of his examples inv olv ed discrete prob lems or the univ ariate case, l ike an u n kno wn p opulation pro- p ortion. Rob ert h ad had only one course in mathematics, in the calculus. But he had raw mathematical abil- ities, p ro vided he could see ho w it migh t b e put to use. He w as singl e-minded in his pursuit of rele v ance to the real world. When I c ame th ere, he was thrilled that here w as a kindred soul that could tutor him in just the kind of mathematics he needed. I sp en t most of m y da ys teac hing Ro b ert Sc h laifer mathematics— first calculus, then linear algebra. I wo uld teac h him something ab out linear alge bra in the morn ing and he w ould sh o w me how it could b e applied in the af- terno on. H e w as not only smarter than other p eople, but h e work ed longer hour s than an yone else. Fien b erg: I’v e heard y ou b eing alluded to as Mr. Decision T ree. What’s the story b ehin d that? Raiffa: Already in m y b o ok with Lu ce on game theory , publish ed in 1957, I used game trees to defin e games in ext ensive form. T here w ere pla yer no d es and c hance no des, but all c h ance n o des had asso ci- ated, ob jectiv e probabilities in the common kno wl- edge domain. When I started w orking on in dividual decision p roblems at th e B-School I mo dified the game tree in to a d ecision tree that featured c hance no des with probabilit y distribu tions sub j ectiv ely as- sessed b y the decision maker. My nonm athemati- cally inclined audiences found it imp ossible to foll o w the logic of th e analysis withou t an accompanying decision tree to k eep trac k of the discussion. My use of decision trees started as a searc h for p edagogi- cal simp licit y , b ut I gradually b ecame dep end en t on them m yself. It’s in teresting to reflect wh y I nev er used decision trees earlier in teac h ing elemen tary CONVERSA TION WITH HOW ARD RAIFF A 9 Fig. 3. Harvar d Statistics Dep artment, 1957. F r om lef t to right : John Pr att, R aiffa, Wil liam Co chr an, Ar thur Dempster and F r e derick M ostel ler. statistics at C olumbia. In Applie d Statistic al D e ci- sion The ory (ASDT), I presen t a sc hematic decision tree d epicting the protot yp ical or canonical statisti- cal d ecision pr oblem. [A t this p oin t Raiffa we nt to the b lac kb oard.] A t mov e 1, a decision no de, the d ecision maker (DM) has a c hoice of exp eriments or information- gathering alternativ es i nclud in g the n ull exp eriment, whic h means acting n o w without gathering fur ther information ab out the unkno wn p opulation p arame- ter, θ . Mo ve 2 is in c hance’s domain and the sample outcome is symbolically d enoted by z . A t mo v e 3 the DM must c h o ose a terminal act a and at m o ve 4 c hance rev eals the true p opulation parameter θ . This r equ ires sp ecifying the marginal probabilit y of z at m ov e 2 and at mo v e 4 the conditional or p os- terior pr obabilit y of θ given z . T o make these p rob- abilit y assessments, th e DM usually starts with a sub j ective ly assessed prior distribution of θ and an ob jectiv e, mo del-based conditional sampling distr i- 10 S. E. FIENBERG Fig. 4. R aiffa dr awing the de cision tr e e. bution of z giv en eac h v alue of θ . The DM then uses Ba yes’ theorem to find the probabilities r equired at mo v e 4 and at 1. This is so standard that analysts using this metho dology are called Ba ye sians. Classi- cists m istak enly c ho ose not to assig n probabilities at mo v es 2 and 4 b ecause these inv olve sub jectiv e pr ob- abilit y inputs and are tab o o. Hence, for them, there is no gain to b e had from considering this s c h ematic decision tree. Fien b erg: In 1957, when y ou really launc hed y our efforts into the Ba ye sian directio n, the w ord Ba ye sian w as not used except p erhaps in a p ejorativ e or math- ematical formalism k in d of w a y . Jimm y Sav age, Jac k Go o d and Dennis L in dley did not call themselv es Ba yesia ns in the early parts of the fifties. Y et by the time you wrote the b o ok with Bob, it had b ecome second nature to ident ify y our self as such. Ho w d id that happ en? Ho w did Ba ye sian inference b ecome kno wn as “Ba y esian”? Raiffa: In the preface to ASDT we refer to “the so-calle d Ba ye sian app roac h .” I’m not su r e wh o is resp onsib le for the nomenclature. But I d islik e us- ing the term “Ba y esian” to refer to d ecision ana- lysts who b eliev e in using sub jectiv e prob ab ilities b ecause, once we generalize from the standard classi- cal statistica l paradigm, the schemat ic decision tree in the shown figure is to o s p ecial and not indicativ e of the b roader class of decision problems; and, in this wider class, sub j ectiv e pr ob ab ility assignmen ts are often made without in v oking Ba ye s’s f orm ula. Fien b erg: So ho w w ould y ou lik e to b e called? Raiffa: I think of myself as a decision analyst who b eliev es in usin g su b jectiv e probabilities. I w ould prefer b eing called a “su b jectivist” than a “Ba y esian.” Rob ert and I divided Ba y esians into t wo groups: en- gineers and scient ists, or “ e cht ” and the “non e cht .” The really true sub jectivists we re the engineers. The non e cht scien tists nev er elicited judgment al ques- tions from an yb o dy . F or them it’s all abstract. T he e cht folk got their hands d irt y . In the early 1960s w e had a series of distinguish ed Ba yesia ns (Lind - ley , Bo x and Tiao), who eac h sp en t a semester at the Business Sc ho ol. T hey w ere pr imarily w onderful statisticia ns of the n on- e cht v ariet y . The most no- table of our e cht visitors was Amos Tve rsky—b ut he visited us in the early 1980s. Next to S c hlaifer, Amos was the p erson who influenced me most. But Amos was not a statistician. Fien b erg: Ho ward, w h y don’t you cont inue with the story of y our early int eractions with Sc hlaifer. Raiffa: Rob er t and I taught an electiv e course to- gether in statistics at t he B-Sc ho ol and it wa s a ball . CONVERSA TION WITH HOW ARD RAIFF A 11 I opted for teac hing b oth th e ob jectivist and sub- jectivist p oint s of v iew side b y side with an op enly declared pr eference for the s ub jective sc ho ol. Rob ert though t that w as a cop-out. He said, “Lo ok, we’ re professors. W e are supp osed to kno w wh at’s righ t. Y ou teac h what’s right; y ou don’t teac h what’s wrong.” “But our stu den ts are going to hav e to read the literature,” I retorted. “Since when do business- men—nev er w omen—read the literature?” He w ouldn’t ha ve an yth in g to do with teac hin g Neyman– P earson theory; it wa s judgmental probability all the w a y righ t do wn the line. The closest I could push him to ward the classical sc ho ol w as to examine de- cision problems fr om the normal—as con trasted to the extensive—form of analysis. Not only was he smart, very smart, but he was the most opinionated p erson I ever met. One ob jection w e encoun tered in using the s ub jec- tivist approac h to statistics w as that it was to o h ard. Rob ert and I explored w a ys to simplify it. In 1959, after I was at th e B-Sc ho ol t wo years, w e started writing a b o ok together on what y ou hav e alluded to as Ba y esian statistic s. It w as not so m uc h a b o ok to b e r ead or eve n a textb o ok but a comp endium of results for the sp ecialist. The theme of our b o ok w as: it doesn’t ha ve to b e too complicated; an ything the classicists can do w e can do also—only b etter. W e discov ered a simple algebraic wa y to go fr om priors to p osteriors for samplin g d istr ibutions that admitted fi xed-dimensional sufficien t statistics, like the exp onen tial distribu tions. Kadane: Y ou mean the use of conjugate priors. Fien b erg: W ell, yo u certainly succeeded in push- ing the ideas. I’m op ening the b o ok right no w, and here, almost at the b eginning, there is a classical re- sult on minimal su fficien t statistics and exp onen tial families. And then sud denly , as if out of nowhere, y ou introdu ce conjugate f amilies and conjugate pr i- ors. Clearly , the ideas w ere around for sp ecial cases, going bac k at least int o the nineteent h cen tury , bu t I ha ve n’t found any other source that laid the ap- proac h out in fu ll generalit y . Ho w did y ou come to this id ea? Raiffa: W ell, it w as p r ett y ob vious that if w e’re going to get a systematic Ba y esian a ppr oac h for the exp onenti al-family distribu tions, w e needed to get somethin g wh er e up dating could b e d on e alge- braically in a formal sort of wa y . W e needed the prior and p osterior to b elong to the same family of distributions and to conform well to the like liho o d function. It’s not h ard to see how the mathematics w ould go. S o I guess I can tak e credit for that. Fien b erg: Y ou should! Raiffa: I ’ll tak e resp onsibilit y for that one. It ju st seemed all so natural. Our effort tur ned into the b o ok you mentio ned at the b eginning, Applie d Sta tistic al De cision The ory , whic h I’m proud to say h as b een republished b y Wi- ley in th eir classics series. O riginally the b o ok w as published not b y a regular pu b lishing hous e but b y the Harv ard Business School Division of Research, whic h h ad nev er p ublished an ything mathematic al b efore. The b o ok m ust hav e sold ma yb e three hun- dred copies. Jimm y Sa v age reviewed the manuscript v ery f a- v orably and he called the notation dazzlingly in- tric ate . He d idn’t like the n otational con ven tions. I tak e resp ons ib ilit y f or the in tricate hieroglyphics. It works for me and seems to w ork also for no vices who hav e not b een brain w ashed w ith usages of ot her notations. T he theory is intric ate enough so that when I’m a wa y from the field for long p erio ds of time and then r eturn, I ha ve a tough time remem b erin g the theo ry and the notation c omes to my rescue. Let me illustrate what I’m talking ab out. Let me go to the blac kb oard and sho w you. W e co nsid er a p op- ulation parameter, designated by the Greek sym b ol µ ; s ince µ is an uncertain qu an tit y , we flag it w ith a tilde sign, ˜ µ . W e distinguish b et w een p rior distri- butions and p osterior distribu tions of ˜ µ b y pr imes and d ou b le primes, giving resp ectiv ely ˜ µ ′ and ˜ µ ′′ . W e distinguish the prior mean—that is, the mean of ˜ µ ′ —whic h w e lab el ¯ µ ′ —from the p osterior mean ¯ µ ′′ . No w we get m ore complicated. F r om a pr ior p oint of view, w e might b e in terested in the as-y et- unknown p osterior mean. Th e d istribution of this quan tit y was dubb ed b y Rob ert, the pr e - p osterior distribution and renamed b y our students as the pr e- p oster ous distribu tion. The prior v ariance of the as- y et-unkno wn p osterior mean is connoted by v ′ . The p osterior distribution of ˜ µ dep ends up on a sufficien t statistic z , whic h w e b r ing into our notational fold, and so it go es. Fien b erg: While that first pr in ting of ASDT by the Business S c ho ol m a y ha v e not sold very man y copies, a later pap erback v ersion brough t out by MIT P ress was widely used and imp ortan t to those of us who tried to tak e th e conjugate prior frame- w ork in to statistic al pr oblems b ey ond d ecision the- ory . But then fairly so on after the b o ok firs t ap- p eared, y ou b egan to turn to related problems. Ho w did this h app en? 12 S. E. FIENBERG Fig. 5. R aiffa r e cr e ating the ASDT notation at the blackb o ar d. Raiffa: ASDT w as wr itten in 1959 and 1960 and published in ’61. Schlaife r and I w ould discus s some ideas and my style w as to start wr iting when I was still confused. Th e task of wr iting fo cused my mind. Sc hlaifer had trouble writing first drafts. He would lo ok at w hat I h ad wr itten and inv ariably his r eac- tion w as: “This is terrible,” and he would tear up m y v er s ion and write something b etter. But he had trouble s tarting without something to criticize. In the academic y ear 1960–196 1, I wa s a w fully busy since I organized, at the request of the F ord F oun dation, a sp ecial 11-mon th program for 40 researc h -orien ted professors, teac hing in managemen t sc ho ols, who f elt the n eed to learn more mathemat- ics. That p r ogram, The I n stitute for Basic Math- ematics for Ap plication to Busin ess (IBMAB), was reputed to b e a huge su ccess and the next generation of deans at suc h prestigious business sc ho ols as Har- v ard, Stanford and Northw estern we re all graduates of that program. Naturally , in th e IBMAB pr ogram I taught statistics from a sub jectivist p ersp ective with a hea vy decision orient ation and the gosp el ra- diated out ward in sc ho ols of managemen t. In the academic y ear 1961– 1962, another radical thing h app ened to me. I w as a s econd reader of a thesis prop osal by J ac k Grayson, a student in the Business Sc ho ol, in the area of finance. Gra yson was in terested in financial d ecisions of oil wildcatters. Through his interact ion with me, h is th esis w as ex- panded from a p urely descrip tiv e to a prescriptive p ersp ectiv e. Ho w should w ildcatters accumulate in- formation fr om geologic su rv eys, seismic sound in gs, exploratory we lls, exp er t j udgments? Th e drilling of an exp loratory well wa s sim ultaneously a term in al- action an d an information-gathering mo v e. Ho w should they form syn dicates for the sharing of risks? This leads to decision problems galore, and the prob - lems d id not easily conform to the classical statisti- cal d ecision paradigm inv olving sampling to gather information ab out an unkno wn p opulation param- eter. The s tatistica l decision paradigm seemed to o restrictiv e, to o h obbling, to o n arro w. Schla ifer con- curred with me and we b egan to think of ourselv es more as decision analysts than as statisticians. I don’t kno w wh y it to ok so long to mak e the s hift, but in m y min d every problem that I thought of up until that p oin t w as cast in the old statistical p aradigm of going f rom a prior to a p osterior distrib ution of a p opulation parameter. With m y new orien ta- tion I s a w problems all ov er the place in business, in medicine, in engineering, in public p olicy w here the decision pr oblems u nder uncertaint y did not fit CONVERSA TION WITH HOW ARD RAIFF A 13 comfortably into the c lassical mold. W e w ere excited ab out our new vision of the w orld of uncertaint y with a v ast new agenda and we started the Decisio n under U ncertaint y Seminar that y ou, Stev e, referr ed to earlier. Fien b erg: Th is w as also around the time that John Pratt work ed w ith you and Rob ert on the introdu c- tory b o ok that app eared in its pr eliminary edition in 1965. It w as called Intr o duction to Statistic al D e- cision The ory —ISDT in con trast to ASDT*. I stud- ied fr om t hat unpublished man uscript. But th en y ou nev er q u ite p olished it up and fin ished it. Wh at hap- p ened? Raiffa: T h e reason why w e didn’t publish this mas- siv e 900-page b o ok after we essentia lly fi nished it w as that Sc hlaifer and I no longer b eliev ed in the cen tralit y of the sta nd ard s tatistica l paradigm as d e- picted in the ab o v e figure. T he b o ok was put aside b ecause w e had a new exciting agenda to explore. But w e should ha ve finished it. Th e fact that the b o ok w as av ailable in a pre-publication form also to ok off the pressure fo r actually publishin g it. That b o ok w asn’t fi nished u nt il 1995, wh en I retired and had more time and more maturit y . Pratt did most of the p olishing and added more theoretic al mate- rial, bu t Sc hlaifer c hose not to get in volv ed in the revisions. W e are no w b ac k in 1963 and with ISDT on a b ac k burn er , we eagerly pu rsued our new agenda. W e had to learn ho w to elicit judgments from p eople— probabilities and utilities. Skeptic s asserted that we couldn’t get real exp erts to p ro vide these s ub jec- tiv e assessment s. W ell, w e demonstr ated to our sat- isfaction th at w as not the case. W e work ed with engineers and mark et exp erts who w ere more than willing to giv e us their sub jectiv e p r obabilit y assess- men ts for real pr oblems. But we re these assessmen ts an y go o d ? Garbage in and garbage out. W e realized that if w e wan ted to elicit jud gmen ts in a credib le manner, th er e was the delicate issue on ho w y ou ask ed the questions. F r aming w as crucial. F or ex- ample, we learned that if we ask ed questions ab out utilit y j udgment s in terms of increment al amoun ts, w e w ould get differen t answ ers than if w e aske d p eo- ple questions ab out th eir asset p ositions. And then w e had to decide whic h set of resp onses should b e used. Schlaifer and I con vinced ourselv es that ques- tions should b e posed in terms o f asset p ositions and not in terms of incremen tal amounts of m oney , b e- cause increment s invit ed all kinds of zero illusions . I had a do ctoral s tu den t who in v estigated the pr ob- lem of ov erconfidence un der my sup ervision. Exp erts did n ot calibrate v ery w ell; they were surpr ised a surpr ising n umb er of times and w e had to m ake our exp erts aw are of this tendency . Our m otiv ation wa s not description, but prescription, but nevertheless our students and we did a lot of work in what is no w called b eha vioral decision making. In the mid-1960s Rob ert in tro duced a required course in the fir st year of the MBA en titled Man- agerial Economics. All 800 students we re exp osed to cases that featured decision making under u n- certain ty . It w as an heroic effort th at was not uni- v ersally app reciated by some of our n onn umerate student s. Bu t it w as lik e an existence theorem: it demonstrated th at decision analysis was relev an t and teac hable to future managers. In retrosp ect I think the effort w as done to o quic kly without enough at- ten tion t o p alatable p edagogy . A t se mester’s end the student s b urned one of Rob ert’s b o oks, on e that I b eliev e deserv ed a prize f or in no v ation. In the mid-1960s, I was offered and accepted a join t c hair b et w een the Business School and the Eco- nomics Departmen t. I relished the f act that I never to ok a course in economics. I taught decision anal- ysis, which in m y mind had a different agenda than courses I once taught in statistical decision th eory . I dr if ted a wa y from Rob ert as I started to w ork on problems m ore in the pub lic sector and to d o re- searc h on multiple confl icting ob jectiv es and nego- tiations. But that researc h , together with my exp e- riences at I IASA, are other c h apters in m y career whic h I’ll discuss tomorrow at the Dic kson Aw ard ceremonies. Fien b erg: As you lo ok bac k o v er the field of statis- tics, don’t y ou hav e a sens e of satisfacti on ab out ho w y our w ork with Rob ert and John h as influenced oth- ers an d the gro wth of the Bay esian sc h o ol? Raiffa: C er tainly I’m proud of what I’ve con tributed in this field. But still I’m a little disapp oin ted. If we made a s urve y of the w a y s tatistics is taugh t across the country , it w ould b e dominated by the old stuff, the Neyman–P earson theory . Carnegie Mellon is a ma v eric k; is an exception. The sub jectivist sc ho ol of decision making is n ot b eing taugh t in many places. Kass: I thin k it’s tur ned a corner. Raiffa: Here, at Carnegie Mellon for s ure, but. . . Kass: W ell, no, I think in the w orld, in the last ten y ears or so I thin k it’s really started to change. And I don’t think it’s only in statist ics but in lots of 14 S. E. FIENBERG other fields, applied areas that really u se statistica l ideas and metho d s . Raiffa: W ell, let’s lo ok at the curricula of m ost of the universitie s. I think that o ve rwh elmingly the classical sc ho ol is still dominan t at most u niv ersities. I w as instru men tal in helpin g to start the Kennedy Sc ho ol of Gov ernment at Harv ard and in the b egin- ning I had some inpu t in what w as taught . In the early da ys the curriculum w as decision and p olicy orien ted and Ba y esian s tatistics and decision anal- ysis w ere taugh t and in tegrated in the curr iculu m. But new teac hers we re hired and they taught what they had learned as student s an d Ba yesianism d is- app eared. Kass: I h a v e another question; it’s almost the same as Stev e’s, inv olving the Ba y es part of it. One of the things that’s so in teresting to many of us is to ex- amine the wa y on e b o dy of work influences another. And in the case of y our b o ok with Sc hlaifer, it’s easy to see ho w that infl uenced, for instance, Mor- rie DeGro ot’s b o ok, wh ic h came a decade or so lat er, and th en, muc h later, J im Berger’s bo ok whic h most student s to day are familiar w ith in mo d ern s tatisti- cal decision theory . In retrosp ect it’s clear h o w yo ur b o ok in spired the m aterial and presen tation in DeG- ro ot’s b o ok. T o see these three b o oks, one righ t after another, it’s very easy to trace the in fl uences bac k- w ards, bu t ho w do we trace things b ac k to see the influences on yo ur wo rk with Rob ert as a Ba y esian? Raiffa: Schlaifer w as driven b y the need to co or- dinate statistics with b usiness decision making and he truly disco v ered from scratc h th e basic ideas of what y ou refer as B a y esianism. I, on the other hand, w as brain washed into the classica l tradition and had to go through a religious con version. Kadane: What ab out Jimmy Sa v age’s w ork and his 1954 b o ok? Raiffa: S omeho w I ju s t w as not a ware o f that b o ok unt il I left Columbia. I already men tioned Herman Chernoff ’s pap er and Herman Ru bin’s sure thin g principle. That had a profound effect on m e. Kass: Not only h as the Ba y esian world b ecome more intimatel y inv olv ed in applications since y our early efforts, b u t s tatistics as a whole has mo ved in this d irection. Raiffa: I hop e y ou are correct b ut it’s painfu lly slo w . I look forward to the da y that there will b e Departmen ts of Decision Sciences in other univ ersi- ties b esides Carn egie-Me llon and Duke . Statistics is a br oad sub ject encompassin g data analysis, mo d- eling and inference as w ell as decisions. I just don’t w an t the decision comp onent to d isapp ear. Fien b erg: W ell, Ho w ard, we all lo ok forward to a con tin uation of this conv ersation when y ou can tell us more ab out y our analytical ro ots as a decision scien tist and ab out y our exp eriences after 1967. Raiffa: I lo ok forward to it. REFERENCES Hammond, J. S., Keeney, R. L. and Rai ff a, H. (19 98). Smart Choic es. Harv ard Business School Press, Boston. Keeney, R. L. and R aiff a, H. (1976). De cisions with Mul- tiple O bje ctives : Pr ef er enc es and V al ue T r ade offs. Wiley , New Y ork. Reprinted, Cambridge Univ. Press, New Y ork (1993). MR0449476 Luce, R. D. and Raiff a, H. (1957). Games and De ci sions : Intr o duction and Critic al Surve y . Wiley , N ew Y ork. P ap er- back rep rint, Dove r, New Y ork. MR0087572 Pra tt, J. W. , Raiff a, H. and Schaifer, R. (1995). Intr o- duction to Statistic al De ci sion The ory . MIT Press, Cam- bridge, MA. MR1326829 Raiff a, H. (1968). De cision Analysis : Intr o ductory L e ctur es on Choic es Under Unc ertainty . Addison-W esley , Reading, MA. Raiff a, H. (1982). The Art and Scienc e of Ne gotiation . Har- v ard Un iv. Press, Cam b ridge, MA. Raiff a, H. (2002). Ne gotiation Analysis. Harv ard U niv. Press, Cam bridge, MA. Raiff a, H., Ri chardson, J. and Metcalfe, D. (2003). Ne- gotiation Analysis : The Scienc e and Art of Col lab or ative De cision. Harva rd Univ. Press, Cam bridge, MA. Raiff a, H. and Schaifer, R. (1961). Applie d Statistic al De cision The ory . Division of Researc h, Harv ard Business School, Bos ton. 1968 pap erback editio n, MIT Press, Cam- bridge, MA. Wiley Classics Library edition (2000).