The Synthesis of Regression Slopes in Meta-Analysis

Statistic al Scienc e 2007, V ol. 22 , N o. 3, 414– 429 DOI: 10.1214 /07-STS243 c  Institute of Mathematical Statistics , 2007 The Synthesis of Reg ression Slop es in Meta-Analysis Betsy Jane Beck er and Meng-Jia Wu Abstr act. Researc h on method s of meta-analysis (the syn thesis of re- lated study results) has dealt with man y simple stud y ind ices, but less atten tion has b een paid to the issu e of sum marizing regression slop es. In p art this is b ecause of the many complications that arise when r eal sets of regression mo d els are accum ulated. W e outline the complexities in v olv ed in syn thesizing slop es, describ e existing metho ds of analysis and presen t a m ultiv ariate generalized least squ ares approac h to th e syn thesis of regression slop es. Key wor ds and phr ases: Generalized least squares, multi v ariate, meta- analysis, regression slop es. W e b egin with a discussion of the rationale f or summarizing regression slop es, a practice that has b ecome more prev alen t in m eta-anal yses in recen t y ears. W e then examine the metho d s for summariz- ing slop es that h a v e b een prop ose d to date, and the assumptions and data requirement s of those meth- o ds. W e conclude b y presenting a generalized least squares (GLS) approac h to the s y nthesis of regres- sion slop es for con tin uous predictors and outcomes, with r emarks on the chall enges and limitatio ns to syn thesis of such estimates. 1. SYNTHESIZING SLOPES While it is by no means common or well und er- sto o d, the syn thesis of regression slop es has receiv ed increased at ten tion in recen t y ears (e.g., Bak er et al. ( 2003 ); P eterson and Bro wn ( 2005 ); Rob e rts ( 2005 ); Betsy Jane Be cker is Pr ofesso r of Me asur ement and Statistics, Col le ge of Educ ation, Florida State University, T al l ahasse e, Florida 32306, USA e-mail: bb e cker@fsu.e du . Meng-Jia Wu is Assistant Pr ofessor of R ese ar ch Metho dolo gy, Scho ol of Educ ation, L oyo la University Chic ago, Chic ago, Il linois 60611, USA e-mail: mwu2@luc.e du . This is an elec tr onic reprint o f the or iginal article published by the Institute of Mathematica l Statistics in Statistic al Scienc e , 200 7 , V ol. 22, No. 3 , 414–4 29 . This reprint differs fro m the o r iginal in paginatio n a nd t yp ogr aphic detail. Rose and Stanley ( 2005 )). T h is gro wing in terest is lik ely relate d to th e increasingly complex mo dels in- v estigated in p rimary researc h, at least in the so cial sciences. Researc hers w ant to mo del th e effects of m ultiple pr edictors as well as to con trol for p ote n- tial confoun ding v ariable s, and in the con text of a primary study this is often ac hiev ed b y including suc h v ariables in complex m o dels. Results of tec h- niques lik e structural equation mo deling, hierarc hi- cal linear mo d eling and multiple regression ha v e of- ten b een omitted from meta-analyse s b ecause of a lac k of kno wledge ab o ut how to syn thesize ind ices from these analyse s, and b ecause of the complexities and assumptions u nderlying the pro c ess of syn the- sis. The main purp o ses of this pap er are to p oint out the complexities and p o tent ial problems in syn the- sizing slop es from regression mo dels, to describ e ex- isting metho ds for summarizing slopes and to present a new synthesis approac h based on generalized least squares estimation. W e fo cus only on the case of m ultiple regression, though clearly other analyses in v olv e regression-lik e mo dels with similar assump- tions. W e b egin with the simple case w here all stud- ies examine very similar mo d els and discuss tec h- niques for estimating a com bined regression mo d el across stud ies. Mo d eling to examine the impact of study features, design differences and stud y qualit y (e.g., Pa ng, Drummond and Song, 1999 ) is touc hed on briefly . Other complications suc h as publication 1 2 B. J. BECKER AN D M.-J. WU bias (e.g., Doucouliag os ( 2005 ); Stanley ( 2005 )) are b ey ond the scop e of our d iscussion. Consider a mo d el in study i relating some pr edic- tors X 1 through X P to an outcome Y for case j . Sp ecifically , in stu d y i , Y ij = β i 0 + β i 1 X ij 1 + · · · + β iP X ij P + e ij (1) for j = 1 to n i cases. The usual assumptions of n or- malit y and homoscedasticit y of errors apply such that e ij ∼ N (0 , σ 2 i ), and linearit y of the X–Y rela- tions is also assu m ed within eac h stud y . O ften in a syn thesis one predictor (let us say X 1 ) is of primary in terest; b elo w we refer to this as the fo cal predic- tor. No w assum e w e hav e a series of stu d ies i = 1 to k , and eac h of them in v olv es a regression with X 1 as a predictor of Y ; t ypically also other predictors (sa y , X 2 through X P ) app ear in these studies. W e ma y wish to summarize the slop es r epresen ting the relation of X 1 to Y (estimates of β 11 through β k 1 ), and on o ccasion p erhaps to summarize all P slop es for X 1 through X P , across the k studies. While syntheses of slop es imp ly a v ariet y of fairly stringen t assumptions, t his has not det erred researc hers from combining regression slop es (though some existing summaries hav e b een d one without re- gard to the underlying assu mptions). Crouc h ( 1995 , 1996 ) summarized slop es from a div ersit y of mo d- els r epresen ting tourism demand and Lau and col- leagues (Lau, Sigelman, Heldman and Babbitt, 1999 ) used regressions to examine the effectiv eness of neg- ativ e p olitica l ad vertisemen ts. F arley , Leh m ann and Sa wy er ( 1995 ) encouraged m ark eting researc hers to syn thesize regression slopes, and more r ecentl y P eterson and Bro wn ( 2005 ) reviewed the use and syn thesis of standardized slop es in meta-a nalysis in the fi eld of psyc hology . Two con tro v ersial and very differen t synt heses of r egression results in edu cation dealt with the topic of whether educational exp en- ditures relate to ac hieve ment outcomes ( Han ushek ( 1989 ); Hedges, Laine and Greenw ald, 1994 ). A re- cen t issue of the Journal of Ec onomic Surveys (e.g., Rob erts ( 2005 ); Rose and Stanley ( 200 5 )) fo cused exclusiv ely on meta-analyses of regressio n co efficien ts on a v ariet y of economic topics, and man y others ha v e syn thesized regressions on div erse topic s in eco- nomics (e.g., Card and Krueger ( 1995 ); Doucoulia- gos and Pa ldam ( 2006 )), in large part thanks to the seminal work of Stanley and J arrell ( 1989 ). In spite of their widespr ead use in economics, meth- o ds for summarizing regression slop es ha v e r eceiv ed less atten tion in the statistic al literature than metho ds for synthesizing other indices used in meta- analysis suc h as standard ized mean differences, cor- relations, and pr op ortions (or transformed prop or- tions such as o dds ratios). Analytic appr oac hes may b e prop osed in the metho ds sections of substantiv e syn theses without muc h attent ion to the statisti- cal b ehavio r of the estimators and tests in v olv ed. In this article w e pro vide a multiv ariate form ulation for the synthesis of slop es, b eginning with discus- sions of th e assump tions required and of problems that meta-analysts ma y encoun ter w h en syn thesiz- ing slop es. W e then briefly review sev eral existing univ ariate and m ultiv ariate app roac hes. Most exist- ing appr oac hes are univ ariate , wh ic h a v oid some, bu t not all, of the issues and assump tions that underlie the syn thesis of sets of regression s lop es. Other ap- proac hes to com bining slop es are more complex, b ut require access to ra w d ata whic h is quite u nusual to ha v e in the meta-anal ysis cont ext. 2. ASS UMPTIONS AND PROBLEMS IN SYNTHESIZING SLOPES The syn thesis of r egression slop es is difficult f or sev eral reasons, and a v ariet y of p roblems m ust b e dealt w ith in the pro cess. Problems include nonequiv- alence of the metrics for the pr edictors an d ou tcomes across stud ies, lac k of information in study rep orts and the estimation of very div erse mo dels across studies. S lop es are identi cally distributed across stud- ies when the outcome Y and the f o cal pr edictor X are measured similarly , wh en the same additional X s app ear in eac h study (i.e., the same mo del is es- timated in eac h study), and when X and Y scores are similarly distributed. Eac h of these conditions is often not met across studies, which is a concern for the meta-analysis of slop es. W e consider eac h con- dition in turn. 2.1 Y Is Measured Similarly Across Studies This consideration is imp ortant b ecause ev en if only a s ingle pr edictor app ears in eac h of a collec- tion of k regression equations, the r aw regression slop e in eac h study d ep ends on the scales of that predictor and the outcome. Th is is eviden t in the language commonly used to describ e the r a w regres- sion slop e—the predicted change in the outcome Y giv en one unit c hange in X . Also this is easily seen in the form ula for the slop e in a biv ariate regression, whic h is b = r X Y ( S Y /S X ). Here r X Y is the correla- tion b etw een X and Y , S Y is the standard deviation THE SY NTHESIS OF R EGRESSION SLOPES IN MET A- ANAL YSIS 3 of the Y scores and S X is th e standard deviation of the X scores. F or t wo raw-sc ale slop es to b e com- parable across studies, the scales of Y a nd X must b e the same (or prop ortional, e.g., b oth X and Y could b e linearly transformed u sing the same trans- formation). Indeed, for total equiv alence of scales, the measures of Y and X should b e equally reliable across stud ies, which is rare ( Amemiy a and F uller ( 1984 ); Hunter and Sc hmidt ( 200 4 )). W e consider an example where Y is a measure of the qualit y of teac hing—often represent ed as stu- den t ac hieve ment. Ou r examples are dra wn from an ongoing syn thesis of studies of the relationship of teac her qu alificatio ns to measures of the qu alit y of teac hing. Across studies stud en t ac hiev emen t is t ypically measured using differen t tests of differen t constructs (math, r eading, etc.), wh ic h ma y b e pre- sen ted as p osttests, difference scores or other mea- sures of change o v er time and the lik e. W e ha v e iden tified ov er 190 s tu d ies that examine measures of studen t learning and to date, from 65 stud ies with measures co ded in detail, we ha ve id entified 79 dif- feren t measures of studen t learning (and co d ing is not complete). At least in this syn thesis, Y is not measured similarly across studies. In some areas, particularly in economics where outputs may b e monetary , outcomes will b e mea- sured similarly or can b e transform ed or adjusted to b e reasonably similar. F or instance, Ashenfelter, Harmon and O osterb eek ( 1999 ) examined r eturns to schooling, where the outcome w as earnings, and earnings scales can b e reasonably we ll equat ed across coun tries and ov er time. How ev er, in m an y areas this will not b e p ossible. 2.2 Focal X Is Measured Simila rly Across Studies This is also a problematic assumption. Again in some realms, such as the study of economic inputs measured in d ollars or other forms of cu r rency , this ma y not b e an issue (e.g., p er pupil exp end itures w ere examined b y Hedges, Laine, and Gr eenw ald ( 1994 )). I n th e Ashenfelter, Harmon and Oosterb eek ( 1999 ) review, sc ho oling w as apparen tly m easur ed in y ears, w hic h wo uld also b e comparable across stud- ies. Ev en in suc h cases, ho wev er, adjustments (e.g., for inflation, for exc hange rates) w ill sometimes b e required. Also wh en the index of stud y results is an elasticit y (common in economics) and repr esents prop ortional c hange in X and Y , the scal e of X ma y not b e as critical. In other areas, how ev er, the fo cal X ’s ma y not b e measured similarly . Shi and Copas ( 2004 ) noted that exp osure (dose) v ariables are of- ten measured categorically in medical dose-resp onse studies. They referred to the p roblem of ha ving suc h catego rizations (whic h can v ary across stu d ies) as the problem of “group ed d ose lev els.” In our s y nthesis of th e literature on teac her qu ali- fications, studies examine s uc h p redictors as degrees earned, coun ts of courses tak en, n umb ers of credits tak en, p erform ance on teac her tests and teac hing exp erience. Some of these (e.g., counts of courses tak en) may b e measured f airly similarly across most studies, w hile others (teac her test p erf orm ance) are not. Ev en suc h things as teac hing exp erience are not alw a ys measured as ratio-scale, contin uous v ariables (e.g., years of exp erience). W e hav e found su c h v ari- ations as dic hotomies repr esenting novic e v ersus ex- p erienced teac hers, categorical r epresen tations (e.g., teac hers with 0–5 y ears, 6–10 ye ars, or more than 10 ye ars exp erience) and ye ars transformed to rep- resen t nonlinear effects (e.g., sq u ared yea rs of exp e- rience). 2.3 Same Additional X ’s A cross Studies This condition is vir tu ally neve r met. In p ractice, studies n early alw a ys estimate d ifferent mo dels. In fact it can b e argued that differences in the mo dels analyzed sh ould b e exp ected across studies, as re- searc hers d ev elop and elab orate on mo d els present in the literature in attempts to refine prediction and to explain additional v ariatio n in the outcomes of in terest. Stanley and Jarrell ( 1989 ) raised a concern ab out differences in mo dels in the con text of mo del sp ecification, and argued that syntheses of r egres- sion results should examine asp ects of mo dels su c h as the functional form of th e v ariables inv olv ed and differences in the indep endent v ariables included in the regressions. Man y economists ha v e dealt w ith this issue by mo deling regression slop es or other in- dices of effect as fun ctions of dum my v ariable pre- dictors that represent differences in mo del sp ecifica- tion (e.g., Doucouliagos and Paldam ( 2006 ); S tanley ( 2001 )). Some examples of h o w mo dels can v ary widely come f rom the literature on teac her qualifications and the qu alit y of teac hing. W u and Bec k er ( 2004 ) examined regression mo dels of the impact of teac her exp erience on stu d en t outcomes based on t wo large- scale survey data sets: the Coleman Equalit y of Ed- ucational Op p ortunity (EEO) data ( Coleman et al. ( 1966 )) and the National Ed ucation Longitudinal 4 B. J. BECKER AN D M.-J. WU Study: 1988 (NELS:88) (e.g., Ingels et al. ( 1992 )). W u and Bec k er found 92 d ifferen t mo dels f or the prediction of studen t achiev emen t in 12 studies us- ing the EEO data s et. Nine of those h ad examined teac her exp erience. Similarly , 55 d ifferen t mo dels ap- p eared in the 11 studies based on the NELS:88 data set; 6 of those mo dels h ad examined teac her exp e- rience. More criticall y , other than teac her exp eri- ence, the 9 mo d els using the EEO d ata set together con tained 122 differen t additional indep end ent v ari- ables, and the 6 mo d els based on th e NELS:88 data set con tained 103 other ind ep endent v ariables. T h e regression models con tained a div ersit y of add itional v aria bles, includ ing so cioeconomic status, teac her salary , teac her/pupil ratio , school c haracteristics, stu- den t and family charact eristics, and the lik e. The question of whether mo d els are similar across studies is imp ortan t b ecause the metric of the raw slop e for X d ep ends on b oth the outcome ( Y ) and X , and b ecause of mo del sp ecificatio n issu es. In par- ticular, eac h slop e’s precision, degree of bias and co- v aria tion with other slop es dep end on the other X ’ s in the mo d el. T o the exten t th at a mod el is not prop- erly sp ecified, all slop es in the mo del are p otent ially biased. Also ho w in tercorrelated the slop es are (i.e., the degree of multico llinearit y) d ep ends on wh at X s are included. The co v ariance matrix Co v ( b i ), wh ere b i is th e ve ctor of slop es for study i , con tains this in- formation. [Notat ion and f orm ulas for Cov( b i ) are in tro duced b elo w.] One simple example suffices to mak e this p oint: Consider the slop e for X 1 when there is only one add itional X in the mo del (sa y X 2 ). The correlation b et w een the slop es for X 1 and X 2 [i.e., Corr( b X 1 , b X 2 )] is th e opp osite of the bi- v aria te correlation Corr( X 1 , X 2 ) b et w een X 1 and X 2 ( Stapleton ( 1995 )). When add itional X ’s are added the slop e cov ariances dep end on the partial correla- tion b et w een X 1 and X 2 , con trolling for other X ’s. Ev en this simple fact rev eals that eac h slop e’s dis- tribution dep ends on other p redictors in the mo del. Ho w ev er, in pr actice , the co v aria nce matrix among the slop es in primary studies is rarely rep orted (though matrices of correlations among predictors are sometimes rep orted). So it will b e unusual to find fu ll Co v( b i ) matrices in pub lished studies, and in suc h cases caution may b e n eeded in synthesizi ng slop es from v ery differen t m o dels. The exten t to whic h differences in the mo dels esti- mated across studies lead to imp ortant differences in slop es across studies is u nclear. Therefore, the qu es- tion of mo del sp ecificatio n is relev ant here. If all of the different v ersions of mo dels are (reasonably) w ell sp ecified, then eac h one should pro vide unbiased and relativ ely ind ep endent estimates of the regres- sion slop es. The impact of m o del differences lik ely dep ends on b oth mo del sp ecification and on the rela- tionships of the fo cal X to the additional v ariables. Let us consider the fo cal predictor or some “base set” of p redictors that app ear in a well sp ecified mo del. If additional v ariables are relativ ely indep en- den t of the predictors in the base set, the slop es of the base set of pr edictors (and their distribu tions) ma y not b e muc h affected by th e addition of th ose new v ariables. Ho w ev er to the extent that added v aria bles are highly correlate d with the base pre- dictors or with the outcome, the slop es of the base predictors will d iffer and will also b e b iased. W e susp ect that th ere will b e some limitations to the application of the estimation approac h sh o wn here when the mo dels used across different studies dif- fer widely , and in particular when some suffer from m ulticollinearit y or other forms of missp ecification. Some empirical inv estigations ha ve attempted to shed ligh t on the r ole of additional primary-study predictors on regression slopes. P eterson and Bro wn ( 2005 ) foun d no impact of either sample size or the n umb er of additional p redictors in regression mo dels on th e relation b etw een the standardized slop e and a corresp onding zero-order correlation. Their analy- ses included slop es from an incredibly wide range of areas and encompassed differen t p r edictors and out- comes fr om stud ies in psyc hology , so ciology , mark et- ing and managemen t. It is p ossible that by lo oking across so many div erse regression mo dels th e im- pact of the nature of the mo dels would b e d iluted. Ashenfelter and colleagues ( 1999 ) attempted a m ore n uanced in v estigation in their review of stud ies of returns to schooling: they assessed the imp ortance of the p resence of cont rols for abilit y and measure- men t err or in the pr imary-study regressions. Their analyses suggested a complicated pattern of imp act of abilit y con trols, with effects f or returns to school- ing in the United States increasing wh en abilit y was con trolled and effects in non-U.S. studies decreas- ing. In contrast , the inclusion of con trols for mea- surement error did not app ear to significan tly affect the slop es. An analysis by Doucouliag os and P aldam ( 2006 ) examined m o dels for the effects of economic d ev el- opmen t aid on the accum ulation of capital in the THE SY NTHESIS OF R EGRESSION SLOPES IN MET A- ANAL YSIS 5 coun tries that receiv e suc h aid. T o explore differ- ences in sample and model sp ecification, their analy- ses exa mined aid-effectiv eness elasticities as the out- come and included as many as 11 dummy v ariable predictors that represent ed study differences. These dummy v ariables represen ted the t yp e of mo del ex- amined in the primary study (e.g., fis cal r esp onse mo dels v ersus growth equations), the nature of the data set u sed (its type and count ries included) and the pr esence of three different con trol v ariables. Con- trols for endogeneit y and the mo del type v ariables had significan t impacts on the elasticiti es, as did sample size. In this analysis, differences in the forms of the m o dels examined in the primary r esearc h pla y ed a large role in the synthesis results. 3. EXISTING METHODS FOR SUMMARIZING SLOPES Next we examine metho ds that h a v e b een pro- p osed for synthesizing regression slop es. These tec h- niques h av e b een describ ed in the literature, bu t some d o not app ear to h a v e b een used in meta- analytic pr actice . 3.1 Summaries of Slopes or F unctions of Slop es Sev eral authors ha v e us ed direct and simple sum- maries of slop es or differences in slop es, although none h as p ro vided a clear statistical justification for the app roac hes used. Jarrell and Stanley ( 1990 ) used slop es for a dummy v ariable th at represen ted union m emb ership (fr om r egression mo d els p redict- ing log w age v alues) in a review of the differences in w ages b et w een un ion and non-union work ers. In a similar analysis, Stanley and Jarrell ( 1998 ) exam- ined the gender gap in wag es. Using ordinary least squares (OLS) regression analyses, Jarrell and Stan- ley examined tw o mo dels for the w age gap due to union mem b ership. One had 20 predictors represent- ing differences in sample and mo del sp ecification, and the other in clud ed 77 pr edictors. The initial 20 predictors represen ted differences in mo del sp ec- ification suc h as the nature of the w age v ariables used and differences in the samples analyzed (e.g., whether b lu e-colla r, white-collar or go v ernment w ork ers w ere included). The other mo del includ ed those 20 pr edictors, p lu s allo w ed th ose pr edictors to v ary o ve r time, and also included 10 indicators iden tifying particular data sets that w ere used and 27 in dicator v aria bles representing multiple fi ndings con tributed by 27 primary-study authors. Jarrell and S tanley app lied OLS in spite of ac- kno wledging that the err ors in their mo del we re likel y to b e heteroscedastic, n oting that “[v]arious efforts to adjust for the problem made little d ifference in this app licatio n” ( Jarrell and Stanley ( 199 0 ), page 56). Similarly S tanley and Jarrell used O LS meth- o ds and tested for homoscedasticit y usin g “con ve n- tional tests” ( Stanley and J arrell ( 1998 ), p age 961). It is not clear wh y these authors d id n ot find het- eroscedastici t y , unless their results arose from roughly equal sized samples, b ecause as will b e shown b elo w, slop es will typical ly not ha ve equal v ariances across studies and thus errors from mo dels with slop es as outcomes will typical ly not b e homoscedas- tic either. 3.2 Summaries of t St atistics Stanley and J arrell ( 1989 ) encouraged economists to summarize regression slop es and suggested u sing the t statistic (i.e., the slop e d ivided by its standard error) as an index. They suggested th is m etric as a w a y to deal with h eteroscedasti cit y of slop es across studies, w hic h could o ccur b ecause of sample size differences and differences in pr ecision. Th ey al so ar- gued that dividing b by its standard error remov es problems d ue to use of differen t scales across stud- ies. While summaries of t v alues ha v e long existed (e.g., W alk er and Sa w ( 1978 )), there are some d ra w- bac ks to their use. First and of greatest concern, the t con tains inf ormation on sample size and preci- sion as we ll as effect magnitude. Thus t can b ecome large either w hen the slop e itself is large or wh en its standard error is small, whic h o ccur s b oth when the sample is large and when there is little v aria- tion in the r egression residuals. Stanley and Jarrell argued that the t “is a standardized measure of the critical parameter of interest ” ( 2005 , page 304), but they did not say what the parameter of in terest is. Clearly t is not an estimator of β . Also these authors do not explain w hether one can u se a summary of the t v alues to obtain a slop e estimate after p o oling or sum marizing the t v alues. Moreo v er, it is some- times difficult to determine the direction of an effect from a t test if a slop e is n ot pr esen ted and the test is not s ignificant or when the researc her rep orts only the absolute v alue of the t . Giv en all of these con- cerns, t v alues are lik ely to b e less meaningful th an other in dices b ased on slop es wh en fin dings are to b e interpreted. While S tanley and Jarr ell did not initially de- scrib e exactly ho w one w ould summarize t v alues, 6 B. J. BECKER AN D M.-J. WU in practice what they and others ha v e often done is to mo del t s or f u nctions of t s in terms of predictors that c haracterize the regression mo dels in their syn - thesis. F or instance, Card and Krueger ( 1995 ) ex- amined log | t | v alues represen ting the effects of dif- feren t lev els of the minim um w age on emp lo ymen t rates. They estimate d ordinary least squares regres- sions for th e log | t | v alues which included as pre- dictors the log of the square ro ot of th e error d e- grees of freedom in the primary study , a du mm y in- dicating w hether the data in cluded a subsample of teenagers and the num b er of explanatory v ariables in the p rimary-study regression mo del. Based on Jarrell and Stanley’s recommendation, Lau, Sigelman, Heldman and Babbitt ( 1999 ) used t v alues in a summary of results from group com- parison studies and regression studies that fo cu s ed on the effect of negativ e p olitical adv ertisemen ts on p olitical campaigns. They foun d that ab out one- quarter of their data p oin ts “come from ordinary least squares (OLS) or logistic regression equations, and th ere is no un iv ersally ac cepted metho d for han- dling su c h data in a m eta-anal ysis” (Lau et al., 19 99 , page 855). T o a v oid losing data, they extracted t statistic s asso ciated with regression co efficien ts that represen ted mean d ifferences b et we en group s exp osed to negativ e adv ertisemen ts and con trol groups (ex- p osed to no adv ertisemen ts or p ositiv e adv ertise- men ts). They conv erted the t v al ues into standard- ized mean differences ( ds ), via d = 2 t/ ( df ) 1 / 2 . T h ey argued that the ds obtained via this transformation could b e com bined with other ds from group com- parisons. Ho w ev er, to the exten t that the pr imary- study regression m o dels included other imp ortant con trol v ariables, these t s lik ely pro duced partial ef- fect sizes, whic h do hav e slight ly different distribu- tions from “t ypical” zero-order effect sizes ( Keef and Rob erts ( 2004 )). Another index that is related to the t v alue is Timm’s (2004 ) “ubiqu itous effect size.” This index can represent a single s lop e or a linear com bination of slop es. Timm’s index r esolv es the problem of de- p endence of the t on sample size b ecause it incorp o- rates a multiplier that reduces the influ ence of the sample size on its v alue. Ho w ev er, to date Timm’s index has not b een u sed in syn theses of slop es and Timm did not provide metho d s for syn thesizing his index. 3.3 Iterative Least Squares Regre ssions An iterativ e GLS app roac h wa s p r op osed by Han u- shek ( 1974 ) to summarize slop es representing re- turns to sc ho oling. Ha nushek’s m etho d optimally re- quires the raw data in order to estimate a co v ariance matrix among the slop es. Ho w ev er, he suggested an alternativ e approac h w hereb y part of the co v ari- ance matrix could b e estimated using OLS regres- sion across stud ies and the estimate obtained from this step w ould then b e added to a fun ction of ra w data from the original (within-study) regressions. T o the exten t that the approac h requires raw data and infrequent ly rep orted summary v alues from the orig- inal studies, it will n ot b e applicable in many meta- analytic settings. 3.4 Dose-Response Mo dels in Epidemiology Greenland ( 1987 ), Greenland a nd Longnec k er ( 1987 ) and Sh i and Copas ( 200 4 ) considered slop es that r elate the amoun t of exp osur e to some sub- stance to o dds-ratio outcomes. T ypical studies re- late lev els of exp osure (e.g., to alcohol, to smok e as in passiv e smoking, etc.) to outcomes including diagnoses of v arious kinds of cancer and other dis- eases. These stud ies fit in to the regression frame- w ork b ecause researc hers w ant to know wh ether the lev el of exp osure to some subs tance predicts higher lev els of p roblematic outcomes (e.g., h igher rates of cancer). Some issues are similar to those for con- tin uous outcomes, b ut the outcome metric differs in these cases b ecause it is typical ly a dichoto my (sur- viv al versus death, presence of some d isease ve rsu s no disease, etc.). I n the epidemiolog y literature t yp- ical fi xed and random-effects syn theses of the dose- resp onse slop es ha ve b een conducted (w eigh ting by the within-study slop e v ariances), and the issue of dep endence has b een addr essed b y incorp orating a within-study co v ariance b et w een o dds ratios (at dif- feren t exp osur e lev els) in to the analyses. This co v ari- ance is different from the co v aria nce b et we en slop es, whic h is incorp orated in our m etho ds b elo w. Shi and Copas ( 2004 ) argued for the use of maxi- m um lik eliho o d est imators of the mean dose-resp onse slop e and a b et wee n-stud ies v ariance comp onent for the slop es, and they also describ e a likel iho o d test of homogeneit y of the dose-resp onse slop es. Sh i and Copas considered a biv ariate regression because only one predictor (exp osure to the dosing v ariable) was used in the within-study m o del. They argued that their appr oac h is also appr o ximate for adjusted o d ds THE SY NTHESIS OF R EGRESSION SLOPES IN MET A- ANAL YSIS 7 ratios (e.g., adju sted for age or other pr edictors), pro vided the adjustments are not great. The ad- justed o dds ratio case is similar to the t ypical sit- uation in most areas of so cial science, where multi- ple con trol v ariables are included in eac h regression mo del. 3.5 Validi ty- Generalization Approache s A considerable literature exists concerning the syn- thesis of test v ali dities (e.g., Hun ter and Sc hmidt ( 2004 )). This area is kno wn as v alidit y generaliza- tion, with a key issue b eing whether test v alidi- ties generalize (i.e., can b e app lied reasonably well) across job t yp es and job settings. T est v alidities are t ypically in d ices that represent the relation b et we en a predictiv e test (e.g ., an emplo yment selec tion test) and some later outcome su c h as job p erformance. A k ey issue in this literat ur e is the effect of differen- tial test reliabilit y across stud ies, th us corrections for measuremen t error are a standard part of th e v al idit y-generalizatio n appr oac h. While test v ali di- ties are most often represen ted b y correlatio n co effi- cien ts, on o ccasion more complex r egression mo d - els are used to examine test v alidit y . Ra j u, F ral- icx and Steinhaus ( 1986 ) estimated the mean slop e and b etw een-studies v ariation in slop es with correc- tions for un reliabilit y in X , where X is a pred ic- tiv e test whose v alidit y is of inte rest. T h eir meth- o ds parallel those p resen ted by Hun ter and Sc hmidt ( 2004 ) for analyzing correlatio n co efficient s, w hic h ha v e b een contro v ersial in the m eta-analysis litera- ture (see, e.g., Hedges ( 1988 )). E ven so, they were used by C rouc h ( 1995 , 1996 ) and Ro ot and col- leagues ( 2003 ). Later Ra ju, P appas and Williams ( 1989 ) conducted an empirical Mon te Carlo study of a v alidit y data base to examine the p erformance of metho ds usin g slop es and correlations and cov ari- ances to represen t v alidities. 3.6 W eighted Least Squares (Uni vari ate) App roaches The weig hted least squ ares (WLS) approac h wa s used by Bini, C o elho and Diniz-Filho ( 2001 ), who cited Hedges and Olkin ( 1985 ) as the basis of their approac h. Greenland and Longnec k er ( 1987 ) also d e- scrib ed this approac h. If we consider the mo del in ( 1 ) ab ov e r elating some X s to Y for p erson j in study i , w e ma y w an t an estimate of the slop e for one predictor, sa y X 1 . Estimating mo del ( 1 ) in eac h of k s tu d ies (using the same estimation m etho d, suc h as ordin ary least squ ares) pro d u ces indep en- den t and appro ximately n ormally d istributed esti- mates of the p opulation slop es β 11 , β 21 , . . . , β k 1 . If w e denote those estimates as b 11 , b 21 , . . . , b k 1 w e can use least squares metho ds to summarize the slop es. Th us, for instance, w e can compute the com bined slop e b · 1 , b · 1 = P k i =1 w i 1 b i 1 P k i =1 w i 1 , (2) where k is th e num b er of slop es com bined, b i 1 is the slop e for X 1 from study i and w i 1 is the w eigh t for that slop e in the i th study , whic h is the recipro cal of the slop e v ariance [ w i 1 = 1 / V ar( b i 1 )]. The v ariance of b · 1 is give n as V ( b · 1 ) = 1 P w i 1 . (3) This approac h could also b e applied to partial corre- lations or standardized regression s lop es. If standard errors were not av ailable, one could w eigh t by sam- ple size, as the relev ant standard errors are t ypically a function of n or of the degrees of freedom for the regression mo del. 3.7 Multiva riate Bay es ian Approach One last p rop osed metho d f or sim ultaneously es- timating a set of regression mo dels was giv en by No vic k and collea gues (No vic k, Jac kson, Tha y er and Cole, 1972 ) in the con text of the v alidit y of college- admissions pr ed iction, wh ere all p redictors are con- sisten tly measured across colleg es. F urthering a Ba y esian metho d attributed to Lind ley , the authors argued for a multistag e Ba y esian form ulation inv olv- ing ra w data, its parameters (the slop es) and h yp er- parameters (e.g., the v ariances of the slop es). Ho w- ev er, w h ile the metho d constitutes an imp r o v emen t b ey ond the m etho ds abov e b ecause it is m ultiv ariate and uses simulta neous estimation, this app r oac h re- quires full acce ss to the ra w data so is not applica ble in the m eta-analytic con text. 4. MUL TIV ARIA TE GLS APPRO A CH Most of the analyses presen ted ab ov e are reason- able if one w an ts to synthesize estimates of a single p opulation slop e and if most of the studies in v olving that slop e examine simple mo dels. Ho wev er, within the i th sample, the P + 1 slop es b i 0 , b i 1 , . . . , b iP are often correlated and there may b e in terest in ob- taining an o v erall regression mo del (rather than a 8 B. J. BECKER AN D M.-J. WU single slop e estimate). T o syn thesize slop e v ectors b 1 , b 2 , . . . , b k , we n eed generalized least squares (GLS) metho d s, primarily b ecause of the unequal v aria nces of effects for studies of d ifferen t sizes. [Stan- ley and Jarrell argued that one could obtain esti- mates of the vec tor of slop es by solving a system of equations with the slop es as endogenous v ariables ( Stanley and Jarrell ( 198 9 ), page 169). Ho w eve r, they d id not discuss exactly h o w to d o so or how to deal with the fact that w ithin eac h study the slop es will b e in tercorrelate d.] An o v erview of the use of GLS for dep enden t standardized-mean-difference ef- fect sizes was giv en by Raudenbush, Bec k er and K ala- ian ( 1988 ), and w e app ly a similar app roac h here to sets of slop es. T o use GLS, w e n eed estimates of the P + 1 slop es from eac h of the k samples (this includes the in- tercept b i 0 ) and their cov ariance matrices Cov( b i ). It is also p ossible to include studies that examine subsets of the P predictors; we comment on ho w this w ould b e done as w e discuss d etails of the ap- proac h. Within sample i , the OLS estimate of β i = ( β i 0 , β i 1 , . . . , β iP ) is f r equen tly rep orted. Th e estima- tor is b i = ( b i 0 , b i 1 , . . . , b iP ) = ( X ′ i X i ) − 1 X ′ i Y i , with Σ i = C ov( b i ) = ( X ′ i X i ) − 1 σ 2 i , where X i is the matrix of predictor v alues in the i th sample, plus a constant if the int ercept is included. Typical ly σ 2 i is n ot kn o wn, b u t rather is estimate d with S 2 i , the m ean squared error (MSE) of the regression in study i . In large samples, b i is normally distributed with mean β i and v ariance Σ i , whic h is the basis for the GLS app roac h. W e w ill assume a common fixed- effects mo del (see Hedges and V ev ea ( 1998 )) whic h presumes that all samples incorp orate the same P predictors in the within-stud y regression mo del, and also assu me th at the v ectors b i estimate a common p opulation slop e v ector β . W e stac k the k sample slop e v ectors and mak e a blo c kwise diagonal matrix of the C ov( b i ) matrices, then apply GLS estimation. First define b =      b 1 b 2 . . . b k      and Σ =     Co v( b 1 ) 0 0 0 0 Cov( b 2 ) 0 0 0 0 · · · 0 0 0 0 Cov( b k )     . Olkin ( 2003 ) p ointed out that in some cases the co- v aria nce matrices Cov( b i ) could b e p o oled; b elo w w e discuss the case wh ere a p o oled MSE is a v ail- able. Then under the assump tion that eac h slop e v ecto r b i is estimating β , we ha ve the mo del b =                b 10 b 11 . . . b 1 P . . . b k 0 . . . b k P                = W β + e =                   1 0 0 0 0 1 0 0 0 0 · · · 0 0 0 0 1 . . . . . . . . . . . . 1 0 0 0 0 1 · · · 0 0 0 0 1                   ∗      β 0 β 1 . . . β P      + e . The slop es are mo d eled as a f u nction of β (the v ec- tor of P + 1 p opu lation slop es) and a design ma- trix W comp osed of zeros an d ones that identify whic h slop es are estimated in eac h samp le. When all s amp les examine the same predictors, a stac k of ( P + 1) × ( P + 1) iden tit y matrices serves as W in the mo del b = W β + e , with Co v ( e ) = Co v ( b ) from ab o v e. I f the samples do not all estimate the same mo del (i.e., some mo dels u se fewer than th e full set of P predictors), we can still use the GLS form u- lation to include those results in the syn thesis. In suc h cases th e comp onen t of W that represent s a sample with few er than P pr ed ictors would not b e a full id en tit y matrix; ro w p + 1 of the id en tit y matrix for sample i would b e omitted if the p th predic- tor wa s not included in study i . Ho w ev er, as men- tioned ab o v e, the in terpretations (a nd distribu tions) of slop es from reduced mo d els w ould n ot b e exactly the same as for slop es fr om mo dels with al l P predic- tors, and estimatio n of su c h quanti ties as σ 2 i will b e more complicate d b ecause σ 2 i and σ 2 i ′ (from samples i and i ′ ) ma y r epresen t differen t p opulation quan ti- ties if differen t sets of predictors w ere examined in samples i and i ′ . It is also p ossible to m o dify this approac h some- what to examine the influence of particular addi- tional pr edictors on a fo cal p redictor’s slop e. F or in- stance, su pp ose the fo cus was on the role of teac her v erbal abilit y as a predictor of stud en t ac hiev emen t. It could b e of int erest to see whether the slop e for teac her verbal abilit y is differen t w hen a measure of studen ts’ pr ior ac hiev ement is in clud ed in the mo d el. This could b e accomplished b y adding a column to THE SY NTHESIS OF R EGRESSION SLOPES IN MET A- ANAL YSIS 9 the W matrix that would con tain a 1 in eac h ro w represen ting a ve rb al-abilit y s lop e that came from a mo del that also included p rior ac hiev emen t. A small example illustrates this idea. Supp ose th at X 1 is teac her v erbal abilit y , X 2 is prior ac hiev emen t and X 3 is so cio economic status. Tw o stud ies are a v ail- able, only one of whic h (say s tu dy 1) includes prior ac hiev emen t. The GLS mo del would b e b =              b 10 b 11 b 12 b 13 − − − b 20 b 21 b 23              = W β + e =              1 0 0 0 0 0 1 0 0 1 0 0 1 0 0 0 0 0 1 0 − − − − − 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0              ∗       β 0 β 1 β 2 β 3 γ 1       + e . The last column of W con tains a 1 in r o w 2 (the ro w for th e verbal-a bilit y s lop e in stud y 1), sho wing that the fi rst study included pr ior ac hieve ment ( X 2 ) in the study-lev el mo del. Ro w 6—the ro w for the v erbal-abilit y slop e in stu d y 2—do es not h a v e a 1 in the last column b ecause stud y 2 did not include X 2 . Also study 2 d o es n ot hav e a ro w of W with a 1 in column 2 b ecause there is no estimate of β 22 . This mo del conta ins a fi fth parameter, denoted γ 1 in the displa y , that repr esen ts the difference in the slop e of teac her ve rb al abilit y when prior achiev e- men t is con trolled. F rom this mo del we can deter- mine that for study 1, the exp ected v alue of b 11 is β 1 + γ 1 , while for study 2, E [ b 21 ] = β 1 . By includ in g additional columns for key con trol v ariables or for other study features, the meta-analyst can examine h yp otheses ab out whether the fo cal slop e is affected b y those elemen ts of the original studies and their regression mo dels. The details of su c h tests are de- scrib ed b elo w. W e caution, ho we ver, th at including large n umbers of added predictors ma y lead to mul- ticolli nearit y , thus meta-analysts would b e wise to carefully examine their m o dels f or the presence of this problem. Regardless of the comp onen ts of W and β , w e estimate β and its co v ariance as ˆ β ∗ = ( W ′ Σ − 1 W ) − 1 W ′ Σ − 1 b and Co v ( ˆ β ∗ ) = ( W ′ Σ − 1 W ) − 1 . Often, as noted ab o ve , w e do not kno w Co v ( b ) = Σ ; th us we typica lly substitute an estimate, wh ic h we shall denote V , and compute instead ˆ β = ( W ′ V − 1 W ) − 1 W ′ V − 1 b (4) and Co v ( ˆ β ) = ( W ′ V − 1 W ) − 1 . (5) With large samples and u nder typical regularit y con- ditions, ˆ β ∼ N( β , C ov ( ˆ β )); th us confid ence in terv als for eac h elemen t of β are a v ai lable, using ˆ β p ± Z 1 − α/ 2 p C pp , where Z 1 − α/ 2 is the u pp er tail 1 − α/ 2 critical v alue of the standard normal distribution and C pp is the p th diagonal ele- men t of the Co v ( ˆ β ) matrix, the v ariance of ˆ β . Also a test of the hyp othesis that the p th slop e β p = 0 can b e obtained via Z = ˆ β p p C pp , whic h is a standard normal deviate und er the null h yp othesis that β p = 0. The v alue of Z is compared to the cutp oints of the standard n ormal distribu tion. Sev eral other tests are a v ail able as w ell. A test of mo del fit, wh ich is essen tially a test of h omogeneit y of the regression int ercepts and slop es across sam- ples and across predictors, is giv en b y Q E = ( b − W ˆ β ) ′ V − 1 ( b − W ˆ β ) , whic h has a large-sample c hi-squared distribu tion with ( k − 1)( P + 1) degrees of f r eedom if all s lop es and in tercepts are included. If F additional columns are added to W to represen t study features, then the degrees of freedom will b e ( k − 1)( P + F + 1). If the m agnitudes of the inte rcepts are n ot of inter- est, a mo dified Q E test can also b e compu ted by including only the predictor slop es, th us reducing the dimension of W and including only those v al- ues of in terest in b , ˆ β and V − 1 . In that case, Q E is chi-squared with ( k − 1) P degrees of freedom. If Q E is large relativ e to cutp oin ts of the approp r i- ate c hi-squared distribution, the slop es v ary b ey ond what one w ould exp ect to see giv en only sampling v aria bilit y . 10 B. J. BECKER AN D M.-J. WU A test of the comp osite h yp othesis that β = 0 is giv en by Q B = ˆ β ′ Co v ( ˆ β ) ˆ β , whic h is c hi-squared with P + 1 degrees of freedom under the n ull h yp othesis that β = 0 or with P de- grees of freedom if only pr edictor slop es are included (see, e.g., Hedges and Olkin ( 1985 )). 4.1 Sp ecial Cases of the GLS App roach The pr oblem with the approac h just d escrib ed its that it is extremely rare to find the full co v ariance matrix of the slop es C o v ( b ) rep orted in a primary researc h study . Thus it is useful to note that the estimator sho wn in ( 4 ) simp lifies to the we ight ed least squares (WLS) un iv aria te estimator give n in ( 2 ) if the off-diagonals in Cov( b ) or V are set equal to zero. Another sp ecial case is one in whic h it is p ossi- ble to p o ol the estimates of σ 2 i across studies. If all studies examine the same mo del and separate es- timates of σ 2 i are a v ailable, then it is p ossible to remo v e th e MSE v alues from the Co v ( b ) m atrices and us e a blo c kwise diagonal matrix X ∗ con taining the ( X ′ i X i ) − 1 matrices in place of V in form ulas ( 4 ) and ( 5 ). It is sho wn in the App end ix that ˆ β = ( W ′ ( X ∗ ) − 1 W ) − 1 W ′ ( X ∗ ) − 1 b (6) pro du ces an estimate of β equiv alen t to the v alue that w ould b e obtained from a p o oled sample. This is b ecause ( X ∗ ) − 1 is a blo ckwise diagonal matrix con taining th e v alues of X ′ i X i , and the p ro duct W ′ ( X ∗ ) − 1 W sums th e X ′ i X i v al ues across the k studies. Similarly ( X ∗ ) − 1 b equals the su m across studies of the v alues of the pro d u cts X ′ i Y i , lead- ing to equiv ale nce with the estimator based on the p o oled samp le. V alues of ( X ′ i X i ) − 1 can be estimated if eac h s tu d y rep orts the co v ariance matrix for the slop es and S 2 i , the estimate of σ 2 i (or other quan tities that allo w computation of S 2 i , e.g., the v ariance of the out- come and the R 2 for the r egression). Eac h elemen t of Co v ( b ) is divided by the estimated MSE: ( X ′ i X i ) − 1 = Co v ( b i ) S − 2 i . T his metho d requires that a p o oled v al ue of the MS E (say S 2 ∗ ) b e obtained and m ul- tiplied by the co v ariance of the synthesiz ed slop e estimator computed using X ∗ , to comp ensate for S 2 i b eing remov ed when X ∗ is substituted f or V . Th us the matrix of co v ariances among the synthe- sized slop es is Co v ( ˆ β ) = ( W ′ ( X ∗ ) − 1 W ) − 1 S 2 ∗ . (7) One p ossible estimator S 2 ∗ could b e S 2 ∗ = X i dfe i S 2 i . X i dfe i , where dfe i is th e degrees of freedom f or error in study i . Unfortunately pr imary r esearc hers do n ot alw a ys rep ort the v al ue of S 2 i , the mean squared error of the regression mo d el in the primary study . Giv en this and the rarit y of find ing full Co v ( b i ) ma- trices, it is exp ected that this sp ecial case will b e relativ ely uncommon. 4.2 Limitations The discussion of sp ecial cases fo cuses our atten- tion to the fact that one w eakness of the prop osed GLS approac h is that it u ses the Cov( b i ) matri- ces that are rarely r ep orted. It is unlik ely , ev en with more stringen t rep orting requiremen ts, that authors will routinely b egin to rep ort these matrices, p artic- ularly in primary researc h stud ies where man y mo d - els with large n umbers of pr edictors are estimated and compared. There are t wo p ossible app roac hes to this prob- lem. O n e is to simp ly assume the slop es are ind ep en- den t, use the squared standard errors of the s lop es as the diagonal elemen ts of Cov( b i ) and set the off- diagonal elements to zero. T his p ro duces w eigh ted least squares estimates. A slightl y more conserv ativ e approac h wo uld b e to assume a common correla- tion v alue among all slopes [e. g., Corr( b ip , b ip ′ ) = 0 . 2] and then compu te the off-diagonal element s of eac h Co v ( b i ) matrix as the pro duct of the slop e standard errors (SE s ) and that common correlation, sp ecifi- cally , Co v( b ip , b ip ′ ) = Corr( b ip , b ip ′ ) ∗ SE( b ip ) ∗ SE( b ip ′ ). One final p oin t regarding th is issue rela tes to mo del sp ecification in the pr imary researc h studies in the meta-analysis. That is, if a mo del is we ll sp ecified in study i , there should b e no serious mult icollinearit y and the degree of co v ariation among the slop es in Co v ( b i ) should not b e great. In su c h cases, setting all off-diagonal element s of C o v( b i ) to zero would not ha ve serious consequences. How ev er, rep orting con v en tions in man y fi elds do n ot require authors to men tion w h ether multico llinearit y was assessed or to rep ort on m ulticol linearit y diagnostics. So the meta- analyst m ust trust that the p rimary study auth ors actually c hec k ed for multico llinearit y and th at an y mo dels rep orted up on are relativ ely free fr om this problem. THE SY NTHESIS OF R EGRESSION SLOPES IN MET A- ANAL YSIS 11 5. EXAMPLE In this example we u se data from the base yea r of the National Ed ucation Longitudinal Stud y of 1988 (NELS:88). NELS:88 is a survey of a national s amp le of h igh-sc ho ol stu den ts from o v er 1000 schools. Th e same measures are used across sc ho ols and when analyzed with prop er weigh ts, the full sample repre- sen ts the U.S. grade 10 high-sc ho ol p opu lation from 1988. In our example w e u se as “studies” 13 sc ho ols with samples of more than 45 stud ents; w e do not use the NELS:88 sampling we ight s that w ould pr o- duce results that reflect the n ational p opulation. Both the sc ho ol-lev el sampling w eigh ts and the within-sc ho ol weig hts that could b e used to mak e eac h s c ho ol’s estimate s reflectiv e of the p opulation of that s c ho ol w ere ignored. 5.1 Mo del Our regression mo del uses three of th e standard- ized cognitiv e tests administered as part of the NELS:88 surv ey—the science, mathematics and reading scales. Th is mo del views science achie ve- men t as a function of math and reading test p erfor- mance. Sp ecifically , Y repr esen ts the NELS:88 sci- ence ac hieve ment test, X 1 is the mathematics test and X 2 is the reading test. Th e mo del estimated in study i is Y ij = β 0 + β 1 X 1 j + β 2 X 2 j + e ij for student j , with error e ij . W e u se ordinary least squares to obtain sc ho ol lev el estimates of this mo del. Computations we re d one u s ing PR OC REG and PR OC IML in S AS . Our analyses are b ased on the item-resp onse-theory estimated num b er-right s cores for these test batter- ies; therefore the ra w slop es can b e inte rpr eted as the pr edicted c hange in the science test score for a one item increase in th e math or reading test score. The science test had 25 items, the math test had 40 and the reading test had 21 items. Means across the 13 schools we re 13.5 or 54% correct on science ( SD = 5 . 7), 24.4 or 61% correct on math ( SD = 10 . 4) and 13.9 or 66% correct for reading ( SD = 5 . 7). The correlat ion b et w een math and read- ing scores w as r MR = 0 . 70, and eac h predictor was also correla ted with the outcome at ab out that same lev el ( r MS = 0 . 70, r RS = 0 . 67) in the full sample. 5.2 Results The regression mo d el with X 1 and X 2 as predic- tors of Y w as estimated within eac h of the sc ho ols, and the slop e estimat es and fitted mo dels are shown in T able 1 . Th e data f rom the 13 sc ho ols w ere also T able 1 Fitte d r e gr essions and MSE values f or ful l sample and 13 scho ol s MSE Sample n i Fitted regression ( S 2 i for sc ho ol i ) F ull 664 2 . 552 + 0 . 245 X 1 + 0 . 358 X 2 14 . 44 1 64 5 . 470 + 0 . 219 X 1 + 0 . 260 X 2 17 . 46 2 59 3 . 591 + 0 . 246 X 1 + 0 . 270 X 2 14 . 24 3 67 5 . 619 + 0 . 040 X 1 + 0 . 638 X 2 14 . 05 4 45 4 . 381 + 0 . 181 X 1 + 0 . 392 X 2 10 . 75 5 47 4 . 305 + 0 . 260 X 1 + 0 . 282 X 2 9 . 32 6 45 2 . 346 + 0 . 185 X 1 + 0 . 195 X 2 14 . 60 7 45 0 . 228 + 0 . 283 X 1 + 0 . 339 X 2 9 . 80 8 56 2 . 289 + 0 . 289 X 1 + 0 . 312 X 2 13 . 32 9 45 3 . 600 + 0 . 248 X 1 + 0 . 263 X 2 12 . 65 10 51 2 . 156 + 0 . 192 X 1 + 0 . 498 X 2 6 . 50 11 48 3 . 621 + 0 . 133 X 1 + 0 . 413 X 2 11 . 02 12 45 3 . 144 + 0 . 250 X 1 + 0 . 382 X 2 17 . 65 13 47 3 . 781 + 0 . 251 X 1 + 0 . 151 X 2 13 . 20 p o oled (used as a single sample) and th e full m o del including in tercepts was estimated across all sc ho ols (for all 664 cases); this resu lt is lab eled “F ull s am- ple.” The estimated mo d el fr om this analysis of the 13 s chools together w as ˆ Y j = 2 . 552 + 0 . 245 X 1 j + 0 . 358 X 2 j and it is s h o wn in the first ro w of T able 1 . (The subscrip t j h as b een omitted from the table en- tries for simplicit y .) Insp ection of the mo dels for the 13 sc ho ols shows some v ariatio n in the slop es and in tercepts; the most unusual lo oking mo d el is for sc ho ol 3. Also casual insp ection of the mean squared errors shows some v ariation in the S 2 i v al ues, with sc ho ol 10 showing the s mallest v alue. How ev er, Lev- ene’s test suggests the error v ariances are n ot dif- feren t [ F (12 , 651) = 1 . 25 , p = 0 . 25], indicating that it is reasonable to p ro ceed with the analysis based on the p o oled MSE. (Although here w e ha v e the ra w data and can compute Lev ene’s test, in prac- tice other tests that do not require r a w data suc h as F max or C o c hran’s C could b e used to test residual v aria nce equalit y .) The upp er triangles of the co v ariance , correlation and X ′ i X i matrices among the slop es for thr ee of the sc ho ols and the fu ll sample are sho wn in T able 2 . The X ′ i X i matrices are used in the third metho d of estimation u s in g the p o oled MSE . The elemen ts of the Co v( b i ) matric es are obtained as the p ro ducts of the entries in ( X ′ i X i ) − 1 times the MSE [e.g., for sc ho ol 1, the first en try in C o v ( b 1 ) is 1.934, whic h is within roundin g error of 17 . 463 × 0 . 110 7 = 1 . 933]. Also the MSE p o oled across the 13 schools is S 2 ∗ = 12 . 83. 12 B. J. BECKER AN D M.-J. WU T able 3 rep eats the OLS results for the p o oled sample (to facilitate comparisons) and also presents the slop es estimated usin g the three synthesis meth- o ds describ ed ab o ve . T he first set of results is based on the GLS estimation metho d with mean and v ari- ance giv en in ( 4 ) and ( 5 ). While th e inte rcept d if- fers somewhat f rom the p o oled-sample int ercept, the slop e coefficien ts are b oth within 0.015 of the v alues estimated in the f u ll sample. Considering that the slop es represent predicted change on a 25-p oint sci- ence test (give n a one-p oin t c hange on X ), these are v ery small differences. The test of homogeneit y of the models usin g Q E defined ab o v e for all slop es and in tercepts sho ws that indeed the slop es and inte r- cepts are not homogeneous ( Q E = 114 . 16, df = 36, p < 0 . 001), and may not hav e come from a single p opulation. Ho wev er, this test asks whether all pa- rameters are equal across sc ho ols; th us the test can also b e large if the in tercepts d iffer. The test can b e computed for th e predictor slop es only (omit- ting b 0 v al ues): when this is done, the Q E v al ue is smaller ( Q E = 21 . 74, df = 24, p = 0 . 59), and indi- cates the math and reading slop es are homogeneous across sc ho ols. Also at least one of the slop es differs from zero, according to the Q B test ( Q B = 518 . 16, df = 2, p < 0 . 001). T able 2 Covarianc e and X ′ X matric es for thr e e studies and ful l sample Sample X ′ X Cov (b) Corr (b) I M R I M R M R F ull I 0 . 0117 5 − 0 . 00018 − 0 . 00042 0 . 1697 − 0 . 0026 − 0 . 0060 − 0 . 32 − 0 . 40 ( n = 664 ) M 0 . 00003 − 0 . 00003 0 . 0014 − 0 . 0005 − 0 . 70 R 0 . 0000 9 0 . 0013 School 1 I 0 . 1107 − 0 . 0037 − 0 . 0017 1 . 9340 − 0 . 0648 − 0 . 0302 − 0 . 61 − 0 . 22 ( n 1 = 64) M 0 . 00 03 − 0 . 0002 0 . 0058 − 0 . 0043 − 0 . 57 R 0 . 0006 0 . 0098 School 2 I 0 . 0914 − 0 . 0015 − 0 . 0034 1 . 3018 − 0 . 0218 − 0 . 0482 − 0 . 36 − 0 . 44 ( n 2 = 59) M 0 . 00 02 − 0 . 0002 0 . 0028 − 0 . 0030 − 0 . 60 R 0 . 0006 0 . 0092 School 3 I 0 . 4267 − 0 . 0103 − 0 . 0058 5 . 9953 − 0 . 1449 − 0 . 0817 − 0 . 65 − 0 . 26 ( n 3 = 67) M 0 . 00 06 − 0 . 0004 0 . 0082 − 0 . 0063 − 0 . 54 R 0 . 0012 0 . 0164 T able 3 R esults of synthesis Metho d of Slop e estimates estimation Intercept Math Readin g Co v (b ) Corr (b) I M R F ull sample 2.552 0.245 0.358 I 0.1697 − 0 . 0026 − 0 . 0060 − 0.32 − 0.40 ( n = 664 ) M 0 . 0004 − 0 . 0005 − 0.70 R 0 . 0013 GLS 2.268 0.247 0.373 I 0.1463 − 0 . 0021 − 0 . 0054 − 0.30 − 0.41 M 0 . 0003 − 0 . 0004 − 0.71 R 0 . 0012 WLS 2.936 0.221 0.343 I 0.1747 0 0 0 0 M 0 . 0004 0 0 R 0 . 0012 GLS using ( X ′ X ) − 1 2.552 0.245 0.358 I 0.1507 − 0 . 0023 − 0 . 0053 − 0.32 − 0.40 M 0 . 0004 − 0 . 0004 − 0.70 R 0 . 0012 THE SY NTHESIS OF R EGRESSION SLOPES IN MET A- ANAL YSIS 13 A t this p oint more d etailed analyses of slop es f or eac h predictor migh t b e of use, and standard u n i- v aria te meta-analysis pro cedu res (e.g., Hedges and Olkin ( 1985 )) could b e applied to eac h set of slop e v alues, or other GLS b ased analyses can b e u sed if it is desired to mo d el the vec tors of slop es (Rauden bush , Bec k er and Kalaian, 1988 ). Also to explore b et we en-studies differences in mo d - els one could then examine mo d erating v ariable s as describ ed ab o v e. If the slop es or parameters for ad- ditional study features still did not app ear homoge- neous, one could estimate b et w een-studies v ariance comp onen ts f or eac h of the slop es. A v ariet y of esti- mators for the b et we en-studies v ariance exist (e.g., Hedges and Olkin ( 1985 ); Sidik and Jonkman, 2005 ) and an estimated b et w een-studies v ariance could then b e added to eac h study’s sampling v ariance to augmen t its uncertain t y . The next set of resu lts wa s obtained b y eliminat- ing the off-diagonal elemen ts from th e Co v ( b ) ma- trices. Th is is equiv alent to estimating the slop es using the univ ariate metho ds s ho wn in displays ( 2 ) and ( 3 ). These v alues also do not deviate far from the fu ll-sample v alues; b oth slop es are within 0.025 p oint s of the slop es from the fu ll sample—deviating only sligh tly more than the GLS v alues. Th is is in spite of the fact that the predictors and outcome sho w mo derate in tercorrelations as can b e seen by insp ection of the Corr( b ) matrices shown in T able 2 . Finally , the third set of results is computed us- ing the X ∗ matrix in p lace of the Co v ( b ) matrix, and the p o oled MS E in place of eac h S 2 i v al ue. As noted ab o v e the slop e computed in this w a y is iden- tical to the slop e for the full sample, and the co v ari- ance matrix differs from the full sample matrix b y a constan t factor equal to the ratio of the estimated p o oled MSE to th e full sample MSE (here that ratio is 12 . 83 / 1 4 . 44 = 0 . 8 9). It is somewhat pr oblematic that the v ariances of slop es from the meta-analysis are less than or equal to the v alues from the full sample (thus suggesting more pr ecision). F rom one application it is not p ossible to determine whether this is a result of the particular nature of the exam- ple data (11 of 13 sc ho ols sh o w MSEs smaller than the MSE of 14.44 for the full data set) or something more p erv asiv e. F u rther examination of the p erf or- mance of these estimati on metho ds via Mon te C arlo metho ds w ill ind icate wh ether a consistent pattern of und erestimatio n is foun d. Our new metho d tak es int o accoun t the interrela- tionships among predictors fr om the p rimary stud - ies, as well as heteroscedasticit y of the slop es, via the v ariance–c o v ariance matrix of the slop es. Both features sh ould represen t imp ro v ement s on ordinary least squares metho ds. Suc h OLS appr oac hes t yp- ically include du mm y v ariables to sho w the pr es- ence of sp ecific predictors or study features, but do not deal with the p ossible dep endence of the pre- dictors in the mo del(s), nor do they accoun t for the heteroscedastic it y inheren t in the slop e estimates. Ev en when off-diagonal elemen ts of Co v ( b ) were set to zero in our analysis, the we ight ed least squares slop es were v ery close to the full samp le slop es. 6. CONCLUSION This p ap er presen ts a review of existing metho ds for the synthesis of regression s lop es and a n ew mul- tiv ariate appr oac h based on generalized least squares estimatio n that is app licable to the meta-analyt ic con text. T able 4 summarizes th e main s tr en gths and w eaknesses of all of the metho ds. Two method s re- quire raw data and thus are not appropriate for the meta-analysis con text. Fiv e others fo cus only on a single fo cal slop e (or some related index such as a t test of that slop e) and thus cannot provide an o v er- all mo del b ased on the syn thesis. Also these meth- o ds ignore dep end en ce among slop es b y omitting all but the fo cal slop e. Some add itionally ignore the in- heren t d ifferen tial precision of slop es across studies b y applying ord inary least squares estimation m eth- o ds. The new m ultiv ariate GLS m etho d addresses these pr oblems, but is itself limited b ecause infor- mation ab out co v ariat ion among slop es is t ypically not giv en in p rimary researc h rep orts. A comparison of the results of three v ariatio ns of the GLS approac h applied to an educational data set is made to the analysis of all data in a single p o oled analysis. The analyses pro duce v ery similar results and in some cases ha v e id en tical results (giv en the a v ai labilit y of sp ecific summary statistics suc h as mean squared errors for the individu al regression mo dels). Our results emp hasize the imp ortance of full r ep orting of sufficient statistics in p r imary re- searc h studies; with less complete information, the full GLS analysis is not p ossible. How ev er, ev en th e less complex w eigh ted least squares app roac h ap- p eared to p ro vide reasonable v alues in one example analysis. APPENDIX: EQUIV ALENCE OF FULL SAMPLE AND SYNTHESIZED RESUL TS WHEN σ 2 i = σ 2 F OR i = 1 TO k Consider k indep end en t samples or studies eac h examining a mo d el r elating p redictors X 1 through 14 B. J. BECKER AN D M.-J. WU X P to an outco me Y for case j . Sp ecifically , in study i , Y ij = β i 0 + β i 1 X ij 1 + · · · + β iP X ij P + e ij for j = 1 to n i . F or later use we also defin e X and Y by stac king the individual X i and Y i matrices: X =      X 1 X 2 . . . X k      and Y =      Y 1 Y 2 . . . Y k      . The OLS regression slope for the full com bined sam- ple is b ∗ = ( X ′ X ) − 1 X ′ Y . (A1) Within stu d y i , the OLS estimate of β i = ( β i 0 , β i 1 , . . . , β iP ) is b i = ( b i 0 , b i 1 , . . . , b iP ) = ( X ′ i X i ) − 1 X ′ i Y i with Σ i = Co v ( b i ) = ( X ′ i X i ) − 1 σ 2 i . If it is reasonable to assume that the error v ariances σ 2 i , for i = 1 to k are equal (e.g., if the k samples are dra wn fr om one p opulation), then w e hav e Σ i = Co v ( b i ) = ( X ′ i X i ) − 1 σ 2 . Next we d efine b =      b 1 b 2 . . . b k      and Σ =     Co v ( b 1 ) 0 0 0 0 Co v ( b 2 ) 0 0 0 0 · · · 0 0 0 0 Co v ( b k )     =     ( X ′ 1 X 1 ) − 1 0 0 0 0 ( X ′ 2 X 2 ) − 1 0 0 0 0 · · · 0 0 0 0 ( X ′ k X k ) − 1     σ 2 , T able 4 Metho ds of summarizing slop es Metho d Data needed Strength W eakness Simple slope summaries Slopes S imple, little data n eeded F o cuses on a single fo cal slope; ignores dep endence and precision of slopes Summaries of t statistics t v alues for slope tests Simple; little data needed ; F ocuses on only a single fo cal slope; t X s and Y s can b e on v alues contain irrelev an t information any scales about sample size; unclear ho w an ind ex of effect is obtained Iterative lea st squares Raw data Accounts for cov ariatio n Iteration needed to get co v ariance approac h among predictors matrix Dose–response mo d els S lopes and standard errors W eights by precision F o cuses on only a single focal slope; (WLS approac h for for mo dels with ignores dep end ence of slop es dic hotomous dic hotomous outcomes outcomes) V alidity generaliza tion Slopes, reliabilities of X Simple; little data needed Reliabilities often not rep orted; approac h and sample sizes ignores dep end ence of slop es Univ ariate WLS approach Slop es and stand ard errors Relativ ely simple; wei ghts F ocuses on only a single focal slop e; by precision ignores dep end ence of slop es Multiv ariate Ba yesian Ra w data Collatera l information can Multistage formulation; requires priors approac h b e shared across and hyperparameters; X s and Y s studies must be on same scales Multiv ariate GLS Slopes and Co v ( b ) W eigh ts by precision; Req u ires cov ari ances among slop es, approac h matrices accoun ts for which are often not rep orted co v ariation; p ro vides entire po oled model THE SY NTHESIS OF R EGRESSION SLOPES IN MET A- ANAL YSIS 15 whic h is lab eled X ∗ σ 2 in the text. When inv erted, this matrix is Σ − 1 =     ( X ′ 1 X 1 ) 0 0 0 0 ( X ′ 2 X 2 ) 0 0 0 0 · · · 0 0 0 0 ( X ′ k X k )     σ − 2 . Also the i th cross-pro duct matrix is X ∗ i X i =            n i X j X ij 1 X j X ij 2 X j X ij 1 X j X 2 ij 1 X j X ij 2 X ij 1 . . . . . . . . . X j X ij P X j X ij 1 X ij P X j X ij 2 X ij P · · · X j X ij P · · · X j X ij 1 X ij P . . . X j X 2 ij P            . The synthesize d GLS slop e estimator is ˆ β ∗ = ( W ′ Σ − 1 W ) − 1 W ′ Σ − 1 b , (A2) where W is a stac k of iden tit y matrices of dimen- sion P + 1 . The first comp onen t of the estimato r is ( W ′ Σ − 1 W ), whic h is a s u m of matrices: ( W ′ Σ − 1 W ) = ( X ′ 1 X 1 ) σ 2 + ( X ′ 2 X 2 ) σ 2 + · · · + ( X ′ k X k ) σ 2 . Equiv ale ntly , W ′ Σ − 1 W =            X i n i X i X j X ij 1 X i X j X ij 2 X i X j X ij 1 X i X j X 2 ij 1 X i X j X ij 2 X ij 1 . . . . . . . . . X i X j X ij P X i X j X ij 1 X ij P X i X j X ij 2 X ij P · · · X i X j X ij P · · · X i X j X ij 1 X ij P . . . X X j X 2 ij P            σ 2 , whic h is simply X ′ X σ 2 for the f ull sample (i.e., the sample p o oled across stud ies), s o W ′ Σ − 1 W = ( X ′ X ) − 1 σ − 2 . Thus we can w r ite ˆ β ∗ = [( X ′ X ) − 1 σ − 2 ] W ′ Σ − 1 b . (A3) Next we consider the term W ′ Σ − 1 b . T he pr o duct W ′ Σ − 1 is a matrix that is σ 2 times a concatenatio n of ( X ′ i X i ) matrices, sp ecifically W ′ Σ − 1 = [ X ′ 1 X 1 | X ′ 2 X 2 | · · · X ′ i X i · · · | X ′ k X k ] σ 2 . Also, b is the stac k ed v ector of the k individual sam- ple slop e v ectors. Th us W ′ Σ − 1 b = X ′ 1 X 1 b 1 σ 2 + X ′ 2 X 2 b 2 σ 2 + · · · + X ′ k X k b k σ 2 . Eac h comp onent of this su m is a ( P + 1) × ( P + 1) matrix. Th en substituting b i = ( X ′ i X i ) − 1 X ′ i Y i in to this equation, w e obtain W ′ Σ − 1 b = X ′ 1 X 1 ( X ′ 1 X 1 ) − 1 X ′ 1 Y 1 σ 2 + · · · + X ′ k X k ( X ′ k X k ) − 1 X ′ k Y k σ 2 = σ 2 h X X ′ i Y i i = σ 2 X ′ Y . Substituting this result int o ( A3 ), we see that ˆ β ∗ = ( X ′ X ) − 1 σ − 2 W ′ Σ − 1 b = ( X ′ X ) − 1 σ − 2 [ σ 2 X ′ Y ] = ( X ′ X ) − 1 X ′ Y , whic h equals b ∗ giv en in (A1). A CKNO WLEDGMENTS This w ork w as sup p orted by NSF Grant s REC- 03356 56 and REC-0634013. Parts of this work were presen ted in the symp osium “Multiv aria te Analy- sis: In C elebration of Ingram Olkin’s 80th Birthd a y” at the Joint Statistical Meet ings, T oronto , C anada, August 2004. REFERENCES Amemiy a, Y. and Fuller, W. A. ( 1984). Estimation for the multi v ariate errors-in-v ariables mod el with estimated error co v ariance matrix. Ann. Statist. 12 497–509. MR074090 8 Ashenfel ter, O., Harmon, C. and Oosterbeek, H. (1999). A review of estimates of the schooling/earnings relationship, with tests for publication bias. L ab our Ec o- nomics 6 453–470 . Baker, C. B., Tweedi e, R., Duv al, S. and Woods, S. W. (2003). Evidence that the SSRI dose resp onse in treating ma jor depression should b e reassessed: A meta-analysis. Depr ess ion and Anxiety 17 1–9. 16 B. J. BECKER AN D M.-J. WU Bini, L. M., Coelho, A. S. G. and Diniz-Filho, J. A. F. (2001). Is th e relationship b etw een population density and b ody size consistent across indep end ent studies? A meta- analytical approac h. R evista Br asileir a de Biolo gia 61 1–6. Card, D. and Krueger, A. B. (1995). T ime-series minim um-wage studies: A meta-analysis. A mer. Ec onom. R ev. (AEA Pap ers and Pr o c e e dings) 85 238–243. Coleman, J. S., Campbell, E. Q., Hobson, C. J., Mc- p ar tland, J., Mood, A . M. and Wei nfeld, F. D. (1966). Survey of Equal Educ ational Opp ortunity . U.S. De- partment of Health, Education and W elfa re, W ashington, DC. Cro uch, G. I. (1995). A meta-analysis of tourism demand. An n. T ourism R ese ar ch 22 103–118 . Cro uch, G. I. (1996). Demand elasticiti es in in ternational marke ting: A meta-analytical application to tourism. J. Business R ese ar ch 36 117–13 6. Doucoulia gos, C. (200 5). Publication bias in the economic freedom and economic growth literature. J. Ec onomic Sur- veys 19 367–387 . Doucoulia gos, C . and P aldam, M . (2006). Aid effective- ness on accum ulation: A meta study . Kyklos 59 227–254. F arley, J. U., Lehmann, D. R. and Sa wyer, A. (1995). Empirical marketing generalization using meta-analysis. Marketing Scienc e 14 G36–G4 6. Greenland, S. (1987). Quantitativ e metho ds in the review of epidemiologic literature. Epidemi olo gic R eviews 9 1–30. Greenland, S . and Longnecker, M. P. (1987). Metho ds for trend estimation from summarized dose-response data, with applica tions to meta-analysis . Amer. J. Epidemiolo gy 135 1301–1 309. Hanushek, E. A. (1974). Efficien t estimators for regressing regressio n coefficients. A mer. Statist. 28 66–6 7. MR034892 9 Hanushek, E. A. (1989). The impact of d ifferen tial exp en- ditures on sc ho ol p erformance. Educ ational R ese ar cher 18 45–65. Hedges, L. V. (1988). The meta-analysis of test va lidity studies: Some new approaches. In T est V alidity (H. W ainer and H. I. Braun, eds.) 191–2 12. Hillsdale, Lawrence Erl- baum, NJ. Hedges, L. V., Laine, R . D. and Green w ald, R. (1994). Does money matter? A meta-analysis of stud ies of the ef- fects of differential school inputs on student outcomes. Ed- uc ational R ese ar cher 23 5–14. Hedges, L. V. and Olkin, I. (1985). Statistic al Metho ds f or Meta-A nalysis. Academic Press, New Y ork. MR079859 7 Hedges, L. V. and Vevea, J. L. ( 1998). Fixed- and random- effects mo dels in meta-analysis. Psycholo gic al Metho ds 3 486–50 4. Hunter, J. E. and Schmidt, F. L. (2004). Metho ds of Meta- An alysis: Corr e cting Err or and Bi as in R ese ar ch Findings. Newbury Park, Sage Publications, CA. Ingels, S. J., Scott, L. A., Li ndmark, J. T., Frankel, M. R. and Myers, S. L. (1992). NELS 88 First F ol low- up: Student Comp onent D ata file User ’ s Manual . 1 . U.S. Department of Education, O ffice of Educational R esearc h and Improv ement. Jarrell, S. B. and St anley, T. D. ( 1990). A meta-analysis of the union-nonunion w age gap. Industrial and L ab or Re - lations R eview 44 54–67. Keef, S. P. and Rober ts, L. A . (2004). The meta-analysis of partial effect sizes. British J. Math. Statist. Psych. 57 97–129 . MR208781 8 Lau, R. R., Sigelman, L., Heldman, C. and Babbitt, P. (1999). The effects of negativ e p olitical advertis ements: A meta-analytic assessmen t. A mer. Pol i tic al Scienc e R eview 93 851–875 . No vick, M. R., Ja ckson, P. H. , Tha yer, D. T . and C ole, N. S. (1972). Estima ting multiple regressions in m groups: A cross-v ali dation stud y . British J. Math. Statist. Psych. 25 33–50. Olkin, I. (2003). P ersonal comm unication. P a ng, F., Dr ummond, M. and Song, F. (1999). The use of meta-analysis in economic ev aluation. D iscussion pap er 173, The Universi ty of Y ork, Centre for Health Economics, NHS Centre for Rev iews and Dissemination. Peterson, R. A. and Bro wn, S. P. (2005). On the use of b eta co efficients in meta-analysis. J. Appl. Psycholo gy 90 175–18 1. Raju, N. S ., Fralicx, R. and Steinh aus, S. D. (1986). Co v ariance and regression slop e models for studying v alid- it y generalization. Applie d Psycholo gic al Me asur ement 10 195–21 1. Raju, N. S., P ap p as, S. and W illiams, C. P. (1989). A n empirical Mon te Carlo test of the accuracy of the correla- tion, co v ariance, and regression slope mod els for assess ing v alidit y generalization. J. Appl. Psycholo gy 74 901–911 . Raudenbush, S. W ., Becker, B. J. and Kalaian, H. (1988). Modeling m ultiv ariate effect sizes. Psycholo gic al Bul l etin 103 111–120. Ro ber ts, C. J. (2005). Issues in meta-regression analysis: An o verview. J. Ec onomic Surveys 19 295–298. Ro ot, T. L., Price, J. T ., Hall, K. R., Schneider, S. H., Ro senzwei g, C. and Pounds, J. A. (200 3). Fingerprints of global w arming on wild animals and plan ts. Natur e 421 57–60. Ro se, A. K. and St anley, T. D. (2005). A meta-analysis of the effect of common cu rrencies on international t rade. J. Ec onomic Surveys 19 347–3 65. Shi, J. Q. and Cop as, J. B. (2004). Meta-analysis for trend estimation. Statistics in M e dicine 23 3–19. Sidik, K. and Jonkman, J. N. (2005). Simple heterogene- it y va riance estimatio n for meta-analysis. Appl. Statist. 54 367–38 4. MR213588 0 St anley, T. D. (2001). Wheat from chaff: Meta-analysis as quantitativ e literature review. J. Ec onomic Persp e ctives 15 131–15 0. St anley, T. D. (2005). Beyond publication bias. J. Ec o- nomic Surveys 19 309–345. St anley, T. D. and Jarrell, S. B. (1989). Meta -regression analysis: A quantitativ e method of literature surveys. J. Ec onomic Surveys 3 161–17 0. St anley, T. D. and Jarrell, S. B. (1998). Gender w age discrimination bias? A meta-regressio n analysis. J. Human R esour c es 33 947–97 3. St anley, T. D. and Jarrell, S. B. (2005). Meta -regression analysis: A quantitativ e method of literature surveys. J. Ec onomic Surveys 19 299–3 08. St apleton, J. (1995). Line ar Statistic al Mo dels . Wiley , New Y ork. MR134348 9 THE SY NTHESIS OF R EGRESSION SLOPES IN MET A- ANAL YSIS 17 Timm, N. H. (2004). Estimating effect sizes in exploratory exp erimenta l studies when using a linear mo del. Amer. Statist. 58 213–217. MR208753 6 W alker, G. A . and Sa w, J. G. (197 8). The distribution of linear com binations of t -v ariables. J. Amer . Statist. Asso c. 73 876–878 . MR0521337 Wu, M.-J. and B ecker, B. J. (2004). Synthesizi ng results from regression stud ies: What can we learn from combining results from studies using large data sets? Pa p er presented at the annual meeting of the A merican Edu cational R e- searc h Association, San D iego, CA.