Exchangeable Random Measures for Sparse and Modular Graphs with Overlapping Communities

Exc hangeable Random Measures for Sparse and Mo dular Graphs with Ov erlapping Comm unities Adrien T o desc hini , Xenia Miscouridou and F ran¸ cois Caron INRIA & Institut de Math´ ematiques de Bor de aux, F r anc e e-mail: Adrien.Todeschini@inria.fr Dep artment of Statistics, University of Oxfor d, UK e-mail: xenia.miscouridou@spc.ox.ac.uk ; caron@stats.ox.ac.uk Abstract: W e propose a nov el statistical model for sparse netw orks with ov erlapping comm unity structure. The model is based on represen ting the graph as an exc hangeable p oin t pro cess, and naturally generalizes existing probabilistic models with overlapping blo c k-structure to the sparse regime. Our construction builds on vectors of completely random measures, and has interpretable parameters, each node b eing assigned a v ector representing its lev el of aﬃliation to some laten t comm unities. W e dev elop methods for simulating this class of random graphs, as well as to p erform posterior inference. W e show that the prop osed approach can recov er in terpretable structure from tw o real-w orld net works and can handle graphs with thousands of no des and tens of thousands of edges. Keyw ords and phrases: Netw orks, Random Graphs, Multiview Netw orks, Multigraphs, Completely Random Measures, L´ evy measure, Multiv ariate Sub ordinator, Sparsity , Non-Negative F actorization, Exchangeabilit y , Poin t Processes. 1. In tro duction There has b een a gro wing in terest in the analysis, understanding and modeling of net work data o ver the recent y ears. A netw ork is comp osed of a set of no des, or vertices, with connections betw een them. Net w ork data arise in a wide range of ﬁelds, and include so cial netw orks, collab oration netw orks, comm unication netw orks, biological net w orks, fo o d webs and are a useful w ay of represen ting in teractions b et ween sets of ob jects. Of particular imp ortance is the elab oration of random graph models, which can capture the salien t prop erties of real-world graphs. F ollo wing the seminal work of Erd¨ os and R ´ en yi ( 1959 ), v arious netw ork mo dels ha ve b een prop osed; see the ov erviews of Newman ( 2003b , 2009 ), Kolaczyk ( 2009 ), Bollob´ as ( 2001 ), Goldenberg et al. ( 2010 ), Fienberg ( 2012 ) or Jacobs and Clauset ( 2014 ). In particular, a large bo dy of the literature has concen trated on mo dels that can capture some mo dular or communit y structure within the net work. The ﬁrst statistical netw ork mo del in this line of researc h is the p opular sto chastic blo ck-model ( Holland et al. , 1983 ; Snijders and No wic ki , 1997 ; Nowic ki and Snijders , 2001 ). The sto chastic block-model assumes that eac h no de belongs to one of p latent communities, and the probability of connection b etw een tw o no des is giv en by a p × p connectivity matrix. This mo del has b een extended in v arious directions, by introducing degree-correction parameters ( Karrer and Newman , 2011 ), by allo wing the num b er of communities to grow with the size of the netw ork ( Kemp et al. , 2006 ), or by considering ov erlapping communities ( Airoldi et al. , 2008 ; Miller et al. , 2009 ; Latouche et al. , 2011 ; Palla et al. , 2012 ; Y ang and Lesko vec , 2013 ). Sto chastic blo c k-mo dels and their extensions ha ve shown to oﬀer a very ﬂexible mo deling framework, with interpretable parameters, and ha ve b een successfully used for the analysis of n umerous real-world net works. How ever, as outlined by Orbanz and Roy ( 2015 ), when one makes the usual assumption that the ordering of the no des is irrelev ant in the deﬁnition of the statistical netw ork mo del, the Ba yesian probabilistic v ersions of those mo dels lead to dense net works 1 : that means that the n umber of edges grows quadratically with the n umber of nodes. This prop erty is rather undesirable, as man y real-w orld netw orks are b elieved to be sparse. Recen tly , Caron and F o x ( 2017 ) prop osed an alternativ e framew ork for statistical netw ork modeling. The framew ork is based on represen ting the graph as an exchangeable random measure on the plane. More precisely , the no des are embedded at some lo cation θ i ∈ R + and, for simple graphs, a connection exists 1 W e refer to graphs whose num b er of edges scales quadratically with the num ber of nodes as dense, and sparse if it scales sub-quadratically . 1 A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 2 b et w een tw o no des i and j if there is a p oint at lo cations ( θ i , θ j ) and ( θ j , θ i ). An undirected simple graph is therefore represen ted b y a symmetric p oint pro cess Z on the plane Z = X i,j z ij δ ( θ i ,θ j ) (1) where z ij = z j i = 1 if i and j are connected, 0 otherwise; see Figure 1 for an illustration. Caron and F ox ( 2017 ) noted that jointly exc hangeable random measures, a notion to b e deﬁned in Eq. ( 21 ), admit a represen tation theorem due to Kallenberg ( 1990 ), providing a general construction for exc hangeable random measures hence random graphs represented b y such ob jects. This connection is further explored b y V eitc h and Roy ( 2015 ) and Borgs et al. ( 2016 ), who pro vide a detailed description and extensive theoretical analysis of the asso ciated class of random graphs, whic h they name Kal lenb er g exchange able gr aphs or gr aphon pr o c esses . Within this class of mo dels, Caron and F o x ( 2017 ) consider in particular the follo wing simple generative mo del, where t w o no des i 6 = j connect with probability Pr( z ij = 1 | ( w ` ) ` =1 , 2 ,... ) = 1 − e − 2 w i w j (2) where the ( w i , θ i ) i =1 , 2 ,... are the p oints of a Poisson p oint pro cess on R 2 + . The parameters w i > 0 can b e interpreted as sociability parameters. Depending on the prop erties of the mean measure of the P oisson pro cess, the authors show that it is p ossible to generate b oth dense and sparse graphs, with p otentially hea vy-tailed degree distributions, within this framework. The construction ( 2 ) is ho wev er rather limited in terms of capturing structure in the net work. Herlau et al. ( 2015 ) prop osed an extension of ( 2 ), which can accommo date a communit y structure. More precisely , in tro ducing latent communit y membership v ariables c i ∈ { 1 , . . . , p } , t wo no des i 6 = j connect with probabilit y Pr( z ij = 1 | ( w ` , c ` ) ` =1 , 2 ,... , ( η k` ) 1 ≤ k,` ≤ p ) = 1 − e − 2 η c i c j w i w j (3) where the ( w i , c i , θ i ) i =1 , 2 ,... are the p oints of a (marked) Poisson p oint pro cess on R + × { 1 , . . . , p } × R + and η k` are p ositive random v ariables parameterizing the strength of interaction b etw een no des in comm unit y k and no des in communit y ` . The model is similar in spirit to the degree-corrected sto c hastic blo ck-model ( Karrer and Newman , 2011 ), but within the p oint process framework ( 1 ), and can thus accommo date b oth sparse and dense netw orks with communit y structure. The model of Herlau e t al. ( 2015 ) how ever shares the limitations of the (degree-corrected) sto c hastic blo ck-model, in the sense that it cannot mo del o verlapping communit y structures, eac h node b eing assigned to a single communit y; see Latouche et al. ( 2011 ) and Y ang and Lesko vec ( 2013 ) for more discussion along these lines. Other extensions with blo c k structure or mixed membership block structure are also suggested by Borgs et al. ( 2016 ). In this pap er, w e consider that each no de i is assigned a set of latent non-negativ e parameters w ik , k = 1 , . . . , p , and that the probability that t wo no des i 6 = j connect is giv en b y Pr( z ij = 1 | ( w ` 1 , . . . , w `p ) ` =1 , 2 ,... ) = 1 − e − 2 P p k =1 w ik w j k . (4) These non-negative weigh ts can b e in terpreted as measuring the level of aﬃliation of no de i to the latent comm unities k = 1 , . . . , p . F or example, in a friendship netw ork, these communities can corresp ond to col- leagues, family , or sp ort partners, and the w eights measure the lev el of aﬃliation of an individual to each comm unit y . Note that as individuals can hav e high weigh ts in diﬀerent communities, the model can capture o v erlapping comm unities. The link probability ( 4 ) builds on a non-negative factorization; it has b een used b y other authors for net work mo deling ( Y ang and Lesko vec , 2013 ; Zhou , 2015 ) and is also closely related to the mo del for multigraphs of Ball et al. ( 2011 ). The main con tribution of this pap er is to use the link probabilit y ( 4 ) within the p oin t pro cess framework of Caron and F ox ( 2017 ). T o this aim, we consider that the no de lo cations and weigh ts ( w i 1 , . . . , w ip , θ i ) i =1 , 2 ,... are drawn from a Poisson p oint pro cess on R p +1 + with a given mean measure ν . The construction of such multiv ariate point pro cess relies on v ectors of completely random measures (or equiv alently multiv ariate sub ordinators). In particular, we build on the ﬂexible though tractable construction recently introduced b y Griﬃn and Leisen ( 2017 ). The proposed mo del generalizes that of Caron and F o x ( 2017 ) b y allo wing the mo del to capture more structure in the net work, while retaining its main features, and is shown to ha v e the following prop erties: A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 3 0 z ij θ i θ j w i 1 w i 2 w i 3 w j 1 w j 2 w j 3 Fig 1 . Repr esentation of a undir e cte d gr aph via a p oint pro c ess Z . Each no de i is emb edde d in R + at some lo c ation θ i and is asso ciate d with a set of p ositive attributes ( w i 1 , . . . , w ip ) . An e dge b etween nodes θ i and θ j is r epr esente d by a p oint at lo c ations ( θ i , θ j ) and ( θ j , θ i ) in R 2 + . • Interpretabilit y: eac h no de is assigned a set of p ositive parameters, which can be interpreted as mea- suring the levels of aﬃliation of a node to laten t communities; once those parameters are learned, they can be used to underco v e r the latent structure in the net work. • Sparsity: we can generate graphs whose num b er of edges gro ws sub quadratically with the num b er of no des. • Exchangeabilit y: in the sense of Kallenberg ( 1990 ). Additionally , we develop a Marko v chain Mon te Carlo (MCMC) algorithm for posterior inference with this mo del, and show exp eriments on tw o real-w orld netw orks with a thousand of nodes and tens of thousands of edges. The article is organized as follows. The class of random graph models is in tro duced in Section 2 . Properties of the class of graphs and simulation are described in Section 3 . W e deriv e a scalable MCMC algorithm for p osterior inference in Section 4 . In Section 5 we pro vide illustrations of the prop osed metho d on simulated data and on tw o net works: a netw ork of citations b etw een political blogs and a netw ork of connections b etw een US airp orts. W e show that the approac h is able to disco ver in terpretable structure in the data and p erforms w ell compared to alternativ es. 2. Sparse graph mo dels with o v erlapping communities In this section, w e present the statistical mo del for simple graphs. The construction builds on vectors of completely random measures (CRM, Kingman , 1967 ). W e only pro vide here the necessary material for the deﬁnition of the netw ork model; please refer to App endix A for additional bac kground on v ectors of CRMs. The model described in this section can also b e extended to bipartite graphs; see Appendix E . 2.1. Gener al c onstruction using ve ctors of CRMs W e consider that each no de i is embedded at some lo cation θ i ∈ R + , and has some set of positive weigh ts ( w i 1 , . . . , w ip ) ∈ R p + . The p oints ( w i 1 , . . . , w ip , θ i ) i =1 ,..., ∞ are assumed to be dra wn from a P oisson process with mean measure ν ( dw 1 , . . . , dw p , dθ ) = ρ ( dw 1 , . . . , dw p ) λ ( dθ ) (5) A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 4 θ 1 θ 2 θ 3 θ 4 3 1 1 1 2 θ 1 θ 2 θ 3 4 2 1 3 θ 1 θ 2 θ 3 θ 4 (a) (b) (c) Fig 2 . An example of (a) the restriction on [0 , 1] 2 of the two atomic measur es D 1 and D 2 , (b) the c orr esp onding multiview dir e cted multigr aphs (top: view 1; b ottom: view 2) and (c) c orresp onding undir e cte d gr aph. where λ is the Leb esgue measure and ρ is a σ -ﬁnite measure on R p + , concentrated on R p + \{ 0 } , which satisﬁes Z R p + min 1 , p X k =1 w k ! ρ ( dw 1 , . . . , dw p ) < ∞ . (6) Under this condition ( Skorohod , 1991 ; Barndorﬀ-Nielsen et al. , 2001 ), we can describ e the set of weigh ts and lo cations using a v ector of completely random measures ( W 1 , . . . , W p ) on R + : W k = ∞ X i =1 w ik δ θ i , for k = 1 , . . . , p. (7) W e simply write ( W 1 , . . . , W p ) ∼ CRM( ρ, λ ) . (8) Mimic king the hierarchical construction of Caron and F ox ( 2017 ), w e introduce integer-v alued random measures D k on R 2 + , k = 1 , . . . , p , D k = ∞ X i =1 ∞ X j =1 n ij k δ ( θ i ,θ j ) (9) where the n ij k are natural in tegers. The vector of random measures ( D 1 , . . . , D p ) can be interpreted as represen ting a m ultiview (a.k.a. multiplex or m ulti-relational) directed multigraph ( V erbrugge , 1979 ; Salter- T ownshend and McCormick , 2013 ), where n ij k represen ts the n umber of interactions from no de i to no de j in the view k ; see Figure 2 for an illustration. Conditionally on the vector of CRMs, the measures D k are indep enden tly dra wn from a P oisson pro cess 2 with mean measure W k × W k D k | ( W 1 , . . . , W p ) ∼ Poisson ( W k × W k ) (10) that is, the n ij k are independently Poisson distributed with rate w ik w j k . 2 Note that we consider a generalized deﬁnition of a Poisson pro cess, where the mean measure is allow ed to ha ve atoms; see e.g. Daley and V ere-Jones ( 2008a , Section 2.4). A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 5 (a) W k × W k (b) Integer p oint pro cesses D k (c) Point pro cess Z Fig 3 . An example, for p = 2 , of (a) the pr o duct me asur es W k × W k , (b) a draw of the dir e cte d multigr aph me asur es D k | W k ∼ Poisson( W k × W k ) and (c) c orr esponding undir e cte d measur e Z = P ∞ i =1 P ∞ j =1 min(1 , P p k =1 n ij k + n j ik ) δ ( θ i ,θ j ) . Finally , the p oint pro cess Z represen ting the graph (Eq. ( 1 )) is deterministically obtained from ( D 1 , . . . , D p ) b y setting z ij = 1 if there is at least one directed connection betw een i and j in an y view, and 0 otherwise, therefore z ij = min(1 , P p k =1 n ij k + n j ik ). T o sum up, the graph model is described as follows: W k = P ∞ i =1 w ik δ θ i ( W 1 , . . . , W p ) ∼ CRM( ρ, λ ) D k = P ∞ i =1 P ∞ j =1 n ij k δ ( θ i ,θ j ) D k | W k ∼ Poisson ( W k × W k ) Z = P ∞ i =1 P ∞ j =1 min(1 , P p k =1 n ij k + n j ik ) δ ( θ i ,θ j ) . (11) The mo del construction is illustrated in Figure 3 . In tegrating out the measures D k , k = 1 , . . . , p , the con- struction can b e expressed as, for i ≤ j z ij | ( w ` 1 , . . . , w `p ) ` =1 , 2 ,... ∼  Ber(1 − exp( − 2 P p k =1 w ik w j k )) i 6 = j Ber(1 − exp( − P p k =1 w 2 ik )) i = j (12) and z j i = z ij ; see Figure 1 . Graph Restrictions. Except in trivial cases, w e hav e W k ( R + ) = ∞ a.s. and therefore Z ( R 2 + ) = ∞ a.s., so the num b er of p oints ov er the plane is inﬁnite a.s. F or α > 0, we consider restrictions of the measures W k , k = 1 , . . . , p , to the interv al [0 , α ] and of the measures D k and Z to the b ox [0 , α ] 2 , and write resp ectively W kα , D kα and Z α these restrictions. Note that condition ( 6 ) ensures that W kα ([0 , α ]) < ∞ a.s. hence D kα ([0 , α ] 2 ) < ∞ and Z α ([0 , α ] 2 ) < ∞ a.s. As a consequence, for a given α > 0, the model yields a ﬁnite n um b er of edges a.s., ev en though there may b e an inﬁnite num b er of p oints ( w i , θ i ) ∈ R + × [0 , α ]; see Section 3 . Remark 1 The mo del deﬁne d ab ove c an also b e use d for r andom multigr aphs, wher e n ij = P p k =1 n ij k is the numb er of dir e cte d inter actions b etwe en i and j . Then we have n ij | ( w ` 1 , . . . , w `p ) ` =1 , 2 ,... ∼ Poisson p X k =1 w ik w j k ! which is a Poisson non-ne gative factorization ( L e e , 1999 ; Cemgil , 2009 ; Psor akis et al. , 2011 ; Bal l et al. , 2011 ; Gop alan et al. , 2015 ). A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 6 Fig 4 . Gr aph sampled fr om the mo del with thr e e latent c ommunities, identiﬁed by c olors r e d, gr e en, blue. F or e ach no de, the intensity of e ach c olor is pr op ortional to the value of the asso ciate d weight in that c ommunity. Pur e r e d/gr e en/blue c olor indic ates the node is only str ongly aﬃliate d to a single c ommunity. A mixtur e of those c olors indic ates b alanc e d aﬃliations to diﬀer ent c ommunities. Graph gener ate d with the softwar e Gephi ( Bastian et al. , 2009 ). Remark 2 The mo del deﬁne d by Eq. ( 12 ) al lows to mo del networks which exhibit assortativity ( Newman , 2003a ), me aning that two no des with similar char acteristics (her e similar set of weights) ar e mor e likely to c onne ct than no des with dissimilar char acteristics. The link function c an b e gener alize d to (se e e.g. Zhou , 2015 ) z ij ∼ Ber 1 − exp − p X k =1 p X ` =1 η k` w ik w j ` !! wher e η k` ≥ 0 , in or der to b e able to c aptur e b oth assortative and dissortative mixing in the network. In p articular, setting lar ger values oﬀ-diagonal than on the diagonal of the matrix ( η k` ) 1 ≤ k,` ≤ p al lows to c aptur e dissortative mixing. The pr op erties and algorithms for simulation and p osterior infer enc e c an trivial ly b e extende d to this mor e gener al c ase. In or der to ke ep the notations as simple as p ossible, we fo cus her e on the simpler link function ( 12 ) . 2.2. Particular mo del b ase d on c omp ound CRMs The k ey comp onent in our statistical netw ork mo del is the m ultiv ariate L ´ evy measure ρ in ( 8 ). V arious ap- proac hes hav e b een developed for constructing multiv ariate L´ evy measures ( T ank ov , 2003 ; Cont and T ank ov , 2003 ; Kallsen and T anko v , 2006 ; Barndorﬀ-Nielsen et al. , 2001 ; Skorohod , 1991 ), or more sp eciﬁcally vectors of completely random measures ( Epifani and Lijoi , 2010 ; Leisen and Lijoi , 2011 ; Leisen et al. , 2013 ; Griﬃn et al. , 2013 ; Lijoi et al. , 2014 ). W e will in this paper consider the following particular form: ρ ( dw 1 , . . . , dw p ) = e − P p k =1 γ k w k Z ∞ 0 w − p 0 F  dw 1 w 0 , . . . , dw p w 0  ρ 0 ( dw 0 ) (13) where F ( dβ 1 , . . . dβ p ) is some sc or e probability distribution on R d + , with moment generating function M ( t 1 , . . . , t p ), ρ 0 is a b ase L ´ evy measure on R + and γ k ≥ 0 are exp onential ly tilting p ar ameters for k = 1 , . . . , p . The mo del deﬁned by ( 5 ) and ( 13 ) is a sp ecial case of the comp ound completely random measure (CCRM) model prop osed b y Griﬃn and Leisen ( 2017 ). It admits the follo wing hierarc hical construction, which makes inter- pretabilit y , c haracterization of the conditionals and analysis of this class of mo dels particularly easy . Let W 0 = ∞ X i =1 w i 0 δ θ i ∼ CRM( e ρ 0 , λ ) (14) A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 7 where e ρ 0 is a measure on R + deﬁned b y e ρ 0 ( dw 0 ) = M ( − w 0 γ 1 , . . . , − w 0 γ p ) ρ 0 ( dw 0 ), and for k = 1 , . . . , p and i = 1 , 2 , . . . w ik = β ik w i 0 where the scores β ik ha v e the follo wing join t distribution ( β i 1 , . . . , β ip ) | w i 0 ind ∼ H ( ·| w i 0 ) (15) with H is an exp onentially tilted version of F : H ( dβ 1 , . . . , dβ p | w 0 ) = e − w 0 P p k =1 γ k β k F ( dβ 1 , . . . , dβ p ) R R p + e − w 0 P p k =1 γ k e β k F  d e β 1 , . . . , d e β p  . (16) Additionally , the set of points ( w i 0 , β i 1 , . . . , β ip ) i =1 , 2 ,... is a Poisson p oint pro cess with mean measure e − w 0 P p k =1 γ k β k F ( dβ 1 , . . . , dβ p ) ρ 0 ( dw 0 ) . (17) Dep endence b etw een the diﬀerent CRMs is b oth tuned by the shared scaling parameter w i 0 and p otential dep endency b etw een the scores ( β i 1 , . . . , β ip ). The hierarc hical construction has the following interpretation: • The w eigh t w i 0 is an individual scaling parameter for no de i whose distribution is tuned by the base L ´ evy measure ρ 0 . It can b e considered as a degree correction, as often used in netw ork models ( Karrer and Newman , 2011 ; Zhao et al. , 2012 ; Herlau et al. , 2015 ). As sho wn in Section 3 , ρ 0 tunes the o verall sparsit y properties of the net work. • The comm unity-related scores β ik tune the level of aﬃliation of node i to comm unity k ; this is con trolled b y b oth the score distribution F and the tilting co eﬃcients γ k . These parameters tune the ov erlapping blo c k-structure of the net work. An example of suc h a graph with three communities is displa yed in Figure 4 . Sp eciﬁc choices for F and ρ 0 . W e now give here speciﬁc choices of score distribution F and base L´ evy measure ρ 0 , which lead to scalable inference algorithms. As in Griﬃn and Leisen ( 2017 ), we consider that F is a pro duct of indep endent gamma distributions F ( dβ 1 , . . . , dβ p ) = p Y k =1 β a k − 1 k e − b k β k b a k k Γ( a k ) dβ k (18) where a k > 0, b k > 0, k = 1 , . . . , p , whic h leads to H ( dw 1 , . . . , dw p | w 0 ) ∝ p Y k =1 w a k − 1 k e − b k w k w 0 − γ k w k dw k whic h is also a pro duct of gamma distributions. ρ 0 is set to b e the mean measure of the jump part of a generalized gamma pro cess ( Hougaard , 1986 ; Brix , 1999 ), which has b een extensively used in BNP mo dels due to its generality , the in terpretability of its parameters and its attractive conjugacy properties ( James , 2002 ; Lijoi et al. , 2007 ; Saeedi and Bouchard-Cˆ ot ´ e , 2011 ; Caron , 2012 ; Caron et al. , 2014 ). The L ´ evy measure in this case is ρ 0 ( dw 0 ) = 1 Γ(1 − σ ) w − 1 − σ 0 exp( − w 0 τ ) dw 0 (19) where the parameters ( σ , τ ) verify σ ∈ (0 , 1) , τ ≥ 0 or σ ∈ ( −∞ , 0] , τ > 0 . (20) The gamma pro cess ( σ = 0), the inv erse Gaussian process ( σ = 1 2 ) and the stable process ( σ ∈ (0 , 1), τ = 0) are sp ecial cases. Using ( 18 ) and ( 19 ), the m ultiv ariate L´ evy measure has the following analytic form ρ ( dw 1 , . . . , dw p ) = 2 e − P p k =1 γ k w k Γ(1 − σ ) " p Y k =1 w a k − 1 k b a k k Γ( a k ) #  τ P p k =1 b k w k  − κ 2 K κ   2 s τ X k b k w k   dw 1 . . . dw p where κ = σ + P p k =1 a k and K is the mo diﬁed Bessel function of the second kind. A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 8 3. Prop erties and Sim ulation The ﬁrst theorem pro vides expressions for the exp ected n umber of edges in the multigraph and simple graph, and for the expected n um b er of no des. The pro of is giv en in App endix C . Theorem 3 The exp e cte d numb er of e dges in the multigr aph D ∗ α , e dges in the undir e cte d gr aph N ( e ) α and observe d no des N α ar e given as fol lows: E [ D ∗ α ] = α 2 µ T µ + α tr (Σ) E [ N ( e ) α ] = α Z R p +  1 − e − w T w  ρ ( dw 1 , . . . , dw p ) + α 2 2 Z R p + ψ (2 w 1 , . . . , 2 w p ) ρ ( dw 1 , . . . , dw p ) E [ N α ] = α Z R p +  1 − e − w T w − αψ (2 w )  ρ ( dw 1 , . . . , dw p ) wher e µ = R R p + w ρ ( dw 1 , . . . , dw p ) , Σ = R R p + w w T ρ ( dw 1 , . . . , dw p ) and ψ ( t 1 , . . . , t p ) = R R p + (1 − e − P p k =1 t i w i ) ρ ( dw 1 , . . . , dw p ) is the multivariable L aplac e exp onent. 3.1. Exchange ability The p oint pro cess Z deﬁned b y ( 11 ) is join tly exc hangeable in the sense of Kallenberg ( 1990 , 2005 ). F or an y h > 0 and an y permutation π of N ( Z ( A i × A j )) d = ( Z ( A π ( i ) × A π ( j ) )) for ( i, j ) ∈ N 2 (21) where A i = [ h ( i − 1) , hi ]. This follows directly from the fact that the vector of CRMs ( W 1 , . . . , W p ) has indep enden t and identically distributed incremen ts, hence ( W 1 ( A i ) , . . . , W p ( A i )) d = ( W 1 ( A π ( i ) ) , . . . , W p ( A π ( i ) )) . (22) The mo del thus falls into the general representation theorem for exchangeable point processes ( Kallen b erg , 1990 ). 3.2. Sp arsity In this section, following the asymptotic notations of Janson ( 2011 ), we deriv e the sparsity prop erties of our graph mo del, ﬁrst for the general construction of Section 2.1 , then for the sp eciﬁc construction on comp ound CRMs of Section 2.2 . Similarly to the notations in Caron and F o x ( 2017 ), let Z α b e the restriction of Z to the b o x [0 , α ] 2 . Let ( N α ) α ≥ 0 and ( N ( e ) α ) α ≥ 0 b e coun ting processes respectively corresp onding to the num b er of nodes and edges in Z α : N α = X i 1 θ i ≤ α 1 ( P j z ij 1 θ j ≤ α ) ≥ 1 N ( e ) α = X i ≤ j z ij 1 θ i ≤ α 1 θ j ≤ α . Note that in the prop ositions b elow, we discard the trivial case R R p + ρ ( dw 1 , . . . , dw p ) = 0 whic h implies N ( e ) α = N α = 0 a.s. General construction. The next prop osition characterizes the sparsity prop erties of the random graph dep ending on the properties of the L´ evy measure ρ . In particular, if Z R p + ρ ( dw 1 , . . . , dw p ) = ∞ (23) then, for any α > 0, there is a.s. an inﬁnite num b er of θ i ∈ [0 , α ] for whic h P k w ik > 0 and the v ector of CRMs is called inﬁnite-activit y . Otherwise, it is ﬁnite-activit y . A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 9 Prop osition 4 Assume that, for any k = 1 , . . . , p , Z R p + w k ρ ( dw 1 , . . . , dw p ) < ∞ (24) Then N ( e ) α =  Θ( N 2 α ) if ( W 1 , . . . , W p ) is ﬁnite-activity o ( N 2 α ) otherwise a.s. as α tends to ∞ . The proof is given in App endix B . Construction based on CCRMs. F or the CCRM L´ evy measure ( 13 ), the sparsit y prop erties are solely tuned by the base L´ evy measure ρ 0 . Ignoring trivial degenerate cases for the score distribution F , it is easily sho wn that the CCRM mo del deﬁned b y ( 5 ) and ( 13 ) is inﬁnite-activity iﬀ the L ´ evy measure ρ 0 v eriﬁes Z ∞ 0 ρ 0 ( dw ) = ∞ . (25) In this case all CRMs W 0 , W 1 , . . . , W p are inﬁnite-activity . Otherwise they are all ﬁnite-activit y and the v ector of CRMs is ﬁnite-activity . In the particular case of a CCRM with indep endent gamma distributed scores ( 18 ) and generalized gamma pro cess base measure ( 19 ), the condition ( 25 ) is satisﬁed whenev er σ ≥ 0. The next prop osition c haracterizes the sparsity of the net work depending on the properties of the base L ´ evy measure ρ 0 . Prop osition 5 Assume that Z ∞ 0 w 0 ρ 0 ( dw 0 ) < ∞ (26) and F is not de gener ate d at 0. Then N ( e ) α =  Θ( N 2 α ) if R ∞ 0 ρ 0 ( dw ) < ∞ o ( N 2 α ) otherwise a.s. as α tends to ∞ . F urthermor e, if the tail L´ evy intensity ρ 0 deﬁne d by ρ 0 ( x ) = Z ∞ x ρ 0 ( dw ) , (27) is a r e gularly varying function, i.e. ρ 0 ( x ) x − σ ` (1 /x ) − → 1 as x → 0 for some σ ∈ (0 , 1) wher e ` is a slow ly varying function verifying lim t →∞ ` ( at ) /` ( t ) = 1 for any a > 0 and lim t →∞ ` ( t ) > 0 , then N ( e ) α = O ( N 2 / (1+ σ ) α ) a.s. as α tends to ∞ . In the p articular c ase of a CCRM with indep endent gamma distribute d sc or es ( 18 ) and gener alize d gamma pr o c ess b ase me asur e ( 19 ) , c ondition ( 26 ) is e quivalent to having τ > 0 . In this c ase, we ther efor e have N ( e ) α =    Θ( N 2 α ) if σ < 0 o ( N 2 α ) if σ ≥ 0 O ( N 2 / (1+ σ ) α ) if σ ∈ (0 , 1) . The pro of is given in Appendix B . Figure 5 (a) pro vides an empirical illustration of Proposition 5 for a CCRM with indep endent gamma scores and generalized gamma based L ´ evy measure. Figure 5 (b) shows empirically that the degree distribution also exhibits a pow er-law b ehaviour when σ ∈ (0 , 1). A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 10 Number of nodes 10 2 10 3 Number of edges 10 2 10 3 10 4 10 5 σ = -0.5 σ = 0.2 σ = 0.5 σ = 0.8 (a) Degree 10 0 10 1 10 2 10 3 Distribution 10 -8 10 -6 10 -4 10 -2 σ = -0.5 σ = 0.2 σ = 0.5 σ = 0.8 (b) Fig 5 . Empiric al analysis of the pr op erties of CCRM b ase d gr aphs gener ate d with p ar ameters p = 2 , τ = 1 , a k = 0 . 2 , b k = 1 p and aver aging over various α . (a) Numb er of e dges versus the numb er of no des and (b) de gre e distributions on a lo g-lo g sc ale for various σ : one ﬁnite-activity CCRM ( σ = − 0 . 5 ) and thre e inﬁnite-activity CCRMs ( σ = 0 . 2 , σ = 0 . 5 and σ = 0 . 8 ). In (a) we note gr owth at a r ate Θ( N 2 α ) for σ = − 0 . 5 and O ( N 2 / (1+ σ ) α ) for σ ∈ (0 , 1) . 3.3. Simulation The p oin t pro cess Z is deﬁned on the plane. W e describ e in this section how to sample realizations of restrictions Z α of Z to the b ox [0 , α ] 2 . General construction. The hierarc hical construction giv en by Eq. ( 11 ) suggests a direct wa y to sample from the mo del: 1. Sample ( w i 1 , . . . , w ip , θ i ) i =1 , 2 ,... from a P oisson pro cess with mean measure ν ( dw 1 , . . . , dw p , dθ ) 1 θ ∈ [0 ,α ] . 2. F or each pair of p oints, sample z ij from ( 12 ). There are tw o ca veats to this strategy . First, for inﬁnite-activity CRMs, the num b er of p oints in R p + × [0 , α ] is almost surely inﬁnite; ev en for ﬁnite-activity CRMs, it ma y be so large that it is not practically feasible. W e need therefore to resort to an approximation, b y sampling from a Poisson process with an approximate mean measure ν ε ( dw 1 , . . . , dw p , dθ ) 1 θ ∈ [0 ,α ] = ρ ε ( dw 1 , . . . , dw p ) λ ( dθ ) 1 θ ∈ [0 ,α ] where Z R p + ρ ε ( dw 1 , . . . , dw p ) < ∞ with ε > 0 controlling the level of approximation. The appro ximation is speciﬁc to the choice of the mean measure, and we describ e suc h an approximation for CCRMs below. The second cav eat is that, for applying Eq. ( 12 ), w e need to consider all pairs i ≤ j , whic h can b e computationally problematic. W e can instead, similarly to Caron and F o x ( 2017 ), use the hierarc hical P oisson construction as follows: 1. Sample ( w i 1 , . . . , w ip , θ i ) i =1 , 2 ,...,K from a Poisson pro cess with mean measure ν ε ( dw 1 , . . . , dw p , dθ ) 1 θ ∈ [0 ,α ] . Let W ε k,α = P K i =1 w ik δ θ i b e the asso ciated truncated CRMs and W ε ∗ k,α = P K i =1 w ik their total masses. 2. F or k = 1 , . . . , p , sample D ∗ k,α | W ε ∗ k,α ∼ Poisson(( W ε ∗ k,α ) 2 ). 3. F or k = 1 , . . . , p , ` = 1 , . . . , D ∗ k,α , j = 1 , 2, sample U k`j | W ε k,α ind ∼ W ε k,α W ε ∗ k,α . 4. Set D ε k,α = P D ∗ k,α ` =1 δ U k` 1 ,k` 2 . 5. Obtain Z from ( D 1 , . . . , D p ) as in ( 11 ). Construction based on CCRMs. The hierarchical construction of comp ound CRMs suggests an algo- rithm to simulate a v ector of CRMS. W e consider the following (truncated) mean measure ρ ε ( dw 1 , . . . , dw p ) = e − P p k =1 γ k w k Z ∞ ε w − p 0 F  dw 1 w 0 , . . . , dw p w 0  ρ 0 ( dw 0 ) (28) A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 11 with ε ≥ 0. W e can sample from the (truncated) CCRM as follo ws 1. (a) Sample ( w i 0 , θ i ) i =1 ,...,K from a Poisson p oint pro cess with mean measure e ρ 0 ( dw 0 ) λ ( dθ ) 1 { w 0 >ε,θ ∈ [0 ,α ] } . (b) F or i = 1 , . . . , K and k = 1 , . . . , p , set w ik = β ik w i 0 where ( β i 1 , . . . , β ip ) | w i 0 is dra wn from ( 15 ). The truncation level ε is set to 0 for ﬁnite-activity CCRMs, and ε > 0 otherwise. W e explain in App endix D ho w to p erform step 1.(a) in the case of a tilted generalized gamma process. 4. P osterior inference In this section, we describe a MCMC algorithm for p osterior inference of the mo del parameters and h yp er- parameters in the statistical net work mo del deﬁned in Section 2 . W e ﬁrst describ e the data augmen tation sc heme and c haracterization of conditionals. W e then describ e the sampler for a general L ´ evy measure ρ , and ﬁnally deriv e the sampler for comp ound CRMs. 4.1. Char acterization of c onditionals and data augmentation Assume that we hav e observ ed a set of connections ( z ij ) 1 ≤ i,j ≤ N α , where N α is the num b er of no des with at least one connection. W e aim at inferring the p ositive parameters ( w i 1 , . . . , w ip ) i =1 ,...,N α asso ciated to the no des with at least one connection. W e also w ant to estimate the p ositive parameters associated to the other no des with no connection. The num b er of such no des may b e large, and even inﬁnite for inﬁnite- activit y CRMs; but under our mo del, these parameters are only identiﬁable through their sum, denoted ( w ∗ 1 , . . . , w ∗ p ). Note that the node locations θ i are not likelihoo d iden tiﬁable, and w e will not try to infer them. W e assume that there is a set of unknown hyperparameters φ of the mean intensit y ρ , with prior p ( φ ). W e assume that the L´ evy measure ρ is absolutely con tin uous with resp ect to the Leb esgue measure on R d , and write simply ρ ( dw 1 , . . . , dw p ; φ ) = ρ ( w 1 , . . . , w p ; φ ) dw 1 . . . dw p . The parameter α is also assumed to b e unkno wn, with some prior α ∼ Gamma( a α , b α ) with a α > 0 , b α > 0. W e therefore aim at appro ximating p (( w 1 k , . . . , w N α k , w ∗ k ) k =1 ,...,p , φ, α | ( z ij ) 1 ≤ i,j ≤ N α ). As a ﬁrst step, w e characterize the conditional distribution of the restricted v ector of CRMs ( W 1 α , . . . , W pα ) giv en the restricted measures ( D 1 α , . . . , D pα ). Prop osition 6 b elow extends Theorem 12 in Caron and F o x ( 2017 ) to the m ultiv ariate setting. Prop osition 6 L et ( θ 1 , . . . , θ N α ) , N α ≥ 0 b e the supp ort p oints of ( D 1 α , . . . , D pα ) , with D kα = X 1 ≤ i,j ≤ N α n ij k δ ( θ i ,θ j ) . The c onditional distribution of ( W 1 α , . . . , W pα ) given ( D 1 α , . . . , D pα ) is e quivalent to the distribution of f W 1 + N α X i =1 w i 1 δ θ i , . . . , f W p + N α X i =1 w ip δ θ i ! (29) wher e ( f W 1 , . . . , f W p ) is a ve ctor of discr ete r andom me asur es, which dep ends on ( D 1 α , . . . , D pα ) only thr ough the total masses w ∗ k = f W k ([0 , α ]) . The set of weights ( w ik ) i =1 ,...,N α ; k =1 ,...,p and ( w ∗ k ) k =1 ,...,p ar e dep endent, with joint c onditional distribution p (( w 1 k , . . . , w N α k , w ∗ k ) k =1 ,...,p | ( n ij k ) 1 ≤ i,j ≤ N α ; k =1 ,...,p , φ, α ) ∝ " N α Y i =1 p Y k =1 w m ik ik # e − P p k =1 ( w ∗ k + P N α i =1 w ik ) 2 " N α Y i =1 ρ ( w i 1 , . . . , w ip ; φ ) # α N α g ∗ α ( w ∗ 1 , . . . , w ∗ p ; φ ) (30) wher e m ik = P N α j =1 n ij k + n j ik and g ∗ α ( w ∗ 1 , . . . , w ∗ p ; φ ) is the pr ob ability density function of the r andom ve ctor ( W 1 ([0 , α ]) , . . . , W p ([0 , α ])) . The pro of can b e straightforw ardly adapted from that of Caron and F ox ( 2017 ), or from Prop osition 5.2 of James ( 2014 ) and is omitted here. It builds on other p osterior characterizations in Ba y esian nonparametric mo dels ( Pr¨ unster , 2002 ; James , 2002 , 2005 ; James et al. , 2009 ). A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 12 Data augmen tation. Similarly to Caron and F o x ( 2017 ), we introduce latent coun t v ariables e n ij k = n ij k + n j ik with ( e n ij 1 , . . . , e n ij p ) | w , z ∼  δ (0 ,..., 0) if z ij = 0 tP oisson(2 w i 1 w j 1 , . . . , 2 w ip w j p ) if z ij = 1, i 6 = j  e n ij 1 2 , . . . , e n ij p 2  | w , z ∼ tP oisson( w 2 i 1 , . . . , w 2 ip ) if z ij = 1, i = j (31) where tP oisson( λ 1 , . . . , λ p ) is the m ultiv ariate P oisson distribution truncated at zero, whose pmf is tP oisson( x 1 , . . . x p ; λ 1 , . . . , λ p ) = Q p k =1 P oisson( x k ; λ k ) 1 − exp( − P p k =1 x k λ k ) 1 { P p k =1 x k > 0 } . One can sample from this distribution by ﬁrst sampling x = P p k =1 x k from a zero-truncated Poisson distri- bution with rate P p k =1 λ k , and then ( x 1 , . . . , x p ) | ( λ 1 , . . . , λ p ) , x ∼ Multinomial  x,  λ 1 P λ k , . . . λ p P λ k  . 4.2. Markov chain Monte Carlo algorithm: Gener al c onstruction Using the data augmentation scheme together with the posterior c haracterization ( 30 ), w e can deriv e the follo wing MCMC sampler, which uses Metrop olis-Hastings (MH) and Hamiltonian Mon te Carlo (HMC) up dates within a Gibbs sampler, and iterates as described in Algorithm 1 . Algorithm 1 Marko v c hain Mon te Carlo sampler for p osterior inference. At each iteration 1. Up date ( w i 1 , . . . , w ip ), i = 1 , . . . ,N α given the rest using MH or HMC. 2. Up date hyperparameters ( φ, α ) and total masses ( w ∗ 1 , . . . , w ∗ p ) given the rest using MH. 3. Up date the latent variables given the rest using ( 31 ). In general, if the L´ evy in tensity ρ can b e ev aluated point wise, one can use a MH up date for step 1, but it w ould scale p o orly with the n umber of no des. Alternatively , if the L´ evy intensit y ρ is diﬀeren tiable, one can use a Hamiltonian Mon te Carlo up date ( Duane et al. , 1987 ; Neal , 2011 ). The c hallenging part of the Algorithm 1 is Step 2. F rom Eq. ( 30 ) w e ha ve p (( w ∗ k ) k =1 ,...,p , φ, α | rest) ∝ p ( φ ) p ( α ) e − P p k =1 ( w ∗ k + P N α i =1 w ik ) 2 " N α Y i =1 ρ ( w i 1 , . . . , w ip ; φ ) # α N α g ∗ α ( w ∗ 1 , . . . , w ∗ p ; φ ) . This conditional distribution is not of standard form and inv olves the multiv ariate p df g ∗ α ( w ∗ 1 , . . . , w ∗ p ) of the random vector ( W 1 ([0 , α ]) , . . . , W p ([0 , α ])) for which there is t ypically no analytical expression. All is a v ailable is its Laplace transform, whic h is giv en b y E h e − P p k =1 t k W k ([0 ,α ]) i = e − αψ ( t 1 ,...,t p ; φ ) (32) where ψ ( t 1 , . . . , t p ; φ ) = Z R p +  1 − e − P p k =1 t k w k  ρ ( dw 1 , . . . , dw p ; φ ) (33) is the m ultiv ariate Laplace exp onent, which inv olves a p -dimensional integral. W e propose to use a Metropolis- Hastings step, with proposal q ( e w ∗ 1: p , e φ, e α | w ∗ 1: p , φ, α ) = q ( e w ∗ 1: p | w ∗ 1: p , e φ, e α ) × q ( e φ | φ ) × q ( e α | α, e φ, w ∗ 1: p ) (34) where q ( e α | α, e φ, w ∗ 1: p ) = Gamma( e α ; a α + N α , b α + ψ ( λ 1 , . . . , λ p ; e φ )) (35) A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 13 and the prop osal for w ∗ 1: p is an exp onentially tilted v ersion of g ∗ α q (( e w ∗ k ) k =1 ,...,p | ( w ∗ k ) k =1 ,...,p , e φ ) = e − P p k =1 λ k e w ∗ k g ∗ e α ( e w 1 , . . . , e w p ; e φ ) e − e αψ ( λ 1 ,...,λ p ; e φ ) (36) where λ k = w ∗ k + 2 P N α i =1 w ik and q ( e φ | φ ) can b e freely sp eciﬁed by the user. This leads to the following acceptance ratio r = p ( e φ ) q ( φ | e φ ) p ( φ ) q ( e φ | φ ) " N α Y i =1 ρ ( w i 1 , . . . , w ip ; e φ ) ρ ( w i 1 , . . . , w ip ; φ ) # " b α + ψ ( e λ 1 , . . . , e λ p ; φ ) b α + ψ ( λ 1 , . . . , λ p ; e φ ) # a α + N α e P p k =1 [ w 2 ∗ k − e w 2 ∗ k ] where e λ k = e w ∗ k + 2 P N α i =1 w ik . This acceptance ratio inv olves ev aluating the multiv ariate exp onent ( 33 ). In the general case, the MCMC algorithm 1 th us requires to b e able to (a) ev aluate point wise the L´ evy intensit y ρ , and p otentially diﬀeren tiate it, (b) ev aluate point wise the Laplace exp onent ( 33 ) and (c) sample from the exp onen tially tilted distribution ( 36 ). Regarding p oint (c), the random v ariable with p df ( 36 ) has the same distribution as the random vector  W 0 1 ([0 , α ]) , . . . , W 0 p ([0 , α ])  where ( W 0 1 , . . . , W 0 p ) ∼ CRM( ρ 0 , λ ) with ρ 0 is an exp onentially tilted v ersion of ρ ρ 0 ( w 1 , . . . , w p ) = e − P k λ k w k ρ ( w 1 , . . . , w p ) . (37) By considering an approximate tilted intensit y ρ ε 0 ( w 1 , . . . , w p ), one can approximately sample from ( 36 ) by sim ulating points from a Poisson pro cess with mean measure αρ ε 0 ( w 1 , . . . , w p ) and summing them up. 4.3. Markov chain Monte Carlo algorithm: Comp ound CRMs The hierarc hical construction of CCRMs enables to derive a certain num b er of simpliﬁcations in the algorithm describ ed in the previous section. Using the construction w ik = β ik w i 0 where the p oints ( w i 0 , β i 1 , . . . , β ip ) i =1 , 2 ,... ha v e L´ evy measure ( 17 ), we aim at approximating the posterior p (( w 10 , . . . , w N α 0 ) , ( β 1 k , . . . , β N α k , w ∗ k ) k =1 ,...,p , φ, α | ( z ij ) 1 ≤ i,j ≤ N α ) . (38) Conditional on the latent count v ariables deﬁned in ( 31 ), w e hav e the follo wing conditional c haracterization, similar to ( 30 ) p (( w 10 , . . . , w N α 0 ) , ( β 1 k , . . . , β N α k , w ∗ k ) k =1 ,...,p | ( n ij k ) 1 ≤ i,j ≤ N α ; k =1 ,...,p , φ, α ) ∝ " N α Y i =1 w m i i 0 p Y k =1 β m ik ik # e − P p k =1 ( w ∗ k + P N α i =1 w ik ) 2 − P N α i =1 w i 0 ( P p k =1 γ k β ik ) × " N α Y i =1 f ( β i 1 , . . . , β ip ; φ ) ρ 0 ( w i 0 ; φ ) # α N α g ∗ α ( w ∗ 1 , . . . , w ∗ p ; φ ) (39) where m i = P p k =1 m ik and f and ρ 0 are densities of F and ρ 0 with respect to the Leb esgue measure. If f and ρ 0 are diﬀerentiable, one can use a HMC up date for Step 1 of Algorithm 1 . In particular, when they tak e the form ( 18 ) and ( 19 ), w e obtain the following simple expressions for the gradient: ∂ U ( q ) d (log w i 0 ) = m i − σ − w i 0   τ + 2 p X k =1 β ik   w ∗ k + N α X j =1 w j 0 β j k     , i = 1 , . . . , N α , ∂ U ( q ) d (log β ik ) = m ik + a k − β ik   b k + 2 w i 0   w ∗ k + N α X j =1 w j 0 β j k     , i = 1 , . . . , N α , k = 1 , . . . , p, A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 14 where U ( q ) = log p ( q | rest) with q = (log w i 0 , log β i 1 , . . . , log β ip ) i =1 ,...,N α . Regarding Step 2 of Algorithm 1 , the Laplace exponent for CCRM takes the simple form ψ ( t 1 , . . . , t p ) = Z ∞ 0 [ M ( − w 0 γ 1 , . . . , − w 0 γ p ) − M ( − w 0 ( t 1 + γ 1 ) , . . . , − w 0 ( t p + γ p ))] ρ 0 ( dw 0 ) (40) whic h only requires ev aluating a one-dimensional in tegral, whatever the num b er p of communities, and this can be done numerically . F or the sp eciﬁc model deﬁned by ( 18 ) and ( 19 ), w e obtain ψ ( t 1 , . . . , t p ) = 1 Γ(1 − σ ) Z ∞ 0 " 1 − p Y k =1  1 + w 0 t k b k + w 0 γ k  − a k # " p Y k =1  1 + w 0 γ k b k  − a k # w − 1 − σ 0 e − w 0 τ dw 0 . Finally , w e need to sample total masses ( w ∗ 1 , . . . , w ∗ p ) from ( 36 ), and this can be done by simulating p oin ts ( w i 0 , β i 1 , . . . , β ip ) i =1 , 2 ,... from a Poisson pro cess with exp onentially tilted L ´ evy in tensity αe − w 0 P p k =1 ( γ k + λ k ) β k f ( β 1 , . . . , β p ) ρ 0 ( w 0 ) (41) and summing up the weigh ts w ∗ k = P i =1 , 2 ,... w i 0 β ik for k = 1 , . . . , p . F or inﬁnite-activity CRMs, this is not feasible, and w e suggest to resort to the approximation of Cohen and Rosinski ( 2007 ). More precisely , w e write ( w ∗ 1 , . . . , w ∗ p ) = X ε + X ε where the random vectors X ε ∈ R p + and X ε ∈ R p + are deﬁned as X ε = P i | w i 0 <ε w i 0 ( β i 1 , . . . , β ip ) and X ε = P i | w i 0 >ε w i 0 ( β i 1 , . . . , β ip ). W e can sample a realization of the random vector X ε exactly b y sim ulating the points of a Poisson pro cess with mean intensit y αe − w 0 P p k =1 ( γ k + λ k ) β k f ( β 1 , . . . , β p ) ρ 0 ( w 0 ) 1 w 0 >ε (42) See Section 3.3 and Appendix D for details. The positive random v ector X ε is approximated by a truncated Gaussian random vector with mean µ ε and v ariance Σ ε suc h that µ ε = α Z R p + w 1: p ρ ε ( dw 1 , . . . , dw p ) Σ ε = α Z R p + w 1: p w T 1: p ρ ε ( dw 1 , . . . , dw p ) where ρ ε ( dw 1 , . . . , dw p ) = e − P p k =1 ( γ k + λ k ) w k Z ε 0 w − p 0 F  dw 1 w 0 , . . . , dw p w 0  ρ 0 ( dw 0 ) . Note that µ ε and Σ ε can b oth be expressed as one-dimensional integrals using the gradient and Hessian of the moment generating function M of F . Theorem 7 in Appendix F , which is an adaptation of the results of Cohen and Rosinski ( 2007 ) to CCRM, gives the conditions on the parameters of CCRM under which Σ − 1 / 2 ε ( X ε − µ ε ) d → N (0 , I p ) as ε → 0 and th us the approximation is asymptotically v alid. The Gaussian approximation is in particular asymptot- ically v alid for the CCRM deﬁned by ( 18 ) and ( 19 ) when σ ∈ (0 , 1), hence is v alid for all inﬁnite-activity cases except σ = 0. Note that due to the Gaussian approximation in the prop osal distribution for ( w ∗ α ), Algorithm 1 do es not actually admit the posterior distribution ( 38 ) as in v arian t distribution, and is an appro ximation of an exact MCMC algorithm targeting this distribution. W e observ e in the experimental section that this appro ximation pro vides v ery reasonable results for the examples considered. A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 15 5. Exp erimen ts 5.1. Simulate d data W e ﬁrst study the conv ergence of the MCMC algorithm on synthetic data simulated from the CCRM based graph mo del described in Section 2 where F and ρ 0 tak e the form ( 18 ) and ( 19 ). W e generate an undirected graph with p = 2 comm unities and parameters α = 200, σ = 0 . 2, τ = 1, b k = b = 1 p , a k = a = 0 . 2 and γ k = γ = 0. The sampled graph has 1121 nodes and 6090 edges. F or the inference, w e consider that b and γ are kno wn and w e assume a v ague prior Gamma(0 . 01 , 0 . 01) on the unknown parameters α and φ = (1 − σ, τ , a ). W e run 3 parallel MCMC c hains with diﬀeren t initial v alues. Each chain starts with 10 , 000 iterations using our mo del with only one comm unity where the scores β are ﬁxed to 1, which is equiv alent to the mo del of Caron and F ox ( 2017 ). W e then run 200 , 000 iterations using our mo del with p comm unities. W e use ε = 10 − 3 as a truncation lev el for sim ulating w ∗ 1: p and L = 10 leapfrog steps for the HMC. The stepsizes of b oth the HMC and the random walk MH on (log(1 − σ ) , log τ , log a ) are adapted during the ﬁrst 50 , 000 iterations so as to target acceptance ratios of 0 . 65 and 0 . 23 resp ectively . The computations tak e around 1h10 using Matlab on a standard desktop computer. T race plots of the parameters log α , σ , τ , a and w ∗ = 1 p P p k =1 w ∗ k and histograms based on the last 50 , 000 iterations are given in Figure 6 . Posterior samples clearly con verge around the sampled v alue. Aiming to study further the rate of conv ergence of our algorithm, we explore the impact of a lo w er threshold v alue ε and a higher n umber of iterations. Firstly , decreasing the threshold ε to a v alue  10 − 3 do es not lead to any noticeable c hange in the MCMC histograms, suggesting that the target distribution of our approximate MCMC is v ery close to the p osterior distribution of in terest. Similarly , our appro ximate MCMC posterior obtained from running the algorithm for 10 6 iterations is v ery close to the one obtained from 200000 iterations. Our mo del is able to accurately recov er the mean parameters of b oth low and high degree no des and to pro vide reasonable credible interv als, as shown in Figure 7 (a-b). By generating 5000 graphs from the p osterior predictive we assess that our mo del ﬁts the empirical p ow er-law degree distribution of the sparse generated graph as sho wn in Figure 7 (c). W e demonstrate the in terest of our nonparametric approac h b y comparing these results to the ones obtained with the parametric v ersion of our model. T o achiev e this, we ﬁx w ∗ k = 0 and force the model to lie in the ﬁnite-activity domain b y assuming σ ∈ ( −∞ , 0) and using the prior distribution − σ ∼ Gamma(0 . 01 , 0 . 01). Note that in this case, the mo del is equiv alen t to that of Zhou ( 2015 ). As shown in Figure 8 (a-b), the parametric model is able to reco ver the mean parameters of nodes with high degrees, and credible interv als are similar to that obtained with the full model; how ever, it fails to pro vide reasonable credible in terv als for nodes with lo w degree. In addition, as shown in Figure 8 (c), the p osterior predictiv e degree distribution does not ﬁt the data, illustrating the unability of this parametric mo del to capture pow er-law b ehaviour. 5.2. R e al-world gr aphs W e now apply our metho ds to learn the latent comm unities of tw o real-w orld undirected simple graphs. The ﬁrst net work to be considered, the p olblogs netw ork ( Adamic and Glance , 2005 ), is the netw ork of the American political blogosphere in F ebruary 2005 3 . Two blogs are considered as connected if there is at least one hyperlink from one blog to the other. Additional information on the p olitical leaning of each blog (left/righ t) is also av ailable. The second net work, named USairp ort , is the netw ork of airp orts with at least one connection to a US airp ort in 2010 4 . T abl e 1 Size of the networks, numb er of c ommunities and computational time. Name Nb no des Nb edges Nb communities p Time polblogs 1224 16 , 715 2 20m USairport 1574 17 , 215 4 1h The sizes of the diﬀerent netw orks are giv en in T able 1 . W e consider γ k = 0 is kno wn and w e assume a v ague prior Gamma(0 . 01 , 0 . 01) on the unkno wn parameters α , 1 − σ , τ , a k and b k . W e take p = 2 comm unities for 3 http://www.cise.ufl.edu/research/sparse/matrices/Newman/polblogs 4 http://www.transtats.bts.gov/DL_SelectFields.asp?Table_ID=292 A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 16 MCMC iterations × 10 5 0 0.5 1 1.5 2 log α 4.4 4.6 4.8 5 5.2 5.4 5.6 5.8 6 Chain 1 Chain 2 Chain 3 True log α 4.8 5 5.2 5.4 5.6 5.8 6 Nb MCMC samples 0 50 100 150 200 250 True (a) log α MCMC iterations × 10 5 0 0.5 1 1.5 2 σ 0.05 0.1 0.15 0.2 0.25 0.3 0.35 Chain 1 Chain 2 Chain 3 True σ 0.05 0.1 0.15 0.2 0.25 0.3 0.35 Nb MCMC samples 0 50 100 150 200 250 True (b) σ MCMC iterations × 10 5 0 0.5 1 1.5 2 τ 0 0.5 1 1.5 2 2.5 3 3.5 Chain 1 Chain 2 Chain 3 True τ 0.5 1 1.5 2 2.5 Nb MCMC samples 0 50 100 150 200 250 True (c) τ MCMC iterations × 10 5 0 0.5 1 1.5 2 a 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 Chain 1 Chain 2 Chain 3 True a 0.14 0.16 0.18 0.2 0.22 0.24 Nb MCMC samples 0 20 40 60 80 100 120 140 160 180 True (d) a MCMC iterations × 10 5 0 0.5 1 1.5 2 w ∗ 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 Chain 1 Chain 2 Chain 3 True w ∗ 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 Nb MCMC samples 0 20 40 60 80 100 120 140 160 180 200 True (e) w ∗ Fig 6 . MCMC tr ac e plots (left) and histo grams (right) of p ar ameters (a) log α , (b) σ , (c) τ , (d) a and (e) w ∗ for a gr aph gener ate d with par ameters p = 2 , α = 200 , σ = 0 . 2 , τ = 1 , b = 1 p , a = 0 . 2 and γ = 0 . polblogs and p = 4 communities for USairport . W e run 3 parallel MCMC c hains, eac h with 10 , 000 + 200 , 000 iterations, using the same pro cedure as used for the simulated data; see Section 5.1 . Computation times are rep orted in T able 1 . The simulation of w ∗ 1: p requires more computational time when σ ≥ 0 (inﬁnite-activity case). This explain the larger computation times for USairport compared to polblogs . W e in terpret the communities based on the minimum Ba yes risk p oint estimate where the cost function is a permutation-in v ariant absolute loss on the weigh ts w = ( w ik ) i =1 ,...,N α ; k =1 ,...,p . Let S p b e the set of p erm utations of { 1 , . . . , p } and consider the cost function C ( w , w ? ) = min π ∈S p " p X k =1 N α X i =1   w iπ ( k ) − w ? ik   + p X k =1   w ∗ π ( k ) − w ? ∗ k   # whose ev aluation requires solving a com binatorial optimization problem in O  p 3  using the Hungarian metho d. W e therefore wan t to solve b w = arg min w ? E [ C ( w , w ? ) | Z ] where E [ C ( w, w ? ) | Z ] ' 1 N P N t =1 C  w ( t ) , w ?  and  w ( t )  t =1 ,...,N are from the MCMC output. F or simplicity , w e limit the searc h of b w to the set of MCMC samples giving b w = arg min w ? ∈ { w (1) ,...,w ( N ) } 1 N N X t =1 C  w ( t ) , w ?  . A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 17 Index of node (sorted by dec. degree) 10 20 30 40 50 Mean sociability parameters 0 0.5 1 1.5 2 2.5 3 95% credible intervals True value (a) 50 nodes with highest degrees Index of node (sorted by dec. degree) 1080 1090 1100 1110 1120 Log mean sociability parameters -12 -11 -10 -9 -8 -7 -6 -5 -4 95% credible intervals True value (b) 50 nodes with low est degrees Degree 10 0 10 1 10 2 10 3 Distribution 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 95% posterior predictive Data (c) Degree distribution Fig 7 . 95% posterior cr e dible intervals and true values of (a) the me an p ar ameters w i = 1 p P p k =1 w ik of the 50 nodes with highest de gr e es and (b) the lo g me an p ar ameters log w i of the 50 no des with lowest de gr e es. (c) Empiric al de gr ee distribution and 95% p osterior pr e dictive cr e dible interval. R esults obtaine d for a gr aph gener ate d with p ar ameters p = 2 , α = 200 , σ = 0 . 2 , τ = 1 , b = 1 p , a = 0 . 2 and γ = 0 . Index of node (sorted by dec. degree) 10 20 30 40 50 Mean sociability parameters 0 0.5 1 1.5 2 2.5 3 95% credible intervals True value (a) 50 nodes with highest degrees Index of node (sorted by dec. degree) 1080 1090 1100 1110 1120 Log mean sociability parameters -12 -11 -10 -9 -8 -7 -6 -5 -4 95% credible intervals True value (b) 50 nodes with low est degrees Degree 10 0 10 1 10 2 10 3 Distribution 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 95% posterior predictive Data (c) Degree distribution Fig 8 . Se e Figur e 7 . R esults obtaine d on the same gener ate d gr aph by inferring a ﬁnite-activity mo del with w ∗ k = 0 and σ ≤ 0 . T able 2 rep orts the no des with highest w eights in each communit y for the polblogs netw ork. Figure 10 also shows the weigh t associated to each of the tw o communit y alongside the true left/righ t class for each blog. The tw o learned comm unities, whic h can be interpreted as “Lib eral” and “Conserv ative”, clearly re- co v e r the political leaning of the blogs. Figure 9 sho ws the adjacency matrices obtained b y reordering the no des by comm unit y membership, where each no de is assigned to the communit y whose weigh t is maxi- m um, clearly sho wing the blo ck-structure of this netw ork. The obtained clustering yields a 93 . 95% accuracy when compared to the ground truth classiﬁcation. Figure 11 (a) sho ws the relativ e communit y proportions for a subset of the blogs. dailykos.com and washingtonmonthly.com are clearly describ ed as lib eral while blogsforbush.com , instapundit.com and drudgereport.com are clearly conserv ativ e. Other more mo der- ate blogs such as danieldrezner.com/blog and andrewsullivan.com hav e more balanced v alues in both comm unities. Figure 12 (a) shows that the posterior predictiv e degree distribution provides a go o d ﬁt to the data. F or USairport , the four learned communities can also b e easily interpreted, as seen in T able 3 . The ﬁrst communit y , lab eled “Hub”, represents highly connected airp orts with no preferred lo cation, while the three others, labeled “East”, “W est” and “Alask a”, are communities based on the lo cation of the airport. In Figure 11 (b), w e can see that some airports hav e a strong lev el of aﬃliation in a single communit y: New Y ork and Miami for “Hub”, Lansing for “East”, Seattle for “W est” and Bethel and Anc horage for “Alask a”. Other airp orts hav e signiﬁcan t weigh ts in diﬀeren t communities: Raleigh/Durham and Los Angeles are h ubs with strong regional connections, Nash ville and Minneap olis share a signiﬁcant num b er of connections with b oth East and W est of the USA. Anchorage has a signiﬁcan t “Hub” weigh t, while most airports in Alask a are A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 18 (a) polblogs (b) USairport Fig 9 . A djac ency matrices of the (a) polblogs and (b) USairport networks, r e or dere d by asso ciating each no de to the c ommunity wher e it has the highest weight. T a ble 2 No des with highest weight in e ach community for the polblogs network. Blo g URLs ar e fol lowe d by known p olitic al le aning: (L) for left-wing and (R) for right-wing. Comm unit y 1: “Liberal” Comm unit y 2: “Conserv ative” dailykos.com (L) atrios.blogspot.com (L) talkingpointsmemo.com (L) washingtonmonthly.com (L) liberaloasis.com (L) talkleft.com (L) digbysblog.blogspot.com (L) newleftblogs.blogspot.com (L) politicalstrategy.org (L) juancole.com (L) instapundit.com (R) blogsforbush.com (R) powerlineblog.com (R) drudgereport.com (R) littlegreenfootballs.com/weblog (R) michellemalkin.com (R) lashawnbarber.com (R) wizbangblog.com (R) hughhewitt.com (R) truthlaidbear.com (R) disconnected from the rest of the w orld as can b e seen in Figure 9 (b). “Alask a” app ears as a separate block while substantial ov erlaps are observed b et ween the “Hub”, “East” and “W est” communities. Figure 12 (b) sho ws that the posterior predictiv e degree distribution also pro vides a goo d ﬁt to the data. 5.3. Comp arisons W e compare the ﬁt of our mo del to the mixed membership sto chastic blockmodel (MMSB) of Airoldi et al. ( 2008 ) and the multiplicativ e latent factor mo del (MLFM) of Hoﬀ ( 2009 ). The tw o mo dels are brieﬂy explained b elo w. T a ble 3 No des with highest weights in e ach community for the USairport network. Comm unit y 1: “Hub” Comm unity 2: “East” Comm unit y 3: “W est” Comm unit y 4: “Alask a” Miami, FL New Y ork, NY Newa rk, NJ Los Angeles, CA Atlanta, GA Washington, DC Chicago, IL Boston, MA Houston, TX Orlando, FL Cleveland, OH Detroit, MI Nashville, TN Chicago, IL Knoxville, TN Atlanta, GA Louisville, KY Indianapolis, IN Memphis, TN Charlotte, NC Denver, CO Las Vegas, NV Los Angeles, CA Salt Lake City , UT Seattle, W A Burbank, CA Phoenix, AZ Oakland, CA Po rtland, OR Albuquerque, NM Anchorage, AK Fairbanks, AK Bethel, AK St. Mary’s, AK King Salmon, AK McGrath, AK Unalakleet, AK Galena, AK Aniak, AK Kotzebue, AK A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 19 w 1 0 1 2 3 4 100monkeystyping.com 12thharmonic.com/wordpress 750volts.blogspot.com 95theses.blogspot.com abbadabbaduo.blogspot.com aboutpolitics.blogspot.com achingandspinning.blogspot.com ackackack.com adamtalib.blogspot.com adviceforlefty.blogspot.com agonist.org aintnobaddude.com ajbenjaminjr.blogspot.com alicublog.blogspot.com allanjenkins.typepad.com allspinzone.blogspot.com alphapredator.com/wdtgw.htm alternateworlds.blogspot.com althippo.blogspot.com alvintostig.typepad.com americablog.blogspot.com americablog.org americanmuslim.blogs.com americanpolitics.com americansforbayh.blogspot.com amleft.blogspot.com amliberal.com/blog amptoons.com/blog anarchyxero.robfindlay.org andifeelfine.blogspot.com andymatic.com angrybear.blogspot.com angrydesi.typepad.com angryfinger.org angryhomo.blogspot.com annatopia.com/home.html anoldsoul.blogspot.com anonymoussources.blogspot.com answerguy.blogspot.com anybodybutbushyall.blogspot.com anywhichway.net arancaytar.blogspot.com archpundit.com arkansastonight.com askhoudari.blogspot.com astroturf.uni.cc atease.blogspot.com atrios.blogspot.com atrios.blogspot.com/ awards5.tripod.com/tarasblog b-patton.blogspot.com bakshi.us/redskunk barkingdingo.blogspot.com bartcop.com battleground-wisconsin.blogspot.com battlegroundstates.blogspot.com beastsbelly.blogspot.com bentcorner.blogspot.com bettyblog.com billmon.org billsrants.blogspot.com blog.dccc.org blog.dennisfox.net blog.glinka.com blog.johnkerry.com blog.kenlan.net blog.mintruth.com blog.veggiedude.com blog01.kintera.com/dnccblog blog01.kintera.com/emilysblog bloganonymous.com blogforamerica.com blogforarizona.com blogfreeordie.blogspot.com blogitics.com blogs.salon.com/0002874 blogs.salon.com/0003364 blogslut.com blosteam.blogspot.com bluegrassroots.blogspot.com bluelemur.com blueoregon.com bluestateredstate.blogspot.com bodyandsoul.typepad.com boffo.typepad.com boloboffin.blogspot.com bopnews.com boston.com/news/blogs/dnc bowles2004.com/weblog boycottsbg.com bradcarson.com/blog bradfriedman.com/bradblog brilliantatbreakfast.blogspot.com browncross.com/usualsuspects btcnews.com/btcnews bullmooseblog.com burntorangereport.com bushlies.net/pages/10/index.htm bushmisunderestimated.blogspot.com busybusybusy.com buzzmachine.com carpetblogger.com cbaker.org/blog cemonitor.blogspot.com centerpiece.blogspot.com changeforamerica.com/blog charleypatton.blogspot.com charlineandjamie.com/dotnetweb01a/blogdisplay.aspx?logname=jamie&logcatid=48 chepooka.com chiron.typepad.com churchofcriticalthinking.com clarified.blogspot.com claudialong.com/blog cleancutkid.com collegedems.com/blog conniptions.net conservativessuck.blogspot.com contrapositive.blogspot.com corrente.blogspot.com counterspin.blogspot.com crankylittleblog.blogspot.com criticalviewer.com crookedtimber.org crooksandliars.com dailyblatt.blogspot.com dailyhowler.com dailykos.com dailywarnews.blogspot.com damntheman.net dashnier.blogspot.com davidsirota.com dawnofnewamerica.blogspot.com deathpenaltyusa.blogspot.com debunker.net dedspace.blogspot.com delraysteve.com/blog demagogue.blogspot.com demleft.blogspot.com democraticunderground.com democratreport.blogspot.com democrats.org/blog democratvoice.org dems2000.net dems2004.org/blog denisdekat.com dewar.journalspace.com digbysblog.blogspot.com digestiblenews.blogspot.com digital-democrat.blogspot.com digitaljay.blogspot.com dimmykarras.blogspot.com dir.salon.com/topics/joe_conason/index.html discourse.net diypolitics.blogdrive.com dneiwert.blogspot.com dogfight04.typepad.com dohiyimir.typepad.com dolphinsdock.com/blog donkey2004.blogspot.com donspoliticalblog.blogspot.com duckdaotsu.blogspot.com dystopia.blog-city.com eastbaykerry.com elayneriggs.blogspot.com elderbearden.blogspot.com electablog.com elemming2.blogspot.com elissa.typepad.com eltcamerica.blogspot.com emergingdemocraticmajority.com emergingdemocraticmajorityweblog.com/donkeyrising emptyisform.com endthenightmare.blogspot.com enemykombatant.blogspot.com ergio.blogspot.com estropundit.blogspot.com etherealgirl.blogspot.com fafblog.blogspot.com fairshot.org farleft.blogspot.com fiercepoet.com flagrancy.net folkbum.blogspot.com forewarned.blogspot.com frederickclarkson.com fromtheroots.org fuckthisblog.blogspot.com gadflyer.com garbage-house.com georgewbuy.blogspot.com gisleson.com/blog global-equality.org/news/blog/index.shtml goose3five.blogspot.com grassrootsmom.blogspot.com grassrootsnation.com greaterdemocracy.org greendogdemocrat.blogspot.com gregpalast.com grupo-utopia.com/blog/isou hal9000report.blogdrive.com hategun.com hellafaded.blogspot.com hellskitchennyc.blogspot.com hereswhatsleft.typepad.com hereswhatsleft.typepad.com/home higherpieproductions.com home.att.net/~gentle.reader home.earthlink.net/~fsrhine home.earthlink.net/~kevin.omeara home.houston.rr.com/skeptical homepage.mac.com/elisa_camahort/iblog hotflashesfromthecampaigntrail.blogspot.com hotflashreport.com howienfriends.blogspot.com hugozoom.blogspot.com ibe.blogspot.com ibrutus.com iddybud.blogspot.com idiosyncratictendencies.com idlehandsmag.com/yellowdog04 iflipflop.com ignatz.blogspot.com ihatepatrobertson.com ilovecynics.com insanepreschoolmom.blogspot.com interestingtimes.blogspot.com interestingtimes.typepad.com ironmouth.com irregulartimes.com/blogger.html islanddave.blogspot.com itlookslikethis.blogeasy.com ivotedforgeorge.com iwantmycountryback.org j-bradford-delong.net/movable_type jadbury.com jadbury.com/blog jadedreality.blogspot.com jameswolcott.com janm.blogspot.com jbcougar.blogspot.com jdeeth.blogspot.com jeremybrendan.blogspot.com jetage.blogspot.com jewssansfrontieres.blogspot.com jimgilliam.com jimtreacher.com jinkythecat.blogspot.com jmbzine.com joebrent.blogspot.com john.hoke.org johnmccrory.com johnnynobody.blogspot.com journals.aol.com/breuvers/madashellnotgoingtotakeitanymore juancole.com julietterossant.com/superchefblog/superchefblog.html juliusblog.blogspot.com justmark.com karlhenderson.blogspot.com karmicsoup.blogspot.com kcdems.blogspot.com kensain.com kerryforpresident2008.blogspot.com kicktheleftist.blogspot.com kiosan.blogharbor.com kippsblog.com kurtkk.blogspot.com kydem.blogspot.com lastdaysoftherepublic.fatoprofugus.net lawdork.blogspot.com leanleft.com left2right.typepad.com lefterer.com lefti.blogspot.com leftinthereign.blogspot.com leftthought.blogspot.com lefttimes.com leftwingnation.com lendmesomesugar.blogs.com/shut_up_already lennonreport.blogspot.com leonards-digest.blogspot.com liberalangst.org liberalavenger.com liberaleric.blogspot.com liberaloasis.com liberalreview.blogspot.com liberalthunderdome.blogspot.com lightupthedarkness.org lightupthedarkness.org/blog/default.asp limericksavant.blogspot.com liquidlist.com livejournal.com/users/hoptkov livejournal.com/users/jmhm loadedmouth.com lorenjavier.blogspot.com loud-liberal.blogspot.com loveamericahatebush.com lowrentrat.blogdrive.com ltobs.blogspot.com ludis.blogspot.com ludovicspeaks.typepad.com ludovicspeaks.typepad.com/real_deal madkane.com/notable.html maggiespants.co.uk mahablog.com marcbrazeau.blogspot.com margaretcho.com/blog/blog.htm margieburns.com markadams.blogdrive.com markarkleiman.com markschmitt.typepad.com/decembrist markwarnerforpresident2008.blogspot.com maruthecrankpot.blogspot.com massachusetts-liberal.com mathewgross.com mathewgross.com/blog matthewyglesias.com maultasch.us maxspeak.org/mt mayflowerhill.blogspot.com mcwil.blogspot.com meanspirit.blogspot.com meateatingleftist.com mediajunkie.com/edgewise mediaprima.com/nv1962 messengerpuppet.com/blog metropol47.blogspot.com michael124.blogspot.com michaelberube.com michaelphillips.blogspot.com mickikrimmel.com/redcarpet midguard.blogspot.com minagahet.blogspot.com mirocat.com misschin.bloggage.com mixtersmix.blogspot.com montages.blogspot.com moxiegrrrl.com mrleft.org msnbc.msn.com/id/3449870 mstabile.blogspot.com mydd.com myleftbrain.com myleftthoughts.blogspot.com mysterypollster.com/main nastyboys.blogspot.com nathancallahan.com nathannewman.org/log nationalreview.com/frum/frum-diary.asp ndnblog.org nearouterheaven.blogspot.com nebursworld.blogspot.com needlenose.com nerofiddled.blogspot.com netpolitik.blogspot.com netweed.com/postmodernanarchist newappeal.blogspot.com newdemocrat.blogdrive.com newdonkey.com newleftblogs.blogspot.com nielsenhayden.com/electrolite nielsenhayden.com/makinglight njfordemocracy.org node707.com nofearoffreedom.blogspot.com nomoremister.blogspot.com norbizness.com noseyonline.com nosmallplans.com/rants notbush.com notgeniuses.com nycitystoop.com obamablog.com obsidianwings.blogs.com oddhours.blogspot.com offthekuff.com/mt oipp.blogspot.com oliverwillis.com one38.org openeyesmemo.com orient-lodge.com ospolitics.org oxblog.blogspot.com pacdaily.com pacificviews.org pandagon.net parabasis.typepad.com patriotboy.blogspot.com pattymurray.com/blog.php paulchasman.com peacegarden.blogspot.com penheaded.blogspot.com pennywit.com/drupal permanentred.com peskytherat.com/pesky phillybri.blogspot.com philoponia.blogspot.com philosophers-stone.blogspot.com philvbblog.blogspot.com phoblographer.com pinkofeministhellcat.typepad.com/pinko_feminist_hellcat planetdave.com/blogs.php3?mid=1 politicalmonitor.us/blog politicalstrategy.org politicalthought.net politicalwire.com politics.feedster.com politicswithrichard.blogspot.com polizeros.com pollsterseek.blogspot.com polstate.com portapulpit.com praguewriter.typepad.com presidentboxer.blogspot.com prestoncaldwellblog.blogspot.com profgoose.blogspot.com progressiveink.com progressivesociety.com/blog proponentofreason.blogspot.com prospect.org/weblog pudentilla.blogspot.com punditician.blogspot.com purplestates.blogspot.com qando.net question-dean.blogspot.com radgeek.com radikal_papi.pitas.com radio.weblogs.com/0131587/categories/flush raedinthemiddle.blogspot.com reachm.com/amstreet readingpost.com realitybasednation.com red-harvest.blogspot.com religiousleft.blogspot.com republicoft.typepad.com resistoppression.blogspot.com restoreamerica.blogspot.com retrogrouch.net rhetoricrhythm.blogspot.com righthandthief.blogspot.com rising-hegemon.blogspot.com risinghegemon.blogspot.com rittenhouse.blogspot.com riverbendblog.blogspot.com roadtosurfdom.com robot-invasion.com robschumacher.blogspot.com rogerailes.blogspot.com rooksrant.com rowhouselogic.com roxanne.typepad.com rudepundit.blogspot.com ruminatethis.com s88172659.onlinehome.us/spockosbrain.html sadlyno.com saveoursenate.blogspot.com sccdp.org/blog.php schadelmann.com scoobiedavis.blogspot.com scottcsmith.net scripting.com scrutinyhooligans.blogspot.com seeingtheforest.com seetheforest.blogspot.com sethdmichaels.com shakespearessister.blogspot.com shamelessantagonist.blogspot.com sharedthought.com/weblog shininglight.us shoegal.modblog.com shotofpolitics.blogspot.com shrillblog.blogspot.com simbaud.blogspot.com simplyappalling.blogspot.com smirkingchimp.com smithersmpls.com snunes.blogspot.com someonetakethewheel.blogspot.com soonerthought.com southknoxbubba.net/skblog spacedog26.blogspot.com spectatorsport.blogspot.com stagefour.typepad.com/commonprejudice stalinism.com/shot-by-both-sides stcfoundation.blogspot.com stephenbates.com/yellowdoggereldemocrat stevecopy.blogharbor.com stevegilliard.blogspot.com steveguy.me.uk stevemacek.blogspot.com stillfighting.blogspot.com stitt.blogspot.com strangedoctrines.typepad.com strangesemantics.blogspot.com suburbanguerrilla.blogspot.com swamiuptown.com swingstateproject.com takebackthemedia.com talkingpointsmemo.com talkleft.com tallone.org tbogg.blogspot.com teenageenthusiast.blogspot.com tesibria.typepad.com/democracy_is_coming thatcoloredfellasweblog.bloghorn.com thatgoodnight.com the-goddess.org the-hamster.com theaggpro.blogspot.com theagitator.blogspot.com theangryliberal.blogspot.com theantibush.org theapologist.blogspot.com theblogwarrior.blogspot.com theblueview.blogspot.com thecarpetbaggerreport.com thecore4.com thefulcrum.blogspot.com theleftcoaster.com thelonedem.com thelucideye.blogspot.com thepeach.blogspot.com thepoorman.net theradiomansays.blogspot.com therogueangel.com/blog thesnooter.blogspot.com thetalentshow.org thetalkingdog.com thewashingtonnote.com theworldaroundyou.com thewritewing.blogspot.com theyetimn.blogspot.com thingsasitis.net think2004.com thinkingrocks.blogspot.com thirdredeye.blogspot.com thislousytshirt.net thismodernworld.com thomasblog.typepad.com/bloggg thunewatch.squarespace.com tinylittledots.com tobyemden.blogspot.com tomburka.com tomcoburnisabigfatjerk.blogspot.com tomherod.blogspot.com toteota.blogspot.com trainorphans.org translucence.org travalex.blogspot.com tristero.blogspot.com trueconservative.typepad.com truedemocrat.blogspot.com turnpikefeminist.com turnspit.com uberdude.blogspot.com uggabugga.blogspot.com unodemocrats.com/blog upper-left.blogspot.com urbandemocracy.blogspot.com urbanecoinc.com/wordpress1 usademocrazy.blogspot.com usliberals.about.com vernsblog.thegillfamily.us:8180 virginiaprogressive.blogspot.com vocis.com/frank waitingfordorothy.blogspot.com wampum.wabanaki.net warandpiece.com washingtonmonthly.com watchingthewatchers.org waveflux.net weblogs.bakshi.us/redskunk whopundit.blogspot.com whoviating.blogspot.com whyareweback.blogspot.com winnegar.blog-city.com winthewest.blogspot.com woneffe.typepad.com wonkette.com worldonfire.typepad.com/world_on_fire wpblog.ohpinion.com wsrblood.typepad.com/wiserblog wtpm.blogspot.com x-pollen.com/many/blog.html xnerg.blogspot.com yarsrevenge.blogspot.com yglesias.typepad.com/matthew yoder.ru younglibs.com Left a100wwe.blogspot.com absurd-canadian.blogspot.com ace.mu.nu aceoftrumpblog.blogspot.com acepilots.com/mt acertainslantoflight.blogspot.com achicknamedmarzi.com activistchat.com/blogiran adeimantus.blogspot.com alamodispatches.blogspot.com alamonation.blogspot.com alarmingnews.com aldaron.modblog.com aldaynet.org alexpelan.blogspot.com alittlemoretotheright.com all-encompassingly.com allahpundit.com allthingsmark.com alphapatriot.com althouse.blogspot.com americanchronicle.blogspot.com americandigest.org americandinosaur.blogspot.com americanfuture.typepad.com americankernel.com americanreport.blogspot.com americanthinker.com americanwarmonger.blogspot.com americanworldview.tripod.com/weltansblog amigo13.blogspot.com amradio.blogspot.com andrewsullivan.com andrightlyso.blogspot.com angrychineseblogger.blog-city.com angrygwn.blogspot.com anncoulter.org annika.mu.nu anothervrwc.blogspot.com antijohnkerry.blogspot.com antiprotester.blogspot.com apologiesdemanded.blogspot.com arkanssouri.blogspot.com armchairgeneralist.typepad.com asmallvictory.net asmodeus-space.com/weblog/blog/index.php astrosoup.blogspot.com athena.blogs.com awideawake.blogspot.com ayerdis.com/rants.php babytrollblog.com baldilocks.typepad.com balloon-juice.com bamapachyderm.blogspot.com batr.net batr.org/commentary.html beautifulatrocities.com beefalwayswins.blogspot.com beldar.blogs.com/beldarblog bellusbellum.info belmontclub.blogspot.com bensgtusmc.com bensworld.patriotforum.org bernardmoon.blogspot.com betsyspage.blogspot.com bidinotto.journalspace.com bigfatroid.blogspot.com bigjourney.blogspot.com billhobbs.com billrice.org billroggio.com/easongate bitheads.blogspot.com bittersweetme.net/blog/index.php bkennelly.com/vox blackmanforbush.blogspot.com blindmindseye.com blithered.blogspot.com blog.nam.org blog.ncsweb.us blog.so-amazing.net blog.willrobinson47.com bloghogger.blogspot.com blogicus.com blogodob.blogspot.com blogsagainsthillary.com blogsforbush.com blogsofwar.com blogspirator.blogspot.com blubberhollow.com/blog blue6golf.blogspot.com bluegoldfish.blogs.com/surface bluestates.blogspot.com bobhayes.net boottotheskull.com borgardblog.blogspot.com brainshavings.com brianscott.blogspot.com brutallyhonest.org buckeyepundit.blogspot.com bullmooserepublicans.com/blog bushblog.us c-pol.blogspot.com californiaconservatives4truth.blogspot.com calirep.blogspot.com calpoly.edu/~doleary/blog.htm captainsquartersblog.com/mt catscape.com cayankee.blogs.com cbcbcbcb.blogspot.com ccs178.com cdharris.net cecile.squarespace.com celluloid-wisdom.com/pw celluloid-wisdom.com/pw/ centinel.blogspot.com chadster.capitalfocus.org chainik_hocker.blogspot.com charlesking.blogspot.com chiefsblog.blogspot.com chrenkoff.blogspot.com christian-patriot.blogspot.com civilcommotion.com claycalhoun.com claytoncorner.blogspot.com clubforgrowth.org/blog cmillertime.blogspot.com coffee.ondragonswing.com commonsenserunswild.typepad.com confederateyankee.blogspot.com conservativeblogger.com conservativecat.com conservativedialysis.com conservativeeyes.blogspot.com conservativelife.com/blog conservativenation.us conservativeobserver.blogspot.com conservativepolitics.blogspot.com conservativepunk.com conservativepush.blogspot.com conservativeresponse.com conservativerevolution.com conservativezone.blogspot.com coxandforkum.com cptnwinky.com crankybastard.blogspot.com crankyneocon.com/crankyneocon cubanstories.blogspot.com cybermenace.blogspot.com cynicalnation.com d-42.com daddypundit.blogspot.com dadougster.blogspot.com dailypundit.com dailyrecycler.com/blog daisycutter.blogspot.com dalythoughts.com danieldrezner.com/blog danzfamily.com dartblog.com datingrepublicans.com deadcantrant.com/blog deanesmay.com deansjournal.blogspot.com decision08.blogspot.com denbeste.nu depressedlonelybored.typepad.com deserttusk.blogspot.com devildawg.blogspot.com dgci.net diggersrealm.com/mt dilusionl.blogspot.com diplomadic.blogspot.com discerningtexan.blogspot.com discountblogger.com disintegrator.blogspot.com dissectleft.blogspot.com dizzy-girl.net/index.php dmobley.blogspot.com dogwooddreams.com/serendipity donaldsensing.com downtownlad.blogspot.com drudgereport.com drumwaster.com drunkensamurai.blogspot.com dummocrats.com dynamist.com/weblog economicswithaface.com/weblog eeg3.com ejectejecteject.com electionprojection.com electricvenom.com elevendayempire.com/movabletype emilythedog.blogspot.com erickohn.com erikack.blogspot.com etalkinghead.com evangelicaloutpost.com everythingiknowiswrong.com evilpundit.com fearfulsymmetry.blogspot.com firstdownpolitics.com franklog.com fraterslibertas.com freedomofthought.com freerepublic.com fringeblog.com fritzs-thoughts.blogspot.com gaypatriot.blogspot.com gccpundit.blogspot.com gcruse.typepad.com georgewbush.com gerberatetra.blogspot.com gevkaffeegal.typepad.com/the_alliance ghostofaflea.com gindy.blogspot.com gmscorner.blogspot.com godubya.com gopandcollege.blogspot.com gopandthecity.blogspot.com gopbloggers.org gopinsight.com gotdesign.blogspot.com greatestjeneration.com grownups.blogspot.com gullyborg.typepad.com happyhomeschooler.blog-city.com hasidicgentile.org hathaby.net/weblog.php headland.blogspot.com healingiraq.blogspot.com heavylifting.blogspot.com hectorvex.blogspot.com henerd.com hiddennook.blogspot.com hillcountryviews.blogspot.com hipublican.com hobsonschoice.us holecard.blogspot.com home.earthlink.net/~techservant home.midsouth.rr.com/joefish homespunbloggers.blogspot.com homicidalmaniak.com horologium.net hughhewitt.com iamaugustus.blogspot.com ikilledcheguevara.blogspot.com imao.us imprescindibile.ilcannocchiale.it incite1.blogspot.com indcjournal.com indepundit.com instapundit.com interested-participant.blogspot.com inthebullpen.com iowageek.blogspot.com iraqthemodel.blogspot.com isdl.blogspot.com isdl.weblogs.us itscomedowntothis.blogspot.com itznewstome.blogspot.com jacklewis.net janegalt.net jchristopherfeng.com jeffblanco.blogdrive.com jewishworldreview.com jg-spot.blogspot.com jihadwatch.org jlynnsmith30.blogspot.com joatmoaf.typepad.com/i_love_jet_noise joepublicus.blogspot.com johnkerrymustlose.com johnnbrown.blogspot.com jokerstotheright.blogspot.com joshferguson.blogspot.com journals.aol.com/lily990016/pissoffliberals journals.aol.com/mark24609/myblog jperspective.com jquinton.com jrogg.blogspot.com juntoboys.blogspot.com justalkin.us kalblog.com kausfiles.com keithy1028.bravejournal.com kellipundit.blogspot.com kenshepherd.blogspot.com kevincourser.blogspot.com kimdutoit.com/dr/weblog.php king-of-fools.com kitchennetwork.blogspot.com la4israel.org/wordpress ladymac.com lahargis.blogspot.com larsonreport.com lashawnbarber.blogspot.com lashawnbarber.com laughatliberals.com laughingwolf.net lawguy.blogspot.com valuerightnews.blogspot.com leftcoastconservative.blogspot.com legendgames.net/myblog.asp libertarianjackass.blogspot.com littlegreenfootballs.com/weblog littlegreenfootballs.com/weblog/weblog.php littletinylies.com livejournal.com/users/bambenek livinginbabylon.blogspot.com lonewacko.com loosecoin.blogspot.com lt-smash.us lucianne.com luxeword.com madanthony.org made4theinternet.blogspot.com madtechspeaks.blogspot.com magnificentfreedom.us manifestcontent.net mantic.squarespace.com mariestwocents.blogspot.com markheimonen.blogspot.com marknicodemo.blogspot.com marksteyn.com martinipundit.com martydee.com/poligov matthewstinson.com/blog mattresspolice.com maybeithinktoomuch.com mdcbowen.org/cobb meckz.net mementomoron.blogspot.com messopotamian.blogspot.com metallicity.com/mb.asp mhking.mu.nu michaeltotten.com michellemalkin.com midwestrightwingers.blogspot.com mikesamerica.blogspot.com millers_time.typepad.com mindspring.com/~gsb3/blogger.html mobile.blogcastlive.com moorewatch.com mostlycajun.com motgs.com mtvirtus.blogspot.com mudandphud.blogspot.com mudvillegazette.com musicfromthebluelight.com/b mwilliams.info my2cent.blogspot.com myaisling.blogspot.com mycatharsis.com mydamascusroad.blogspot.com myechochamber.blogspot.com mypetjawa.mu.nu nashvilletruth.blogspot.com nationalbird.net nationalcenter.org/blog.html nationalnewsregister.com/modules/wordpress nationalreview.com/thecorner nationalreview.com/thecorner/corner.asp neoconservatives.blogspot.com neoconswatch.blogspot.com neonbrainiac.nu/mt/blog nerepublican.blogspot.com neverwrongnever.blogspot.com newcounterculture.com/log newsbloggers.blogspot.com newsjungle.blogspot.com newsnik.com nicedoggie.net nickiegoomba.blogspot.com nicoladellarciprete.blogs.com noguru.tblog.com nomayo.mu.nu nonboxthinking.blogspot.com nykola.com o4tom.blogspot.com officesmileman.com ofthemind.blogspot.com ogresview.blogspot.com okieonthelam.com oldfashionedwhippersnapper.blogspot.com onthewire.blogspot.com opiniontimes.blogspot.com opiniontimes.com oppb.com orthogonian.blogspot.com outsidethebeltway.com pajamaeditors.blogspot.com pajamahadin.com pajamajournalist.blogspot.com palominas.blogspot.com papadoc.net/pinkflamingobar.html pardonmyenglish.com patrickmcclarty.blogspot.com patrickruffini.com patriot-paradox.com patriotblog.com patriotism101.blogspot.com patriotsforbush.com patterico.com pejmanesque.com pennsylvanianinexile.blogspot.com perryvsworld.blogspot.com philomathean.blogs.com piano-tuning.net/blog poeticvalues.blogspot.com poliblogger.com polipundit.com politicalanimals.blogspot.com politicaldogs.org politicalmusings.net politicalpie.modblog.com politicaltracker.com politicaltracker.com/blog/weblog.php politicalvicesquad.blogspot.com poorjustin.blogspot.com poweragerambleson.blogspot.com powerlineblog.com powerpundit.com powervacuum.org presidentreagan.info/discussion professorbainbridge.com prolifeblogs.com propagandamachine.blogspot.com prosandcons.us punditguy.com punditz.journalspace.com qoae.net quante.blogspot.com quibbles-n-bits.com quimundus.modblog.com quimundus.squarespace.com r21online.com r2korn.blogdrive.com rabner.blogspot.com radioequalizer.blogspot.com raincrossblog.com randomjottings.net rant.st ratherbiased.com/news rathergate.com realclearpolitics.com realordeal.blogspot.com rebrand-democrat.txfx.net reckersau.blogspot.com redmindbluestate.blogspot.com rednyc.blogspot.com redstate.org redstatealliance.blogspot.com reggiescorner.blogspot.com regimechangeiran.com renpro.kwcom.net reporterette.com republog.net respublica2004.blogspot.com retiredwarrant.blogdrive.com revone.net/jarhead rfburnhertz.net/rf_blog richmar.blogspot.com right-thinking.com rightmoment.blogspot.com rightonred.net rightperspective.net rightrainbow.com rightstate.blogspot.com rightvoices.com rightwingbastardo.blogspot.com rightwingbob.com rightwingdude.squarespace.com rightwingjournal.blogspot.com rightwingnews.com rightwingthinking.blogspot.com rightwingtroll.blogspot.com rightword.blogspot.com riteturnonly.blogspot.com robbernard.com robertp17.blogspot.com rockpaperscissor.blogspot.com rockpaperscissors.blogs.com rogerlsimon.com romeocat.typepad.com ron-james.com/blog rosenblog.com roseoftheknights.com/yank/pride.html roughstock.blogspot.com ryanshead.com samizdata.net/blog sarcasticsoutherner.blogspot.com sayanythingblog.com scha-den-freu-de.blogspot.com scrappleface.com scrawlville.com sebourn.com/weblog section-31.blogspot.com secureliberty.org/ sedenteanimo.com selfish-souls.com/blog sgtstryker.com shadydowns.blogspot.com shakingspears.blogspot.com shallows.blogspot.com sharpeblog.cybercatholics.com shortfatcorporal.blogspot.com sickofliberals.blogspot.com sinfactor.blogspot.com sistertoldjah.com sixmeatbuffet.com slantpoint.com slingsnarrows.erudite-absurdity.com slipofthepen.blogspot.com slowplay.com slublog.blogspot.com smarthomeschool.com smashingzero.blogspot.com smellyduck.blogspot.com sneakeasysjoint.com sondrak.com soundingthetrumpet.blogspot.com southernappeal.blogspot.com southernrightwingextremist.blogspot.com southernwatch.blogspot.com spreadingunderstanding.blogspot.com spydrzweb.blogspot.com squiggler.braveblog.com stan-istan.blogspot.com stealthebandwagon.blogspot.com stillmanvalley.org/aoo stix1972.blogspot.com stmonk.blogspot.com stolenthunder.blogspot.com strategicintelligence.blogspot.com streetlog.typepad.com summation.net sweetgrrrl.blogspot.com swfwithattitude.blogspot.com swiftreport.blogs.com tagorda.com talesofawanderingmind.blogspot.com talkshowamerica.blogspot.com techievampire.net/wppol techtrend.com/blog/taranicole templeofjennifer.com/blog tepid-observer.blogspot.com texasbug.blogspot.com thatliberalmedia.com the-torch.com theamericanmind.com thebloghill.blogspot.com thecommonvirtue.com theconservativezone.com thecrankfiles.blogspot.com thedailygoodnews.com thedispatch.blogspot.com thedonovan.com theglitteringeye.com thejessefactor.blogspot.com thelastword.blogdrive.com theliberalslies.blogspot.com themeofabsence.com theneo-progressive.blogspot.com thenudnikfile.blogspot.com thepatriette.com thepaytons.org/essays/considerettes thepoliticalanimal.com thepoliticalteen.net therightblog.blogspot.com therightcoast.blogspot.com therightwingconspiracy.org therinsecycle.com therostra.blogspot.com thesakeofargument.com thespoonsexperience.com thetarpit.blogspot.com thetemplarpundit.blogspot.com theunderscore.blogspot.com theurbangrind.blogspot.com thevaticanofliberalism.com thinkingright.us thisliberal.com thomasgalvin.blogspot.com thomaswharry.com timblair.net timblair.spleenville.com tokenyoungconservative.blogspot.com tom-hanna.org toneofvoice.blogspot.com totels.com/bush04 townhall.com/clog truebluegal.blogspot.com truthlaidbear.com truthliesandcommonsense.blogspot.com truthprobe.blogspot.com tseytlin.blogspot.com tsrightdominion.blogspot.com tuckereskew.typepad.com twentyfirstcenturyrepublican.blogspot.com txfx.net unconventionalwisdom.typepad.com undercaffeinated.com underscorebleach.net/content/jotsheet unfetteredmusings.blogspot.com unusualhabits.blogspot.com usdlaw.blogspot.com variousorthodoxies.com victorhanson.com viewfromaheight.blogspot.com viewfromtonka.blogspot.com viewpointjournal.com vikingpundit.blogspot.com virginiapolitics.blogspot.com vodkapundit.com volokh.com warriorscreed.000k3.com watchandwait.blogspot.com watdanook.blogspot.com web.utk.edu/~agroves wehavesomeplanes.blogspot.com whatsakyer.blogspot.com wholewheatblogger.com wildweasel.blog-city.com williamwebbdotorg.blogspot.com winceandnod.blogspot.com windingroad.typepad.com windsofchange.net wizbangblog.com writehouse.us writingcompany.blogs.com/this_isnt_writing_its_typ xanga.com/lockjawtheogre xtremerightwing.net youngconservative.blogspot.com zebrax.blogs.com zeke01.blogspot.com zeke01.typepad.com zeph1z.tripod.com/blog Right Liberal w 2 0 1 2 100monkeystyping.com 12thharmonic.com/wordpress 750volts.blogspot.com 95theses.blogspot.com abbadabbaduo.blogspot.com aboutpolitics.blogspot.com achingandspinning.blogspot.com ackackack.com adamtalib.blogspot.com adviceforlefty.blogspot.com agonist.org aintnobaddude.com ajbenjaminjr.blogspot.com alicublog.blogspot.com allanjenkins.typepad.com allspinzone.blogspot.com alphapredator.com/wdtgw.htm alternateworlds.blogspot.com althippo.blogspot.com alvintostig.typepad.com americablog.blogspot.com americablog.org americanmuslim.blogs.com americanpolitics.com americansforbayh.blogspot.com amleft.blogspot.com amliberal.com/blog amptoons.com/blog anarchyxero.robfindlay.org andifeelfine.blogspot.com andymatic.com angrybear.blogspot.com angrydesi.typepad.com angryfinger.org angryhomo.blogspot.com annatopia.com/home.html anoldsoul.blogspot.com anonymoussources.blogspot.com answerguy.blogspot.com anybodybutbushyall.blogspot.com anywhichway.net arancaytar.blogspot.com archpundit.com arkansastonight.com askhoudari.blogspot.com astroturf.uni.cc atease.blogspot.com atrios.blogspot.com atrios.blogspot.com/ awards5.tripod.com/tarasblog b-patton.blogspot.com bakshi.us/redskunk barkingdingo.blogspot.com bartcop.com battleground-wisconsin.blogspot.com battlegroundstates.blogspot.com beastsbelly.blogspot.com bentcorner.blogspot.com bettyblog.com billmon.org billsrants.blogspot.com blog.dccc.org blog.dennisfox.net blog.glinka.com blog.johnkerry.com blog.kenlan.net blog.mintruth.com blog.veggiedude.com blog01.kintera.com/dnccblog blog01.kintera.com/emilysblog bloganonymous.com blogforamerica.com blogforarizona.com blogfreeordie.blogspot.com blogitics.com blogs.salon.com/0002874 blogs.salon.com/0003364 blogslut.com blosteam.blogspot.com bluegrassroots.blogspot.com bluelemur.com blueoregon.com bluestateredstate.blogspot.com bodyandsoul.typepad.com boffo.typepad.com boloboffin.blogspot.com bopnews.com boston.com/news/blogs/dnc bowles2004.com/weblog boycottsbg.com bradcarson.com/blog bradfriedman.com/bradblog brilliantatbreakfast.blogspot.com browncross.com/usualsuspects btcnews.com/btcnews bullmooseblog.com burntorangereport.com bushlies.net/pages/10/index.htm bushmisunderestimated.blogspot.com busybusybusy.com buzzmachine.com carpetblogger.com cbaker.org/blog cemonitor.blogspot.com centerpiece.blogspot.com changeforamerica.com/blog charleypatton.blogspot.com charlineandjamie.com/dotnetweb01a/blogdisplay.aspx?logname=jamie&logcatid=48 chepooka.com chiron.typepad.com churchofcriticalthinking.com clarified.blogspot.com claudialong.com/blog cleancutkid.com collegedems.com/blog conniptions.net conservativessuck.blogspot.com contrapositive.blogspot.com corrente.blogspot.com counterspin.blogspot.com crankylittleblog.blogspot.com criticalviewer.com crookedtimber.org crooksandliars.com dailyblatt.blogspot.com dailyhowler.com dailykos.com dailywarnews.blogspot.com damntheman.net dashnier.blogspot.com davidsirota.com dawnofnewamerica.blogspot.com deathpenaltyusa.blogspot.com debunker.net dedspace.blogspot.com delraysteve.com/blog demagogue.blogspot.com demleft.blogspot.com democraticunderground.com democratreport.blogspot.com democrats.org/blog democratvoice.org dems2000.net dems2004.org/blog denisdekat.com dewar.journalspace.com digbysblog.blogspot.com digestiblenews.blogspot.com digital-democrat.blogspot.com digitaljay.blogspot.com dimmykarras.blogspot.com dir.salon.com/topics/joe_conason/index.html discourse.net diypolitics.blogdrive.com dneiwert.blogspot.com dogfight04.typepad.com dohiyimir.typepad.com dolphinsdock.com/blog donkey2004.blogspot.com donspoliticalblog.blogspot.com duckdaotsu.blogspot.com dystopia.blog-city.com eastbaykerry.com elayneriggs.blogspot.com elderbearden.blogspot.com electablog.com elemming2.blogspot.com elissa.typepad.com eltcamerica.blogspot.com emergingdemocraticmajority.com emergingdemocraticmajorityweblog.com/donkeyrising emptyisform.com endthenightmare.blogspot.com enemykombatant.blogspot.com ergio.blogspot.com estropundit.blogspot.com etherealgirl.blogspot.com fafblog.blogspot.com fairshot.org farleft.blogspot.com fiercepoet.com flagrancy.net folkbum.blogspot.com forewarned.blogspot.com frederickclarkson.com fromtheroots.org fuckthisblog.blogspot.com gadflyer.com garbage-house.com georgewbuy.blogspot.com gisleson.com/blog global-equality.org/news/blog/index.shtml goose3five.blogspot.com grassrootsmom.blogspot.com grassrootsnation.com greaterdemocracy.org greendogdemocrat.blogspot.com gregpalast.com grupo-utopia.com/blog/isou hal9000report.blogdrive.com hategun.com hellafaded.blogspot.com hellskitchennyc.blogspot.com hereswhatsleft.typepad.com hereswhatsleft.typepad.com/home higherpieproductions.com home.att.net/~gentle.reader home.earthlink.net/~fsrhine home.earthlink.net/~kevin.omeara home.houston.rr.com/skeptical homepage.mac.com/elisa_camahort/iblog hotflashesfromthecampaigntrail.blogspot.com hotflashreport.com howienfriends.blogspot.com hugozoom.blogspot.com ibe.blogspot.com ibrutus.com iddybud.blogspot.com idiosyncratictendencies.com idlehandsmag.com/yellowdog04 iflipflop.com ignatz.blogspot.com ihatepatrobertson.com ilovecynics.com insanepreschoolmom.blogspot.com interestingtimes.blogspot.com interestingtimes.typepad.com ironmouth.com irregulartimes.com/blogger.html islanddave.blogspot.com itlookslikethis.blogeasy.com ivotedforgeorge.com iwantmycountryback.org j-bradford-delong.net/movable_type jadbury.com jadbury.com/blog jadedreality.blogspot.com jameswolcott.com janm.blogspot.com jbcougar.blogspot.com jdeeth.blogspot.com jeremybrendan.blogspot.com jetage.blogspot.com jewssansfrontieres.blogspot.com jimgilliam.com jimtreacher.com jinkythecat.blogspot.com jmbzine.com joebrent.blogspot.com john.hoke.org johnmccrory.com johnnynobody.blogspot.com journals.aol.com/breuvers/madashellnotgoingtotakeitanymore juancole.com julietterossant.com/superchefblog/superchefblog.html juliusblog.blogspot.com justmark.com karlhenderson.blogspot.com karmicsoup.blogspot.com kcdems.blogspot.com kensain.com kerryforpresident2008.blogspot.com kicktheleftist.blogspot.com kiosan.blogharbor.com kippsblog.com kurtkk.blogspot.com kydem.blogspot.com lastdaysoftherepublic.fatoprofugus.net lawdork.blogspot.com leanleft.com left2right.typepad.com lefterer.com lefti.blogspot.com leftinthereign.blogspot.com leftthought.blogspot.com lefttimes.com leftwingnation.com lendmesomesugar.blogs.com/shut_up_already lennonreport.blogspot.com leonards-digest.blogspot.com liberalangst.org liberalavenger.com liberaleric.blogspot.com liberaloasis.com liberalreview.blogspot.com liberalthunderdome.blogspot.com lightupthedarkness.org lightupthedarkness.org/blog/default.asp limericksavant.blogspot.com liquidlist.com livejournal.com/users/hoptkov livejournal.com/users/jmhm loadedmouth.com lorenjavier.blogspot.com loud-liberal.blogspot.com loveamericahatebush.com lowrentrat.blogdrive.com ltobs.blogspot.com ludis.blogspot.com ludovicspeaks.typepad.com ludovicspeaks.typepad.com/real_deal madkane.com/notable.html maggiespants.co.uk mahablog.com marcbrazeau.blogspot.com margaretcho.com/blog/blog.htm margieburns.com markadams.blogdrive.com markarkleiman.com markschmitt.typepad.com/decembrist markwarnerforpresident2008.blogspot.com maruthecrankpot.blogspot.com massachusetts-liberal.com mathewgross.com mathewgross.com/blog matthewyglesias.com maultasch.us maxspeak.org/mt mayflowerhill.blogspot.com mcwil.blogspot.com meanspirit.blogspot.com meateatingleftist.com mediajunkie.com/edgewise mediaprima.com/nv1962 messengerpuppet.com/blog metropol47.blogspot.com michael124.blogspot.com michaelberube.com michaelphillips.blogspot.com mickikrimmel.com/redcarpet midguard.blogspot.com minagahet.blogspot.com mirocat.com misschin.bloggage.com mixtersmix.blogspot.com montages.blogspot.com moxiegrrrl.com mrleft.org msnbc.msn.com/id/3449870 mstabile.blogspot.com mydd.com myleftbrain.com myleftthoughts.blogspot.com mysterypollster.com/main nastyboys.blogspot.com nathancallahan.com nathannewman.org/log nationalreview.com/frum/frum-diary.asp ndnblog.org nearouterheaven.blogspot.com nebursworld.blogspot.com needlenose.com nerofiddled.blogspot.com netpolitik.blogspot.com netweed.com/postmodernanarchist newappeal.blogspot.com newdemocrat.blogdrive.com newdonkey.com newleftblogs.blogspot.com nielsenhayden.com/electrolite nielsenhayden.com/makinglight njfordemocracy.org node707.com nofearoffreedom.blogspot.com nomoremister.blogspot.com norbizness.com noseyonline.com nosmallplans.com/rants notbush.com notgeniuses.com nycitystoop.com obamablog.com obsidianwings.blogs.com oddhours.blogspot.com offthekuff.com/mt oipp.blogspot.com oliverwillis.com one38.org openeyesmemo.com orient-lodge.com ospolitics.org oxblog.blogspot.com pacdaily.com pacificviews.org pandagon.net parabasis.typepad.com patriotboy.blogspot.com pattymurray.com/blog.php paulchasman.com peacegarden.blogspot.com penheaded.blogspot.com pennywit.com/drupal permanentred.com peskytherat.com/pesky phillybri.blogspot.com philoponia.blogspot.com philosophers-stone.blogspot.com philvbblog.blogspot.com phoblographer.com pinkofeministhellcat.typepad.com/pinko_feminist_hellcat planetdave.com/blogs.php3?mid=1 politicalmonitor.us/blog politicalstrategy.org politicalthought.net politicalwire.com politics.feedster.com politicswithrichard.blogspot.com polizeros.com pollsterseek.blogspot.com polstate.com portapulpit.com praguewriter.typepad.com presidentboxer.blogspot.com prestoncaldwellblog.blogspot.com profgoose.blogspot.com progressiveink.com progressivesociety.com/blog proponentofreason.blogspot.com prospect.org/weblog pudentilla.blogspot.com punditician.blogspot.com purplestates.blogspot.com qando.net question-dean.blogspot.com radgeek.com radikal_papi.pitas.com radio.weblogs.com/0131587/categories/flush raedinthemiddle.blogspot.com reachm.com/amstreet readingpost.com realitybasednation.com red-harvest.blogspot.com religiousleft.blogspot.com republicoft.typepad.com resistoppression.blogspot.com restoreamerica.blogspot.com retrogrouch.net rhetoricrhythm.blogspot.com righthandthief.blogspot.com rising-hegemon.blogspot.com risinghegemon.blogspot.com rittenhouse.blogspot.com riverbendblog.blogspot.com roadtosurfdom.com robot-invasion.com robschumacher.blogspot.com rogerailes.blogspot.com rooksrant.com rowhouselogic.com roxanne.typepad.com rudepundit.blogspot.com ruminatethis.com s88172659.onlinehome.us/spockosbrain.html sadlyno.com saveoursenate.blogspot.com sccdp.org/blog.php schadelmann.com scoobiedavis.blogspot.com scottcsmith.net scripting.com scrutinyhooligans.blogspot.com seeingtheforest.com seetheforest.blogspot.com sethdmichaels.com shakespearessister.blogspot.com shamelessantagonist.blogspot.com sharedthought.com/weblog shininglight.us shoegal.modblog.com shotofpolitics.blogspot.com shrillblog.blogspot.com simbaud.blogspot.com simplyappalling.blogspot.com smirkingchimp.com smithersmpls.com snunes.blogspot.com someonetakethewheel.blogspot.com soonerthought.com southknoxbubba.net/skblog spacedog26.blogspot.com spectatorsport.blogspot.com stagefour.typepad.com/commonprejudice stalinism.com/shot-by-both-sides stcfoundation.blogspot.com stephenbates.com/yellowdoggereldemocrat stevecopy.blogharbor.com stevegilliard.blogspot.com steveguy.me.uk stevemacek.blogspot.com stillfighting.blogspot.com stitt.blogspot.com strangedoctrines.typepad.com strangesemantics.blogspot.com suburbanguerrilla.blogspot.com swamiuptown.com swingstateproject.com takebackthemedia.com talkingpointsmemo.com talkleft.com tallone.org tbogg.blogspot.com teenageenthusiast.blogspot.com tesibria.typepad.com/democracy_is_coming thatcoloredfellasweblog.bloghorn.com thatgoodnight.com the-goddess.org the-hamster.com theaggpro.blogspot.com theagitator.blogspot.com theangryliberal.blogspot.com theantibush.org theapologist.blogspot.com theblogwarrior.blogspot.com theblueview.blogspot.com thecarpetbaggerreport.com thecore4.com thefulcrum.blogspot.com theleftcoaster.com thelonedem.com thelucideye.blogspot.com thepeach.blogspot.com thepoorman.net theradiomansays.blogspot.com therogueangel.com/blog thesnooter.blogspot.com thetalentshow.org thetalkingdog.com thewashingtonnote.com theworldaroundyou.com thewritewing.blogspot.com theyetimn.blogspot.com thingsasitis.net think2004.com thinkingrocks.blogspot.com thirdredeye.blogspot.com thislousytshirt.net thismodernworld.com thomasblog.typepad.com/bloggg thunewatch.squarespace.com tinylittledots.com tobyemden.blogspot.com tomburka.com tomcoburnisabigfatjerk.blogspot.com tomherod.blogspot.com toteota.blogspot.com trainorphans.org translucence.org travalex.blogspot.com tristero.blogspot.com trueconservative.typepad.com truedemocrat.blogspot.com turnpikefeminist.com turnspit.com uberdude.blogspot.com uggabugga.blogspot.com unodemocrats.com/blog upper-left.blogspot.com urbandemocracy.blogspot.com urbanecoinc.com/wordpress1 usademocrazy.blogspot.com usliberals.about.com vernsblog.thegillfamily.us:8180 virginiaprogressive.blogspot.com vocis.com/frank waitingfordorothy.blogspot.com wampum.wabanaki.net warandpiece.com washingtonmonthly.com watchingthewatchers.org waveflux.net weblogs.bakshi.us/redskunk whopundit.blogspot.com whoviating.blogspot.com whyareweback.blogspot.com winnegar.blog-city.com winthewest.blogspot.com woneffe.typepad.com wonkette.com worldonfire.typepad.com/world_on_fire wpblog.ohpinion.com wsrblood.typepad.com/wiserblog wtpm.blogspot.com x-pollen.com/many/blog.html xnerg.blogspot.com yarsrevenge.blogspot.com yglesias.typepad.com/matthew yoder.ru younglibs.com Left a100wwe.blogspot.com absurd-canadian.blogspot.com ace.mu.nu aceoftrumpblog.blogspot.com acepilots.com/mt acertainslantoflight.blogspot.com achicknamedmarzi.com activistchat.com/blogiran adeimantus.blogspot.com alamodispatches.blogspot.com alamonation.blogspot.com alarmingnews.com aldaron.modblog.com aldaynet.org alexpelan.blogspot.com alittlemoretotheright.com all-encompassingly.com allahpundit.com allthingsmark.com alphapatriot.com althouse.blogspot.com americanchronicle.blogspot.com americandigest.org americandinosaur.blogspot.com americanfuture.typepad.com americankernel.com americanreport.blogspot.com americanthinker.com americanwarmonger.blogspot.com americanworldview.tripod.com/weltansblog amigo13.blogspot.com amradio.blogspot.com andrewsullivan.com andrightlyso.blogspot.com angrychineseblogger.blog-city.com angrygwn.blogspot.com anncoulter.org annika.mu.nu anothervrwc.blogspot.com antijohnkerry.blogspot.com antiprotester.blogspot.com apologiesdemanded.blogspot.com arkanssouri.blogspot.com armchairgeneralist.typepad.com asmallvictory.net asmodeus-space.com/weblog/blog/index.php astrosoup.blogspot.com athena.blogs.com awideawake.blogspot.com ayerdis.com/rants.php babytrollblog.com baldilocks.typepad.com balloon-juice.com bamapachyderm.blogspot.com batr.net batr.org/commentary.html beautifulatrocities.com beefalwayswins.blogspot.com beldar.blogs.com/beldarblog bellusbellum.info belmontclub.blogspot.com bensgtusmc.com bensworld.patriotforum.org bernardmoon.blogspot.com betsyspage.blogspot.com bidinotto.journalspace.com bigfatroid.blogspot.com bigjourney.blogspot.com billhobbs.com billrice.org billroggio.com/easongate bitheads.blogspot.com bittersweetme.net/blog/index.php bkennelly.com/vox blackmanforbush.blogspot.com blindmindseye.com blithered.blogspot.com blog.nam.org blog.ncsweb.us blog.so-amazing.net blog.willrobinson47.com bloghogger.blogspot.com blogicus.com blogodob.blogspot.com blogsagainsthillary.com blogsforbush.com blogsofwar.com blogspirator.blogspot.com blubberhollow.com/blog blue6golf.blogspot.com bluegoldfish.blogs.com/surface bluestates.blogspot.com bobhayes.net boottotheskull.com borgardblog.blogspot.com brainshavings.com brianscott.blogspot.com brutallyhonest.org buckeyepundit.blogspot.com bullmooserepublicans.com/blog bushblog.us c-pol.blogspot.com californiaconservatives4truth.blogspot.com calirep.blogspot.com calpoly.edu/~doleary/blog.htm captainsquartersblog.com/mt catscape.com cayankee.blogs.com cbcbcbcb.blogspot.com ccs178.com cdharris.net cecile.squarespace.com celluloid-wisdom.com/pw celluloid-wisdom.com/pw/ centinel.blogspot.com chadster.capitalfocus.org chainik_hocker.blogspot.com charlesking.blogspot.com chiefsblog.blogspot.com chrenkoff.blogspot.com christian-patriot.blogspot.com civilcommotion.com claycalhoun.com claytoncorner.blogspot.com clubforgrowth.org/blog cmillertime.blogspot.com coffee.ondragonswing.com commonsenserunswild.typepad.com confederateyankee.blogspot.com conservativeblogger.com conservativecat.com conservativedialysis.com conservativeeyes.blogspot.com conservativelife.com/blog conservativenation.us conservativeobserver.blogspot.com conservativepolitics.blogspot.com conservativepunk.com conservativepush.blogspot.com conservativeresponse.com conservativerevolution.com conservativezone.blogspot.com coxandforkum.com cptnwinky.com crankybastard.blogspot.com crankyneocon.com/crankyneocon cubanstories.blogspot.com cybermenace.blogspot.com cynicalnation.com d-42.com daddypundit.blogspot.com dadougster.blogspot.com dailypundit.com dailyrecycler.com/blog daisycutter.blogspot.com dalythoughts.com danieldrezner.com/blog danzfamily.com dartblog.com datingrepublicans.com deadcantrant.com/blog deanesmay.com deansjournal.blogspot.com decision08.blogspot.com denbeste.nu depressedlonelybored.typepad.com deserttusk.blogspot.com devildawg.blogspot.com dgci.net diggersrealm.com/mt dilusionl.blogspot.com diplomadic.blogspot.com discerningtexan.blogspot.com discountblogger.com disintegrator.blogspot.com dissectleft.blogspot.com dizzy-girl.net/index.php dmobley.blogspot.com dogwooddreams.com/serendipity donaldsensing.com downtownlad.blogspot.com drudgereport.com drumwaster.com drunkensamurai.blogspot.com dummocrats.com dynamist.com/weblog economicswithaface.com/weblog eeg3.com ejectejecteject.com electionprojection.com electricvenom.com elevendayempire.com/movabletype emilythedog.blogspot.com erickohn.com erikack.blogspot.com etalkinghead.com evangelicaloutpost.com everythingiknowiswrong.com evilpundit.com fearfulsymmetry.blogspot.com firstdownpolitics.com franklog.com fraterslibertas.com freedomofthought.com freerepublic.com fringeblog.com fritzs-thoughts.blogspot.com gaypatriot.blogspot.com gccpundit.blogspot.com gcruse.typepad.com georgewbush.com gerberatetra.blogspot.com gevkaffeegal.typepad.com/the_alliance ghostofaflea.com gindy.blogspot.com gmscorner.blogspot.com godubya.com gopandcollege.blogspot.com gopandthecity.blogspot.com gopbloggers.org gopinsight.com gotdesign.blogspot.com greatestjeneration.com grownups.blogspot.com gullyborg.typepad.com happyhomeschooler.blog-city.com hasidicgentile.org hathaby.net/weblog.php headland.blogspot.com healingiraq.blogspot.com heavylifting.blogspot.com hectorvex.blogspot.com henerd.com hiddennook.blogspot.com hillcountryviews.blogspot.com hipublican.com hobsonschoice.us holecard.blogspot.com home.earthlink.net/~techservant home.midsouth.rr.com/joefish homespunbloggers.blogspot.com homicidalmaniak.com horologium.net hughhewitt.com iamaugustus.blogspot.com ikilledcheguevara.blogspot.com imao.us imprescindibile.ilcannocchiale.it incite1.blogspot.com indcjournal.com indepundit.com instapundit.com interested-participant.blogspot.com inthebullpen.com iowageek.blogspot.com iraqthemodel.blogspot.com isdl.blogspot.com isdl.weblogs.us itscomedowntothis.blogspot.com itznewstome.blogspot.com jacklewis.net janegalt.net jchristopherfeng.com jeffblanco.blogdrive.com jewishworldreview.com jg-spot.blogspot.com jihadwatch.org jlynnsmith30.blogspot.com joatmoaf.typepad.com/i_love_jet_noise joepublicus.blogspot.com johnkerrymustlose.com johnnbrown.blogspot.com jokerstotheright.blogspot.com joshferguson.blogspot.com journals.aol.com/lily990016/pissoffliberals journals.aol.com/mark24609/myblog jperspective.com jquinton.com jrogg.blogspot.com juntoboys.blogspot.com justalkin.us kalblog.com kausfiles.com keithy1028.bravejournal.com kellipundit.blogspot.com kenshepherd.blogspot.com kevincourser.blogspot.com kimdutoit.com/dr/weblog.php king-of-fools.com kitchennetwork.blogspot.com la4israel.org/wordpress ladymac.com lahargis.blogspot.com larsonreport.com lashawnbarber.blogspot.com lashawnbarber.com laughatliberals.com laughingwolf.net lawguy.blogspot.com valuerightnews.blogspot.com leftcoastconservative.blogspot.com legendgames.net/myblog.asp libertarianjackass.blogspot.com littlegreenfootballs.com/weblog littlegreenfootballs.com/weblog/weblog.php littletinylies.com livejournal.com/users/bambenek livinginbabylon.blogspot.com lonewacko.com loosecoin.blogspot.com lt-smash.us lucianne.com luxeword.com madanthony.org made4theinternet.blogspot.com madtechspeaks.blogspot.com magnificentfreedom.us manifestcontent.net mantic.squarespace.com mariestwocents.blogspot.com markheimonen.blogspot.com marknicodemo.blogspot.com marksteyn.com martinipundit.com martydee.com/poligov matthewstinson.com/blog mattresspolice.com maybeithinktoomuch.com mdcbowen.org/cobb meckz.net mementomoron.blogspot.com messopotamian.blogspot.com metallicity.com/mb.asp mhking.mu.nu michaeltotten.com michellemalkin.com midwestrightwingers.blogspot.com mikesamerica.blogspot.com millers_time.typepad.com mindspring.com/~gsb3/blogger.html mobile.blogcastlive.com moorewatch.com mostlycajun.com motgs.com mtvirtus.blogspot.com mudandphud.blogspot.com mudvillegazette.com musicfromthebluelight.com/b mwilliams.info my2cent.blogspot.com myaisling.blogspot.com mycatharsis.com mydamascusroad.blogspot.com myechochamber.blogspot.com mypetjawa.mu.nu nashvilletruth.blogspot.com nationalbird.net nationalcenter.org/blog.html nationalnewsregister.com/modules/wordpress nationalreview.com/thecorner nationalreview.com/thecorner/corner.asp neoconservatives.blogspot.com neoconswatch.blogspot.com neonbrainiac.nu/mt/blog nerepublican.blogspot.com neverwrongnever.blogspot.com newcounterculture.com/log newsbloggers.blogspot.com newsjungle.blogspot.com newsnik.com nicedoggie.net nickiegoomba.blogspot.com nicoladellarciprete.blogs.com noguru.tblog.com nomayo.mu.nu nonboxthinking.blogspot.com nykola.com o4tom.blogspot.com officesmileman.com ofthemind.blogspot.com ogresview.blogspot.com okieonthelam.com oldfashionedwhippersnapper.blogspot.com onthewire.blogspot.com opiniontimes.blogspot.com opiniontimes.com oppb.com orthogonian.blogspot.com outsidethebeltway.com pajamaeditors.blogspot.com pajamahadin.com pajamajournalist.blogspot.com palominas.blogspot.com papadoc.net/pinkflamingobar.html pardonmyenglish.com patrickmcclarty.blogspot.com patrickruffini.com patriot-paradox.com patriotblog.com patriotism101.blogspot.com patriotsforbush.com patterico.com pejmanesque.com pennsylvanianinexile.blogspot.com perryvsworld.blogspot.com philomathean.blogs.com piano-tuning.net/blog poeticvalues.blogspot.com poliblogger.com polipundit.com politicalanimals.blogspot.com politicaldogs.org politicalmusings.net politicalpie.modblog.com politicaltracker.com politicaltracker.com/blog/weblog.php politicalvicesquad.blogspot.com poorjustin.blogspot.com poweragerambleson.blogspot.com powerlineblog.com powerpundit.com powervacuum.org presidentreagan.info/discussion professorbainbridge.com prolifeblogs.com propagandamachine.blogspot.com prosandcons.us punditguy.com punditz.journalspace.com qoae.net quante.blogspot.com quibbles-n-bits.com quimundus.modblog.com quimundus.squarespace.com r21online.com r2korn.blogdrive.com rabner.blogspot.com radioequalizer.blogspot.com raincrossblog.com randomjottings.net rant.st ratherbiased.com/news rathergate.com realclearpolitics.com realordeal.blogspot.com rebrand-democrat.txfx.net reckersau.blogspot.com redmindbluestate.blogspot.com rednyc.blogspot.com redstate.org redstatealliance.blogspot.com reggiescorner.blogspot.com regimechangeiran.com renpro.kwcom.net reporterette.com republog.net respublica2004.blogspot.com retiredwarrant.blogdrive.com revone.net/jarhead rfburnhertz.net/rf_blog richmar.blogspot.com right-thinking.com rightmoment.blogspot.com rightonred.net rightperspective.net rightrainbow.com rightstate.blogspot.com rightvoices.com rightwingbastardo.blogspot.com rightwingbob.com rightwingdude.squarespace.com rightwingjournal.blogspot.com rightwingnews.com rightwingthinking.blogspot.com rightwingtroll.blogspot.com rightword.blogspot.com riteturnonly.blogspot.com robbernard.com robertp17.blogspot.com rockpaperscissor.blogspot.com rockpaperscissors.blogs.com rogerlsimon.com romeocat.typepad.com ron-james.com/blog rosenblog.com roseoftheknights.com/yank/pride.html roughstock.blogspot.com ryanshead.com samizdata.net/blog sarcasticsoutherner.blogspot.com sayanythingblog.com scha-den-freu-de.blogspot.com scrappleface.com scrawlville.com sebourn.com/weblog section-31.blogspot.com secureliberty.org/ sedenteanimo.com selfish-souls.com/blog sgtstryker.com shadydowns.blogspot.com shakingspears.blogspot.com shallows.blogspot.com sharpeblog.cybercatholics.com shortfatcorporal.blogspot.com sickofliberals.blogspot.com sinfactor.blogspot.com sistertoldjah.com sixmeatbuffet.com slantpoint.com slingsnarrows.erudite-absurdity.com slipofthepen.blogspot.com slowplay.com slublog.blogspot.com smarthomeschool.com smashingzero.blogspot.com smellyduck.blogspot.com sneakeasysjoint.com sondrak.com soundingthetrumpet.blogspot.com southernappeal.blogspot.com southernrightwingextremist.blogspot.com southernwatch.blogspot.com spreadingunderstanding.blogspot.com spydrzweb.blogspot.com squiggler.braveblog.com stan-istan.blogspot.com stealthebandwagon.blogspot.com stillmanvalley.org/aoo stix1972.blogspot.com stmonk.blogspot.com stolenthunder.blogspot.com strategicintelligence.blogspot.com streetlog.typepad.com summation.net sweetgrrrl.blogspot.com swfwithattitude.blogspot.com swiftreport.blogs.com tagorda.com talesofawanderingmind.blogspot.com talkshowamerica.blogspot.com techievampire.net/wppol techtrend.com/blog/taranicole templeofjennifer.com/blog tepid-observer.blogspot.com texasbug.blogspot.com thatliberalmedia.com the-torch.com theamericanmind.com thebloghill.blogspot.com thecommonvirtue.com theconservativezone.com thecrankfiles.blogspot.com thedailygoodnews.com thedispatch.blogspot.com thedonovan.com theglitteringeye.com thejessefactor.blogspot.com thelastword.blogdrive.com theliberalslies.blogspot.com themeofabsence.com theneo-progressive.blogspot.com thenudnikfile.blogspot.com thepatriette.com thepaytons.org/essays/considerettes thepoliticalanimal.com thepoliticalteen.net therightblog.blogspot.com therightcoast.blogspot.com therightwingconspiracy.org therinsecycle.com therostra.blogspot.com thesakeofargument.com thespoonsexperience.com thetarpit.blogspot.com thetemplarpundit.blogspot.com theunderscore.blogspot.com theurbangrind.blogspot.com thevaticanofliberalism.com thinkingright.us thisliberal.com thomasgalvin.blogspot.com thomaswharry.com timblair.net timblair.spleenville.com tokenyoungconservative.blogspot.com tom-hanna.org toneofvoice.blogspot.com totels.com/bush04 townhall.com/clog truebluegal.blogspot.com truthlaidbear.com truthliesandcommonsense.blogspot.com truthprobe.blogspot.com tseytlin.blogspot.com tsrightdominion.blogspot.com tuckereskew.typepad.com twentyfirstcenturyrepublican.blogspot.com txfx.net unconventionalwisdom.typepad.com undercaffeinated.com underscorebleach.net/content/jotsheet unfetteredmusings.blogspot.com unusualhabits.blogspot.com usdlaw.blogspot.com variousorthodoxies.com victorhanson.com viewfromaheight.blogspot.com viewfromtonka.blogspot.com viewpointjournal.com vikingpundit.blogspot.com virginiapolitics.blogspot.com vodkapundit.com volokh.com warriorscreed.000k3.com watchandwait.blogspot.com watdanook.blogspot.com web.utk.edu/~agroves wehavesomeplanes.blogspot.com whatsakyer.blogspot.com wholewheatblogger.com wildweasel.blog-city.com williamwebbdotorg.blogspot.com winceandnod.blogspot.com windingroad.typepad.com windsofchange.net wizbangblog.com writehouse.us writingcompany.blogs.com/this_isnt_writing_its_typ xanga.com/lockjawtheogre xtremerightwing.net youngconservative.blogspot.com zebrax.blogs.com zeke01.blogspot.com zeke01.typepad.com zeph1z.tripod.com/blog Right Conservative Fig 10 . L evel of aﬃliation of each blo g of the polblogs network to the c ommunities identiﬁed as “Liber al” (top) and “Conserva- tive” (b ottom). The names of the blo gs ar e gr oup e d ac c or ding to the left-right wing gr ound truth. L eft-wing blo gs ar e r epr esente d in blue on the left, right-wing blo gs in r ed on the right. blogsforbush.com (R) instapundit.com (R) drudgereport.com (R) tagorda.com (R) danieldrezner.com/blog (R) andrewsullivan.com (R) iwantmycountryback.org (L) democraticunderground.com (L) wonkette.com (L) washingtonmonthly.com (L) dailykos.com (L) atrios.blogspot.com (L) Liberal Conservative (a) polblogs Bethel, AK Anchorage, AK Seattle, WA Los Angeles, CA Denver, CO Minneapolis, MN Nashville, TN Louisville, KY Lansing, MI Miami, FL Raleigh/Durham, NC New York, NY Hub East West Alaska (b) USairport Fig 11 . Relative values of the weights in e ach c ommunity for a subset of the no des of the (a) polblogs and (b) USairport networks. Multiplic ative latent factor mo del Let X ∈ R N × N denote a (symmetric) random matrix of eﬀects for a set of N nodes. Under this mo del, X is explained as the sum of systematic patterns and random noise. Y denotes the adjacency matrix. F or 1 ≤ i < j ≤ N X ij = ξ 0 + ζ i + ζ j + M ij + E ij Y ij = 1 X ij > 0 where ξ 0 ∈ R is the in tercept, ζ i ∈ R is the additiv e no de eﬀect, M ∈ R N × N is the matrix of m ultiplicativ e eﬀects. The square symmetric matrix M has a laten t decomp osition of the form M = U Λ U T , where U ∈ R N × p and Λ ∈ R p × p is a diagonal matrix. E ∈ R N × N is the matrix of standard normal noise; E ij iid ∼ N (0 , 1). Denoting by u i the i th ro w of U w e hav e M ij = u T i Λ u j . W e use the priors implemented in the pac k age amen . F or the additiv e eﬀect ζ i it is assumed that ζ i i.i.d. ∼ N (0 , σ 2 ζ ), with 1 /σ 2 ζ ∼ Gamma  1 2 , 1 2  . F or the m ultiplicativ e eﬀects it is assumed that u ij ind ∼ N (0 , σ 2 j ) with 1 /σ 2 j ∼ Gamma(2 , 1). F or ξ 0 w e use an improper prior p ( ξ 0 ) ∝ 1 /ξ 0 . A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 20 Degree 10 0 10 1 10 2 10 3 Distribution 10 -8 10 -6 10 -4 10 -2 10 0 95% posterior predictive Data (a) polblogs Degree 10 0 10 1 10 2 10 3 Distribution 10 -7 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 95% posterior predictive Data (b) USairport 10 0 10 1 10 2 10 3 Degree 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 Distribution 95% posterior predictive Data (c) polblogs 10 0 10 1 10 2 10 3 Degree 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 Distribution 95% posterior predictive Data (d) USairport 10 0 10 1 10 2 10 3 Degree 10 -7 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 Distribution 95% posterior predictive Data (e) polblogs 10 0 10 1 10 2 10 3 Degree 10 -7 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 Distribution 95% posterior predictive Data (f ) USairport Fig 12 . Empiric al de gr e e distribution (r e d) and p osterior pr edictive (blue) of the (left) polblogs and (right) USairport networks under our (top r ow) CCRM mo del, (midd le r ow) MMSB and the (bottom r ow) MLFM. Mixe d memb ership sto chastic blo ckmo del Let N be the n umber of no des in the netw ork and Y the adjacency matrix. F or eac h node i = 1 , . . . , N , let π i ∼ Dirichlet( ς , . . . , ς ) A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 21 0 0.05 0.1 0.15 0.2 0.25 0 50 100 150 (a) polblogs 0 0.05 0.1 0.15 0.2 0.25 0 50 100 150 (b) USairport 0 0.1 0.2 0.3 0.4 0 50 100 150 CCRM MLFM MMSB empirical (c) polblogs 0 0.1 0.2 0.3 0.4 0 50 100 150 (d) USairport Fig 13 . Posterior pr e dictive plots for the polblogs and USairport networks under the CCRM, MLFM and MMSB mo dels. T op r ow: standar d deviation of degr e e; b ottom r ow: cluster c oeﬃcient b e a p dimensional mixed mem b ership probability vector for no de i , with ς > 0. F or each pair of no des i < j , let c ij | π i ∼ π i c j i | π j ∼ π j Y ij | c ij , c j i , B ∼ Bernoulli  (1 − ρ ) B c ij c j i  where c ij ∈ { 1 , . . . , p } is the emission indicator v ariable, ρ is a parameter that controls the prop ortion of zeros that should not b e explained by the blo c kmo del and B is the p × p matrix that con tains the Bernoulli rates of the link probabilities b etw een diﬀerent communities, i.e. B k,l is the probabilit y of a connection b et w een a member of group k and one of group l . W e assume ς ∼ Gamma(1 , 1), ρ ∼ Beta(1 / 2 , 1 / 2) and B k,` ∼ Beta(1 , 1). F or eac h mo del, we run three MCMC chains for p osterior inference, using the amen R pac k age ( Hoﬀ et al. , 2017 ) for MLFM. Under MMSB, w e use 200000 MCMC iterations, of which 100000 discarded as burn in. Under MLFM, which required more iterations to conv erge, we ran 1 M iterations, of whic h 500000 discarded as burn-in. In b oth cases we thinned the output to obtain 500 samples approximately distributed from the p osterior distribution. A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 22 P osterior predictive chec ks. In order to ev aluate the go o dness of ﬁt of eac h model, w e lo ok at some statistics of a replicated netw ork for a new set of nodes, sampled under the p osterior predictive. Let Y ∗ b e a N b y N adjacency matrix corresp onding to the edges of a set of N new no des, sampled from Pr( Y ∗ | Y ) =  R Pr( Y ∗ | ξ 0 , Λ) Pr( dξ 0 , d Λ | Y ) for MLFM R Pr( Y ∗ | ς , ρ, B ) Pr( dς , dρ, dB | Y ) for MMSB Note that the p osterior predictive setting is diﬀerent from that of Hoﬀ ( 2009 ), which replicates a netw ork for the same set of no des and therefore conditions on the parameters u i , ζ i . W e generate 500 samples from the p osterior predictive under each mo del. W e are interested in some standard summary statistics: the degree distribution, standard deviation of the degrees and cluster coeﬃcient. Posterior degree distributions under eac h mo del for polblogs and USairport are presen ted in Figure 12 and p osterior predictiv e distributions of the standard deviation of the degree and clustering coeﬃcient in Figure 13 . Ov erall, the MMSB did not p erform w ell on the t wo datasets considered. While b eing a v ery ﬂexible mo del, applied successfully to a wide range of real-world netw orks, the MMSB do esn’t explicitly capture degree heterogeneit y; the latent comm unities reco v ered (not shown) do not corresp ond to those recov ered by CCRM or MLFM: it tends to cluster no des according to their degree, explaining the shap e of the p osterior degree predictiv e in Figure 12 (c-d) . Such limitations, due to the lac k of degree correction, hav e been ac knowledged b y previous authors ( Karrer and Newman , 2011 ; Gopalan et al. , 2013 ). Note that degree-corrected MMSB ha v e b een prop osed by Gopalan et al. ( 2013 ), similar to the general class of mo dels discussed in ( Hoﬀ , 2009 ). MLFM on the other hand incorp orates degree heterogeneity and thus giv es b etter ﬁt on the degree predic- tiv e distribution. It also giv es similar results than CCRM and reco v ers similar latent communities. How ever, b y construction this mo del cannot capture sparsit y or hea vy tailed degree distributions. It underestimates the prop ortion of no des with degree one as shown in Figure 12 (top ro w). While our CCRM mo del ov er-estimates the prop ortion of no des of degree one, it tends to giv e a b etter ﬁt to the empirical degree distribution ov erall. Finally , in Figure 13 , we rep ort the go o dness of ﬁt statistics used in the analyses of Hoﬀ ( 2009 ). The ﬁrst one is the standard deviation of the degree shown in the middle ro w, on which MMSB performs po orly , whereas CCRM and MLFM ha v e similar results, marginally close to the empirical v alue. The bottom ro w rep orts the cluster co eﬃcient, also known as triadic dependence. In this case, CCRM giv es a better ﬁt on the sparse USairport dataset while MFLM gives a b etter result than CCRM on the dense polblogs dataset. Ac knowledgmen ts The authors thank George Deligiannidis for p ointing out the article of Asmussen and Rosi ´ nski ( 2001 ). FC ac kno wledges the supp ort of the European Commission under the Marie Curie Intra-European F ello wship Programme. P art of this w ork has been supp orted by the BNPSI ANR pro ject no ANR-13-BS-03-0006-01. App endix A: Bac kground on completely random measures A.1. Completely r andom me asur es Completely random measures (CRM) w ere introduced by Kingman ( 1967 , 1993 ) and are now standard to ols for constructing ﬂexible Bay esian nonparametric (BNP) mo dels; see for example the surveys of Lijoi and Pr ¨ unster ( 2010 ) or Daley and V ere-Jones ( 2008b , Section 10.1). A CRM W on R + is a random measure suc h that, for any collection of disjoin t measurable subsets A 1 , . . . , A n of R + , W ( A 1 ) , . . . , W ( A n ) are indep endent. A CRM can b e decomp osed into a sum of three indep enden t parts: a non-random measure, a countable collection of atoms with random masses at ﬁxed lo cations, and a countable collection of atoms with random masses and random lo cations. Here, we will only consider CRMs with random masses and random lo cations, whic h tak e the form W = ∞ X i =1 w i δ θ i (43) where the w i ∈ R + are the masses and θ i ∈ R + are the lo cations. The law of W can actually b e characterized b y a Poisson point pro cess N = { ( w i , θ i ) i =1 , 2 ,... } on R 2 + with mean measure ν ( dw , dθ ). W e fo cus here on A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 23 the case where the CRM is homogeneous with independent incremen ts. This implies that the location θ i are indep enden t of the w eights w i and the mean measure decomp oses as ν ( dw , dθ ) = ρ ( dw ) λ ( dθ ) where λ is the Leb esgue measure and ρ is a measure on R + suc h that Z ∞ 0 (1 − e − w ) ρ ( dw ) < ∞ . (44) W e write W ∼ CRM( ρ, λ ). Note that W ([0 , T ]) < ∞ a.s. for any real T while W ( R + ) = ∞ a.s. if ρ is not degenerate at 0. If Z ∞ 0 ρ ( dw ) = ∞ (45) then there will b e a.s. an inﬁnite n umber of jumps in any interv al [0 , T ] and we refer to the CRM as inﬁnite- activit y . Otherwise, it is called ﬁnite activit y . Let ρ b e the tail L´ evy intensit y deﬁned as ρ ( x ) = Z ∞ x ρ ( dw ) (46) for x > 0. This function corresponds to the expected n umber of p oints ( w i , θ i ) suc h that w i > x and θ i ∈ [0 , 1], and its asymptotic properties pla y an imp ortant role in the c haracterization of the graph properties. A.2. V e ctors of CRMs Multiv ariate extensions of CRMs ha ve b een prop osed recently by v arious authors ( Epifani and Lijoi , 2010 ; Leisen and Lijoi , 2011 ; Leisen et al. , 2013 ; Griﬃn et al. , 2013 ; Lijoi et al. , 2014 ). These mo dels are closely re- lated to L´ evy copulas ( T ank ov , 2003 ; Con t and T ank ov , 2003 ; Kallsen and T ank ov , 2006 ) and multiv ariate sub- ordinators on cones ( Barndorﬀ-Nielsen et al. , 2001 ; Skorohod , 1991 ). A vector of CRMs ( W 1 , . . . , W p ) on R + is a collection of random measures W k , k = 1 , . . . , p , such that, for any collection of disjoin t measurable sub- sets A 1 , . . . , A n of R + , the vectors ( W 1 ( A 1 ) , . . . , W p ( A 1 )), ( W 1 ( A 2 ) , . . . , W p ( A 2 )),. . . ,( W 1 ( A n ) , . . . , W p ( A n )) are mutually indep enden t. W e only consider here vectors of CRMs with b oth random weigh ts and lo cations. In this case, the measures W k , k = 1 , . . . , p , are a.s. discrete and take the form W k = ∞ X i =1 w ik δ θ i . (47) The law of the vector of CRMs can b e characterized b y a Poisson p oin t process on R p +1 + with mean measure ν ( dw 1 , . . . , dw p , dθ ). W e fo cus again on homogeneous vectors of CRMs with indep endent increments where the mean measure can b e written as ν ( dw 1 , . . . , dw p , dθ ) = ρ ( dw 1 , . . . , dw p ) λ ( dθ ) . (48) where ρ is a measure on R p + , concen trated on R p + \{ 0 } , whic h satisﬁes Z R p + min 1 , p X k =1 w k ! ρ ( dw 1 , . . . , dw p ) < ∞ . (49) W e use the same notation as for (scalar) CRMs and write simply ( W 1 , . . . , W p ) ∼ CRM( ρ, λ ). A key quan tity is the multiv ariate Laplace exponent deﬁned b y ψ ( t 1 , . . . , t p ) := − log E h e − P p k =1 t k W k ([0 , 1]) i (50) = Z R p +  1 − e − P p k =1 t k w k  ρ ( dw 1 , . . . , dw p ) (51) Note that this quan tity in volv es a p -dimensional in tegral which ma y not be analytically computable, and may b e expensive to ev aluate numerically . As for CRMs, if Z R p + ρ ( dw 1 , . . . , dw p ) = ∞ (52) A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 24 then there will b e an inﬁnite num b er of θ i ∈ [0 , T ] for whic h P k w ik > 0 and the v ector of CRMs is called inﬁnite-activity . Otherwise, it is called ﬁnite-activity . Note that some (but not all) CRMs may still b e marginally ﬁnite-activit y . App endix B: Pro of of Prop ositions 4 and 5 The pro of of ( Caron and F o x , 2017 , App endix C) can b e directly adapted to the m ultiv ariate generalization presen ted in this pap er. W e only provide a sk etch of the proof. First, as Z is a jointly exc hangeable p oint pro cess v erifying ( 21 ) and under the moment condition ( 24 ), it follows from the law of large num b ers that N ( e ) α = Θ( α 2 ) a.s. as α → ∞ . Finite-activit y case. If the vector of CRMs is ﬁnite-activity , the jump lo cations arise from an homogeneous P oisson process with ﬁnite rate, and N α = Θ( α ) a.s. It follo ws that N ( e ) α = Θ( N 2 α ) a.s. as α → ∞ . Inﬁnite-activit y case. Consider no w the inﬁnite-activity case. F ollo wing Caron and F ox ( 2017 ), one can lo w er b ound the no de counting pro cess N α b y a coun ting pro cess e N α whic h is conditionally P oisson, and the same pro of applies. F or inﬁnite-activity CCRM, we use the fact that ψ ( W 1 ([0 , α ]) , . . . , W p ([0 , α ])) → ∞ a.s., it follo ws that N α = Ω( α ) a.s., and therefore N ( e ) α = o ( N 2 α ) a.s. as α → ∞ . Finally , for comp ound CRMs with regularly v arying ρ 0 with exp onent σ and slowly v arying function such that lim t →∞ ` ( t ) > 0, Proposition 8 in App endix G implies that N α = ω ( α 1+ σ ) a.s. and N ( e ) α = O ( N 2 / (1+ σ ) α ) a.s. as α → ∞ . App endix C: Pro of of Theorem 3 Let D ∗ α = P p k =1 D kα ([0 , α ]) b e the num b er of edges in the directed graph of size α , W ∗ k,α = W k ([0 , α ]) and W ∗ α = ( W ∗ 1 ,α , . . . , W ∗ p,α ) T . Using Campb ell’s theorem, E [ D ∗ α ] = E [ E [ D ∗ α | W ∗ α ]] = E  ( W ∗ α ) T W ∗ α  = E [ W ∗ α ] T E [ W ∗ α ] + tr(cov ( W ∗ α )) = α 2 µ T µ + α tr(Σ) (53) where w e deﬁne µ = Z R p + w ρ ( dw 1 , . . . , dw p ) , Σ = Z R p + w w T ρ ( dw 1 , . . . , dw p ) . Let w i = ( w i 1 , . . . , w ip ). W e hav e, using the extended Slivn yak-Mec ke theorem ( Møller and W aagep etersen , 2003 , Theorem 3.3) E [ N ( e ) α ] = E h E [ N ( e ) α | W 1 , . . . , W p ] i = E   X i 1 θ i ≤ α    1 − e − w T i w i  + 1 2 X j 6 = i 1 θ j ≤ α  1 − e − 2 w T i w j      = α Z R p +  1 − e − w T w  ρ ( dw 1 , . . . , dw p ) + α 2 2 Z R p + ψ (2 w 1 , . . . , 2 w p ) ρ ( dw 1 , . . . , dw p ) . (54) A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 25 Using the extended Slivn yak-Mec ke theorem then Campb ell’s theorem, E [ N α ] = E [ E [ N α | W 1 , . . . , W p ]] = E " X i  1 − e − 2 w T i ( P j 6 = i w j 1 θ j ≤ α ) − w T i w i  1 θ i ≤ α # = α Z R p + E  1 − e − 2 w T ( P j w j 1 θ j ≤ α ) − w T w  ρ ( dw 1 , . . . , dw p ) = α Z R p +  1 − e − w T w − αψ (2 w 1 ,..., 2 w p )  ρ ( dw 1 , . . . , dw p ) (55) By monotone conv ergence, w e ha ve, as α tends to inﬁnit y , E [ N α ] ∼ α Z R p + ρ ( dw 1 , . . . , w p ) if the CRM is ﬁnite-activity and E [ N α ] = ω ( α ) otherwise. App endix D: Sim ulation from a tilted truncated generalized gamma pro cess W e w ant to sample points from a Poisson pro cess with truncated mean measure ρ ε ( dw ) = h ( w ) w − 1 − σ e − τ w 1 w>ε dw (56) where h is a monotone decreasing and b ounded function, and ( τ , σ ) v erify either τ ≥ 0 and σ ∈ (0 , 1), or τ > 0 and σ ∈ ( − 1 , 0]. W e will resort to adaptiv e thinning ( Lewis and Shedler , 1979 ; Ogata , 1981 ; F av aro and T eh , 2013 ). F or τ > 0, consider the family of adaptive b ounds g t ( s ) = h ( t ) t − 1 − σ exp( − τ s ) with g t ( s ) > ρ ( s ) for s > t . W e hav e, G t ( s ) = Z s t g t ( s 0 ) ds 0 = h ( t ) τ t − 1 − σ (exp( − τ t ) − exp( − τ s )) and G − 1 t ( r ) = − 1 τ log  exp( − τ t ) − r τ t − 1 − σ h ( t )  . F or τ = 0, w e consider b ounds g t ( s ) = h ( t ) s − 1 − σ and w e obtain G t ( s ) = h ( t ) σ ( t − σ − s − σ ) G − 1 t ( r ) =  t − σ − r σ h ( t )  − 1 /σ . The adaptiv e thinning sampling sc heme is as follows: 1. Set N = ∅ , t = ε 2. iterate un til termination A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 26 (a) Draw r ∼ Exp(1) (b) If r > G t ( ∞ ), terminate; else set t 0 = G − 1 t ( r ); (c) with probabilit y ρ ε ( t 0 ) /g t ( t 0 ) accept sample t 0 and set N = N ∪ { t 0 } (d) set t = t 0 and con tinue 3. Return N a dra w from the P oisson random measure with intensit y ρ ε on [ ε, + ∞ ) The eﬃciency of this approach dep ends on the acceptance probabilit y , which is giv en, for τ > 0, by ρ ε ( s ) g t ( s ) = h ( s ) s − 1 − σ h ( t ) t − 1 − σ < 1 for s > t . App endix E: Bipartite net w orks It is possible to use a construction similar to that of Section 2 to mo del bipartite graphs, and extend the mo del of Caron ( 2012 ). A bipartite graph is a graph with t wo types of no des, where only connections b etw een no des of diﬀerent t yp es are allow ed. No des of the ﬁrst type are embedded at lo cations θ i ∈ R + , and no des of the second t yp e at lo cation θ 0 j ∈ R + . The bipartite graph will b e represen ted b y a (non-symmetric) p oint pro cess Z = X i,j z ij δ ( θ i ,θ 0 j ) (57) where z ij = 1 if there is an edge b et ween no de i of type 1 and no de j of type 2. Statistical Mo del. W e consider the mo del W 1 , . . . , W p ∼ CRM( ρ, λ ) W 0 1 , . . . , W 0 p ∼ CRM( ρ 0 , λ ) and for k = 1 , . . . , p , D k | W k , W 0 k ∼ Poisson ( W k × W 0 k ) D k = X i,j n ij k δ ( θ i ,θ 0 j ) and z ij = min(1 , P p k =1 n ij k ). P osterior inference. W e derive here the inference algorithm when ( W 1 , . . . , W p ) and ( W 0 1 , . . . , W 0 p ) are comp ound CRMs with F and ρ 0 taking the form ( 18 ) and ( 19 ). Assume that w e observe a set of connections z = ( z ij ) i =1 ,...,N α ; j =1 ,...N 0 α . W e in tro duce laten t v ariables n ij k , for 1 ≤ i ≤ N α , 1 ≤ j ≤ N 0 α , k = 1 , . . . , p , ( n ij 1 , . . . , n ij p ) | w , w 0 , z ∼  δ (0 ,..., 0) if z ij = 0 tP oisson( w i 1 w 0 j 1 , . . . , w ip w 0 j p ) if z ij = 1 . W e w ant to appro ximate p (( w 10 , . . . w N α 0 ) , ( β 1 k , . . . , β N α k , w ∗ k ) k =1 ,...,p , ( w 0 10 , . . . , w 0 N 0 α 0 ) , ( β 0 1 k , . . . , β 0 N 0 α k , w 0 ∗ k ) k =1 ,...,p , φ, α , φ 0 , α 0 | z ) Denote m ik = P N 0 α j =1 n ij k and m i = P p k =1 m ik . The MCMC algorithm iterates as follows: 1. Up date ( α, φ ) | rest using a Metrop olis-Hastings step. A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 27 2. Up date w i 0 | rest ∼ Gamma   m i − σ, τ + p X k =1 β ik   γ k +   N 0 α X j =1 w 0 j k   + w 0 ∗ k     . 3. Up date β ik | rest ∼ Gamma   a k + m ik , b k + w i 0   γ k +   N 0 α X j =1 w 0 j k   + w 0 ∗ k     . 4. Up date ( w ∗ 1 , . . . , w ∗ p ) | rest. 5. Up date the latent v ariables n ij k | rest. 6. Rep eat steps 1-4 to update ( α 0 , φ 0 ), ( w 0 10 , . . . , w 0 N 0 α 0 ), ( β 0 1 k , . . . , β 0 N 0 α k ) k =1 ,...,p and ( w 0 ∗ 1 , . . . , w 0 ∗ p ). App endix F: Gaussian appro ximation of the sum of small jumps Theorem 7 Consider the multivariate r andom variable X ε ∈ R p + with moment gener ating function E [ e − t T X ε ] = exp " − α Z R p +  1 − e − P p k =1 t k w k  ρ ε ( dw 1 , . . . , dw p ) # wher e α > 0 and ρ ε ( dw 1 , . . . , dw p ) = e − P p k =1 γ k w k Z ε 0 w − p 0 F  dw 1 w 0 , . . . , dw p w 0  ρ 0 ( dw 0 ) with ε > 0, ρ 0 is a L´ evy measure on R + and F is a probabilit y distribution on R p + with densit y f v erifying Z ∞ 0 f ( z u 1 , . . . , z u p ) dz > 0 U -almost ev erywhere Z R p + k β 1: p k 2 f ( β 1 , . . . , β p ) dβ 1: p < ∞ where U is the uniform distribution on the unit sphere S p − 1 . Then if ρ 0 is a regularly v arying L´ evy measure with exponent σ ∈ (0 , 1), i.e. Z ∞ x ρ 0 ( dw 0 ) x ↓ 0 ∼ x − σ ` (1 /x ) where ` : (0 , ∞ ) → (0 , ∞ ) is a slowly v arying function then Σ − 1 / 2 ε ( X ε − µ ε ) d → N (0 , I p ) as ε → 0, where µ ε = α Z R p + w ρ ε ( dw 1 , . . . , dw p ) Σ ε = α Z R p + w w T ρ ε ( dw 1 , . . . , dw p ) with µ ε ∼ α E [ β ] σ 1 − σ ε 1 − σ ` (1 /ε ) as ε → 0 Σ ε ∼ α E [ β β T ] σ 2 − σ ε 2 − σ ` (1 /ε ) as ε → 0 where β is distributed from F . A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 28 Pro of. W e write the mo del in spherical form. Let r = p P w 2 k and u k = w k r for k = 1 , . . . , p − 1. The determinan t of the Jacobian is r p − 1 √ 1 − P p − 1 k =1 u 2 k and so e ρ ε ( r , u 1 , . . . , u p − 1 ) = r p − 1 u p e − r P p k =1 γ k u k Z ε 0 w − p 0 f  r u 1 w 0 , . . . , r u p w 0  ρ 0 ( dw 0 ) dr du 1: p − 1 := µ ε ( dr | u 1: p − 1 ) U ( du 1: p − 1 ) where u p = q 1 − P p − 1 k =1 u 2 k , µ ε ( dr | u ) = r p − 1 e − r P p k =1 γ k u k R ε 0 w − p 0 f  ru 1 w 0 , . . . , ru p w 0  ρ 0 ( dw 0 ) dr and U ( du ) = 1 u p du 1: p is the uniform distribution on the unit sphere S p − 1 . In order to apply Theorem 2.4 in Cohen and Rosinski ( 2007 ) (see also Asm ussen and Rosi ´ nski , 2001 ), we need to show that there exists a function b ε : (0 , 1] → (0 , + ∞ ) such that lim ε → 0 σ ε ( u ) b ε > 0, U -almost everywhere (58) where σ 2 ε ( u ) = Z ∞ 0 r 2 µ ε ( dr | u ) and for every κ > ε lim ε → 0 1 b 2 ε Z k w 1: p k >κb ε k w 1: p k 2 ρ ε ( dw 1 , . . . , dw p ) = 0 (59) Assume that R ∞ 0 f ( z u 1 , . . . , z u p ) dz > 0 U -almost everywhere. With the change of v ariable z = r w 0 , and the dominated conv ergence theorem we obtain σ 2 ε ( u ) = Z ∞ 0 z p +1 f ( z u 1 , . . . , z u p )  Z ε 0 e − z w 0 P p k =1 γ k u k w 2 0 ρ 0 ( dw 0 )  dz ∼  Z ∞ 0 z p +1 f ( z u 1 , . . . , z u p ) dz   Z ε 0 w 2 0 ρ 0 ( dw 0 )  as ε → 0 ∼  Z ∞ 0 z p +1 f ( z u 1 , . . . , z u p ) dz  σ 2 − σ ε 2 − σ ` (1 /ε ) as ε → 0 Let b ε = ε 1 − σ / 2 p ` (1 /ε ), w e ha ve lim ε → 0 σ 2 ε ( u ) b 2 ε =  Z ∞ 0 z p +1 f ( z u 1 , . . . , z u p ) dz  σ 2 − σ > 0, U -almost everywhere (60) No w consider, for an y κ > 0, I ε = Z k w 1: p k >κb ε k w 1: p k 2 ν ε ( dw 1 , . . . , dw p ) = Z ε 0 Z k β 1: p k > κb ε w 0 w 2 0 k β 1: p k 2 e − w 0 P p k =1 γ k β k f ( β 1 , . . . , β k ) ρ 0 ( dw 0 ) dβ 1: p F or w 0 ∈ (0 , ε ), w e ha ve κb ε w 0 ≥ κb ε ε = ε − σ / 2 ` (1 /ε ) > κ 2 ε − σ / 4 for ε small enough as t δ ` ( t ) → 0 for any δ > 0 as t → ∞ . So for ε small enough I ε > Z ε 0 Z k β 1: p k >κ 2 ε − σ/ 4 w 2 0 k β 1: p k 2 e − w 0 P p k =1 γ k β k f ( β 1 , . . . , β k ) ρ 0 ( dw 0 ) dβ 1: p > " Z k β 1: p k >κ 2 ε − σ/ 4 k β 1: p k 2 f ( β 1 , . . . , β k ) dβ 1: p #  Z ε 0 w 2 0 ρ 0 ( dw 0 )  A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 29 As  R ε 0 w 2 0 ρ 0 ( dw 0 )  ∼ σ 2 − σ b 2 ε when ε → 0, w e conclude that lim ε → 0 I ε = lim ε → 0 σ 2 − σ Z k β 1: p k >κ 2 ε − σ/ 4 k β 1: p k 2 f ( β 1 , . . . , β k ) dβ 1: p = 0 (61) Equations ( 60 ) and ( 61 ) with Theorem 2.4 of Cohen and Rosinski ( 2007 ) yield Σ − 1 / 2 ε ( X ε − µ ε ) d → N (0 , I p ) as ε → 0, where µ ε = α Z R p + w 1: p ρ ε ( dw 1 , . . . , dw p ) = α Z R p + Z ε 0 w 0 β 1: p e − w 0 P p k =1 γ k β k ρ 0 ( dw 0 ) f ( β 1 , . . . , β p ) dβ 1: p ∼ α E [ β 1: p ] σ 1 − σ ε 1 − σ ` (1 /ε ) as ε → 0 and Σ ε = α Z R p + w 1: p w T 1: p ρ ε ( dw 1 , . . . , dw p ) ∼ α E [ β 1: p β T 1: p ] σ 2 − σ ε 2 − σ ` (1 /ε ) as ε → 0 using the dominated con vergence theorem and lemmas 9 and 10 . App endix G: T ec hnical lemmas Prop osition 8 L et ν b e a L´ evy me asur e deﬁne d by Eq. ( 5 ) and ( 13 ) and ψ b e its multivariate L aplac e exp onent. Assume that ρ 0 is a r e gularly varying function with exp onent σ ∈ (0 , 1) : ρ 0 x ↓ 0 ∼ x − σ ` (1 /x ) (62) Then ψ is (multivariate) r e gularly varying ( R esnick , 2013 ), with exp onent σ. Mor e pr e cisely, for any ( x 1 , . . . x p ) ∈ (0 , ∞ ) p , we have ψ ( tx 1 , . . . , tx p ) = Z R p +  1 − e − t P p k =1 x k w k  ν ( dw 1 , . . . , dw p ) t ↑∞ ∼ t σ Γ(1 − σ ) ` ( t ) E " p X k =1 x k β k ! σ # . Pro of. ψ ( tx 1 , . . . , tx p ) = Z R p +  1 − e − t P p k =1 x k w k  ν ( dw 1 , . . . , dw p ) = Z R p +  1 − e − t P p k =1 x k w k  ν ( dw 1 , . . . , dw p ) = Z R p + f ( β 1 , . . . , β p )  Z ∞ 0  1 − e − w 0 t P p k =1 x k β k  e − w 0 P p k =1 γ k β k ρ 0 ( dw 0 )  dβ 1: p whic h giv es, using Lemmas 9 , 10 , and the dominated con vergence theorem ψ ( tx 1 , . . . , tx p ) t ↑∞ ∼ t σ Γ(1 − σ ) ` ( t ) Z (0 , ∞ ) p p X k =1 x k β k ! σ f ( β 1 , . . . , β p ) dβ 1: p . A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 30 Lemma 9 If Z ∞ x ρ ( dw ) x ↓ 0 ∼ x − σ ` (1 /x ) Then Z ∞ x e − cw ρ ( dw ) x ↓ 0 ∼ x − σ ` (1 /x ) Pro of. Z ∞ x e − cw ρ ( dw ) = Z ∞ x ρ ( dw ) − Z ∞ x (1 − e − cw ) ρ ( dw ) x ↓ 0 ∼ x − σ ` (1 /x ) as R ∞ 0 (1 − e − cw ) ρ ( dw ) < ∞ for any c > 0. Lemma 10 ( Gne din et al. , 2007 ; Bingham et al. , 1989 ). L et ρ b e a L´ evy me asur e with r e gularly varying tail L ´ evy intensity Z ∞ x ρ ( dw ) x ↓ 0 ∼ x − σ ` (1 /x ) (63) wher e σ ∈ (0 , 1) and ` is a slow ly varying function (at inﬁnity). Then ( 63 ) is e quivalent to Z x 0 w k ρ ( dw ) x ↓ 0 ∼ σ k − σ x k − σ ` (1 /x ) (64) Z ∞ 0 (1 − e − tw ) ρ ( dw ) t ↑∞ ∼ Γ(1 − σ ) t σ ` ( t ) (65) for any k ≥ 1 . References L. A. Adamic and N. Glance. The political blogosphere and the 2004 US election: divided they blog. In Pr o c e e dings of the 3r d international workshop on Link disc overy , pages 36–43. A CM, 2005. E. M Airoldi, D. Blei, S. E Fien b erg, and E. Xing. Mixed membership sto chastic blo c kmo dels. The Journal of Machine L e arning R ese ar ch , 9:1981–2014, 2008. S. Asmussen and J. Rosi´ nski. Appro ximations of small jumps of L´ evy pro cesses with a view to wards sim ula- tion. Journal of Applie d Pr ob ability , pages 482–493, 2001. B. Ball, B. Karrer, and M. E. J. Newman. Eﬃcien t and principled metho d for detecting comm unities in net w orks. Physic al R eview E , 84(3):036103, 2011. O. E. Barndorﬀ-Nielsen, J. P edersen, and K.-I. Sato. Multiv ariate sub ordination, self-decomp osability and stabilit y . A dvanc es in Applie d Pr ob ability , 33:160–187, 2001. M. Bastian, S. Heymann, and M. Jacomy . Gephi: An op en source softw are for exploring and manipulating net w orks. In International AAAI Confer enc e on Weblo gs and So cial Me dia , 2009. N. H. Bingham, C. M. Goldie, and J. L. T eugels. R e gular variation , volume 27. Cam bridge universit y press, 1989. B. Bollob´ as. R andom gr aphs , v olume 73. Cam bridge Universit y Press, 2001. C. Borgs, J. T. Chay es, H. Cohn, and N. Holden. Sparse exchangeable graphs and their limits via graphon pro cesses. ArXiv pr eprint arXiv:1601.07134 , 2016. A. Brix. Generalized gamma measures and shot-noise Cox pro cesses. A dvanc es in Applie d Pr ob ability , 31(4): 929–953, 1999. F. Caron. Ba yesian nonparametric mo dels for bipartite graphs. In F. Pereira, C.J.C. Burges, L. Bottou, and K.Q. W einberger, editors, A dvanc es in Neur al Information Pr o c essing Systems 25 , pages 2051–2059. Curran Associates, Inc., 2012. F. Caron and E. B. F ox. Sparse graphs using exc hangeable random measures. Journal of the R oyal Statistic al So ciety: Series B (Statistic al Metho dolo gy) , 79, 2017. A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 31 F. Caron, Y.W. T eh, and T.B. Murph y . Bay esian nonparametric Plac kett-Luce mo dels for the analysis of preferences for college degree programmes. The Annals of Applie d Statistics , 8(2):1145–1181, 2014. A. T. Cemgil. Bay esian inference for nonnegative matrix factorisation mo dels. Computational Intel ligenc e and Neur oscienc e , 2009, 2009. S. Cohen and J. Rosinski. Gaussian appro ximation of multiv ariate L´ evy pro cesses with applications to sim ulation of temp ered stable pro cesses. Bernoul li , 13(1):195–210, 2007. R. Con t and P . T anko v. Financial mo del ling with jump pr o c esses , v olume 2. CRC press, 2003. D.J. Daley and D. V ere-Jones. An Intr o duction to the The ory of Point Pr o c esses. V olume I: Elementary The ory and Metho ds . Springer V erlag, 2nd edition edition, 2008a. D.J. Daley and D. V ere-Jones. An Intr o duction to the The ory of Point Pr o c esses. V olume II: Gener al The ory and Structur e . Springer V erlag, 2nd edition edition, 2008b. S. Duane, A. D. Kennedy , B. J. P endleton, and D. Ro weth. Hybrid Mon te Carlo. Physics L etters B , 195(2): 216–222, 1987. I. Epifani and A. Lijoi. Nonparametric priors for vectors of surviv al functions. Statistic a Sinic a , pages 1455–1484, 2010. P . Erd¨ os and A. R ´ enyi. On random graphs. Public ationes Mathematic ae , 6:290–297, 1959. S. F a v aro and Y.W. T eh. MCMC for normalized random measure mixture mo dels. Statistic al Scienc e , 28(3): 335–359, 2013. S. E Fienberg. A brief history of statistical mo dels for net work analysis and open c hallenges. Journal of Computational and Gr aphic al Statistics , 21(4):825–839, 2012. A. Gnedin, B. Hansen, and J. Pitman. Notes on the o ccupancy problem with inﬁnitely many b oxes: general asymptotics and p ow er la ws. Pr ob ab. Surv , 4(146-171):88, 2007. A. Goldenberg, A.X. Zheng, S.E. Fienberg, and E.M. Airoldi. A survey of statistical net work mo dels. F oun- dations and T r ends in Machine L e arning , 2(2):129–233, 2010. P . Gopalan, J. M. Hofman, and D. M. Blei. Scalable recommendation with hierarchical Poisson factorization. In Confer enc e on Unc ertainty in Artiﬁcial Intel ligenc e , 2015. P . K. Gopalan, C. W ang, and D. Blei. Modeling o verlapping communities with no de popularities. In A dvanc es in neur al information pr o c essing systems , pages 2850–2858, 2013. J. E. Griﬃn and F. Leisen. Comp ound random measures and their use in Bay esian non-parametrics. Journal of the R oyal Statistic al So ciety: Series B (Statistic al Metho dolo gy) , 79(2):525–545, 2017. J. E. Griﬃn, M. Kolossiatis, and M. F. J. Steel. Comparing distributions b y using dep enden t normalized random-measure mixtures. Journal of the R oyal Statistic al So ciety: Series B , 75(3):499–529, 2013. T. Herlau, M. N. Schmidt, and M. Mørup. Completely random measures for mo delling blo c k-structured sparse net works. T echnical rep ort, arXiv preprint arXiv:1507.02925, 2015. P . Hoﬀ, B. F osdic k, A. V olfo vsky , and Y. He. amen: A dditive and Multiplic ative Eﬀe cts Mo dels for Networks and R elational Data , 2017. URL https://CRAN.R- project.org/package=amen . R pac k age v ersion 1.3. P . D. Hoﬀ. Multiplicativ e latent factor mo dels for description and prediction of so cial netw orks. Computational and Mathematic al Or ganization The ory , 15(4):261–272, 2009. P . W. Holland, K. B. Laskey , and S. Leinhardt. Sto c hastic blo ckmodels: First steps. So cial networks , 5(2): 109–137, 1983. P . Hougaard. Surviv al mo dels for heterogeneous p opulations derived from stable distributions. Biometrika , 73(2):387–396, 1986. A. Z. Jacobs and A. Clauset. A uniﬁed view of generative mo dels for netw orks: mo dels, methods, opp ortunities and c hallenges. T ec hnical rep ort, arXiv:1411.4070, 2014. L. James. Ba yesian Poisson process partition calculus with an application to Bay esian L´ evy mo ving a v erages. The Annals of Statistics , pages 1771–1799, 2005. L. James. Poisson latent feature calculus for generalized Indian buﬀet pro cesses. T echnical rep ort, arXiv:1411.2936, 2014. L. F. James. P oisson pro cess partition calculus with applications to exc hangeable mo dels and bay esian nonparametrics. arXiv pr eprint math/0205093 , 2002. L.F. James, A. Lijoi, and I. Pr ¨ unster. P osterior analysis for normalized random measures with indep endent incremen ts. Sc andinavian Journal of Statistics , 36(1):76–97, 2009. S. Janson. Probabilit y asymptotics: notes on notation. T echnical rep ort, arXiv:1108.3924, 2011. O. Kallenberg. Exc hangeable random measures in the plane. Journal of The or etic al Pr ob ability , 3(1):81–136, A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 32 1990. O. Kallen b erg. Pr ob abilistic symmetries and invarianc e principles . Springer, 2005. J. Kallsen and P . T anko v. Characterization of dep endence of multidimensional L ´ evy pro cesse s using L´ evy copulas. Journal of Multivariate Analy sis , 97(7):1551–1572, 2006. B. Karrer and M. E. J. Newman. Sto chastic blo ckmodels and communit y structure in net works. Physic al R eview E , 83(1):016107, 2011. C. Kemp, J. B. T enenbaum, T. L. Griﬃths, T. Y amada, and N. Ueda. Learning systems of concepts with an inﬁnite relational mo del. In AAAI , v olume 21, page 381, 2006. J.F.C. Kingman. Completely random measures. Paciﬁc Journal of Mathematics , 21(1):59–78, 1967. J.F.C. Kingman. Poisson pr o c esses . Oxford Universit y Press, USA, 1993. E. D. Kolaczyk. Statistic al A nalysis of Network Data: Metho ds and Mo dels . Springer, 2009. P . Latouche, E. Birmel ´ e, and C. Ambroise. Overlapping stochastic block mo dels with application to the Frenc h political blogosphere. The Annals of Applie d Statistics , pages 309–336, 2011. H. S. Lee, D. D.and Seung. Learning the parts of ob jects by non-negative matrix factorization. Natur e , 401 (6755):788–791, 1999. F. Leisen and A. Lijoi. V ectors of t wo-parameter Poisson–Diric hlet pro cesses. Journal of Multivariate Anal- ysis , 102(3):482–495, 2011. F. Leisen, A. Lijoi, and D. Span´ o. A v ector of Diric hlet pro cesses. Ele ctr onic Journal of Statistics , 7:62–90, 2013. P . A. Lewis and G. S. Shedler. Simulation of nonhomogeneous Poisson processes by thinning. Naval R ese ar ch L o gistics Quarterly , 26(3):403–413, 1979. A. Lijoi and I. Pr ¨ unster. Mo dels b eyond the Diric hlet pro cess. In N. L. Hjort, C. Holmes, P . M ¨ uller, and S.G. W alk er, editors, Bayesian nonp ar ametrics , v olume 28, page 80. Camb. Ser. Stat. Probab. Math, 2010. A. Lijoi, R. H. Mena, and I. Pr ¨ unster. Con trolling the reinforcemen t in Ba yesian non-parametric mixture mo dels. Journal of the R oyal Statistic al So ciety: Series B (Statistic al Metho dolo gy) , 69(4):715–740, 2007. A. Lijoi, B. Nip oti, and I. Pr ¨ unster. Ba yesian inference with dep endent normalized completely random measures. Bernoul li , 20(3):1260–1291, 2014. K. Miller, T. Griﬃths, and M. Jordan. Nonparametric latent feature mo dels for link prediction. In NIPS , 2009. J. Møller and R. W aagepetersen. Statistic al infer enc e and simulation for sp atial p oint pr o c esses . CRC Press, 2003. R. M. Neal. MCMC using Hamiltonian dynamics. In S. Bro oks, A. Gelman, G. Jones, and X.-L. Meng, editors, Handb o ok of Markov Chain Monte Carlo , v olume 2. Chapman & Hall / CR C Press, 2011. M. Newman. Networks: an intr o duction . OUP Oxford, 2009. M. E. J. Newman. Mixing patterns in net works. Physic al R eview E , 67(2):026126, 2003a. M.E.J. Newman. The structure and function of complex netw orks. SIAM r eview , pages 167–256, 2003b. K. Nowic ki and T. Snijders. Estimation and prediction for sto chastic blo c kstructures. Journal of the Americ an Statistic al Asso ciation , 96(455):1077–1087, 2001. Y. Ogata. On Lewis’ simulation metho d for p oint pro cesses. IEEE T r ansactions on Information The ory , 27 (1):23–31, 1981. P . Orbanz and D. M. Ro y . Bay esian models of graphs, arrays and other exchangeable random structures. IEEE T r ans. Pattern Anal. Mach. Intel ligenc e (P AMI) , 37(2):437–461, 2015. K. Palla, D. A. Knowles, and Z. Ghahramani. An inﬁnite laten t attribute mo del for netw ork data. In ICML , 2012. I. Pr ¨ unster. R andom pr ob ability me asur es derive d fr om incr e asing additive pr o c esses and their applic ation to Bayesian statistics . PhD thesis, Univ ersity of P avia, 2002. I. Psorakis, S. Rob erts, M. Eb den, and B. Sheldon. Ov erlapping comm unity detection using Bay esian non- negativ e matrix factorization. Physic al R eview E , 83(6):066114, 2011. S. Resnic k. Extr eme values, r e gular variation and p oint pr o c esses . Springer, 2013. A. Saeedi and A. Bouchard-Cˆ ot´ e. Priors o ver recurrent con tinuous time pro cesses. In A dvanc es in Neur al Information Pr o c essing Systems , pages 2052–2060, 2011. M. Salter-T o wnshend and T. H. McCormick. Latent space models for multiview netw ork data. T ec hnical rep ort, T ec hnical Rep ort, 2013. A. V. Skorohod. R andom pr o c esses with indep endent incr ements , volume 47. Springer, 1991. A. T o deschini, X. Misc ouridou and F. Car on/Exchange able R andom Me asures for Sp arse and Mo dular Gr aphs 33 T. A. B. Snijders and K. Nowic ki. Estimation and prediction for sto chastic blockmodels for graphs with laten t block structure. Journal of classiﬁc ation , 14(1):75–100, 1997. P . T anko v. Dep endence structure of sp ectrally p ositive multidimensional L ´ evy pro cesses. Unpublishe d manuscript , 2003. V. V eitc h and D. M. Roy . The class of random graphs arising from exchangeable random measures. arXiv pr eprint arXiv:1512.03099 , 2015. L. M. V erbrugge. Multiplexity in adult friendships. So cial F or c es , 57(4):1286–1309, 1979. J. Y ang and J. Lesko vec. Ov erlapping communit y detection at scale: a nonnegative matrix factorization approac h. In Pr o c e e dings of the sixth A CM international c onfer enc e on Web se ar ch and data mining , pages 587–596. A CM, 2013. Y. Zhao, E. Levina, and J. Zhu. Consistency of communit y detection in net works under degree-corrected sto c hastic block mo dels. The A nnals of Statistics , 40(4):2266–2292, 2012. M. Zhou. Inﬁnite edge partition models for o verlapping communit y detection and link prediction. In A rtiﬁcial Intel ligenc e and Statistics (AIST A TS2015), JMLR W&CP , volume 38, 2015.

Exchangeable Random Measures for Sparse and Modular Graphs with Overlapping Communities

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment