A Modeling Framework for Gossip-based Information Spread

A Mo deling F ramew ork for G o ssip-based Information Spread Rena Bakhshi 1 , Daniela Gavidia 2 , W an F okkink 1 , and Maarten v an Steen 1 1 V rije Univ ersiteit Amsterdam, D e partment of Computer Science, De Bo el elaan 1081a, 1081 HV Amsterd am, The Net h erl ands { rbakhshi ,wanf,stee n } @few.vu .nl 2 Chess, P .O. Box 5021, 2000 CA Haarlem dgavidia@g mail.com Abstract. W e present an analytical framew ork for gossip proto co ls based on the pairwise information exchange betw een in teracting no des. This framew ork allo ws for s tudying the impact of proto col parameters on the p erf ormance of the proto col. Previously , gossip-based information dis- semination protocols hav e been analyzed under the assumption of p er- fect, lossless communication channels. W e extend our framew ork for the analysis of netw orks with lossy c h a nnels. W e sho w ho w th e presence of message loss, coupled with sp ecific top o logy configurations, impacts the expected beh a vior of the proto col . W e va lidate the obtained mo dels against simulations for tw o protocols. 1 In t r oduction Gossip proto cols h av e emerged as an attractive solution for distributing informa - tion in larg e-scale systems, due to their simplicity and efficiency . Randomness and the distr ibu ted nature of go ssip proto cols consider a bly incre a se their ro- bustness compared to deterministic proto cols, a lso in the presence of failures and data loss . Although gos sip pro t o cols are characteriz e d by their super fic ia l simplicity , the larg e-scale behavior of a g ossip system is not easily predictable. F rom the per spective o f proto col design and behavior prediction, this is an undesirable situation, and with the growing n umber of gossip pro t o cols, there is an increasing demand for their analy s is in an insig h tful and systematic wa y . How ever, the formal analy s is o f these pro t o cols is still a rather unexplor ed resear ch field with many challenges, in part caused by the fact that traditiona l techn iques quickly lead to a s t ate-space explosio n ( see, e.g ., [14, 10, 2]). In [6], we hav e pro posed a mo delling fra mew ork for gossip-based information dissemina tio n proto col, in which each no de per io dically sele c t s a r andom p eer and shuffles its lo cal data. In this pap er, w e show that this appro ac h is generally applicable to gossip proto cols for information dissemination, and we extend the a n alysis metho d to netw o rks with lossy channels. An exc ha ng e of data be t ween no des is mo delled as a state tra nsition capturing the presence or absence of a selected data item b efore and after a gossip b et ween t wo nodes . Our mo delling fra mework allows for the pr e d iction of the system behavior in large-s cale environmen ts. W e prop ose a mo de l based o n lo cal pairwise interac- tions which in com bina t ion with mo del c hecking (cf. [5]) or with the mean- fie ld framework (cf. [3]) a llo ws for the analysis of the emergent b eha vio ur of the sys- tem. The mo del can als o b e used for optimiza tio n of system par ameters and per formance (cf. [6]), and for fast even t-bas ed simulation, as it is done in this pap er. W e intro d uce our theo r etical fr amew or k initially assuming no communica- tion failures in the netw or k. Later, we show that the framework is applicable to netw orks where no des communicate ov er an unrelia ble medium. An imp or- tant par t of our prop osed framework is the pro babilit y P dr op that an observed item in a no de’s lo cal sto r age has b een replaced b y another item a f ter a gossip. W e derive the express ions for this probability under the assumption o f perfect communication medium for tw o pro t o cols, Newsc ast [17 ] a nd Shuffle [15]. In the presence of message los s , how ever, the dep endence of the probability on the t yp e of under lying netw ork gr a ph emerges. W e deconstruct the express io n for P dr op int o its main co mponents, and identify the o nes that dep end on s pecific scenario configura tions, i.e., message loss and top ology combinations. One such comp onen t is the pr obabilit y o f an observed item found a t one goss ip ing no de, to be als o found at its peer. Since it is not feasible to co ver a ll p ossible netw o rk top ologies in one formula, the computation of this pro ba bilit y is based on sta - tistical data that can b e o btained from Monte Ca rlo sim ulations o f the proto col at hand. Since the work o f Demer s et al. [11], gossip pro t o cols often follow bidirec- tional data exchange (push-pull) for b ett er per fo rmance. As observed in [1 3], the p erformance of these proto cols is usually ev a lua ted under the assumption of a perfect, error - f ree co mm unication medium. Thus, data exchange op erations betw een no des a re ge n erally c o nsidered to b e atomic opera t ions, e.g., [21, 7, 22, 18, 24 , 15, 12, 6 ]. That is, if a no de initiates a gos sip with a no th er no de, b oth no des ba se their loca l decis io ns up on each other ’s data, a n d in some proto cols even guar an tee the pr eserv ation of each other’s data. In practice, how ever, im- plement ing data exchange as an atomic op eration is har d to achiev e assuming communication over unreliable media [23 ]. In this resp ect, our pap er makes an additional contribution; it investigates the impac t of a realistic environment with lossy comm unica t ion channels on a push-pull-based goss ip proto col. F urthermore , w e pr esen t in more detail the following o bserv ations: • Int ro ducing mes sage loss in the net work model affects the emergent b eha vior of the proto col in a s pecific manner: with the introduction of loss y communi- cation c hannels , the cor relation betw een lo cal storage conten t of neighbor ing no des increa ses, a nd the degree of the cor relation depe n ds on the netw ork top ology . • The fewer neigh bor s no des ha ve in the underlying netw ork , the stronge r the effect of message lo ss on the emerg ent behavior of the pr oto col. F or fully connected netw ork s , message loss do es not hav e an impact on the distribution 2 of da ta over the netw ork, which r e mains uniform (as observed when there are no losses). Two a reas of research are relev ant to our w o r k: generic modelling frameworks for gossip proto cols and the perfor mance of gossip pr oto cols in the presence of message lo ss. Automated mean-field fra mework for dynamic g ossip netw o rks has bee n pr esented in [4]. Due to the under lying mea n- field metho d, used by this framework, the results are accur ate only fo r v e r y large netw orks and for av era ge behavior o f the mo delled system. The mean-field framework can b e combined with our framework, as it is shown in [3 ] for Sh uffle. There is previo us work [16] on a simple gossip-ba sed member s hip proto col with no na tomic proto co l a ctions in the presence of message los s of up to 1%. The a uthors pr op osed to us e a push-based gossip proto col, in which only an active no de sends da ta to its peer , a nd immedia tely r emov es the sent data from its cache. Moreover, the no de sends only tw o items, IP addresses a nd p orts of itself and of a random p eer from its lo cal cache. In the proto c ols that we study , each go ssiping pair exc hanges a random subset of the data items. The authors show the proto col corr ectness modelling it by gr aph transfor mations, similar to the a pproach in [10]. The proto co l pr op erties analyzed include the exp ected nu m ber of neighbors a no de has a nd the uniformity of the neighborho o d lists. The theoretica l results in [16] were no t co mpared with exp er iment al ev alua tio ns of the gossip proto c ol. The scop e of their pape r differs from ours : it focuses on the correctnes s analysis, lea ving out perfor mance ana lysis of the prop os ed pr oto col. The pap er is or ganized in the following sections . Sec. 2 outlines the class of gossip pr oto cols that can b e analyzed with our fra mework, presented in Sec. 3. W e consider tw o case studies, Newscas t and Sh uffle, and demo nstrate the mo d- elling and de r iv ation of the state transition ma trix for them, in Sec. 4 a nd Sec. 5, resp ectively . Sec. 4 demonstr ates the mo delling o f push and pull v er sions of News- cast. In Sec. 6, we mo del and analyze Shuffle in the presence o f messag e los s. Sec. 7 stretc hes the scop e of our theoretica l framework to g ossip-bas ed member- ship proto co ls lik e Cyclo n. Sec. 8 concludes the pap er . 2 Bac kground In this sec tio n, we sp ecify a class of go s sip proto cols for which our mo delling framework is a pplicable. W e co nsider larg e netw or ks, wher e no des interact in a p eer-to-p eer style. All no des hav e a common agree ment on the frequency o f gossiping. E ach no de stor e s lo cal data in its c ache and executes t wo different threads, an a ctive and a pas s ive one (see Fig. 1). The active thread per io dically initiates a contact with a randomly chosen p eer p by sending it a (sub)set σ of its cache, and waits for a reply . Upo n rec e ptio n of the reply σ p , the no de upda tes its cache based on the cache, σ and σ p . The passiv e thread w aits for a message sen t b y the active thread of a neighbor and replies to it with its data σ . The node then updates its cache ba sed on the stored, received and sent data. The e x change mes sage with its c onten t is c alled 3 wait ( ∆ t time u nits) p ← RandomPeer () ; σ ← PrepareMsg () ; send σ to p ; wait until receive( σ p ) σ ← Upd ate ( σ , σ p ) ; wait until receive( σ p ) σ ← PrepareMsg () ; send σ to sender( σ p ); σ ← Up date ( σ, σ p ) ; (a) activ e thread (b) passive thr ead Fig. 1. S keleton of a gossip protocol an exchange buffer . No des tha t ex e cute their active thr e ad are initiators , and no des that execute their pa s sive threa d are called c ontacte d . The generic proto col, describ ed a bove, is a push-pul l goss ip proto col. O ther v a riations of gossip pr o to cols ar e o nes in which the initiator only sends lo c al data to its go s sip partner ( push ), a nd ones in which an initiator only req uests data from its p eer ( pul l ). All three versions of go ssip proto c o ls are cov ered b y our framework. The r andom p eer selection RandomPeer () is based on the set of neighbors a s determined by the underlying netw o rk gra ph. The nature of the da ta and the result o f the generic op erations in Fig. 1 a r e application- depe ndent. In goss ip- based infor mation dis s emination pr oto cols like [18, 15 ], a finite list of news items comp oses the lo cal cac he of a no de. The op eration σ ← PrepareMsg () in Fig. 1 selects a ra ndom (or predefined) set of items fro m the cache of the no de. The metho d Up date merge s the list o f old items with the lis t of r eceived items. Pro p- erties o f the proto co ls that can b e analy z ed within o ur fr a mework include the nu m ber of co pies of an item in the netw o rk ov er time and the sp eed a t which items spread throughout the netw ork. In gossip-ba sed membership mana gement pro to cols, such as [2 4, 1], a cache of e a ch no de consists o f a finite set of its peer IP addresses, so-called the p artial view of the mem b ership of the net work. The metho d Up date pro duces a sample of the union of the old and the received views. The p erfor mance metric s of these proto cols include a distribution of the partial view s ize, and the num ber of no des reached in the presence of no de failures. W e refer to [9] for other applications o f gossip proto cols. As we demonstrate later in this pap er, o ur framework models go s sip-based information dissemination pro to cols, but it can b e also a pplied to other type of applications, such a s gossip-ba sed members hip proto co ls . 3 P airwise Interaction Mo del An exchange o f data items b etw een no des is modelled as a state transition, capturing the pres e nce or absence of an obs e rved data item d b efo r e and after a gossip int eraction b etw een two no des: an initiator A and a c ontacte d n o de B . Each state is then a pair ( a, b ) of bits, each indicating the presence (if equal to 1) or the absence (if equa l to 0) of the item in the cache of A and B , res pec tively . 4 (0 , 0) (0 , 1) (1 , 0) (1 , 1) P (10 | 01) P (01 | 10) P (11 | 10) P (10 | 11) P (01 | 11) P (11 | 01) P (00 | 01) P (00 | 10) P (00 | 11) P (00 | 00) P (01 | 01) P (10 | 10) P (01 | 01) Fig. 2. Pairwis e interactio n mo del The model has four p o ssible states of the caches of A and B : (1 ) when bo th hold d , (2–3) either of the ca ches holds the item d , and (4) neither cache ho lds d . These corresp ond to the states (0 , 0), (0 , 1), (1 , 0), and (1 , 1) in the state transition diagra m, sho wn as Fig. 2. T ransitions from one state to another ar e lab elled b y the resp ective tra nsition pro babilities P ( a 2 b 2 | a 1 b 1 ), where a 1 b 1 is the state b efore a gossip interaction, and a 2 b 2 is the state after the interaction, with a i , b i ∈ { 0 , 1 } . a 1 , a 2 and b 1 , b 2 corres p o nd to states of no de s A a nd B . F or instance, P (01 | 10) means that no de A had d b efor e the interaction, whic h it passed on to B , afterw a rds. The building blo cks of a ll transitio n probabilities P ( a 2 b 2 | a 1 b 1 ) of the s tate diagram are t w o probabilities: (i) the pro bability P sele ct of an item to b e se le cted by a no de fro m its cache for a gossip, and (ii) the probability P dr op that an item is replaced b y another one, received by its no de in the gossip. The expressions for P sele ct and P dr op depe nd on sp ecifics of the op era tions PrepareMsg and Up date , resp ectively . Moreover, these probabilities are functions of the numb er of items n , exchange buffer size s , and c ache size c , and 0 < s ≤ c . W e as sume that c < n , i.e. all n items cannot b e stor ed in a single lo cal cac he. W e explain our framework by consider ing state transitions and computing the corresp onding pr obabilities for several ca se studies. Our state diag ram differs from a Marko v c hain of the s y stem: it e xpresses a state of gos s iping pair rather than state o f all no des. The transition proba bili- ties are used with other frameworks, e.g ., mean-field framework and numerical simulations, to study the emergent b ehaviour of system. 4 Case study: Newsc ast W e now briefly describ e o ur fir s t ca se study , a simple push-pull information propaga tion pro to col. It is a v ariation of the News cast proto col [1 7]. In the original version of Newsca s t, each item is paire d with a timestamp indicating when it was crea ted. The original Newsca s t proto co l can als o b e mo de lle d using our analytical framework; w e will come bac k to the deta ils in Sec. 7. 5 The basic idea o f the proto col is a perio dic ex change of data items published by no des in the net work. Items can be, for example, a n umber or net work addr ess (IP and p ort) of a node. F or now, we a ssume that nodes do not cr ash a nd communication channels are failure-free. 4.1 Sp ecification The pr oto col oper ates in a wide ar ea netw or k 3 . A no de p e rio dically pic k s a random p eer and exchanges s r andom items with it. The no de then selects c random items for its new cache among the re ceived items and those in its cache. The metho ds in Fig. 1 for Newscast are summarised in the following table: metho d ope ration RandomPee r() sele ct a p eer u niformly at random Prepa reMsg() select s random items Upda te( σ, σ p ) s tore only c rand om items, selected among the items in its cac h e and the received items σ p 4.2 T ra nsition diagram W e analyse the spr ead of a generic item, whic h w e call d , using the fra mework int ro duced in Sec. 3. As stated previously , there are four po s sible states of the caches of A a nd B . The state (0 , 0) has o nly one outgoing transition, back to itself. P (00 | 00 ) = 1 since if A and B do not hav e d b efor e their exchange then they clearly still do not have d after the ex change. W e now deter mine v alue s for other probabilities P ( a 2 b 2 | a 1 b 1 ). Note that due to the symmetry o f information exc ha nge b e t ween gossiping no des in the proto c o l, P ( a 2 b 2 | a 1 b 1 ) = P ( b 2 a 2 | b 1 a 1 ); a nd the proba bili- ties P dr op and P sele ct are the same for bo th initia tor a nd contacted no de. W e us e the following notations for complementary probabilities: P ¬ select for 1 − P sele ct , and P ¬ dr op for 1 − P dr op . State (0 , 1) Before gossip, d is only in the cac he o f no de B . a 2 b 2 = 01 : B neither sends nor drops d from its cache, or B sends a nd k eeps d , and A drops it, i.e., the pro bability is P (01 | 01) = P ¬ select · P ¬ dr op + P sele ct · P ¬ dr op · P dr op . a 2 b 2 = 10 : B selects and drops d , and A keeps it; i.e ., the probabilit y is P (10 | 0 1) = P sele ct · P dr op · P ¬ dr op . a 2 b 2 = 11 : B se nds d , and b oth no des k eep it; i.e., P (11 | 01) = P sele ct · P ¬ dr op · P ¬ dr op . a 2 b 2 = 00 : B sends d and b oth nodes dr op it, or B do es not send and dro ps d ; P (00 | 01) = P sele ct · P dr op · P dr op + P ¬ select · P dr op . State (1 , 1) Before gossip, d is in the caches of bo th nodes. 3 In the netw ork of goss iping n o des, an information dissemination is fully dictated b y gossiping frequency instead of communicatio n latencies (cf. [17]). 6 a 2 b 2 = 01 : node A drops d while up dating its cache, but B keeps it; i.e., P (01 | 11) = P ¬ dr op · P dr op a 2 b 2 = 10 : this transition is symmetric to the previous one. a 2 b 2 = 11 : neither A nor B dro ps the item d ; i.e., P (11 | 1 1) = ( P ¬ dr op ) 2 . a 2 b 2 = 00 : both A and B drop d ; i.e., P (00 | 1 1 ) = ( P dr op ) 2 . All transition proba bilities are summarized in Fig. 3 P (01 | 01) = P (10 | 10) = ( P ¬ select + P select · P drop ) · P ¬ drop P (10 | 01) = P (01 | 10) = P select · P drop · P ¬ drop P (11 | 01) = P (11 | 10) = P select · P ¬ drop · P ¬ drop P (00 | 01) = P (00 | 10) = ( P select · P drop + P ¬ select ) · P drop P (01 | 11) = P (10 | 11) = P ¬ drop · P drop P (11 | 11) = ( P ¬ drop ) 2 P (00 | 11) = ( P drop ) 2 P (00 | 00) = 1 Fig. 3. T ransition probabilities 4.3 Building blo c ks: P sele ct and P dr op W e now deriv e the ex pressions for the proba bilities P sele ct and P dr op . The follow- ing analysis a ssumes tha t all no de caches are full; that is, the netw ork is already running for a while. Consider no des A and B engag ed in a n ex change, a nd let B receive the exchange buffer S A from A . Let k be the num ber o f duplica tes (see Fig . 4), i.e., the items of an intersection of the node cache C B and the exchange buffer of its gossip partner S A (i.e., S A ∩ C B ). C B S A k = S A ∩ C B (a) S A C B S B b s = S B ∩ S A (b) Fig. 4. a) Items sent b y A t h at are in th e cache of B ; b ) Items in common for the exchange buffers of both A and B . The pr obability of selecting an item d in the cache is the num ber o f se le cted items s divided by the total num b er of items in the c ache c : P sele ct = s c . Thus, the probability that a n item d in the cac he is not selected is: P ¬ select = c − s c . Among the c items in C B , there a re k items also in S A ; th us, only c random items of the tota l c + s − k items in C B ∪ ( S A \ C B ) c a n b e k ept: P ¬ dr op ( k ) = c c + s − k . 7 Thu s, the probability P dr op that a n item in C B ∪ ( S A \ C B ) is dropp ed from C B , given k items in S A ∩ C B is as follows: P dr op ( k ) = 1 − c c + s − k . (1) Assuming uniform sampling o f items, the a verage v alue of k is s · c n . Th us, the probability of dr opping an item after the exchange P dr op is 1 1+ cn/ ( s ( n − c )) . Later we discuss a case when the assumption of uniform sampling do es not ho ld. 4.4 V alidation T o v alida te our t heoretical results, we sim ulated Newscast in a round-based fash- ion similar to simulations in PeerSim [1 9]. A new item d is initia lly in tr o duced in a net work o f 2500 nodes at one no de. In this fully connected net work, caches of all nodes are full and unifor mly p opulated by n = 500 items. After each gossip round, we measure the total num b er o f copies of d in the net work ( r epli- c ation pr op erty), a nd how man y no des in total hav e seen d over time ( c over age prop erty). Simulations with the simplifie d Newsc ast Each no de in the net work has a ca che size o f c = 10 0. Once in a r ound, every no de selects uniformly at r andom one no de from the netw or k of 2500 nodes a nd exc hanges s = 5 0 random items. T o make a fair comparis on with the simulations with the model, we let the no des gossip for 1000 rounds items other than d ; items are replica ted and the replicas fill the c a ches of all no des. At the ro und 1 000, the o bserved item d is inser ted int o the netw ork a t a r andom lo ca tion. F rom that ro und o n, replication a nd cov er a ge a re measured by the end o f each ro und. Simulations with the mo del F or the simulations with the mo del, the sy s tem parameters n, c and s are set to 500 , 100 and 50, r e s p e c tively . Each no de in the net work only maintains a state v ariable whic h indicates the pres ence or absence of d . Nodes up date their state in pair s acco rding to the tra nsition proba bilities in Fig. 3. While in the actual protoc ol, node s up date the co nten ts of their cac hes, in the mo del simulations, no des up date only their state v a riables. Since we do not need a startup time for the simulations with the model, at the ro und 0, we set the sta te of a r andom no de to 1, and all others to 0. F rom that round on, we track the states of the nodes . The gra phs in Fig. 5 depict the p erformanc e o f the simplified Newscast and of the ana ly tical mo del in ter ms of r eplication (left) and cov erage (right) of d . The results ar e collected from 1 000 simulation runs of the proto col and 10 00 runs of the mo del. In approximately 72% of the all runs, the obs e r ved item disapp ears. In the gr aphs, howev er, we o nly c o nsidered the runs wher e the item did not disapp ear ; e.g., to obtain 1000 runs of the proto col displa yed in Fig. 5, we p erfor med a to tal of 3700 runs. Assuming a norma l distribution of t he results, the confidence interv al for 95% confidence is 1 6 times nar r ow er than the standard deviations drawn on the graphs. 8 0 100 200 300 400 500 600 0 100 200 300 400 500 number of replicas rounds protocol and model 0 500 1000 1500 2000 2500 3000 0 100 200 300 400 500 number of nodes rounds protocol and model Fig. 5. Clique: R eplication (left) and co verage (right) of item d for Newscast and its mod el, with c = 100, n = 500, s = 50. Fig. 5 show the av er a ge and standar d deviation of the successful r uns with Newscast. The standard devia tion bars o n the gra phs are the standard devia tion of the r eplicas and num ber of no des that hav e seen item in the system. F or a ll successful runs, the netw ork reaches the equilibrium, in which there are around 500 replicas of d . Due to rando m gossiping, item d is even tua lly seen by all no des in the netw or k, when cov era ge re a ches a ll 250 0 nodes. W e have chosen this type of network graph for a ba sic intro duction of our framework since a uniform distribution o f items o ver the netw o rk is c le arly a v a lid a ssumption in a fully connected net work o f no des executing Newscast. W e will turn to other types of netw ork gra phs in the next case study . Our framework allows for mo delling of push-only as w e ll pull-only gossip proto cols, see Appendix A. 5 Case Study: Shu ffle W e now mov e on to another case , the Shuffle pro to col introduced in [15] a nd originally a nalyzed in [6] using our fra mework. This section is intended to make the reader familiar with the a nalysis of this pro to col. 5.1 Sp ecification Sh uffle a ims a t the conserv atio n of da ta in an a d ho c net w ork. Each no de main- tains a cac he o f data items that are disseminated thro ughout the netw ork. The dissemination is done b y p er io dic exc hange of s ra ndom items betw een tw o gos - siping p e e rs. The pro ce dur e of selectio n of items is similar to the one in Newsc ast, but items a re disca rded according to a differen t p olicy: (i) a no de cannot disca rd an item unles s it is sent to the gossip pa rtner; (ii) the partner is not a llow ed to drop received items . A node can keep only c items in its cache after the exchange. A pe e r will fa vor dropping items it has just sent ov er other items in its cache. Note that in this wa y , 9 an exc ha nged item will a lwa ys b e preser ved, and po s sibly ev en replicated. Since the peer, in its turn, has agreed to store received items in its cache, discarding items do es not lead to the loss of information in the netw ork (if there is no message loss). The gener ic routines in Fig. 1 can b e summarized for Shu ffle as follows: metho d operation RandomPee r() sele ct a p eer uniformly at random Prepa reMsg() select s random items Upda te( σ, σ p ) • add σ p receiv ed entries to the cache; • remov e duplicated items; • remov e items among σ \ σ p uniformly at rand om until th e cache has c items. 5.2 T ra nsition Diagram Unlik e in Newscast, nodes are not allowed to dis card the items received from their gossip partner s. T aking this difference into account, the transition proba bilities for Shuffle are quite easy to derive from the transitions in Fig. 3. Note that for this pr oto col the state (0 , 0) has only a self-tra nsition, and no other outgoing or incoming tr ansitions, b e c ause o f the preser v ation nature of the proto co l. That is, if either no des send an item, its partner keeps this copy as w ell, and if a n item is not among the selected for a shuffle, the item is no t replaced by another one. The transition proba bilities can b e ea sily computed, see Fig. 6 . F or more details on deriv atio n of these transition proba bilities, we refer to [6]. P (01 | 01) = P (10 | 10) = P ¬ sele ct P (10 | 01) = P (01 | 10) = P sele ct · P dr op P (01 | 11) = P (10 | 11) = P sele ct · P ¬ sele ct · P dr op P (00 | 00) = 1 P (11 | 10) = P (11 | 01) = P sele ct · P ¬ dr op P (11 | 11) = 1 − 2 · P sele ct · P ¬ sele ct · P dr op Fig. 6. T ransition probabilities for Shuffle P sele ct remains the same as for Newscast, P sele ct = s c . Howev er, P dr op is different for Sh uffle, and we derive it in the next section. 5.3 Probabilit y of Dropping an item P dr op represents the probability that a n item that can be ov er wr itten is indeed ov erwr itten by an item received by its no de in the e x change. P dr op depe nds on the num b er of items bo th gos siping no des hav e in common, in particula r , how many of such items the contacted no de re c eives during the exchange. T o derive the express ion for P dr op , we assume a unifor m distr ibution of items over the netw o rk; in the a bsence of messa ge loss, this assumption is suppo rted by experiments in [15, 20 , 6] and analys is in [8]. 10 Consider ag ain Fig . 4. By a ssumption, C A and C B are a unifor m selection from the en tire p opula tio n of n items. S A and S B are chosen uniformly at random from C A and C B , resp ectively . After the exchange, no de B has to allo cate items from S A in its cache C B , taking int o the account the duplicated items in S A ∪ C B and S A ∪ S B . Basically , the items in S B that are no t in S A are repla ced by items in S A that are no t in C B . So , every item from S A \ S B that is in C B means that one item from S B \ S A can be kept in C B . In other words, the probability P dr op that an item from S B ∪ S A is dropp ed from C B equals to the probability that an item from S A \ S B is not in C B . By uniform distribution, the pr obability is n − c n − s . Thus, P dr op ≈ n − c n − s . This approximation of P dr op for Sh uffle has b een successfully established through exp eriments [6], and used for mo delling and o ptimization in [6, 5 ]. 6 Sh uffling through Lossy Channels So far, the analy s is has relied on the as s umption that no des interact in a p erfect, lossless c o mmun ication e nvironmen t. W e now drop this assumption, and consider ad ho c net works where no des are cont in ually communicating with each other ov er an unreliable medium. W e now explore the impact o f lo s sy channels on a gossip proto col with a push-pull information exchange. 6.1 Assumptions Every mess age s e nt now ha s a fixed, pos itive pr obability to b e lost due to a disturbance of the comm unication medium. As ex plained in the introductio n, we no lo nger assume that the shuffle proc e dure is atomic. There a r e three genera l case s in pa irwise co mmun ication with res p ect to message delivery , as depicted in Fig. 7: (a) no de A initiates a gossip with B by sending a messa ge, but the message is los t, (b) no de A succes s fully initiates a gossip with B , but a messag e from B is lost o n its way to A , and (c) a gossiping pair receives mes sages from each o ther. 6.2 T ra nsitions W e take the analytical mo del of Shuffle descr ibed in Sec. 5 as starting p oint. In our current mo del, the state ( 0 , 0 ) is no longer isolated, since there is a pos s ibilit y to remov e the only copy of d from the cache of the no de B , in the scenario shown in Fig. 7(b). A B (a) A B (b) A B (c) Fig. 7. Scenario s of comm un ication with lossy c hann els: (a) lo ss of the request mes sage, (b) loss of the reply message, (c) gossip without loss. 11 W e again expre s s the transition proba bilities P ( a 2 b 2 | a 1 b 1 ) in ter ms of P sele ct , P dr op . T o analy ze the pro to col in the presence of messa ge loss, we introduce into the for mal mo del an additional input parameter, the pro bability P loss that a message is not delivered to a no de due to c hannel loss. W e use P ¬ loss for 1 − P loss . State (0 , 0) Before shuffle, neither A nor B hav e d in their cache. a 2 b 2 = 00 : neither A nor B hav e item d after a shuffle, b eca use neither of them had it in the caches befor e the shuffle: P (00 | 00) = 1 . a 2 b 2 ∈ { 01 , 10 , 11 } : canno t occur , beca use none of the no des have item d . State (1 , 0) Before shuffle, d is only in the cache of no de A . a 2 b 2 = 01 : oc curs only when bo th the request of A and the r eply of B a re suc- cessful and node A selects a nd dr ops d : P (01 | 10) = ( P ¬ loss ) 2 · P sele ct · P dr op . a 2 b 2 = 10 : B do es not hav e d b ecause: (a) A did not select d , or (b) A selected d but the r equest message got lost on the w ay to B : P (10 | 10) = P ¬ select + P loss · P sele ct . a 2 b 2 = 11 : both no des A and B hav e a copy of d b eca use: (a) either b oth no des received the gossip messages and A selected d and kept it, or (b) A selected d and the reply messag e from B got lost; that is, P (11 | 10 ) = ( P ¬ loss ) 2 · P sele ct · P ¬ dr op + P ¬ loss · P loss · P sele ct a 2 b 2 = 00 : cannot o ccur, as A would only dro p d if it r eceived a reply , which implies that B would keep d . State (0 , 1) Before shuffle, a copy of d is only in the ca che of node B . a 2 b 2 = 01 : only B has d b ecause the mes s age from A g ot lost o r, B r eceived the message, and: (a) it did not select d or (b) B selected d , k ept it, and reply got lost; i.e . the probability is P (01 | 0 1) = P ¬ loss · ( P ¬ select + P loss · P sele ct · P ¬ dr op ) + P loss . a 2 b 2 = 10 : both request and reply mess ages were succe ssfully delivered a nd B selected a nd dr opp ed d , which a mounts to the probability P (10 | 0 1) = ( P ¬ loss ) 2 · P sele ct · P dr op . a 2 b 2 = 11 : both mess ages were deliv er ed successfully , and B selected and kept d ; that is, P (11 | 0 1) = ( P ¬ loss ) 2 · P sele ct · P ¬ dr op . a 2 b 2 = 00 : neither of the no des have d , because B selected and dropp ed d , but A did not receive the r eply messa ge. P (00 | 01) = P ¬ loss · P loss · P sele ct · P dr op . State (1 , 1) Before shuffle, d is in the c a ches o f A a nd B . a 2 b 2 = 01 : only B has d since b oth messa ges were successfully received, A se- lected and dro ppe d d while B did not select it: P (01 | 1 1) = ( P ¬ loss ) 2 · P sele ct · P dr op · P ¬ select . a 2 b 2 = 10 : only no de A has d b eca us e no de B selected d , dropp ed it, and no de A did not selec t d : P (10 | 1 1) = P ¬ loss · P ¬ select · P sele ct · P dr op . a 2 b 2 = 11 : after the shuffle b oth nodes have d , b ecause: a) the messag e from A did not arrive at B , i.e. P loss ; or b) the mess a ge from A was suc c essfully r eceived by B , but the reply mes sage got lost, and: 12 ⋆ A did not s elect d , while B (i) did not selec t d as w e ll, or (ii) selected a nd kept d . P ¬ loss · P ¬ select · P loss · ( P ¬ select + P sele ct · P ¬ dr op ); and ⋆ no de A selected d : P ¬ loss · P sele ct · P loss ; or c) bo th no des r eceived each other messages, and: ⋆ A did not select d , while B (i) also did not select it, or (ii) selected and kept it: ( P ¬ loss ) 2 · P ¬ select · ( P ¬ select + P sele ct · P ¬ dr op ); and ⋆ (i) A selected and kept d , while B did not s elect d , or (ii) b o th no des selected d . ( P ¬ loss ) 2 · P sele ct · ( P ¬ select · P ¬ dr op + P sele ct ) . Hence, P (11 | 11) = 1 − (2 − P loss · (3 − P loss )) · P sele ct · P ¬ select · P dr op . a 2 b 2 = 00 : discarding of an item by b oth nodes cannot occur . While P loss expresses the r eliability of the communication channels, P sele ct and P dr op are der ived based on the b ehavior of the proto c o l. P sele ct is a gain the random selection of items from the cac he; an item has s c chance of b eing selected, regar dless of message loss. The calculation of P dr op is complex, and w e discus s it in the rema inder of this section. 6.3 The Probability of Dropping an Item W e now ana lyze ho w messag e loss a ffects one of the building blocks of o ur mode l: P dr op . In Sec. 5.3, we hav e a lready found a n expres sion for P dr op in the case of Shu ffle, relying on the assumption of a unifor m distribution of items. Here, we revisit P dr op and der ive a gener al for mula witho ut making an y assumptions ab out the distribution of items. Later on, we will explore ho w message lo ss a ffects the distribution of items, and will determine that it is the coupling of messag e loss a nd the topo lo gy of the netw ork that affects P dr op . Having isolated the comp onent of P dr op that is affected by the messa ge loss/top olog y coupling, w e will prop ose the use of statistical data to calcula te that sp ecific part of the P dr op expression. When a node selects an item to be s ent to a neighbor, there is a pro bability P dr op that the item will b e dr opp ed from the no de’s cache a fter the gossip exchange. The selected item may b e dro ppe d only when there is a need to create space for an item rec eived from a gossip partner. Therefore, the pr obability P dr op depe nds on the rela tionship b etw een a) the new items received from the gossip partner (for which the no de needs spa c e, the shaded area in Fig. 8(b), r eferred to as A 1 ), and b) the items that the no de has selected to send to the gossip partner and is a llow ed to disca rd (the shaded area in Fig . 8(c), refer red to as A 2 ). In order to find an expr ession for P dr op , we need to calculate the probability of an item b eing in A 1 and the proba bility of a n item b eing in A 2 , whic h we will denote as P ( A 1 ) and P ( A 2 ), resp ectively . Given t wo no des , A and B , that engag e in a gossip exc ha nge, we can r epresent their ca ches as sets C A and C B , and the sets of items they exchange with each other b y S A and S B , resp ectively . The sets C A and C B may hav e common elements (items), s ee Fig. 8(a ). W e define P inx as the pro bability of a ny item found in o ne of the c a ches to b e also found in the other. Knowing P inx , we ca n 13 formulate P ( A 1 ) as P sele ct · (1 − P inx ) and P ( A 2 ) as P sele ct · (1 − P sele ct · P inx ). The probability P dr op can then b e expres s ed a s: P dr op = P ( A 1 ) P ( A 2 ) = 1 − P inx 1 − P sele ct · P inx (2) In order to calcula te a v a lue for P dr op , it is neces s ary to have a v alue for the prob- ability P inx . F or the case of C A and C B being random sa mples of a p opulation of n items, that is, under the assumption that items are uniformly distributed ov er the netw ork w e can analytically deduce that P inx = c n . F or netw orks with per fect comm unication channels, repea ted execution of Shuffle r esults in a uni- form distribution of items. In the next section, w e lo ok at the effect that lossy channels have on the distr ibutio n of items. 6.4 Uniform Distributio n of Items : Do es it Still Hol d? As alr e ady men tioned, the calcula tion of P dr op in the pr esence of messa ge loss requires us to sp ecula te a b out the co nt ent s of the goss ip partner ’s cache. As- suming that the items in the caches of b oth gossip pa r tners are r andom samples of the totality of items in the net work, we can easily estimate P inx as c n and, therefore, hav e an analy tical expressio n for P dr op ≈ n − c n − s . In this section, we ver- ify whether the uniform distribution of items, observed under no messag e loss, is still a v alid ass umption. The Imp ortance of R andom ness T o understand the effect of message loss on abstract level, consider o ne no de in the net work, executing Shuffle. W e examine the scenario fro m Fig. 7(b), where a reply mes sage from a node to the initiating node is lo st due to c hannel failure. When tw o no des A and B gos sip their loca l items to each o ther, the probability that the messa ge fro m B to A is lo st, is P ¬ loss · P loss . If the message of B is not delivered to A , items b c ∪ S A will be co mmon to cache C A and cache C B after the shuffle, since B , unaw ar e of the failure, purges the sent items S B , and A , which received no r eply , keeps its S A items (see Fig. 9). Over time neighbor ing no des hav e a gr owing intersection of items in their cache. If a no de has a small neighborho o d, the random sample b ecomes mor e C A C B b c b c = C A ∩ C B (a) C B S B S A A 1 = S A \ C B (b) S A C B S B A 2 = S B \ S A (c) Fig. 8. a) Items in common for the caches of b oth n od es A an d B ; b) Items sent by A that are not in the cac he of B ; c) Items sent by B that are not in the exc hange bu ffer of A . 14 C A C B b c b c = C A ∩ C B A B − − − − − − − − − − → C A C B b c ∪ S A b c = C A ∩ C B Fig. 9. Correlation of the caches increases due t o message loss biased to wards the collection of items le ft in the neigh bor ho o d. Limited by the access to the storage space of the neigh b o r ho o d, a set of neighbors main tains only a subset of all items. In general, the stronger the correlation b etw een the caches conten t o f the neighbours, the slower the spr ead o f observed item thro ughout the netw ork . F or the fully connected gr aph, since it is the entire collection of the items ev- ery node has access to, the uniform distribution assumption in general r emains. A la rger neig hborho o d r educes the pro bability of rep eated c o mmun ication b e- t ween tw o no des, and due to t he increased commun ication range, there is a faster exchange b etw een distan t ar eas of the netw or k. Exp erimental Observ ations T o supp ort our claims that messa ge loss affects the distribution o f items, we conducted s imulation exp er iment s. Each s imulation exp er iment has a startup per io d o f 1000 rounds, during which N = 2500 no des goss ip n = 50 0 differen t items under no message lo ss. W e use three differen t net work topo logies: a fully connected netw ork , a sq uare grid with ea ch int ernal no de having 4 neig hbors, and a netw or k where e very no de has 4 r a ndomly chosen neighbor s (outdegr ee 4). By the end of the sta r tup p erio d, the items hav e b een replicated, ac hieving uniform distribution. That is, every item ha s a probability c n = 100 500 = 0 . 2 of being present in a giv en no de ’s cache. After the sta r tup p er io d has finished, the communication channel is set up to fail with a probability P loss . W e calculate, per round, the proba bilities P (11), P (10) and P (01) that an exchange inv o lves bo th no des having item x , only the initiator having item x , or only the gossip partner having item x , respectively . Fig. 10 shows the av er age v a lue s of P (11), P (10) and P (01) over 1 000 rounds for different v alues of P loss using the three top ologies men tio ned earlier. As exp ected, for the case o f no messag e loss ( P loss = 0) the probabilities sugg est a uniform distribution of items in the net work. W e can analytically deduce their exp ected v a lue , which ma tches the exp erimental results, with P (1 1 ) as c n · c n = 0 . 04 and P (10) = P (01) as c n · (1 − c n ) = 0 . 16. Howev er, as the probability of message loss increa ses, we observe some c hanges in P (11), P (1 0 ) a nd P (01). F or the case of the fully connected netw ork , the pro babilities remain stable, suggesting tha t the uniform distributio n o f items remains unaffected by message loss. With the other top ologies, howev er, as message lo ss increases . the num ber of gos sip interactions b etw een no des that bo th have item x also rises. Since 15 0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 P loss P(01) fully connected grid range 1 outdegree 4 0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 P loss P(10) fully connected grid range 1 outdegree 4 (a) (b) 0 0.05 0.1 0.15 0.2 0.25 0.3 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 P loss P(11) fully connected grid range 1 outdegree 4 (c) Fig. 10. Probabilit y that a gossip exc hange occurs b etw een a) a node that do es not hav e item x and a nod e t hat has the item, b) a nod e that has item x and one that do es not, and c) tw o no des h a ve item x , for different top ologies. item x is repre s entativ e of all items in the ne tw ork, the graphs sugg est that, as a consequence o f message lo ss, goss ip partners hav e more items in common than they would hav e if items were uniformly distributed. In other words, due to message loss, a no de is mor e lik ely to ha ve items in common with a neigh bo r tha n with another random node in the net work. Hence, the topolog y of the underlying net work now pla ys a role in calcula ting P dr op . 6.5 Calculating P dr op based on Statistical Data By no w, w e have esta blished that in case of message loss the structure of neigh- bo rho o ds pla ys an impo r tant role in determining the distribution of items, whic h, in turn, deter mines the pr o bability of an item found in a no de’s cache to be a lso found in the gos sip partner’s cache, P inx . Our calcula tion of P dr op depe nds on finding a v alue for P inx . This top ology v ar ying compo nent ca n be modelled a n- alytically or mea sured exp erimentally from a single run of the proto co l, for a given top o lo gy . Here, we opt for obtaining P inx for a given netw ork gra ph fro m statistical data collected fr o m exp eriments. With P inx , we can pro ceed to calcu- late P dr op , obtaining the final building blo ck of the mo del needed for v alidation. 16 W e can calculate P inx from the probabilities P (1 1), P (10) and P (01) measured exp erimentally in the previous section: P inx = P (11) P (10) + P (11) The left g raph of Fig. 11 shows the v alues for P inx calculated for e ach exp eriment using a different P loss . Based on these v alues, we compute P dr op using (2), as seen in right g r aph of Fig. 11. As exp ected, the calculated v alue s show that, in the face of message loss, different top ologies yield different probabilities o f dropping an item. P dr op drops more ha rshly in the mor e cluster e d to po logies, which suffer mor e from neighboring no des ha ving similar items as mess age lo ss increases. 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0 0.2 0.4 0.6 0.8 P loss P inx fully connected grid range 1 outdegree 4 0.7 0.75 0.8 0.85 0.9 0 0.2 0.4 0.6 0.8 P loss P drop fully connected grid range 1 outdegree 4 Fig. 11. Probabilities P inx (left) and P dr op (right), for different top ologies. 6.6 Exp erimental Ev aluation The exper iment s in this s e ction sim ulate the case wher e a new item d is intro- duced by o ne no de in the network in which all caches a r e full and unifor mly po pulated by n = 50 0 items. Each exp eriment takes as input the top olo gy of the net work to determine whic h pairs of no des can gossip. In a ll case s , we use a net- work of N = 250 0 no des a rrang e d in either of the three top olo gies mentioned in the previous section (a fully connected netw ork, a square grid or a graph where every node has 4 r andomly chosen neighbo rs). In the exp er iments that follow, after each g ossip r ound, w e measure the total num b er of occur rences of d in the net work (replica tio n), and how many node s in total hav e seen d (cov er age). Simulations with Shuffle Each node in the netw ork ha s a cache size of c = 100, and sends s = 50 items when g ossiping. In each r ound, every no de randomly selects one of its neighbor s and shuffles. In order to make a fair comparis on with the simulations with the mo del, we let the no des gos sip for 1 000 rounds with P loss = 0, to ensure that no ne of the n = 500 initial items is lost in the sta rtup per io d b efore initiating the mea surements of the prop erties. After this star tup 17 per io d of 1000 rounds, items a re replicated and the r eplicas fill the ca ches of all no des. At ro und 100 0, P loss is set to the desire d v alue and item d is inserted int o the net work at a random lo cation. F rom that momen t on, w e track its replica tio n and cov era ge. Simulations with the mo del F or the simu lations with the mo del, n, c and s are system para meter s set to 500, 100 and 50, resp ectively . P inx is determined ex - per imentally on the netw ork, and P dr op is calculated using equa tio n (2). Instead of ma int aining a ca che, ea ch no de in the net work only main ta ins a v aria ble that represents whether it holds item d or not (state 1 or 0, resp ectively). No des upda te their state in pa irs according to the tra nsition proba bilities introduce d befo re, see Fig. 2. This mimics a n actual exchange of items b etw een a pair of no des according to Shuffle. While in the pro to col this results in b oth nodes up- dating the conten ts o f their caches, in a s imu lation using the ana lytical mo del upda ting the state of a node refer s to up dating o nly one v ar ia ble: whe ther the no de is in po s session of item d or not. At round 0 we set the s ta te of a random no de to 1 (while all the others ha ve state 0) and track the state of the no des for the remainder of the simulation. Figs 12 and 13 show the behavior of Sh uffle (top row) and how it compares to the analytical model (b ottom row) in terms of replication (left) and co verage (right) of d , fo r different v alues of P loss . F or ea ch v a lue o f P loss , 100 sim ula tion 0 100 200 300 400 500 600 0 500 1000 1500 2000 number of replicas rounds Ploss = 0.2 Ploss = 0.4 Ploss = 0.6 0 500 1000 1500 2000 2500 0 500 1000 1500 2000 number of nodes reached rounds Ploss = 0.2 Ploss = 0.4 Ploss = 0.6 20 40 60 80 100 0 500 1000 1500 2000 rounds P loss =0.6 (diff) std. dev. 20 40 60 80 100 replication diff (shuffle-model) P loss =0.4 (diff) std. dev 20 40 60 80 100 P loss =0.2 (diff) std. dev. 100 200 300 400 0 500 1000 1500 2000 rounds P loss =0.6 (diff) std. dev. 100 200 300 400 coverage diff (shuffle-model) P loss =0.4 (diff) std. dev 100 200 300 400 P loss =0.2 (diff) std. dev. Fig. 12. Grid, range 1: T op : R eplication (left) and cov erage (right) of item d for the shuffle. Bottom: difference betw een the shuffle and the mo del for replication (left) and co vera ge (righ t). 18 0 100 200 300 400 500 600 0 200 400 600 800 1000 number of replicas rounds Ploss = 0.2 Ploss = 0.4 Ploss = 0.6 0 500 1000 1500 2000 2500 0 200 400 600 800 1000 number of nodes reached rounds Ploss = 0.2 Ploss = 0.4 Ploss = 0.6 20 40 60 80 100 0 200 400 600 800 1000 rounds P loss =0.6 (diff) std. dev. 20 40 60 80 100 replication diff (shuffle-model) P loss =0.4 (diff) std. dev 20 40 60 80 100 P loss =0.2 (diff) std. dev. 100 200 300 400 500 0 200 400 600 800 1000 rounds P loss =0.6 (diff) std. dev. 100 200 300 400 500 coverage diff (shuffle-model) P loss =0.4 (diff) std. dev 100 200 300 400 500 P loss =0.2 (diff) std. dev. Fig. 13. T opology with outdegree 4: T op: R ep lication (left) and cov erage (right) of item d for the shuffle. Bottom: difference b etw een th e shuffle and the mo del for replication ( left) and co verage (right). runs (for b oth Shuffle a nd its model) were executed. Due to message loss, it is po ssible fo r item d to dis app ear after b eing intro duced into the netw ork (usually in the first few rounds). As P loss increases, this situation is mor e lik ely . In the graphs, we only ta ke into account the successful runs, i.e., where the item did not disapp ear , but spread. The top rows of Figs 12 a nd 13 s how the average and s tandard deviation of the successful runs with Sh uffle. W e compare this data with the av erage of the successful r uns of the mo del, and pres ent the differe nc e (in abs o lute v alue) in the b ottom rows of Figs 12 and 13. W e include the s tandard deviation for the shuffle in the b ottom row for co mparison. It clearly shows that the difference betw een the re s ults o btained from the shuffle and the model fall w ell within the standard deviation o f the shuffle results, confir ming the ability o f the mo del in predicting the av erage behavior of the proto co l. In all successful runs (despite message loss), the netw ork con verges to a sit- uation in which ther e ar e roug hly 50 0 copies o f d , indicating that after r ep e ated execution of the proto col d receives a fair shar e of the stor a ge space in the net- work; 2500 · 100 cac he slots divided b etw e e n 5 00 items. Also , as e xp ected, due to random gossiping item, d is even tually seen by all nodes in the netw or k, when cov er a ge r eaches 100 %. 19 7 Broadening the Scop e: a P eer-Sampling Proto col In the same w ay that we have developed mo dels for information dissemination algorithms, we ca n apply our metho dology to other t ype s of g ossip proto cols . In this section, we describ e, in broad terms, how to apply our mo delling approach to Cyclon, a go ssip-based peer- sampling proto co l. 7.1 The Cyclon Proto col Peer sampling is a ser vice that no des in larg e - scale distributed s ystems can call to o btain a random peer to g ossip w ith. Cyclo n implements a pee r sa mpling service by constructing and maintaining an unstructured ov er lay us ing g o ssiping mem ber ship infor ma tion. As opp os ed to Shuffl e, the data items exchanged by Cyclon a re r eferences to other no des in the netw ork . The r eferences, or links, that a no de stores in its cac he represent the no des that it can gossip with. The collection of links in the net work co ns titutes an ev er-changing overla y o ver which links ar e exchanged. The aim of Cyclo n is to keep the ca ches of the nodes in the net work p opulated with a r a ndom selection o f links to other no des. The fo llowing table summarize s the pro to col: metho d ope ration RandomPee r() Select s random items and place t h em in σ . F rom σ , select a gossip partner un iformly at random. Prepa reMsg() I f the no de initiates the exchange, replace the link to the gossip p artn er with a link to itself in σ . Return σ . Upda te( σ, σ p ) Dis card links to the no de itself and items that are already in the cac h e from σ p . Include remaining items in cache by 1 ) using empty s lots and 2) replacing entries among the ones in σ . Note that the steps taken within the routines of Cyclo n dep end on the r o le that a no de has in the go ssip ex change. The no de that initiates the exchange alwa ys includes a link to itself in the message , while the contacted no de do es not. This is reflected in the Prepa reMsg() routine, which is s lightly different for the initiator. 7.2 Mo del ling C yclon : States and T ransitions According to Cyclon, ev er y link d has a publisher D , and w hene ver D gossips , it sends the link d to its par tner. Thus, when mo delling the dissemination o f link d thro ughout the net work, we have to take into acco unt whether D (the source o f link d ) is inv olved in the exchange. Therefor e, we define the state of a no de as a tw o-bit string, wher e the first bit indicates whether the no de is D and the second bit indica tes whether link d is in the no de’s c a che (as done in the previous mo dels). As no des up date their s tates in pairs, and with each no de having a state represented by tw o bits, the sta te of a pair of no des engaged in a Cyclon e x- change consis ts of a four -bit string. While this op ens up to the p o ssibility fo r 20 00 , 00 00 , 01 01 , 00 01 , 01 P (0100 | 0001) P (0001 | 0100) P (0101 | 0100) P (0100 | 0101) P (0001 | 0101) P (0101 | 0001) P (0000 | 0000) P (0001 | 0001) P (0100 | 0100) P (0001 | 0001) Fig. 14. State transitions when no de A and B are not produ cers of item d . 10 , 00 10 , 01 01 , 10 00 , 10 P (1001 | 1000) P (0010 | 0110) P (0110 | 0110) P (1001 | 1001) Fig. 15. State transitions when either nod e A or B is the pro ducer of item d . 16 sta tes , only 8 states are v alid. Cyclon precludes a no de from storing its o wn link, eliminating sev en states (where o ne of the no des has state 11 ). In addition, the state 1010 , w her e D contacts itself, is a lso inv alid. Fig. 14 shows the tra nsitions b etw e e n states where D is not in volv ed. F or simplicity , w e a ssume that there are no link fa ilur es, resulting in a transition diagram simila r to what we would encounter for Sh uffle. Note that the four states depicted hav e the for m 0X0X, indicating that neither the initiator nor the gossip partner is D . When D is inv olved in the exc hange, there are only four p o s sible states, as shown in Fig. 1 5 . Since D a s the initiator alwa ys includes a link to itself in the set of items sent, the outcome will alw ays be that the other node has a link d . On the other hand, w he n D is the contacted no de (see b o ttom of figure), the initiator might or migh t not drop link d fro m its ca che as a result. In all cases, D alwa ys k e e ps the state 10 , a s it can never k eep a link to itself in its cache. Describing the transitions in terms of the building blo cks P sele ct and P dr op can b e do ne in a similar wa y as with Shuffle (when D is not inv olved in the exchange). The v ariations in the routines, sp ec ific a lly PrepareMsg() , due to the role taken by the no de in the exchange, ha ve to be given specia l consider ation, as they will cause P sele ct to b e slig ht ly differ ent for the initiator and the contacted no de. When no de D is inv olved, Cyclon is very clear ab out the outcome. The transitions can b e easily calcula ted. If node D is the initiator, there is only one p oss ible outcome (probability = 1 ). If no de D is the contacted no de, the transitions dep end so lely on P dr op . 7.3 Other Proto cols As mentioned b efore, in the original version of Newsca st, data items are times- tampe d a ccording to their creation time. A no de allo cates the space in its cache for the received items b y discarding corr esp onding num b er of oldest items. If an item is received that is also in the no de’s cache, the recent version of the item is 21 preferred. Lik e for Cyclon, a state o f the trans ition diagr am of Newscast is then a pa ir of tuples ( a, b ), where a, b ∈ { 0 , (1 , i ) } . The first bit indicates if the data item is pr esent in the gos siping no de’s cache. The integer i is a timestamp of the c o rresp onding item. The express io n for proba bilit y P sele ct remains the same, but P dr op is then a function of the corr esp onding timestamp. It can b e either derived thr ough rigo rous analysis, o r can b e measured exp erimentally similar to the approa ch presented in Sec . 6. 8 Conclusions T raditional analysis metho ds for computer systems such a s mo del chec king with its exhaustiv e state spa ce search, fail to cop e with larg e netw o rks. Alt hough quite useful for s tudying certain b ehavior of small- sized net w orks, these metho ds do not scale w ell for go ssip proto cols . O n the other hand, s tandard metho ds for mo delling of epidemics scale well, but abstra ct aw ay the details of a proto co l, showing only simple emergent b ehavior of the system. A challenge is to develop analytical mo dels that ca pture (part of ) the b ehavior of a system, and then subsequently o ptimize design parameters, at the right level of abstra c tio n. In this paper , we presented an analytical framework for a class of gossip-based information dissemination pro to cols. Mo delling a gos sip proto col at the level of lo cal pa irwise int eractions, as demonstrated in this pa p er, is a scala ble appr oach to analyze suc h proto cols. On the one hand, this analytical model includes the mechanics of communication b etw een no des on the level of the proto col details . On the other hand, such a mo del allows for studying the impact of system parameters on the per formance of the proto col, and can b e used to optimally design and fine-tune it. F urthermore, we hav e presented the analysis o f a gossip pro to col in the pres - ence of transient communication failure s. F or o ur fra mework, we introduced a hybrid metho d to compute P dr op that co mbines b oth rigoro us mo delling of the proto col and statistical data sa mpling from large- s cale Mo nte Car lo simulations for differe nt netw ork top olo gies. T o b e mo re precise, we derive analytical ex pres- sions for comp onents of our framework that are in v a riant with r e s p e ct to the scenarios w e wan t to mo del. Having decomp osed our model into its basic c o m- po nents, w e identify the ones that are affected by specific scena rio configuratio ns (in this case, message loss and top olo gy c ombinations). Finding analy tical ex- pressions for these comp onents (in this work, only P inx ) would require modelling each sp ec ific sce nario config uration, r educing the applicability o f the framework to those v e ry sp ecific co nfigurations. Instead, w e isolate the fr a mework from the scenario- sp ecific comp onent and use s tatistical data sampling to obtain a v alue for it. W e opt for this combined approach since it allows us to build a framework that captures the b ehavior of the gos sip protoc ol without requiring us to inco r - po rate scenar io-sp ecific elements into the model. The challenge in this approach lies in being able to decomp ose the model into its minimal co mpo nents, in such a wa y that the ones which a re dependent on par ticular scenarios (for which ex- 22 pressions that encompass all scenario s cannot b e derived) c an b e isolated and computed sepa r ately . In effect, we striv e for a golden mean b etw een high-level mo dels s uch a s for epidemics s howing o nly the emergent behavior and the low- level mo dels of the pr oto col that depend on particular implemen tatio n settings. With respec t to gossip-based diss e mination, our study revealed that for net- works with lossy c o mmunication c hannels , the assumption of a unifor m distri- bution of data is v alid only if ev ery node can gossip with an y other no de in the net work. In future, we plan to study the impact of different goss iping frequency on the p erformance r e sults. References 1. A. Allav ena, A. Demers, and J. Hop croft. Co rrectness of a gossip based membership protocol. In Pr o c. of PODC , pages 292–30 1. ACM , 2005 . 2. R. Ba khshi. Gossiping Mo dels: F ormal Analysis of Epidemic Pr oto c ols . PhD thesis, V rije Universiteit Amsterdam, 2011. http://hdl.han dle.net/18 71/18387 . 3. R. Bakhshi, L. Cloth, W. F okk ink, and B. R. Hav erkort. Mean-field framewo rk for p erformance ev aluation of push -pull gossip proto cols. Performanc e Evaluation , 68(2):157– 179, 2011. 4. R. Bakhshi, J. End rullis, S. Endru llis, W. F okkink, and B. Haver kort. A utomating the mean-field metho d for large dynamic gossip netw orks. In Pr o c. of QEST , pages 241–250 . IEEE Computer So ciet y , 2010. 5. R. Bakhshi and A. F ehnker. On the impact of modelling c hoices for distributed information s pread. In Pr o c. of QEST , pages 41–50 . IEEE Computer Society , 2009. 6. R. Bakhshi, D. Ga vidia, W. F okkink, and M. va n S teen. An analytical model of information dissemination for a gossip-based protocol. Comp. Netw. , 53(13):228 8– 2303, 2009. 7. S. Bo yd , A. Ghosh, B. Prabhak ar, and D. Shah. Randomized gossi p algorithms. IEEE/ACM T r ans. on Networking (TON) , 14(SI):2508–25 30, 2006. 8. Y. Bu sn el, R . Beraldi, and R. Baldoni. A formal characterizatio n of u niform p eer sampling based on view shuffling. In Pr o c. of PDCA T , pages 360–36 5, 2009. Ex- tended version to appear in JPDC. 9. P . Costa, V. Gramoli, M. Jelasit y , G. P . Jesi, E. Le Merrer, A. Montreso r, and L. Querzoni. Ex ploring the in terd isciplinary connections of gossip-based systems. ACM SIGOPS Op er. Syst . R ev. , 41(5):51– 60, 2007. 10. P . Crouzen, J. va n de P ol, and A. R ensink. Applying formal metho ds to gos- siping netw orks with mCRL and GROO VE. A CM SIGMETRICS Performanc e Evaluation R eview , 36(3):7–16, 2008. 11. A. D emers, D . Greene, C. Hauser, W. Irish, J. Larson, S. Shenker, H . Sturgis, D. Swinehart, and D. T erry . Epidemic algori thms for replicated database main te- nance. In Pr o c. of PODC , pages 1–12 . ACM Press, 1987. 12. A. Dimakis, A. Sarw ate, and M. W ainw right. Geographic gossi p: efficient aggre- gation for sensor netw orks. In Pr o c. of IPSN , pages 69–7 6. AC M Press, 2006. 13. N. D rost, E. Ogston, R. v an Nieuwpo ort, and H. Bal. ARRG: Real-W orld Gossip- ing. In Pr o c. of HPD C , pages 147–1 58. ACM, 2007. 14. A. F ehnker an d P . Gao. F ormal verification and simulati on for p erformance an alysis for probabilistic broadcast protocols. In Pr o c. of ADHOC-NOW , volume 4104 of LNCS , pages 128–1 41. Springer, 2006 . 23 15. D. Gavidia, S. V oulgaris, and M. va n Steen. A gossip-based distributed news service for wireless mesh netw ork s. In Pr o c. of WONS , pages 59–67. IEEE, 2006. 16. M. Gurevich and I . Keidar. Correctness of goss ip-based membership under message loss. In Pr o c. of PO DC , pages 151–16 0. AC M, 2009 . 17. M. Jelasit y , W. Kow alczyk, and M. v an Steen. N ewscast computing. T echnical Rep ort IR - CS-006, V rije Universiteit, 2003. 18. M. Jelasit y , W. Kow alczyk, and M. v an Steen. N ewscast computing. T echnical Rep ort IR - CS-006, V rije Universiteit Amsterdam, 2003. 19. M. Jelasity , A. Mon tresor, G. P . Jesi, and S. V oulgaris. PeerSim: A p eer-to-p eer sim u lator. http://peersim. sourcefor ge.net/ . 20. M. Jelasity , S. V oulgaris, R. Guerraoui, A .-M. Kermarrec, and M. v an Steen. Gossip-based p eer sampling. ACM T r ans. on Com put. Syst. , 25(3):8, 2007 . 21. M.-J. Lin and K. Marzullo . Directional gossip: Gossi p in a wide area n etw ork. In Pr o c. of EDCC-3 , pages 364–379. Springer, 1999. 22. A. Stavrou, D. R ub enstein, and S . S ahu. A light weigh t, robu st p2p system t o handle flash cro wds. In Pr o c. of ICNP , pages 226–235. IEEE, 2002. 23. A. T anenbaum and M. v an Steen. Distribute d Systems: Principles and Par adigms . Prenti ce Hall, 2007. 24. S. V oulgaris, D. Ga v idia, and M. v an Steen. Cyclon: Inexp ensive membership man- agemen t for unstru ct ured p2p o verl a ys. J. Network and Syst. Manage. , 13(2):197– 217, 2005. A Pull-only and push-onl y v ariations Our framework is not limited to push-pull gossip proto cols, but allows for mod- elling of push-only as w ell pull-only go ssip proto cols. T o illustra te this, we use t wo v a r iations o f the simplified Newscast, and briefly demonstrate the calculation of transition probabilities for push and pull versions of the protoco l. Push-only proto col Recall that in push-based g ossip pro to cols, only the initia tor no de A sends its message to B , but do e s not up da te its own c a che. Ther e fore, the tra ns itions where A changes its state are not p oss ible a nymore, and the pr obability o f s uch transitions is 0. The tra nsition pro bability P (00 | 00 ) = 1 since items do not app ear from nowhere. State (0 , 1) Before gossip, only B has d in its cache. a 2 b 2 = 01 : node B do es no t drop d from its cache; P (0 1 | 01) = P ¬ dr op . a 2 b 2 = 00 : node B drops d during the update o f its cache; P (00 | 01 ) = P dr op . State (1 , 0) Before gossip, d is in the cache of A . a 2 b 2 = 10 : node A either do es not send d , o r sends it, but B dr ops d , i.e., the probability is P (10 | 1 0) = P ¬ select + P sele ct · P dr op . a 2 b 2 = 11 : node A sends the item d , and no de B keeps it: P (11 | 1 0) = P sele ct · P ¬ dr op . State (1 , 1) Before gossip, d is in the caches of bo th nodes. a 2 b 2 = 10 : node B drops d from its cac he during the up date; P (10 | 11 ) = P dr op . a 2 b 2 = 11 : node B keeps the item d after the up date; i.e., P (11 | 1 1) = P ¬ dr op . 24 Pull-only proto col In contrast to the push version, in pull-based gossip proto cols the initiator no de only requests data from its p eer . In terms of state transitions , only initiator A can up date its state, but not the contacted no de B . Th us , the pro bability of transitions in which the state of B b efore go s sip differs fr o m its state after the g ossip is 0. The state (0 , 0) again has a s elf-transition with pr obability 1: P (00 | 00) = 1. State (0 , 1) Before gossip, only B has d in its cache. a 2 b 2 = 01 : node A do es not get d from B , or B sends the item, but A dr o ps it; P (01 | 01) = P ¬ select + P sele ct · P dr op . a 2 b 2 = 11 : node B sends d and no de A keeps it; P (11 | 0 1) = P sele ct · P ¬ dr op . State (1 , 0) Before gossip, d is in the cache of A . a 2 b 2 = 10 : node A do es not drop d a fter its cache up date, i.e ., the proba bility is P (10 | 10) = P ¬ dr op . a 2 b 2 = 00 : node A does no t keep the item for its up dated ca che: P (00 | 10) = P dr op . State (1 , 1) Before gossip, d is in the caches of bo th nodes. a 2 b 2 = 01 : node A drops d from its cache during the up date; P (01 | 11) = P dr op . a 2 b 2 = 11 : node A k ee ps the item d after the up da te with the probabilit y P (11 | 11 ) = P ¬ dr op . 25