Spatial Interactions of Peers and Performance of File Sharing Systems

Spatial Interactions of Peers and Performance of File Sharing Systems Franc ¸ ois Baccelli INRIA– ´ ENS France Fabien Mathieu INRIA–Univ ersity Paris 7 France Ilkka Norros VTT Finland Abstract —W e propose a new model for peer -to-peer network- ing which takes the network bottlenecks into account bey ond the access. This model allows one to cope with key features of P2P netw orking lik e degr ee or locality constraints or the fact that distant peers often ha ve a smaller rate than nearby peers. W e show that the spatial point process describing peers in their steady state then exhibits an interesting r epulsion phenomenon. W e analyze two asymptotic regimes of the peer-to-peer network: the fluid regime and the hard–core regime. W e get closed form expressions for the mean (and in some cases the law) of the peer latency and the download rate obtained by a peer as well as for the spatial density of peers in the steady state of each regime, as well as an accurate approximation that holds f or all regimes. The analytical r esults are based on a mix of mathematical analysis and dimensional analysis and ha ve important design implications. The first of them is the existence of a setting where the equilibrium mean latency is a decreasing function of the load, a phenomenon that we call super -scalability . I . I N T RO D U C T I O N Peer-to-peer (P2P) architectures have been widely used over the Internet in the last decade. The main feature of P2P is that it uses the av ailable resources of participating end users. In the field of content distribution (file sharing, liv e or on- demand streaming), the P2P paradigm has been widely used to quickly deploy low-cost, scalable, decentralized architec- tures. For instance, the ideas and success of BitT orrent [1] hav e sho wn that distrib uted file-sharing protocols can provide practically unbounded scalability of performance. Although there are currently many other architectures that compete with P2P (dedicated Content Distribution Networks, Cloud-based solutions, . . . ), P2P is still unchallenged with respect to its low-cost and scalability features, and remains a major actor in the field of content distribution. The Achilles’ heel of todays’ P2P content distribution is the access upload bandwidth, as ev en high-speed Internet access connections are often asymmetric with a relati vely low uplink capacity . Therefore, most theoretical models of P2P content distribution presented so f ar hav e been ‘traditional’ in the sense of assuming a common, relatively low access bandwidth, in particular concerning the upload direction, which functions as the main performance bottleneck. Howe ver , in a near future the deployment of very high speed access (e.g. FTTH) will challenge the justification of this assumption. This raises the need of ne w P2P models that describe what happens when the access is not necessarily the main/only bottleneck and that allow one to better understand the fundamental limitations of P2P . A. Contributions A new model. The first contrib ution of the present paper is the model presented in Section III, which features the following two ke y ingredients which were lacking in previ- ous models of the literature on P2P dynamics: 1) a spatial component thanks to which the topology of the peer locations is used to determine their interactions and their pairwise exchange throughput; 2) a networking component allo wing one to represent the capacity of the network elements as well as the transport protocols used by the peers and to determine the actual exchange throughput between them. More precisely , we consider a scenario where peers randomly appear in some metric space, typically the Euclidean plane representing the physical distance, and download from their neighbors with a throughput that may depend on some distance or R TT (it can be the case for e.g. TCP transport). The typical P2P application we hav e in mind is a BitT orrent-like file-sharing system. Ho wev er , the high abstraction lev el of our model also allows for interpretations beyond this framew ork. Using proper QoS requirements, it could be extended to any kind of P2P content distribution services (like li ve and on-demand streaming). The space could also be a representation of the peers’ interests, the position of a peer representing its own centers of interest. In such a space, two close peers share common interests, and therefore are likely to exchange more data. A promising form of scalability . The rationale that is usually brought forward to explain P2P scalability is that the ov erall service capacity gro wths with the number of peers. This allows the system to reach an equilibrium point no matters ho w popular the service is. This equilibrium was first analytically studied in [2], under the traditional assumption mentioned abov e that the upload/download capacity is the bottleneck determining the exchange throughput obtained by peers. The model proposed in [2] leads to an equilibrium point which exhibits the expected scaling property in that the service latency can be sho wn to remain constant when the system load increases. In our new model, the equilibrium point may exhibit a stronger form of scalability than that in [2], that we propose to call super -scalability , where the service latency actually decr eases with the system load. Conditions for super -scalability to hold. As we shall see in Sections II and IV, this super-scalability phenomenon is not difficult to understand from a pure queuing theory or graph theory viewpoint. Roughly speaking, super-scalability can be shown to hold in a queue whenev er the service rate of a typical customer scales like the number of customers in the system (rather than like a constant as in [2]). Equiv alently , it is not difficult to see that it holds if the peer interaction graph is complete at any gi ven time. Howe ver , in practice, the network cannot sustain arbitrary high rates. Also, interactions between peers are limited by degree constraints and by the requirement to select peer connections with good throughput. Section VI combines our model together with an abstract network model to determine the conditions on the peering rules, on the network capacity and on the transport protocols for which the mathematical analysis makes sense and for which the super-scalability property can possibly surviv e. The laws of super -scalability . The paper also provides a full analytical quantification of the system at the equilibrium point: in addition to the latency formula, it also provides closed form expressions for e.g. the density of peers present in the P2P ov erlay or the rate obtained by each peer , as functions of the peering rules and the network parameters. These equilibrium laws, which take specific forms for each type of transport protocol, are the main analytical contributions of the paper . These are gathered in Section IV for the simplest scenarios and in Sections VII and VIII for a few variants that can be b uilt on our model: generic rate functions, auxiliary servers, seeding beha vior of users, access bottleneck condition, etc. These laws have important P2P implications. In particular , they allow one to determine optimal tuning of the parameters of the P2P algorithms e.g. the optimal peering degree or the best parameters of the transport protocols to be used within this context. One theoretically nov el feature of our model is the proof of a r epulsion phenomenon which was empirically observed in [3]: as close peers get faster rates, they quit the system earlier , so a node “sees” fewer peers in its immediate vicinity than one would expect by considering the spatial entrance distribution alone. All these results are validated through simulations in Section V. B. Related W ork Our main scenario is inspired by a BitT orrent-like file- sharing protocol. In BitT orrent [1], a file is segmented into small chunks and each do wnloader (called leecher ) exchanges chunks with its neighbors in a peer-to-peer overlay network. A peer may continue to distribute chunks after it has completed its o wn do wnload (it is called a seeder then). Theoretical studies and modeling have already provided relati vely good understanding of BitT orrent performance. Qiu and Srikant [2] analyzed the effecti veness of P2P file- sharing with a simple dynamic system model, focusing on the dynamics of leechers and seeders. Massouli and V ojno vic [4] proposed an elegantly abstracted stochastic chunk-level model of uncoordinated file-sharing. In the case of non- altruistic peers (who do not continue as seeders), their results indicated that if the system has high input rate and starts with a large and chunk-wise sufficiently balanced population, it may perform well very long times without any seeder . Howe ver , instability may be encountered in the form of the “missing piece syndrome” identified by Mathieu and Reynier [5], where one (and exactly one!) chunk keeps existing in very few copies while the peer population gro ws unboundedly . Hajek and Zhu [6], [7] proved that the syndrome is unavoidable, if the non- altruistic peers enter empty-handed and if the peer arriv al rate is lar ger than the chunk upload rate offered by persistent seeders. On the other hand, they also proved that the system becomes stable for an y input rate, if the peers ha ve enough altruism to stay as seeders as long as it takes to upload one chunk. The missing piece syndrome can be avoided ev en in the case of non-altruistic peers by using more sophisticated download policies at the cost of somewhat increased download times, see [8], [9], [10]. The above results were obtained in a homogeneous, potentially fully connected network model. The present paper introduces a much less trivial family of peer interaction models, focusing on a bandwidth-centered approach similar to the one proposed by Benbadis et al. [11]. T o av oid excessi ve layers of complexity , we neglect chunk- lev el modeling in this phase, although realizing that meeting the rare chunk problem will modify and enrich the picture in future research. The natural feature of large variation of transfer speeds in P2P systems has been considered in a large number of papers. For e xample, part of the peers can rely on cellular network access that is an order of magnitude slower than fixed network access used by the other part. Such scenarios differ howe ver substantially from our model, where the transfer speeds depend on pair-wise distances b ut not on the nodes as such. There are some earlier papers considering P2P systems in a spatial framework. As an example, Susitaiv al et al. [12] assume that the peers are randomly placed on a sphere, and compare nearest peer selection with random peer selection in terms of resource usage proportional to distance. Howe ver , the distance has no ef fect on transfer speed in their model. Our paper seems to be the first where a peer’ s downloading rate is a function of its distances to other peers. I I . S U P E R – S C A L A B I L I T Y T O Y E X A M P L E Consider a system in steady state where jobs arri ve to get some service. This system will be said to be super-scalable if the mean job latency decreases when the arri val rate increases and all other system parameters remain fixed. In order to understand how super-scalability can arise, we propose the follo wing two toy examples: consider a system where peers arriv e and want to do wnload some file of size F . Peers arrive in the system with intensity λ and leav e the system as soon as their own download is completed. In our first toy example, the access upload bandwidth is considered as the main bottleneck. If we neglect issues related to data/chunk availability , and if U is the typical upload bandwidth of a peer, then it makes sense to assume that U is also the typical download throughput experienced by each peer . In particular, in the steady state (if any), the mean latency W and the a verage number of peers N should be such that W = F U and N = λW = λF U (Little’ s Law). (1) Although very simple, (1) contains a core property of standard P2P systems: the mean latenc y is independent of the arriv al rate. This is the scalability property , which is one of the main motiv ations for using P2P . Now , imagine a second toy example based on a complete shift of the bottleneck paradigm. Let the main resource bot- tleneck be the (logical, directed) links between nodes instead of the nodes themselv es. W e should then consider the typical bandwidth C fr om one peer to another as the ke y limitation. If each peer is connected to e very other one (the interaction graph is complete at any time), then the equilibrium Equation (1) should be replaced by W = F ( N − 1) C and N = λW , which leads to N = r λF C + 1 4 + 1 2 and W = s F λC +  1 2 λ  2 + 1 2 λ . For λF C  1 , this can be approximated by N ≈ r λF C and W ≈ r F λC . (2) The behavior of this new system is quite dif ferent from the previous one. Among other things, the service time is no w in versely proportional to the square root of the arriv al intensity , so that super-scalability holds. In this to y example, the central reason for super -scalability is rather obvious: the number of edges in a complete graph is of the order of the square of the number of nodes, and so is the ov erall service capacity . The main question addressed in the present paper is to better understand the fundamental limitations of P2P systems and in particular to check whether super-scalability can possibly hold in future, network-limited, P2P systems, where the throughput between peers will be determined by transport protocols and network resource limitations rather than the upload capacity alone. This requires the definition of a ne w model allowing one to take both the latter and the former into account as well as the limitations inherent to P2P ov erlays like e.g. the constraints on the degree of the peering graph, the av ailability of data/chunks, etc. I I I . N E T W O R K L I M I T E D P 2 P S Y S T E M S The aim of this section is to define a model meeting all the abov e requirements. A. Dynamics Our peers liv e in a spatial domain D . The domain can be some general Euclidean or ev en abstract metric space. It can describe physical distance between peers, distances deri ved from metrics in the underlying physical network, or ev en represent some semantic space. For simplicity , we focus on a basic model where D is the Euclidean plane R 2 , but there is no basic difficulty in extend- ing this framework. W e also use sometimes an arbitrarily large torus as an approximation of D . Assume that new peers arrive according to some time-space random process. The set of the positions of peers present at time t is denoted by Φ t . Each peer p has an individual service requirement F p > 0 . In the basic example where the service required by e very peer consists of downloading one and the same file, F p would most naturally be modeled as a constant F describing the size of the file. W e assume that two peers at locations x and y serve each other at rate f ( || x − y || ) , where f is a non-negativ e function which we call the bit rate function of the model 1 . This function describes the network transport and connectivity limitations. W e will see later ho w these limitations can be taken into account. In order to focus on bandwidth aspects, we do not explicitly take into account issues related to chunk av ailability . Follo w- ing the approach proposed by [2], we assume that filesharing effecti veness can be affected by some factor η ≤ 1 because sometimes, a peer may not hav e any chunk that a neighbor would want. In the following, we omit η by assuming that file sizes are always scaled by a factor 1 η . W e are aware, ho wev er , that handling chunk av ailability through a constant η has some limitations, and we will point out the scenarios where chunk av ailability can become a real issue. The services recei ved from se veral peers are additi ve, so that the total download rate of a peer at x is µ ( x, φ t ) = X y ∈ φ t \{ x } f ( || x − y || ) . By symmetry , µ ( x, φ t ) is also the upload rate of a peer at x . In order for the access not to be a further limitation, the access capacity of a peer at x should exceed µ ( x, φ t ) . This is our default assumption here (access as a possible bottleneck is considered in Section VIII). A peer p born at point x p at time t p leav es the system when its service requirement has been fulfilled, i.e. at time τ p = inf { t > t p : Z t t p µ ( x p , φ s ) ds ≥ F p } . A peer is usually called a leecher if it has not completed its download, and seeder if it has. Although this paper is mainly focused on leechers-only system, the situation where peers 1 W e implicitly assume that bandwidth rates are automatically adjusted by the system, at the network layer, in a TCP-like fashion, or at the applicative layer , using a UDP-like approach. continue as seeders after having completed their service will be considered in Section VIII. B. Examples of Bit Rate Functions W e will consider two basic cases throughout the paper: 1) peers use a TCP-like congestion control mechanism; 2) peers use UDP . In P2P , UDP is often used in place of TCP . Howe ver , P2P- ov er-UDP protocols try to be TCP–friendly [13], [14]: they are designed to respect TCP flows and actually mimic TCP 2 . Consider first the case where peers use TCP Reno. On the path between two peers, let ϑ denote the packet loss probability and R TT denote the round trip time. Then the square root formula [15] stipulates that the rate obtained on this path is ξ R TT √ ϑ , with ξ = ∼ 1 . 309 . Assuming the R TT to be proportional to distance r yields a transfer rate of the form g ( r ) = C r . (3) W e can refine (3) by assuming that R TT is not simply linear in r but some af fine function of it, namely R TT = ar + b , where a accounts for propagation delays in the Internet path and b accounts for the mean delay in the two access networks. Then the transfer rate between two peers with distance r becomes g ( r ) = C r + q , with C = ξ a √ ϑ , q = b a . (4) Another natural model is that where one accounts for an ov erhead cost of c bits per second. The transfer rate between two peers at distance r is then g ( r ) =  C r + q − c  + , with ( . ) + := max( ., 0) . (5) In the case where peers use UDP , on the path between two peers, the transfer rate is of the form g ( r ) = C , where C is a constant. (6) C. Connectivity Limitation Having specified some transfer rate function g , we notice that a peer cannot interact with all other peers of the overlay network: it would result in a full mesh ov erlay , impossible to handle for large networks. Therefore, peers usually limit their neighborhood, for instance by selecting only peers within a certain distance and/or by limiting its total number of neighbors. This constraint is ev en more meaningful in the wireless contex, as it can correspond to some transmission range. This leads to the following choices for the bit rate function: • Constant Range model: take f ( r ) = g ( r )1 r ≤ R ( R is called the range ), so that µ ( x, Φ) = X x i ∈ Φ ,x i 6 = x 1 || x i − x ||≤ R g ( || x i − x || ) , (7) 2 For instance, TFRC (www .ietf.org/rfc/rfc3448.txt) recommends that UDP flows use the square root formula to predict the transfer rate that a TCP flow would get and use this rate for throttling their traffic. The TCP model is hence directly applicable to such a setting. Name Description Units C T CP Speed parameter bits · s − 1 · m C U DP Speed parameter bits · s − 1 F Mean file size bits R Peering range m λ Leecher arrival rate m − 2 · s − 1 W Mean latency s µ Mean rate bits · s − 1 U Upload bottleneck bits · s − 1 T ABLE I T A B L E O F N OTA T I O N where g is one of the functions considered abov e. • Constant Number of Near est P eers model: take the L closest peers as the set of communicating neighbors. This rule is non-symmetric and difficult to deal with exactly . T o begin with, computing the ef fective rate f between to peers at x and y is not a function of || x − y || only , but of the configuration φ t . In this paper, the main model will be that where the transfer rate between two communicating peers is giv en by (3) or (6) and where the range is constant. More general rate functions (e.g. as defined in (4) and (5)) and an approximation of connectivity defined by the number of peers will be analyzed in Section VIII. Let us stress again that the frame work can be extended to more general metric spaces and/or to more general rate functions. For instance, in a noise limited wireless network of the Euclidean plane, it makes sense to assume that the rate between two peers at distance r is determined by some Signal to Noise Ratio condition and is hence proportional to log  1 + C r α  with α > 2 the path loss e xponent. Of course the additiv e assumption on the point-to-point rates only makes sense in rather particular cases (e.g. orthogonal channels) and more general models should be considered within this wireless setting. W e will not pursue the general wireless setting in the present paper . W e will howe ver consider more general rate functions than the above TCP and UDP functions in Section VIII including the above additi ve wireless setting, which will be referred to as the SNR model. D. Mathematical Assumptions W e assume that new peers arri ve according to a Poisson pro- cess with space-time intensity λ ( ‘P oisson rain’ ). λ , expressed in m − 2 .s − 1 , describes the birth rate of peers: the number of peer arriv als taking place in a domain of surface A (expressed in m 2 ) in an interval [ s, t ] (in seconds) is a Poisson random variable with parameter λA ( t − s ) . For the sake of mathematical tractability , we assume the F p ’ s to be independent and identically distributed random variables with finite expectation, denoted by F = E ( F p ) . More specifically , we assume in this paper that their common distribution is exponential of mean F in order to gain in mathematical tractability . Pr oposition 1: If the domain D in which the peers liv e is compact, then φ t is a Markov process which is ergodic for any birth rate λ > 0 . The proof, which can be found in Appendix X, is based on a domination argument which can easily be extended to unbounded domains. The e xistence of stationary regimes for φ t in the case of an unbounded domain then follows from this and a tightness argument. Howe ver , the ergodicity of φ t and the uniqueness of its stationary regimes cannot be established as easily in this case. Garcia and Kurtz [16] prov ed the e xistence and er godicity of a wide class of attr active spatial birth-and-death processes in infinite domains. Extending their approach to our repulsive case (see below for the terminology) seems feasible b ut goes way beyond what can be done within the space limitations of the present paper . In what follows, for results stated on the (infinite) Euclidean plane case, we conjecture that the spatial birth-and-death processes of interest admit a unique stationary regime. In any case, all our results can be rephrased on a large torus where this conjecture is not needed. I V . M A T H E M A T I C A L A N A L Y S I S In this section, we focus on the main model under TCP (3) with fixed range R . The results on UDP (6) are provided as well. W e adopt the same strategy concerning the proofs as abov e for Proposition 1: we give proofs in the torus case, so as to provide the main ideas, but refrain from discussing their extensions to the infinite Euclidean plane. The limiting argu- ments for these extensions are left for future work. The final formulas are howe ver alw ays gi ven in the infinite Euclidean plane where they ha ve a particularly simple form. For the main model, the system has 4 basic parameters: the range R in meters ( m ), the typical filesize F in bits, the peer arriv al rate λ in m − 2 · s − 1 and a rate constant C in bits · m · s − 1 ( bits · s − 1 in the UDP case). According to Proposition 1, the model admits a steady state regime where the peers (in the basic model all leechers) form in R 2 a stationary and ergodic point process [17]. W e denote by β o the density of the peer (leecher) point process, by µ o the mean rate of a typical peer , by W o the mean latency of a typical peer , and by N o the mean number of peers in a ball of radius R around a typical peer , all in the steady state regime of the P2P dynamics. In the follo wing, we will also consider sev eral approxima- tions of the main model: • a fluid r egime/limit , where the corresponding quantities will be denoted by a f subscript (e.g. β f ); • a har d–core r e gime/limit for which we will use the notation . h (e.g. β h ); • a heuristic description of the main model with a hat notation (e.g. ˆ β 0 ) In any of these regimes, Little’ s law tells that the av erage density verifies β = λW . A. Fluid Limit The fluid limit consists in assuming that the density is uniformly distributed in space at any time. In particular, in the fluid limit, the presence of one single peer in a giv en point does not impact the system. From Campbell’ s formula [17], the mean total bit rate of a typical location of space (or equiv alently of a newcomer peer) is µ f = β f 2 π Z R r =0 ( C /r ) r dr = β f 2 π C R . (8) Now , the fluid limit hypothesis allows one to assume that a peer sees µ f during its whole lifetime. W e get that the mean latency of a peer is W f = F µ f . (9) Hence β µ = λF. (10) From (8), (9) and (10), we hav e β f = r λF 2 π C R , µ f = √ λF 2 π C R, W f = r F λ 2 π C R . (11) In the fluid limit, the mean number of peers in a ball of radius R around a typical peer is N f = π R 2 β f = r π 2 r λF R 3 C . (12) For UDP , the same reasoning gi ves: β f , UDP = q λF π C R 2 , µ f , UDP = √ λF π C R 2 , W f , UDP = q F λπ C R 2 , N f , UDP = √ π r λF R 2 C (13) As we see in the expression for the mean latency in (11) and (13) both the TCP and the UDP fluid limits exhibit the same super-scalability as the toy example: in spite of the fact that the interactions are not as in the complete graph and depend on the distance, the mean latency decreases in 1 √ λ when λ tends to infinity and ev erything else is fixed. B. Dimensional Analysis At this point of the paper , the fluid limit is a thought experi- ment, not necessarily related to the actual model. Dimensional analysis [18] helps to address this issue. W e first use the π -theorem [18] to strip our problem from redundant variables: if we choose R as a ne w distance unit, then the arriv al intensity becomes l = λR 2 , the download constant becomes c = C /R and the other parameters are unchanged. If we now define F as an information unit, then the download speed constant becomes c = C / ( RF ) and the other parameters are unchanged. Finally , if we take a time unit such that the download speed constant is 1 , we get a system where all parameters are equal to 1 but for the arriv al rate which is equal to l = λF R 3 C . As the system itself is not af fected by the choice of measurement units, all its properties only depend on the (dimensionless) parameter ρ = ρ TCP = λF R 3 C . (14) The π -theorem allows some freedom in the choice of the parameter . By noticing that N f = p π 2 √ ρ , we can use N f , which has a physical interpretation (the number of neighbors predicted by the fluid limit), instead of ρ . By similar arguments, we ha ve ρ UDP = λF R 2 C , (15) so we use N f , UDP = √ π √ ρ UDP . The π -theorem tells that all systems that share the same parameter N f are similar . No w consider the union of two independent systems that use the same parameters ( λ , F , C , R ): the real model, with latency W o , and the fluid model, with latency W f . The ratio W o W f is a dimensionless property of the ov erall system, therefore it is a function of N f only . In other words, there exists a dimensionless function M ( N f ) such that: W o = M ( N f ) W f . (16) From Little’ s law , we also deduce the density: β o = β f M ( N f ) . (17) These equations are true for both the TCP and UDP rates (with a different M function in each case). T o summarize, although our system may be subject to com- plex interactions and is defined by four independent parame- ters, dimensional analysis allows one to express its general behavior through a one-parameter function M (unknown), which expresses how far the real system is from its fluid limit. C. Fluid as a Bound W e no w giv e a better understanding of the behavior of the real system through the following theorem. Theor em 1 (Repulsion): In the steady state, E [ X x i ∈ Φ f ( || x i || )] ≥ E 0 [ X x i ∈ Φ \{ 0 } f ( || x i || )] , (18) where E 0 denotes expectation w .r .t. P 0 , the Palm probability [17] w .r .t. the point process Φ . The proof is giv en in Appendix XI in the torus case. Theorem 1 says that there are less points (in terms of their f –weight) in a ball of radius R around a typical peer (i.e. under the Palm probability) than in a ball of the same radius around a typical location of the Euclidean plane (i.e. under the stationary probability P ). This is what we call a repulsion ef fect. Cor ollary 1: M ≥ 1 . Pr oof of cor ollary: Theorem 1 is equiv alent to saying that µ o ≤ β o 2 π C R . This, the relation W o ≥ F /µ o (which is obtained by a direct con vexity argument) and Little’ s law β o = λW o imply that β o ≥ λ F β o 2 π C R which in turn implies β o ≥ β f and M ≥ 1 . In other words, repulsion implies that the fluid regime is actually a lower (resp. upper) bound for the mean latency and the peer density (resp. the mean rate). No w , the following theorem tells that the bound is tight. Theor em 2: When N f tends to infinity , M tends to 1 , and the law of a typical peer latency con ver ges weakly to an exponential random v ariable of parameter W f . The sketch of proof is given in Appendix XII, where it is shown that this regime is such that not only the traffic is high but the peers also stay long enough to make the fluctuations slow and weak. By the almost constancy of the rate at any point, we get the almost exponentiality . Theorem 2 says that when the number of neighbors pre- dicted by the fluid limit tends towards infinity , the system behav es like its fluid limit. D. Har d–Cor e Re gime A stationary point process is hard–cor e with exclusion radius R if there is no pair of points in the point process with a distance less than R . Conjectur e 1: When N f tends to 0, N f M ( N f ) tends to 1, and the stationary peer point process tends to a hard–core point process with exclusion radius R , with intensity β h and latency W h defined as follows: β h = 1 π R 2 , W h = 1 λπ R 2 , (19) Moreov er , the cdf of the latency con verges weakly to 1 − e − t 2 W h 2 , t > 0 . (20) Conjecture 1 is supported by simulations (cf. Section V), and by the following insight on the hard–core behavior: when two peers are at distance r ≤ R , the av erage time under TCP for one of them to disappear is less than rF C ≤ RF C . If N f  1 , (11), (12) and Corollary 1 giv e RF C  W o . In other words, when two peers are in range, one of them disappears almost instantly compared to the typical latency of the system, so when we take a snapshot of the system at a gi ven time and finite area of space, it is likely that we see only peers out of range R from each other . A similar reasoning stands for UDP . It is worthwhile mentioning that according to (19), the volume fraction of the associated sphere packing model is 1/4 (since we ha ve a density 1 π R 2 of non-intersecting balls of radius R/ 2 ). This volume fraction is hence the same as that of the Mat ´ ern hard–ball model in the so called jamming regime (see e.g. [19]). Let us stress that this hard-core regime is hardly desirable (performance largely belo w the one predicted by the fluid limit and extreme unf airness). Moreover , peer data exchanges are very sparse, so the fluid assumption on the exchange of chunks fails to hold. Chunk a vailability becomes probably a bottleneck as important as bandwidth under these conditions, suggesting that the performance will be even worse should we take chunk exchanges e xplicitly into account. For all these reasons, the hard–core regime, which we presented for completing the description of our model, should be avoided by all means. The discussions on design will hence be in part focused on the tuning of the system parameters that av oid this regime. E. Heuristic For intermediate values of N f , where fluid and hard–core limits do not apply , we propose a first order approximation. For TCP , it consists in approximating M by ˆ M , the unique solution in [1 , ∞ ) of ˆ M 2 1 − ˆ M 2 N f ln  1 + 2 N f ˆ M  ! = 1 . (21) In order to derive (21), we use a heuristic factorization of the factorial moment measure of order 3 [17] which is described in Appendix XIII. Informally , the method consists in computing an approximation ˆ u o of the a verage rate of a peer assuming that: (i) a neighbor at distance r from that peer “sees” a rate ˆ u o + C r ; (ii) in return, the peer “sees” at distance r a density of neighbors λF ˆ u o + C r (using (10)). Under this approximation, the fluid equation (8) no w be- comes µ o ≈ λF 2 π C R R 0 1 µ o + C r dr = λF 2 π C  R µ o − C µ 2 o ln(1 + µ o R C )  , (22) which leads to ˆ µ 2 o = µ 2 f  1 − C ˆ µ o R ln  1 + ˆ µ o R C  . (23) Using ˆ µ 0 = µ f ˆ M and noticing that µ f R C = 2 N f , equation (21) follows. This heuristic is in line with Theorem 2 and Conjecture 1. When N f tends to ∞ , it follows from (21) that ˆ M ∼ 1 . This is in line with Theorem 2. When N f tends to 0, expanding the log in (21) gives ˆ M ∼ 1 N f , which substantiates Conjecture 1. In the UDP case, the same heuristic leads to ˆ M = 1 ˆ M + 1 N f , so that (24) ˆ M = s 1 +  1 2 N f  2 + 1 2 N f , (25) which also supports both Theorem 2 and Conjecture 1. F . T oy Example Re visited W e revisit the example of Section II within the more precise framew ork considered in the present section (Poisson arriv als, exponential file size). This toy example can be seen as the UDP case on the torus, when the range is large enough for all pairs of peers to be within range. Assume the surface of the torus to be 1. Then, geometry disappears and we hav e a birth and death process for the total population with birth rate λ and death rate in state i equal to µ ( i ) = C F i ( i − 1) . The state space is that of positiv e integers. The local balance equations read π ( i − 1) λ = π ( i ) µi ( i − 1) , i ≥ 2 . The solution is π ( i ) = π (1) 1 i !( i − 1)! ρ i − 1 , i ≥ 2 , where ρ = λF C . Hence the mean is β = 1 + P i ≥ 2 1 (( i − 1)!) 2 ρ i − 1 1 + P i ≥ 2 1 i !( i − 1)! ρ i − 1 = √ ρ B (0 , 2 √ ρ ) B (1 , 2 √ ρ ) , with B the BesselI function. In words, we ha ve β = β f M ( N f ) with        β f = N f = r λF C and M ( X ) = B (0 , 2 X ) B (1 , 2 X ) . (26) W e recognize the approximation (2), which implies super - scalability in 1 √ λ , corrected with an M function like in (17). W e remark that for this toy example, we ha ve the exact v alue of M and not an asymptote or a heuristic, and we can verify that Corollary 1 and Theorem 2 hold. Notice also that if δ ( i ) = µi , then π ( i ) = ρ i i ! e − ρ for all i ≥ 0 so that N = ρ and W = 1 /µ . In this case, which in essence is that of [2], or equiv alently that of the M/M/ ∞ queue, where we hav e scalability but no super -scalability . V . S I M U L A T I O N R E S U L T S In this section, we validate our results and substantiate our results by means of simulations. For sake of computability , we approximate the infinite space by a torus of radius 1 . For concision, we only present here the simulation results for the TCP case, but UDP results are completely similar . As stated by the dimensional analysis, all systems can be described by the function M . The goal of the simulation is then to sample that function. W e just have to fix three independent parameters and use the fourth one to run through all possible scenarios. W e decide to choose the following fixed parameter: R = . 1 , which giv es a good trade-off between the torus as an approxi- mation of the plan; C = 1 (arbitrary choice); W f = 100 . The last choice means that remaining free parameters are adjusted so that (11) values to 100 in each experiment. That way , for all simulations, the fluid model will predict the same mean latency , so the measured latencies will gi ve M directly , up to a constant factor W f . W e naturally use N f (defined by (12)) as the variable parameter . W e use N f instead of ρ as main dimensionless parameter because it is strictly equiv alent from the point of view of dimensional analysis, yet it gives a direct meaning to the variable (av erage number of neighbors in the fluid model). The remaining input parameters of the system are then completely defined: λ = N f π R 2 W f , F = 2 N f C W f R . (27) W e choose to use a discrete time simulator, with elementary time step set to τ = 1 . W ith our settings, the resulting step transitions are empirically small enough for the discrete model to be a good approximation of the continuous model. In the end, we get a simulator that achiev es the needed trade- off between speed and accuracy (an event-based simulator, 10 −1 10 0 10 1 10 0 10 1 N f M ( N f ) From simulation Hard core Heuristic Fluid Fig. 1. M ( N f ) in the TCP case. for instance, would give exact rendering of the continuous model but would require a lot more of computation). For each considered setting, the simulation runtime was adjusted so that about 20000 peers could be observed in stationary state. All results presented are obtained through 10 runs per setting. A. Pr operties of the M Function W e propose to start with a global study of the function M . W e made simulations for N f varying from 1 / 32 to 64 . Results are displayed Figure 1. The empirical results are compared with 1) the fluid limit, 1 , 2) the hard–core limit, 1 N f , and 3) the heuristic formula (21). Figure 1 allo ws us to check almost all results from previous section in one look: • the fluid limit is a lower bound of the actual system (which is equiv alent to Theorem 1); • as N f goes to ∞ , the fluid bound becomes tight (this is Theorem 2); • as N f goes to 0 , the system behavior con ver ges tow ards the hard–core limit (this is Conjecture 1). Additionally , one checks that the heuristic (21) gives a good approximation of M for intermediate v alues of N f , while con ver ging to the hard–core and fluid limits when N f goes to 0 and ∞ respectiv ely . B. Fluid Model W e now propose to focus on the case N f = 64 , in order to analyze the system in detail when it reaches the fluid limit. The value M (64) gi ven by simulations is 1 . 007 , which is higher than 1 yet very close to it, as predicted by Theorem 2. If one looks at the latency distribution, it is almost in- distinguishable from an exponential distribution of mean W f (Figure 2) as predicted by Theorem 2. 0 100 200 300 400 500 600 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Latency CDF From simulation Fluid exponential Fig. 2. Cdf of latency for N f = 64 . In the fluid model, it is quite dif ficult to distinguish the sys- tem from a spatial birth and death process of birth parameter λ and death parameter 1 /W f , namely a Poisson point process of intensity β f . Differences can only be spotted if small distances are in volved. More precisely , two peers at distance r hav e a mutual latency influence of rF C , so one can expect Palm effect to become less visible when rF C is lar ge enough compared to W f . This allo ws us to sho w that R N f is the critical distance below which the Palm effects become dif ficult to neglect. For N f = 64 , this giv es R 64 ≈ 0 . 016 . In our case, the best way to differentiate the actual process from a Poisson process is to consider ho w far the closest neighbor of a peer is. While for a Poisson process the distance should be 1 2 √ λW f ≈ 0 . 0111 in average, simulation shows an actual average distance of 0 . 0115 : the nearest neighbor is slightly farther away by about 4% . If we go into detail by comparing the two distributions, it appears that the main gap appears for small distances (cf. Figure 3), which supports the concept of critical Palm distance: if a peer gets a very close neighbor , both rates will be higher than usual, so one of them is likely to lea ve sooner , lo wering the probability of finding very close neighbors in a random configuration. As N f tends tow ards ∞ , we expect this dif ference to become negligible: the probability to get a neighbor so near that it will significantly affect the total rate becomes arbitrary low , so the repulsion effect becomes negligible. C. Har d–Cor e Model W e conduct the same type of detailed study for N f = 1 / 32 . For these parameters, the value M (1 / 32) is now 31 . 6 , to compare with the hard–core model prediction M h = 32 ; so the accuracy of the model is pretty good. Figure 4 displays the latency distribution, using for compar- ison the hard–core distribution and the exponential distribution of parameter W o . One observes a close fit to the one proposed 0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Distance from nearest neighbor CDF From simulation Equivalent Poisson Fig. 3. Cdf of nearest neighbor distance for N f = 64 . by the distrib ution function (20) of Conjecture 1: when a peer arrives, with probability one half, it disappears instantly; otherwise it follows an exponential distrib ution of average 2 W h . In other words, not only the mean latency is much larger than in the fluid model (by a ratio 1 N f ), but half of the peers will get a service time arbitrary larger compared to the other half (as N f goes tow ards 0 ). The distrib ution of the closest neighbor is also of interest (cf. Figure 5); the distribution has been truncated to the maximal distance R , as a peer does not “see” beyond R . W e see here the repulsion ef fect at its paroxysm: there are many orders of magnitude between the empirical distribution and the equiv alent Poisson distribution. For instance, Poisson says that the probability to have at least one neighbor in range is 1 − e − λπ W R 2 ≈ 62 . 6% . In the stationary re gime, this probability is only 0 . 078% , whereas the hard–core conjecture tells us that it will continue to decrease as N f goes to 0 . D. Intermediate V alues W e hav e no good formal description of the actual laws observed for intermediate values of N f , these distributions show a compromise between the equi valent fluid and hard– core distributions. In order to compare with the fluid and hard-core limits, we giv e the latency distrib ution (Figure 6) and the closest neighbor distribution (Figure 7) for N f = 1 . One can see that these distributions sho w a compromise between the equi valent fluid and hard–core distributions. E. Summary of Simulations For both the fluid and hard–core limits, simulations v alidate that we have a good description of the av erage system perfor - mance defined by M , but also of the latency distribution. For intermediate states, although the bounds still hold, it is better to rely on the heuristic, which gi ves quite accurate results on M , but with no details on the distrib ution. 0 0.5 1 1.5 2 2.5 3 x 10 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Latency CDF From simulation Hard−core Equivalent exponential Fig. 4. Cdf of latency for N f = 1 / 32 . 0 0.02 0.04 0.06 0.08 0.1 10 −8 10 −7 10 −6 10 −5 10 −4 10 −3 10 −2 10 −1 10 0 Distance from nearest neighbor CDF From simulation Equivalent Poisson Fig. 5. Cdf of nearest neighbor for N f = 1 / 32 . V I . N E T W O R K C A PAC I T Y C O N S T R A I N T S The aim of this section is to determine the capacity required for the network elements in order to achiev e the super-scalable regime identified abo ve. More precisely , so far , the only assumptions on the network were that 1) the access is not a limitation anymore (or not the only bottleneck); 2) the network is a bottleneck, resulting into a rate between peers that depends on their distance and some range or degree constraints. This section introduces an abstract network model on which the P2P traffic will be mapped through some natural shortest path routing mechanism. W e then determine the mean flow that traverses a typical network element. This flow of course depends on the protocols used in the network which in turn determine the bit rate function. For simplicity , we limit the study to the fluid limit of the 0 100 200 300 400 500 600 700 800 900 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Latency CDF From simulation Equivalent hard−core Equivalent exponential Fig. 6. CDF of latency for N f = 1 . 0 0.02 0.04 0.06 0.08 0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Distance from nearest neighbor CDF From simulation Equivalent Poisson Fig. 7. CDF of closest neighbor for N f = 1 . system. A. Network Capacity Model W e consider an underlying network made of routers and links between them. A simple example is that where (i) routers form a Poisson point process of intensity θ in the plane; (ii) links are the Delaunay edges (see e.g. [19], Chapt. 4) on this point process; (iii) each peer is directly connected to the closest router and the path between two routers is the shortest path (with minimal hop count) on the Delaunay graph. In this case, the number of links between two peers is asymptotically proportional to the distance between them [19]. For all straight lines of the plane, the point process of intersections of this line with the edges forms a stationary point process of intensity µ e := 2 √ θ on the line. Denoting by K the capacity of an edge, we get a total capacity per unit distance of Ξ := µ e K . Now , in order to simplify the ev aluation of the P2P load on the underlying network, we will assume that (a) θ is large enough so that the hop-count between two peers can be seen as proportional to their distance and the flow between them as a straight line; (b) Any rate smaller than Ξ l can be transported through a segment of length l . Remark In order to further justify the formula f ( r ) = C r 1 r ≤ R for the rate of two peers at distance r within the refined network model presented above, one can use the bandwidth sharing formalism of [20]. A connection of Euclidean length r uses approximately γ r L links where L = 2 3 √ θ is the mean length of a Delaunay edge of a Poisson point process of intensity θ (see [21] p. 477) and where γ is the (stretch) constant of the shortest path algorithm (see [19], V ol 2, Prop 20.7). W e assume that each link is of capacity K . W e consider the network as an open bandwidth sharing network [20] with connections of various classes arriving to the network, transferring a file of mean size F and leaving the network. W e write the bandwidth optimization problem in any gi ven state in this network as max X i log( ν i ) under the constraints X i ∈ C j ν i ≤ K, where ν i is the rate of connection i and C j is the collection of connections that trav erse link j in this state. Denoting by α j the Lagrange multiplier associated with constraint j , we get that at the optimum point, for all i 1 ν i − X j : i ∈ C j α j = 0 . In the steady state regime (in both time and space), the sequence α j should be stationary and ergodic. So, when denoting by α its mean, when θ is large, car d { j : i ∈ C j } is large too and we get from spatial ergodicity that if connection i is of length r , namely uses γ r/ε ρ links, then ν i ≈ 1 α 1 l ( ν i ) , with l ( ν i ) = car d { j : i ∈ C j } ≈ γ r L . Hence ν i ≈ L 1 αγ 1 r for r ≤ R . B. Flow Equations For the sake of easy e xposition, we start with the model on the line. The flow through the origin is ψ = 2 E X X i ,X j ∈ Φ f ( | X i − X j | )1 X i < 0 1 X j > 0 . In the fluid model, we can use the fact that the second moment measure of Φ is β 2 times the Lebesgue measure on R 2 and Campbell’ s formula to get that ψ = 2 Z x< 0 Z y > 0 f ( y − x ) β dxβ dy = 2 β 2 Z r> 0 r f ( r ) dr. The last expression comes from the change of v ariables r := y − x, x := x . Consider now the model on the plane. Let X i = ( x 1 i , x 2 i ) . W e make here the assumption that the bit flow between any two peers follo ws a straight line in the plane, and that the network capacity is defined by some constant Ξ , expressed in bits.s − 1 .m − 1 , such that the maximal flo w rate that can go through a segment. Let Ψ( ε ) be the rate that goes through a segment S of length ε . W e can choose for instance S = [(0 , − ε 2 ) , (0 , ε 2 )] . Let H − denote the left half-plane and H + the right half plane. Then Ψ( ε ) is Ψ( ε ) = 2 E X X i ∈ Φ ∩ H − , X j ∈ Φ ∩ H + f ( | X i − X j | )1 [ X i ,X j ] ∩ S 6 = ∅ = 2 Z Z X ∈ H − , Y ∈ H + f ( | X − Y | )1 [ X,Y ] ∩ S 6 = ∅ β dX β d Y = 2 β 2 Z Z X ∈ H − , Z ∈ H + f ( | Z | )1 [ X,Z + X ] ∩ S 6 = ∅ dX d Z = 4 β 2 Z Z r > 0 , θ ∈ [0 , π 2 ] f ( r ) r sin( θ ) εr dr dθ = 4 β 2 ε Z r> 0 r 2 f ( r ) dr where the third line comes from the change of variables Z := Y − X , X := X . So, by isotropy , the flow per unit length through any line of the plane is Ψ = Ψ(1) = 4 β 2 Z r> 0 r 2 f ( r ) dr . Using the fluid expression of the density β = β f = s λF 2 π R r> 0 r f ( r ) dr we get the following key relation Ψ = Ψ(1) = 2 π λF R r> 0 r 2 f ( r ) dr R r> 0 r f ( r ) dr . (28) In the TCP case ( f ( r ) = C r 1 r ≤ R ), we get Ψ TCP = 2 C β 2 εR 2 = 1 π λF R. (29) In the UDP case ( f ( r ) = C 1 r ≤ R ), we get Ψ UDP = 4 3 C β 2 εR 3 = 4 3 π λF R. (30) For the netw ork to sustain the rate generated by our model, it is required that Ψ ≤ Ξ . (31) If one can assume, under some joint fluid limit, that both the flo w and the number of links going through a segment are asymptotically deterministic, then Condition (31) is also sufficient for stability . Studying the v alidity conditions of this hypothesis is, ho wev er , beyond the scope of this paper . Note that for both TCP and UDP , the condition (31) does not depend on C . This surprising result means that in the fluid limit, we can arbitrarily scale the individual rate of connections (thus decreasing the latency) without changing the b urden on the underlying network. Of course, there is a flaw in that reasoning, which is that increasing C e ventually impairs the v alidity of the fluid limit. In details, as C increases, N f gets smaller so we tend to the hard-core limit where (i) there is unfairness as half of the peers get almost instant service compared to the other half; (ii) the a verage latency reaches an asymptotic value 1 λπ R 2 , so further increase of C is meaningless. V I I . M O R E G E N E R A L R A T E F U N C T I O N S While we focused on TCP-lik e (3) and UDP-like (6) func- tions, all our results can easily be generalized in the fluid limit to any rate function f such that R r> 0 r f ( r ) dr < ∞ . Even if f has no maximal range R , we just hav e to replace C R in (8) by R r> 0 r f ( r ) dr and proceed. This gi ves µ f = β f γ , with γ = 2 π Z r> 0 r f ( r ) dr . (32) Once γ is known, we can generalize (11) by β f = s λF γ , µ f = p λF γ , W f = s F λγ . (33) Notice that the scaling in 1 √ λ still holds. W ithout a range R , N f , which is π R 2 β f , is not properly defined, which impairs a direct introduction of M . Howe ver , if we hav e R r> 0 r 2 f ( r ) dr < ∞ , we can use ˜ R := R r> 0 r 2 f ( r ) dr R r> 0 r f ( r ) dr o (34) instead of R and extend the dimensional analysis accordingly ( ˜ R being interpreted as the typical range of f ). Let us illustrate this method with a fe w concrete examples of type f ( r ) = g ( r )1 r ≤ R . A. Affine RTT If g is giv en by (4), then then the mean bit rate of a typical location of space is µ f = β 2 π Z R r =0 C r + q r dr = β f 2 π C  R − q ln  1 + R q  , so that we hav e γ = 2 π Z R r =0 C r + q r dr = 2 π C  R − q ln  1 + R q  . (35) B. Overhead For g as in (5), after noticing the necessary condition R ≤ C c (each connection needs to use a minimal bandwidth c for the ov erhead), we get µ f = β f 2 π Z R ∧ C c r =0  C r − c  r dr so that γ = 2 π Z R r =0  C r − c  r dr = 2 π  RC − R 2 c 2  . (36) The best value for R is R = C c , which giv es γ = πC 2 /c. C. P er Flow Rate Limitation The protocol or some physical constraints may limit the individual rates. If one assumes a maximal rate U for each flow , we have g ( x ) = ( C /x ) ∧ U . This giv es γ = 2 π Z R r =0  C r ∧ U  r dr = ( π U R 2 if C ≥ U R π  2 C R − C 2 U  otherwise. (37) W e find back (11) and (13) as special cases of (33) for U ≡ ∞ and U ≤ C R (up to notation for the latter). D. SNR W ireless Model The setting is that where the bit rate function is f ( r ) = 1 2 log  1 + C r α  1 r ≤ R (38) with α > 2 the path loss exponent, C the Signal to Noise Ratio at distance 1 and R the transmission range. In the case when R is finite, we will limit ourselves to the fluid case and to the special case where α = 4 (the reason for the las assumption being that the rele vant integral, namely R R 0 log  1 + C r α  r dr , can be then explicitly computed). In this case, direct computations giv e that γ = π  R 2 log(1 + C R 4 ) + √ C arctan( R 2 √ C )  . (39) The e valuation of the mean number of neighbors of a typical node, namely N f = π R 2 β f , allows one to identify the mean number of orthogonal channels per unit space required to cope with the P2P load, namely β f N f = π R 2 λF γ . (40) In an infinite plane, this would require an infinite number of orthogonal channels, which is of course not feasible. Howe ver , it then makes sense to reuse spectrum in this case, to the cost of an decrease of C (resulting from an increase of the noise power due to the presence of distant interference). In this sense, this scheme makes sense under appropriate spectrum bandwidth assumptions, in the same way as the TCP scheme mak es sense under appropriate network capacity assumptions. Notice that the integral R ∞ 0 log  1 + C r α  r dr is finite. This allows us to consider the wireless SNR model with an infinite range. In this case, the result is much simpler: for all α > 2 , γ = π 2 C 2 α 2 sin  2 π α  . (41) V I I I . E X T E N S I O N S O F T H E B A S I C M O D E L The aim of this section is to show that our analysis can be extended in several ways and tak e important practical phe- nomena into account. Unless otherwise stated, we will place ourselves in the fluid re gime, but the dimensional analysis approach can be used with all extensions to relate the fluid limit to the real system through some function M . The only cav eat is that if an extension introduces ne w parameters, M can be a function of several dimensionless variables instead of N f only . This is illustrated by our first extension. A. P ermanent Servers Assume that there exists some servers, or eternal seeders 3 . The motiv ation for considering this is for instance: (i) per- manent servers can solve the issue of chunk av ailability by being able to provide any asked chunk; (ii) this allows one to consider hybrid systems which combine classical server solutions and a P2P approach; (iii) with our model, the latency goes to ∞ when λ goes to 0 (cf. (19)), which is not a desirable effect; servers or permanent seeders seem a perfect solution to prev ent this. W e focus on the TCP case. The servers are characterized by their density of bitrate U C , expressed in bit.s − 1 .m − 2 , so that if β f is the peer density , a typical peer gets U C β f from the servers. T o describe the system, we need another dimensionless pa- rameter in addition to N f . W e con veniently choose χ C := U C λF . χ C expresses the ratio between the density of rate needed by the system and the density of rate pro vided by the servers. If χ C ≥ 1 , then the permanent rate from servers is suf ficient to serve the peers, otherwise P2P is needed for stability . Let us consider two limiting cases: the system is mainly client/server ( χ C  1 ), or the system is mainly P2P with a small server -assistance ( χ C  1 ). The case χ C  1 can be seen as a scenario where servers are here mainly for insuring chunk av ailability . If χ C  1 , then almost all resources come from the servers. W e can deduce that the point process is hard–core (ev en if N f is large, if it is fixed and if χ C grows, the servers can make newcomers lea ve before they hav e the occasion to reach another peer), so if a peer can collect all the a vailable bandwidth in its range, the av erage latency will be W C ≈ F π R 2 U C . (42) For χ C  1 , we focus on the fluid limit ( N f  1 ). Adapting (8), the rate of a peer is then µ f ,C = 2 π RC β f ,C + U C β f ,C , (43) from which we deduce W f ,C = s F − U C λ λ 2 π C R = W f p 1 − χ C ≈ W f . (44) 3 This is distinct from the case where leechers can seed for some time after they complete their download, which is addressed in VIII-D Let us point out that the behavior of (44) for χ C close to 1 is not expected to be realistic, as the impact of the client/server behavior becomes prominent. For the hard–core process, one could also express W h,C as something that tends to W h if χ C tends to 0 , which suggests that M C ( N f , χ C ) admits a limit M C ( N f , 0) = M ( N f ) when χ C tends to 0 . In words, the results presented in pre vious sections still hold if one assumes the existence of servers with relativ ely small bandwidth introduced to inject chunks into the system. B. Abandonment Here we consider the case where all leechers hav e some abandonment rate. Let a denote this rate. In the stationary state, we have λ = ( µ f F + a ) β f . From (8), we deduce µ 2 f + µ f aF = 2 π RC λF . The positiv e solution of this equation is µ f = s 2 π RC λF +  aF 2  2 − aF 2 . (45) The analysis can hence be extended without dif ficulties. F or instance, the abandonment ratio is giv en by aF µ f + aF . C. P er P eer Rate Limitation Due to the asymmetric nature of certain access networks (e.g. ADSL), the uplink rate is often the most important access rate limitation. Let U denote (here) the a verage upload capacity of a peer; then the average rate in the fluid limit should be such that µ f = √ λF 2 π C R ≤ U. (46) A natural dimensioning rule would then be to choose R = U 2 λF 2 π C in order to use all the av ailable capacity . D. Leechers and Seeders When a leecher has obtained all its chunks, it can become a seeder and remains such for a duration T S . In this setting, there is a density of seeders λT S in the stationary regime. In the fluid limit with seeders, (8) becomes µ f ,S = ( β f ,S + λT S )2 π C R . (47) Using (10) and F = W f ,S µ f ,S , we get W 2 f ,S + W f ,S T S = W 2 f . (48) The positiv e solution of this equation is W f ,S = s W 2 f +  T S 2  2 − T S 2 . (49) In particular , we hav e W f ,S ≈ W f for T S  W f and W f ,S ≈ W 2 f T S for T s  W f . By comparing (49) and (45), one can interpret seeding as the exact opposite of abandonment: seeders, which improve the system, impact the latency the same way that abandonment, which degrades the system, impacts the rate. W e also remark that in a fluid model where rates are only determined by the upload access, we hav e (see [11] for details) W f ,S + T S = W f . (50) W e can see (48) as the e xtension of (50) to the network-limited model. At last, we propose to study the hard–core limit. W ithout seeder , a leecher can leav e only if it finds a peer within range, and instant service happens with probability one half. W ith seeders, a leecher is certain to complete its do wnload if there is another peer in its neighborhood, as the latter will not leave the system before the former finishes. W e can then notice that the configuration of peers (leechers and seeders) includes a spatial Poisson distribution of density λT S . In particular , the probability for a newcomer to find a peer within range R is at least 1 − e − λT S π R 2 . Therefore, for any  > 0 , if T S ≥ − log(  ) λπ R 2 , then leechers will get instant service with a probability greater than 1 −  . This suggests that seeders may be a good antidote for systems where a hard–core beha vior cannot be avoided: a seeding time of the same order of magnitude than the average latency in absence of seeders is enough to guarantee that most of the peers get instant download. E. Adaptive Range Consider the constant number of near est peers model of Section III. In the fluid limit, which can be reached by increasing L until it identifies to N f , an approximate version of this model is obtained by considering a range model with radius R such that R , the density β and the target number of neighbors L verify π R 2 β = L. (51) In this case, µ ( x, Φ) is as in (7) but with R = q L π β . In this section, we consider a general model with R = κβ − α with α a real parameter . The constant radius ball corresponds to the case α = 0 and the L nearest neighbor case to α = 1 2 . Note that as β depends on R , R = κβ − α has to be seen as a fixed point equation for α > 0 . By dimensional analysis, one gets that for all α 6 = 1 2 , all properties of the system only depend on the parameter ρ = λF C κ 3 1 − 2 α . (52) For α = 1 2 (nearest peers), the parameter is ρ = κ (or equiv alently L ). The fluid analysis giv es µ f = 2 π C κβ 1 − α , so that β f =  λF 2 π C κ  1 2 − α W f = λ − 1 − α 2 − α F 1 2 − α (2 π C κ ) − 1 2 − α µ f = (2 π C κ ) 1 2 − α ( λF ) 1 − α 2 − α . (53) Notice that the algorithm which leads to this hence consists in choosing a radius of the form R = κ  λF 2 π C κ  α α − 2 . F or instance in the constant number of near est peers TCP case, we get W TCP =  F 2 C  2 3  1 π λL  1 3 . (54) This is an interesting result: it means that in the fluid limit, TCP can achieve super–scalability even if each peer has a limited number of neighbors. This is not the case for UDP , where the latency is W UDP = F LC (we still hav e scalability though). W e conclude this subsection by an asymptotic analysis where all parameters are fixed b ut for λ which tends to infinity . W e assume we are in the fluid regime (which will lead to some restrictions on the set of parameters). In vie w of (53), we will call d = 1 2 − α the density exponent , l = α − 1 2 − α the latency exponent and r = α / ( α − 2) the radius exponent . W e ha ve the conservation rule d − l = 1 , which is just a rephrasing of Little’ s law . Similarly N f = K λ 1 − 2 α 2 − α , with K a constant. So, for λ tending to ∞ , the fluid regime requires that either α > 2 or α < 1 2 . Hence, there are 2 regimes when λ → ∞ : • For α > 2 , (which corresponds to 1 < r < ∞ ) one gets at the same time d < 0 and l < 0 , which means a peer density and a latenc y which both tend to 0 when λ tends to ∞ . This is a rather surprising regime: the load per unit time and space tends to infinity; the density tends to 0 (there are no peers around for deliv ering service); nev ertheless, latency tends to 0 (i.e. when a peer arrives, it is instantly served by in visible peers located at infinity). W e will call this re gime Heaven’ s–flash . • For α < 1 2 (which corresponds to − 1 / 3 < r < 1 ), one gets d > 0 and l < 0 , which means a peer density that tends to infinity and a latency which tends to zero when λ tends to ∞ . This is the swarm–flash regime. Notice the possible existence of a critical–flash regime, with r = 1 , α = ∞ , d = 0 and l = − 1 , where the density is a constant and the latenc y tends to 0. Another interesting though critical case is that where α = 1 / 2 , where the structural properties of the system do not depend on λ anymore as shown by dimensional analysis. F . Mixed Extensions The proposed extensions, presented separately for sake of clarity , can easily be interleaved, at least in the fluid limit. For instance, combining (33) and (49), the average latency of a system with seeders and a rate function parameter γ (cf VII) is W f = s F λγ +  T S 2  2 − T S 2 , (55) In order to illustrate the fact that the above extensions are compatible, we analyze this case in the setting where the uplink limitation is taken into account. µ f = ( β f + λT S )2 π Z R r =0 ( C /r ) ∧ U r dr = ( β f + λT S ) ξ ( C , R, U ) . (56) From Little’ s law applied to the leechers, β f = λF /µ f . Hence β f ( β f + λT S ) = λF ξ ( C , R, U ) . The positiv e solution of this equation is β f = λT S 2 s 1 + 4 F λT 2 S ξ ( C , R, U ) − 1 ! , (57) which is an increasing function of λ . Since W f = β f /λ , W f = T S 2 s 1 + 4 F λT 2 S ξ ( C , R, U ) − 1 ! . (58) One can then mary this with the various ways of defining R as a function of λ . I X . C O N C L U S I O N The following general law quantifying P2P super-scalability was identified: in a P2P system with rate function g and range R , according to our model, the stationary latency is of the form W o = M s π 2 R 4 λF γ ! s F λγ , (59) with γ = 2 π R R 0 g ( r ) dr and with M ( x ) a function which is larger than 1 and tends to 1 when x tends to infinity (if there is no range, (59) can still be used with the typical range ˜ R defined in VII). Both in the TCP case, i.e. for g ( r ) = C r , and in the UDP case, i.e. for g ( r ) = C , the function x → M ( x ) is decreasing (and has an explicit approximation). W ith a decreasing M , Equation (59) exhibits two causes of super-scalability . First, there is the 1 √ λ super–scalability that comes from the fluid term W f = q F λγ . This is the same type of super–scalability that was observed in the toy example. But there is also a super-scalability that comes from M , which expresses the surprising fact that increasing the arriv al rate reduces the slow-do wn due to the repulsion phenomenon identified in the paper . For N f large enough, the main cause of scalability is W f , but otherwise, the effect of M on super– scalability is not to be neglected. The conditions for the super-scalability formula (59) to hold were also identified: First, the network should have the capacity to cope with the P2P traffic. This translates into the requirement K √ θ > 2 λF γ Z R 0 r 2 g ( r ) dr, (60) where θ is the spatial intensity of routers and K the typical link capacity . In words, the linear capacity of the network should scale like λ if other parameters are unchanged. Secondly the access should not be the bottleneck, which translates into the requirement U > p λF γ , (61) where U the (total) upload capacity of each peer . In words, the latter should scale like the square root of λ . W e remark that the link capacity requirement is larger than the access requirement, which intuitiv ely supports our initial motiv ation, which was that in future (wired) networks, the bottleneck should not be the access anymore. Note that we are fully aware of the fact that, in the the hard- core regime, our model might fail due to the lack of adequate representation of the chunk level. W e expect chunk av ailability to become a crucial bottleneck in hard–core. So, if N f := R 4 λF γ  1 , our conclusions are probably overestimating the actual performance. One of the future challenges in the research started by this paper is the extension to chunk-lev el modeling. Considering chunks leads to the issue of data a vailability , and a chunk- based system may be, in some scenarios, less stable that the models considered in this paper . For instance, a missing piece syndrome may be encountered in the form of growing spatial subpopulations missing at least one chunk. Parameters like the degree of altruism and the spatial intensity of permanent seeders can be expected to appear in the characterization of a stable regime. X . A P P E N D I X : P RO O F O F P RO P O S I T I O N 1 ( S K E T C H ) Choose a number z 0 > 0 such that f ( z 0 ) > 0 and split D into cells with diameters at most z 0 . Then all peers in a cell with population higher than one receiv e service at least at rate f ( z 0 ) . It follo ws that the population of each cell is stochastically dominated by an M / M / ∞ queue that is modified so that a lone customer cannot lea ve. Since such queues are stable with any input rate, the distribution of ( | Φ t | : t ≥ 0) is tight, whatev er the initial state Φ 0 . The ergodicity can no w be sho wn by a standard coupling argument: two realizations with different initial states but same arriv al process couple in finite time.  X I . A P P E N D I X : P R O O F O F T H E O R E M 1 W e work here on the torus T of area D . Let d denote the distance on T and m the Haar measure. Let f : (0 , ∞ ) → (0 , ∞ ) be a positi ve function, and let Φ t be the state of the SBD at time t . For x 0 ∈ T , let a = Z T f ( k x − x 0 k ) m ( d x ) . By translation in variance, a is independent of the choice of x 0 . Further, the left hand side of the claim can be expressed as E [ X x i ∈ Φ f ( || x i || )] = E ( N 0 ) a D . (62) Consider no w the P2P dynamics on T in steady state. For all X ∈ Φ t , let A t ( X ) = X Y ∈ Φ t ,Y 6 = X f ( k X − Y k ) (63) A t = X X ∈ Φ t A t ( X ) , (64) where A t ( X ) is the death rate of point X and A t is the total death rate of the SBD (here we assume that the mean file size F is equal to 1). The right hand side of the claim can be written as E 0 [ X x i ∈ φ \{ 0 } f ( || x i || )] = E 0 ( A 0 (0)) . (65) By the rate conservation principle (e.g., [22], 1.3.3), applied to the stochastic process N t = Φ t ( T ) , we get λD = E ( A 0 ) = E ( N 0 ) E 0 ( A 0 (0)) , (66) with E 0 the (spatial) Palm probability of Φ 0 . This relation says that the birth rate r ↑ = λD should balance the death rate r ↓ = E ( A ) . The relation E ( A 0 ) = E ( N 0 ) E 0 ( A 0 (0)) follows from the definition of the Palm probability . Let E ↑ denote the (time) Palm probability of the SBD at birth epochs and E ↓ that at death epochs. The rate conservation principle applied to the stochastic process (total rate) A t , that we assume cadlag, giv es r ↑ E ↑ ( I ) = r ↓ E ↓ ( D ) with I = A 0+ − A 0 the total rate increase and D = A 0 − A 0+ the (absolute v alue of the) total rate decrease. Since r ↑ = r ↓ , we get that E ↑ ( I ) = E ↓ ( D ) . From the P AST A property [22], and the fact that births are uniform on T , E ↑ ( I ) = 2 E ( N 0 ) a D . The (total) death point process admits a stochastic intensity w .r .t. the filtration F t = σ (Φ s , s ≤ t ) equal to A t . Hence, it follows from P apangelou’ s theorem (e.g., [22], Theorem 1.9.2) that d P ↓ d P | F 0 − = A 0 E ( A 0 ) . Since the decrease (in state Φ 0 − ) is of magnitude A 0 ( X ) (w .r .t. Φ 0 − ) with probability A 0 ( X ) A 0 (w .r .t. Φ 0 − ), we get E ↓ ( D ) = 2 E A 0 E ( A 0 ) X X ∈ Φ 0 A 0 ( X ) A 0 A 0 ( X ) ! = 2 E P X ∈ Φ 0 ( A 0 ( X )) 2 ! E P X ∈ Φ 0 A 0 ( X ) ! . Hence, when using the fact that E P X ∈ Φ 0 ( A 0 ( X )) 2 ! E P X ∈ Φ 0 A 0 ( X ) ! = E 0  ( A 0 (0)) 2  E 0 ( A 0 (0)) , the rate conservation principle for total rate gives: E ( N 0 ) a D = E 0  ( A 0 (0)) 2  E 0 ( A 0 (0)) . (67) Recalling (62) and (65), we now note that Theorem 1 follows from the fact that E 0  ( A 0 (0)) 2  ≥ E 0 ( A 0 (0)) 2 . X I I . A P P E N D I X : S K E T C H O F P R O O F O F T H E O R E M 2 Assume for simplicity that f is bounded. W e proceed as in the fluid limit of a queue, by scaling the arriv al and service rates appropriately , and consider a sequence of systems index ed by n , where n is a parameter that tends to infinity . Our assumption is that the arriv al rate in system n is λ n = λn , and the mean file size in system n is F n = F n . W e tessellate the plane with a grid made of squares of side δ , and time with a grid of width η . Hence, the mean number of arriv als in a typical square and a typical time interval is λnδ 2 η for all n . In addition, the strong la w of large numbers (SLLN) shows that the random number of arriv als A t n in a typical square in the time interval ( t, t + η ) is such that A t n /n tends a.s. to the constant λη δ 2 when n tends to infinity . The ne xt task is to show that the number of peers N t n ( k , l ) present at time t in the square with coordinates ( k δ, l δ ) is such that N t n ( k , l ) /n con ver ges a.s. to some deterministic limit β t ( k , l ) δ 2 . W e then get that the number of deaths in this square in the time interval ( t, t + η ) , denoted by D t n ( k , l ) , satisfies lim n →∞ 1 n D t n ( k , l ) = 1 F β t ( k , l ) X p,q f [( pδ, q δ ) , ( k δ, l δ )] β t ( p, q ) δ 4 η . This follows from the fact that the probability that a typical peer in the square ( k δ, l δ ) dies approximately with probability η F n X p,q f [( pδ, q δ ) , ( k δ, l δ )] N t n ( p, q ) that tends to 1 F X p,q f [( pδ, q δ ) , ( k δ, l δ )] β t ( p, q ) δ 2 η , so that the number of deaths tends to the announced limit. (Notice ho wev er that this discretization does not make sense for , e.g., f ( x, y ) = C / | x − y | , as f [( k δ, lδ ) , ( k δ, l δ )] = ∞ .) Hence, by letting δ and η tend to 0, we get that the function β t ( x ) which is the value of the density at x ∈ R 2 at time t in the fluid regime satisfies the dif ferential equation d dt β t ( x ) = λ − β t ( x ) F Z R 2 f ( x, y ) β t ( y ) dy . (68) The steady state of this is λ β ( x ) = 1 F Z R 2 f ( x, y ) β ( y ) dy . A translation in variant solution of this is β 2 = λF R R 2 f ( x, y ) dy , which is the “fluid solution”. X I I I . A P P E N D I X : J U S T I FI C A T I O N O F T H E H E U R I S T I C In order to deriv e the heuristic of Section IV -E, we use the balance equation for the second order factorial moment density , which reads 2 β o λ = 2 m [2] ( x, y ) C F 1 || x − y ||≤ R || x − y || (69) + C F Z D m [3] ( x, y , z )  1 || x − z ||≤ R || x − z || + 1 || y − z ||≤ R || y − z ||  dz , for all x and y . W e then use the follo wing approximations: m [3] ( x, y , z ) ≈ m [2] ( x, y ) m [2] ( x, z ) β o m [3] ( x, y , z ) ≈ m [2] ( x, y ) m [2] ( y , z ) β o . Then, we get from (69) that β o λ ≈ m [2] ( x, y ) C F 1 || x − y ||≤ R || x − y || + m [2] ( x, y ) C F 1 2 Z D 1 || x − z ||≤ R || x − z || m [2] ( x, z ) β o dz + m [2] ( x, y ) C F 1 2 Z D 1 || y − z ||≤ R || y − z || m [2] ( y , z ) β o dz , that is m [2] ( x, y ) ≈ λF β o C 1 || x − y ||≤ R || x − y || + µ o . (70) with µ o =: C R B (0 ,R ) m [2] (0 ,z ) β o 1 || z || dz . So µ o ≈ λF 2 π C R R 0 1 µ o + C r dr = λF 2 π C  R µ o − C µ 2 o ln(1 + µ o R C )  , (71) which is our departure point in Section IV -E. R E F E R E N C E S [1] B. Cohen, “BitT orrent specification, ” 2006, http://www .bittorrent.org. [2] D. Qiu and R. Srikant, “Modeling and performance analysis of BitT orrent-like peer-to-peer networks, ” ACM SIGCOMM Computer Communication Review , v ol. 34, no. 4, pp. 367–378, 2004. [3] J. S. Otto, M. A. S ´ anchez, D. R. Choffnes, F . E. Bustamante, and G. Siganos, “On blind mice and the elephant: understanding the net- work impact of a lar ge distrib uted system, ” in SIGCOMM , S. Kesha v , J. Liebeherr, J. W . Byers, and J. C. Mogul, Eds. A CM, 2011, pp. 110–121. [4] L. Massoulie and M. V ojnovic, “Coupon replication systems, ” IEEE/ACM T rans. Networking , vol. 16, no. 3, pp. 603–616, 2005. [5] F . Mathieu and J. Reynier, “Missing piece issue and upload strate gies in flashcrowds and P2P-assisted filesharing. ” in AICT/ICIW’06 , 2006. [6] B. Hajek and J. Zhu, “The missing piece syndrome in peer-to-peer communication, ” 2010, http://arxiv .org/abs/1002.3493. [7] J. Zhu and B. Hajek, “Stability of a peer-to-peer communication system, ” 2011. [8] H. Reittu, “ A stable random-contact algorithm for peer-to-peer file sharing, ” in IFIP IWSOS , 2009, pp. 185–192. [9] I. Norros, H. Reittu, and T . Eirola, “On the stability of two-chunk file- sharing systems, ” Queueing Systems , vol. 67, pp. 183–206, 2011. [10] B. O ˘ guz, V . Anantharam, and I. Norros, “Stable, scalable, decentralized P2P file sharing with non-altruistic peers, ” 2011, [11] F . Benbadis, F . Mathieu, N. Hegde, and D. Perino, “Playing with the bandwidth conservation law , ” in IEEE P2P , 2008, pp. 140–149. [12] R. Susitai val, S. Aalto, and J. V irtamo, “ Analyzing the dynamics and resource usage of P2P file sharing systems by a spatio-temporal model, ” in P2P-HPCS06, in conj. with ICCS , May . 2006, pp. 420–427. [13] A. Norberg, “Bittorrent enhancement proposals on uTorrent transport protocol, ” 2009, http://bittorrent.org/beps/bep 0029.html. [14] S. Shalunov , “Lo w extra delay background transport (LEDB A T), ” IETF Draft, 2010, http://datatracker .ietf.org/wg/ledbat/charter/. [15] T . Ott, J. K emperman, and M. Mathis, “The stationary beha vior of ideal TCP congestion avoidance, ” Internetworking: Research and Experience , vol. 11, pp. 115–156, 1992. [16] N. Garcia and T . Kurtz, “Spatial birth and death processes as solutions of stochastic equations, ” ALEA Lat. Am. J. Probab . Math. Stat. , vol. 1, pp. 281–303, 2006. [17] D. J. Daley and D. V ere-Jones, An Intr oduction to the Theory of P oint Pr ocesses . Springer , 1988. [18] E. Buckingham, “The principle of similitude, ” Nature , vol. 96 (2406), pp. 396–397, 1915. [19] F . Baccelli and B. Błaszczyszyn, Stochastic Geometry and W ireless Networks, V olume I and II , ser . Foundations and Trends in Networking. NoW Publishers, 2009. [20] F . P . Kelly , L. Massoulie, and N. S. W alton, “Resource pooling in congested networks: proportional fairness and product form. ” Queueing Syst. , vol. 63, no. 1-4, pp. 165–194, 2009. [21] R. Schneider and W . W eil, Stochastic and Integral Geometry . Springer , 2008. [22] F . Baccelli and P . Br ´ emaud, Elements of Queueing Theory . Berlin: Springer V erlag, 2003.