Relaxation-based coarsening and multiscale graph organization

RELAXA TION-BASED CO ARSENING AND MUL TISCALE GRAPH OR GANIZA TION DORIT RON ∗ , IL Y A SAFRO † , AND ACHI BRANDT ‡ Abstract. In this paper we generalize and improv e the multiscale organization of graphs by introducing a new measure that quantifies the “closeness” b etw een tw o nodes. The calculation of the measure is linear in the num b er of edges in the graph and in volv es just a small num ber of relaxation sweeps. A similar notion of distance is then calculated and used at each coarser level. W e demonstrate the use of this measure in multiscale methods for several imp ortant com binatorial optimization problems and discuss the multiscale graph organization. 1. In tro duction. A general approac h for solving many large-scale graph prob- lems, as well as most other classes of large-scale computational science problems, is through multilev el (multiscale, m ultiresolution, etc.) algorithms. This approach gen- erally inv olves c o arsening the problem, pro ducing from it a sequence of progressiv ely coarser levels (smaller, hence simpler, related problems), then recursively using the (appro ximate) solution of eac h coarse problem to pro vide an initial approximation to the solution at the next-finer level. A t eac h level, this initial appro ximation is first impro ved b y what w e generally call “lo cal pro cessing” (LP). This is an inexp ensiv e sequence of short steps, eac h inv olving only a few unkno wns, together co vering all unkno wns of that level several times o ver. The usual examples of LP are few sweeps of classical (e.g., Gauss-Seidel or Jacobi) relaxation in the case of solving a system of equations, or a few Monte Carlo passes in statistical-physics simulations. F ollowing the LP , the resulting appro ximation may b e further improv ed by one or several cy- cles, each using again a coarser-level approximation follo wed b y LP , applying them at eac h time to the r esidual problem (the problem of calculating the err or in the current appro ximation). See, for example, references [ 6 , 7 , 11 , 12 , 13 , 14 , 38 , 42 ]. A t eac h level of coarsening one needs to define the set of coarse unkno wn v ariables and the equations (or the sto chastic relations) that they should satisfy (or the energy that they should minimize). Eac h coarse unknown is defined in terms of the next- finer-lev el unkno wns ( define d , not c alculate d : they are all unkno wns un til the coarse lev el is approximately solved and the fine lev el is in terp olated from that solution). The following are examples: • The set of coarse unknowns can simply represent a chosen subset of the fine- lev el set. • If the fine-lev el v ariables are real num b ers or vectors, eac h coarse v ariable can represen t a weigh ted av erage of sev eral of them. • If the fine-level v ariables are Ising spins (having only v alues of +1 or − 1), eac h coarse v ariable can again be an Ising spin, representing the sign of the sum of sev eral fine spins. • A coarse v ariable can be defined from several fine v ariables by a sto c hastic pro cess ([ 5 ], for example). • In the case of graph problems, each no de of the coarse graph can represent ∗ F aculty of Mathematics and Computer Science, The W eizmann Institute of Science, dorit.ron@weizmann.ac.il † Mathematics and Computer Science Division, Argonne Nationa l Laboratory , safro@mcs.anl.gov ‡ F aculty of Mathematics and Computer Science, The W eizmann Institute of Science, achi.brandt@weizmann.ac.il 1 an aggr e gate of sev eral fine-level no des or a weighte d aggr e gate of such no des, that is, allo wing each fine-level node to b e split b et ween sev eral aggregates. The c hoice of an adequate lo cal pro cessing at a fine lev el and the choice of an adequate set of v ariables at the next-coarser level are strongly coupled. The general guiding rule [ 10 ] is that this pair of choices is go od if (and to the exten t that) a fine- lev el solution can alw ays b e recov ered from the corresp onding set of coarse v ariables b y a short iterativ e use of a suitably modified version of the LP . That version is called c omp atible LP (CLP). Examples are compatible Monte Carlo (CMC), introduced in [ 13 ], and compatible relaxation (CR), introduced in [ 8 ]. The CLP , needed in several imp ortan t upscaling pro cedures (such as the selec- tion of the coarse v ariables, the acceleration of the fine-level simulations, and the pro cessing of fine-level windows within coarse simulations; see [ 10 ]) can also b e used for p erforming the interpolation from the coarse solution to obtain the first approxi- mation at the fine level. When p ossible, how ever, the construction of a more explicit in terp olation is desired in order to apply it for the direct formulation of equations (or an energy functional) that should gov ern the coarse lev el, as in Galerkin coarsening. In the process of defining the set of coarse v ariables and in constructing an explicit in terp olation, it is imp ortant to know ho w “close” tw o given fine-level v ariables are to eac h other at the stage of switc hing to the coarse level. W e need to know, in other w ords, to what extent the v alue after the LP of one v ariable implies the v alue of the other. If they are sufficiently close, they can, for example, be aggregated to form a coarse v ariable. The central issue addressed in the presen t article is ho w to measure this “close- ness” b etw een tw o v ariables in a system of equations or b et ween tw o no des in a given graph. (W e consider the latter to be a sp ecial case of the former, by asso ciating the graph with the system formed by its Laplacian.) More generally , we wan t to define the distance of one v ariable x i from a small subset S of sever al v ariables, in order to measure how w ell x i can b e interpolated from S following the LP . In classical Algebraic Multigrid (AMG), aimed at solving the linear system Ax = b o r n X j =1 a ij x j = b i , ( i = 1 , ..., n ) , (1.1) the closeness of t wo unknowns x i and x j is measured simply b y the relativ e size of their coupling a ij , for example, by the quantit y | a ij | / max( X k | a ik | , X k | a kj | ) (1.2) (or similarly by the relative size of their coupling in some p ower of A ). Although this definition has work ed well for the coarsening pro cedures of discretized scalar elliptic differen tial equations, it is not really effectiv e, and sometimes meaningless, for systems lac king sufficient diagonal dominance (including many discretized nonsc alar elliptic systems). Moreo ver, ev en for systems with a fully dominan t diagonal (suc h as the Laplacian of a graph), the classical AMG definition may result in wrong coarsening, for example, in graphs with nonlo cal edges (see example in Sec. 3 ). Instead, we prop ose to define the “closeness” b etw een tw o v ariables exactly , by measuring how well their v alues are correlated at the coarsening stage, namely , fol- lo wing the LP relaxation sweeps. Since the coarse level is actually applied to the r esidual system, the tw o v ariables will b e considered close if their err ors hav e nearly 2 the same ratio in all relaxed vectors. W e will thus create a sequence of K normalize d r elaxe d err or ve ctors x (1) , ..., x ( K ) , each obtained b y relaxing the homogeneous system Ax = 0 from some (e.g., random) start and then normalizing the result. W e will then define the algebr aic distanc e (recipro cal of “closeness”) b et ween an y tw o v ariables x i and x j as min η  K X k =1 | η x ( k ) i − | η − 1 | x ( k ) j | p  1 /p , (1.3) where p ≥ 2 in order to attach larger w eights to larger differences (using usually either p = 2 or the maximum norm ( p → ∞ )). This use of η gives a symmetric measure of ho w well x i can b e interpolated from x j or vice v ersa. F or the graph Laplacian (and other zero-sum A ) this can b e simplified to a distance defined as  K X k =1 ( x ( k ) i − x ( k ) j ) 2  1 / 2 or K max k =1 | x ( k ) i − x ( k ) j | . (1.4) More generally , the distance of a no de x i from a subset S of sev eral nodes can similarly b e defined as the deviation of the b est-fitted interpolation from S to x i , where the deviation is the L 2 norm of the v ector of K errors obtained up on applying the interpolation to our K normalized relaxed error vectors, and the best-fitted in- terp olation is the one ha ving the minimal deviation. (This least-square interpolation is the one in tro duced in bo otstrap AMG (BAMG) [ 9 ] for the coarse-to-fine explicit in terp olation.) An essential asp ect of the “algebraic distance” defined here is that it is a crude lo c al distanc e . It measures meaningful closeness only b etw een neighboring no des; the closer they are the less fuzzy is their measured distance. F or no des that should not b e considered as neighbors, their algebraic distance just detects the fact that they are far apart; its exact v alue carries no further meaning. The important p oin t is that this crude lo cal definition of distance is fast to calculate and is all that is required for the coarsening purp oses. A similar notion of distance is then similarly calculated at eac h coarser level. Indeed, w e argue that meaningful distances in a general graph should, in princi- ple, b e define d (not just c alculate d ) only in such a multiscale fashion. This essen tial viewp oin t, and relations to diffusion distances and spectral clustering are discussed in Section 5 . In particular, w e adv o cate the replacement of spectral metho ds by AMG- lik e m ultilevel algorithms, which are b oth faster and more tunable to define b etter solutions to man y fuzzy graph problems (see, for example, [ 41 , 42 ]). The pap er is organized as follows. The graph problems w e use to demonstrate our approach are in tro duced in Sec 2 . The calculation of the “algebraic distance” and its use within the m ultiscale algorithm is describ ed in Section 3 . Results of tests are summarized in Section 4 . Finally , the relations of our approac h to diffusion distances and sp ectral clustering are discussed in Section 5 . 2. Notation and problem definitions. Given a weigh ted graph G = ( V , E ), where V = { 1 , 2 , ..., n } is the set of nodes (vertices) and E is the set of edges. Denote b y w ij the non-negative weigh t (coupling) of the undirected edge ij b et ween no des i and j ; if ij / ∈ E , then w ij = 0. W e consider as our examples the following t wo optimization problems. 3 2.1. Linear ordering. Let π b e a bijection π : V − → { 1 , 2 , ..., n } . The purp ose of linear ordering problems is to minimize some functional o ver all p os- sible p erm utations π . The following functional should b e minimized for the minimum p -sum problem 1 (M p SP): σ p ( G, π ) = X ij w ij | π ( i ) − π ( j ) | p . (2.1) In the generalized form of the problem that emerges during the multilev el solver, eac h v ertex i is assigned with a v ol ume (or leng th ), denoted v i . Giv en the vector of all v olumes, v , the task now is to minimize the cost σ p ( G, π , v ) def = σ p ( G, x ) = X ij w ij | x i − x j | p , where x i = v i 2 + P k,π ( k ) <π ( i ) v k ; that is eac h v ertex is positioned at its center of mass, capturing a segment on the real axis that equals its length. The original form of the problem is the special case where all the v olumes are equal. In particular, we would like to concentrate on the minim um linear arrangemen t (where p = 1) and the minim um 2-sum problem (M2SP) that w ere pro ven to be NP-complete in [ 23 , 24 ] and whose solution can serv e as an appro ximation for man y differen t linear ordering problems replacing the spectral approaches [ 41 , 42 ]. 2.2. P artitioning. The goal of the 2-partitioning problem is to find a partition of V in to tw o disjoint nonempty subsets Π 1 and Π 2 suc h that minimize X i ∈ Π 1 ,j ∈ Π 2 w ij , sub ject to | Π k | ≤ (1 + α ) · | V | 2 , ( k = 1 , 2) , (2.2) where α is a given imb alanc e factor . Graph partitioning is an NP-hard problem [ 22 ] used in many fields of computer science and engineering. Applications include VLSI design, minimizing the cost of data distribution in parallel computing and optimal tasks scheduling. Because of its practical imp ortance, man y different heuristics (sp ectral [ 36 ], com binatorial [ 31 , 21 ], ev olutionist [ 15 ], etc.) ha v e b een dev elop ed to pro vide an approximation in a reason- able (and, one hop es, linear) computational time. How ever, only the introduction of m ultilevel metho ds for partitioning [ 30 , 35 , 2 , 34 , 46 , 3 , 37 , 4 , 27 , 29 , 1 ] has really pro vided a breakthrough in efficiency and quality . 3. The coarsening algorithm. In the multilev el framew ork a hierarc hy of de- creasing size graphs G 0 , G 1 , ..., G k is constructed. Starting from the given graph, G 0 = G , w e create by recursive c o arsening the sequence G 1 , ..., G k , then solve the coarsest level directly , and uncoarsen the solution back to G . In general, the AMG-based coarsening is interpreted as a pro cess of weigh ted aggregation of the graph no des to define the no des of the next coarser graph. In w eighted aggregation each no de can b e divided into fractions, and different fractions 1 W e use this definition for simplicity . The usual definition of the functional is σ p ( G, π ) = ( P ij w ij | π ( i ) − π ( j ) | p ) 1 /p , which yields the same minimization problem. 4 b elong to different aggregates. The construction of a coarse graph from a giv en one is divided into three stages. First a subset of the fine no des is chosen to serve as the se e ds of the aggregates (the no des of the coarse graph). Then the rules for aggregation are determined, thereby establishing the fraction of eac h nonseed no de b elonging to eac h aggregate. Finally , the graph couplings (or edges) betw een the coarse no des are calculated. The en tire coarsening scheme is shown in Algorithm 1 . The AMG-based multilev el framework for graph optimization problems is dis- cussed, for example, in [ 42 ]. In the presen t work w e generalize the coarsening part of the AMG-based framework. The problem-dep enden t solution of the coarsest lev el and the uncoarsening are not changed here. They are fully described in [ 42 ] and references therein. The principal difference b et ween the previous AMG-based coarsening approaches [ 42 , 28 , 17 ] and the new r elaxation-b ase d approac h is the impro ved measure, the algebr aic c oupling , assigned to each edge, or, more generally , b et ween any t wo nodes, in the graph. The algebraic coupling is the reciprocal of the calculated algebr aic distanc e in tro duced b elow. Algebraic distance and coupling. The need for an improv ed measure for the graph couplings can b e explained by observing the graph depicted in Figure 3.1 : one additional edge ij (connecting no des i and j ) is added to a regular t wodimensional mesh. While coarsening, no des i and j clearly should not b elong to the same aggregate unless their coupling is m uc h stronger than other graph couplings. How ever, if the w eight of ij is just somewhat larger than all other graph edges, and if the black dots are some of the seeds of the coarse aggregates (c hosen b y some AMG-based criterion; see, for example, Algorithm 2 ), no de i will tend to be aggregated with no de j , rather than with any of its neighbors. Such a decision will create bad coarse-lev el appro ximations in many optimization problems (e.g., linear ordering and partitioning). Moreo ver, at the next-coarser levels the approximation may further deteriorate by making similar wrong decisions, making the entire neighborho o d of i close to j , thereb y promoting linear arrangemen ts in whic h many local couplings would unnecessarily b ecome long- range ones. T o prev ent this situation we would like to ha ve a measure that not only ev aluates the coupling b et ween i and j according to the dir e ct coupling betw een them but also tak es into accoun t the contribution of connections betw een the neighb orho o ds of i and j . That is, if the immediate (graph) neighbors of i are connected to those of j , the coupling b et ween i and j should b e enhanced, while if i ’s neighbors are not connected to those of j , as in Figure 3.1 , a significant w eakening of the ij coupling is due. This measure will prev ent p ossible errors while coarsening. W e in tro duce the notion of algebr aic distanc e , whic h is based on the same set of test ve ctors (TVs) b eing used in the b o otstrap AMG (BAMG) [ 9 ]. The key new ingredien t of the adaptive BAMG setup is the use of sev eral TVs, collectively repre- sen ting algebraically smo oth error, to define the in terp olation weigh ts. When a priori kno wledge of the nature of this error is not a v ailable, sligh tly relaxed random v ectors are used for this task. A set of some K lo w-residual TVs { x ( k ) } K k =1 can first b e ob- tained by relaxation. Namely , eac h x ( k ) is a result of r fine-lev el relaxation sweeps on the homogeneous equation Ax = 0, starting from a r andom approximation, where A is the Laplacian of the graph. In particular, w e hav e used a small num b er (usually r=10) of Jacobi under relaxation sw eeps with ω = 0 . 5. That is, the new v alue for eac h x ( k ) , k = 1 , ..., K (in our tests K = 20) is x ( k ) N E W = (1 − ω ) x ( k ) + ωx ( k ) J AC , (3.1) 5 j i q Fig. 3.1 . Mesh graph with an additional e dge b etwe en no des i and j . The black dots mark some of the no des sele cted to serve as the se e ds of the c o arse aggr e gates; se e Algorithm 2 . where x ( k ) J AC = D − 1 ( D − A ) x ( k ) , (3.2) D b eing the diagonal of A . The algebraic distance from no de i to no de j is defined o ver the K relaxed TVs by d ij = K max k =1 | x ( k ) i − x ( k ) j | . (3.3) Other definitions, s uc h as d ij = K X k =1 ( x ( k ) i − x ( k ) j ) 2 (3.4) are also possible. Hence, only if d ij is small ma y nodes i and j b e aggregated in to the same coarse no de. The algebraic coupling b etw een i and j , c ij , is defined as the recipro cal of d ij : c ij = 1 /d ij . (3.5) Data : Q , ν Result : coarse graph F or ev ery edge ij derive its algebraic distance d ij ( 3.3 ) or ( 3.4 ) and algebraic coupling c ij ( 3.5 ); SelectCoarseNo des( Q , ν ); Define the coarse graph using the matrix P in Equation ( 3.7 ); Algorithm 1 : Coarsening scheme W e return to the example in Figure 3.1 and demonstrate the outcome of Definition ( 3.3 ) b y comparing d ij with d i ∗ = min { d is | s a nearest neighbor of i } . W e show that i will not tend to b e connected to j unless w ij equals the sum of i ’s other couplings. 6 F urthermore, we sho w that ev en if i is connected to j as a result of strong w ij , i ’s other neigh b ors will not tend to b e connected to i as well but will prefer other neighbors; hence the neighborho o ds of i and j will not tend to be connected to eac h other. Consider T able 3.1 . The num b er K of TVs is giv en in the leftmost column. The n umber r of Jacobi relaxation sweeps v aries from 10 to 100 as sho wn in the second to the left column. Each of the four columns to the righ t presen ts the (natural) logarithm of d ij /d i ∗ , av eraged ov er 100 independent runs, for graph couplings w uv = 1 when u and v are nearest neigh b ors, and w ij = 1 , 2 , 3 , or 4 as sho wn. The num b ers in paren theses are the corresp onding standard deviations. Clearly the strength of the coupling b etw een i and j is relativ ely decreased when measured b y the algebraic distance. F or instance, if the graph coupling b et ween i and j is 1 (as are all other couplings in the graph), then after 20 relaxation sweeps (with K = 10) d ij is three times bigger than the minim um of the (algebraic distance of the) edges to i ’s four nearest neigh b ors. Thus, the algebraic coupling b etw een i and j is not the strongest coupling of i (not even close to it), and hence it is guaranteed that i and j will not b elong to the same coarse no de. The imp ortance of using more than just 1 TV can b e seen from the v alues of the standard deviations: The use of 1 TV results in standard deviations similar to the av erage, which means that ln ( d ij /d i ∗ ) has a significan t chance to become negative, so ij has a significant chance to b e the strongest coupling of i . With 10 TVs this c hance b ecomes muc h smaller, at least for w ij ≤ 2. Ev en with 10 TVs, ho wev er, the c hance gro ws with w ij , b ecoming more than 50% roughly when w ij ≥ 4. Thus, the aggregation of i with j b ecomes likely . This by itself is fine and justified. What we really need to a void is that entire neigh b orho ods of i and j will, as a result, b e aggregated at some coarser level. Hence, it is important to see whether the neigh b ors of i will tend to b e aggregated with i (and th us also with j ) or will prefer their other neighbors. T o see that, w e calculate the (natural) logarithm of d q i /d q i ∗ , where d q i ∗ = min { d q s | s a nearest neighbor of q other than i } . As shown in T able 3.2 , q would rather b e aggregated with one of its other-than- i neighbors. F or example, for K = 10, r = 20 and w ij = 4 out of the 100 runs, in 95 q w ould hav e been connected with i . The main conclusion is that no des i and j do not tend to b e connected as long as w ij is smaller than the sum of all other couplings of i or of j . When the coupling is of the same strength, they will be connected ab out half the time, but then, not less imp ortan t, the neigh b ors of i (and similarly of j ) will not tend to join them but will prefer to b e connected to other nearest neighbors no des. Similar results are obtained when using ( 3.4 ) to calculate d ij . With the notion of the algebraic coupling in mind, the coarse no des selection and the calculation of the aggregation weigh ts are mo dified as follo ws. Seed selection. The construction of the set of seeds C and its complement F is guided by the principle that each F -no de should b e “strongly coupled” to C . W e will include in C no des with exceptionally large volume or no des exp ected (if used as seeds) to aggregate around them an exceptionally large total v olume of F -nodes. W e start with C = ∅ , hence F = V , and then sequentially transfer nodes from F to C , as follows. As a measure of ho w large an aggregate seeded by i ∈ F might grow, we define its futur e volume ϑ i b y ϑ i = v i + X ij ∈ E v j · c j i P j k ∈ E c j k . (3.6) No des with future volume larger than ν times the av erage of the ϑ i ’s are first trans- ferred to C as most “representativ e” (in our tests ν = 2). The insertion of additional 7 w uv = 1 for ( u, v ) nearest neighbors K r w ij = 1 w ij = 2 w ij = 3 w ij = 4 1 10 2.47(1.51) 1.88(1.74) 1.38(1.85) 1.14(1.69) 20 2.74(1.74) 2.1(1.59) 1.4(1.26) 1.44(1.59) 50 2.65(1.36) 1.98(1.76) 1.92(1.41) 1.5(1.59) 100 3.03(1.72) 2.14(1.32) 1.51(1.42) 1.16(1.78) 5 10 1(0.502) 0.628(0.416) 0.24(0.417) -0.0484(0.397) 20 1.34(0.442) 0.825(0.415) 0.435(0.358) 0.208(0.332) 50 1.68(0.342) 1.04(0.338) 0.643(0.306) 0.362(0.296) 100 1.78(0.467) 1.06(0.392) 0.743(0.369) 0.396(0.359) 10 10 0.821(0.281) 0.443(0.294) 0.022(0.293) -0.244(0.313) 20 1.09(0.268) 0.624(0.239) 0.298(0.235) 0.0126(0.253) 50 1.49(0.263) 0.86(0.235) 0.504(0.226) 0.2(0.204) 100 1.69(0.315) 1.01(0.275) 0.572(0.264) 0.285(0.271) T able 3.1 Statistic al r esults (over 100 runs) for the aver age (and, in p ar entheses, the standar d deviation) of ln ( d ij /d i ∗ ) , c alculate d with K TVs and r Jac obi r elaxation sweeps for different relative strengths of w ij . w uv = 1 for ( u, v ) nearest neighbors K r w ij = 1 w ij = 2 w ij = 3 w ij = 4 1 10 0.975(1.67) 0.939(1.7) 1.14(1.63) 1.07(1.89) 20 0.911(1.62) 1.09(1.31) 1.02(1.46) 0.931(1.64) 50 1.37(1.77) 1.14(1.79) 1.28(1.49) 1.24(1.45) 100 0.897(1.55) 1.23(1.45) 1.29(1.53) 1.31(1.44) 5 10 0.382(0.534) 0.482(0.428) 0.416(0.52) 0.587(0.487) 20 0.434(0.444) 0.472(0.366) 0.592(0.486) 0.663(0.458) 50 0.498(0.436) 0.755(0.53) 0.784(0.526) 0.813(0.455) 100 0.501(0.522) 0.746(0.544) 0.812(0.549) 0.816(0.535) 10 10 0.283(0.312) 0.299(0.316) 0.376(0.307) 0.401(0.357) 20 0.362(0.281) 0.419(0.295) 0.449(0.288) 0.441(0.327) 50 0.448(0.311) 0.531(0.35) 0.672(0.351) 0.604(0.333) 100 0.464(0.377) 0.682(0.348) 0.839(0.374) 0.749(0.39) T able 3.2 Statistic al r esults (over 100 runs) for the aver age (and, in p ar entheses, the standar d deviation) of ln ( d qi /d q i ∗ ) (se e Figur e 3.1 ), c alculate d with K TVs and r Jac obi relaxation sweeps for different r elative str engths of w ij . F -nodes to C dep ends on a “strength of coupling to C ” threshold Q (in our tests Q = 0 . 5), as sp ecified in Algorithm 2 . Coarse no des. Eac h node in the c hosen set C b ecomes the seed of an aggregate that will constitute one coarse-level no de. Next it is necessary to determine for each i ∈ F a list of j ∈ C to whic h i will belong. Define c alib er , l , to b e the maximal n umber of C -p oints allow ed in that list. The selection we prop ose here is based on b oth measures: the graph couplings w ij ’s and the algebraic couplings c ij ’s. Define for each i ∈ F a coarse neigh b orhoo d ¯ ¯ N C ( i ) = { j ∈ C : ij ∈ E } . Set D to b e the maximal c ij in ¯ ¯ N C ( i ). Construct a possibly smaller coarse neighborho o d b y including only no des with strong algebraic coupling ¯ N C ( i ) = { j ∈ ¯ ¯ N C ( i ) : c ij ≥ β ∗ D } , we use 8 Data : Q , ν Result : set of seeds C C ← ∅ , F ← V ; Calculate ϑ i ( 3.6 ) for eac h i ∈ F , and their av erage ϑ ; C ← no des i with ϑ i > ν · ϑ ; F ← V \ C ; forall i ∈ F in desc ending or der of ϑ i do if ( P j ∈ ( C ∩ N ( i )) c ij / P j ∈ N ( i ) c ij ) ≤ Q or ( P j ∈ ( C ∩ N ( i )) w ij / P j ∈ N ( i ) w ij ) ≤ Q then mov e i from F to C ; end Algorithm 2 : SelectCoarseNo des( Q , ν ) β = 0 . 5. If ¯ N C ( i ) > l , then the final coarse neighborho o d N C ( i ) will include the first l largest w ij ’s in ¯ N C ( i ). If ¯ N C ( i ) ≤ l , then N C ( i ) ← ¯ N C ( i ). This construction of the coarse neighborho od N C ( i ) of i ∈ F is summarized in Algorithm 3 . (In the results b elo w we hav e used only l = 1 and l = 2.) The classical AMG in terp olation matrix P (of size | V | × | C | ) is then defined by P ij =      w ij / P k ∈ N C ( i ) w ik fo r i ∈ F, j ∈ N C ( i ) 1 fo r i ∈ C, j = i 0 otherwise . (3.7) P ij represen ts the fraction of i that will b elong to the j th aggregate. Data : l , i , β ¯ ¯ N C ( i ) ← { j ∈ C : ij ∈ E } ; D = max j ∈ ¯ ¯ N C ( i ) c ij ; ¯ N C ( i ) = { j ∈ ¯ ¯ N C ( i ) : c ij ≥ β ∗ D } ; if l < | ¯ N C ( i ) | then N C ( i ) ← the l largest w ij ’s in ¯ N C ( i ); if l ≥ | ¯ N C ( i ) | then N C ( i ) ← ¯ N C ( i ); Algorithm 3 : The coarse neighborho o d N C ( i ) Coarse graph couplings. The coarse couplings are constructed as follo ws. Let I ( k ) be the ordinal num b er in the coarse graph of the node that represents the aggregate around a seed whose ordinal n umber at the fine level is k . F ollo wing the w eighted aggregation sc heme used in [ 43 ], the edge connecting tw o coarse aggregates, p = I ( i ) and q = I ( j ), is assigned with the w eight w ( coarse ) pq = P k 6 = l P ki w kl P lj . The v olume of the i th coarse aggregate is P j v j P j i . 4. Computational results. W e demonstrate the p o wer of our new relaxation- based coarsening sc heme b y comparing its experimental results with those of the classical AMG-based coarsening for three important NP-hard optimization problems: the minimum 2-sum (( 2.1 ) with p = 2), the minimum linear arrangement (( 2.1 ) with p = 1), and the minim um 2-partitioning ( 2.2 ) problems. In all cases the results are 9 obtained by taking the lightest p ossible uncoarsening schemes, so that differences due to the differen t coarsening schemes are least blurred. W e hav e implemented and tested the new coarsening scheme b y using the linear ordering pac k ages developed in [ 41 ] and in [ 40 ] and the Scotch pack age [ 35 ] on a Lin ux machine. The implementation is nonparallel and has not b een optimized. The results should be considered only qualitativ ely and can certainly be improv ed by more adv anced uncoarsening. Thus, no intensiv e attempt to achiev e the b est-kno wn results for the particular test sets w as done. The details regarding the uncoarsening schemes for the abov e problems are given in [ 41 , 40 , 17 ]. 4.1. The minimum p -sum problem. W e presen t the n umerical comparison for t wo minim um p -sum problems: the minim um 2-sum problem and the minimum linear arrangemen t. F or these problems w e ha ve designed a full relaxation-based coarsening solv er and ev aluated it on a test set of 150 graphs of differen t nature, size ( | V | ≤ 5 · 10 6 and | E | ≤ 10 7 ) and prop erties. The test graphs are taken from [ 19 ] and from real- life netw ork data such as so cial netw orks, p ow er grids, and p eer-to-p eer connections. Our solv ers are free and can b e downloaded with detailed solutions for every graph from [ 39 ]. T o emphasize the difference in the minimization results b et ween the tw o coarsening schemes (the relaxation-based and the classical AMG-based schemes), w e measure the results obtained at the end of the multilev el cycle b efor e the final lo cal optimization p ostpro cessing (Gauss-Seidel relaxation and the lo cal pro cessing in [ 41 , 40 ]), as well as after its application. Moreo ver, we use small calib ers, l = 1 , 2, since these demonstrate more sharply the qualit y of matc hing b et ween the F -p oin ts and the C -p oin ts. F or higher calibers it is also imp ortan t to use the adaptiv e BAMG scheme [ 9 ] for calculating the in terp olation weigh ts, whic h is b ey ond the scop e of this w ork. Small calib ers are imp ortant for maintaining the lo w complexit y of the multilev el framew ork, whic h is vital, for example, for hypergraphs and expanders. The minimum 2-sum problem (M2SP). A comparison of the relaxation- based and AMG-based coarsenings with calibers 1 and 2 is presented in Figures 4.1 (a) and 4.1 (b), respectively . Each x-axis scale division corresp onds to one graph from the test set. The y-axis corresp onds to the ratio b etw een the av erage cost obtained b y 100 runs of the AMG-based coarsening and the one obtained b y 100 runs of the relaxation-based coarsening. Eac h figure con tains tw o curv es: the dashed curv es with cost measuremen ts before applying the p ostpro cessing of lo cal optimization (e.g., Gauss-Seidel relaxation, Window Minimization [ 41 ]) and the regular curves with cost measuremen ts after adding such optimization steps. Clearly most graphs b enefit from the relaxation-based coarsening, sho wing a ratio greater than 1. The ratio decreases when more optimization is used, especially since the Gauss-Seidel relaxation is p ow erful algorithmic component for this problem and th us brings the results of the t wo coarsening schemes closer to each other as is indicated by the regular curves. All the these results w ere obtained with K = 10 TVs. When we low ered K to 5, we observ ed no significant c hange in the results. Our num b er of Jacobi ov errelaxation sw eeps r = 20 cannot b e reduced by more than twice since this relaxation sc heme is exp ected to ev olve slo wly . The detailed analysis of the conv ergence prop erties are presen ted in [ 16 ]. The minim um linear arrangemen t problem. Similarly to the previous prob- lem, w e designed a relaxation-based solv er and established a series of exp erimen ts for the minimum linear arrangement problem. The exp erimen tal setup w as iden tical to that of the M2SP . It w as based on the solv er designed in [ 40 ]. In this case we can observ e even more significant improv ement when employing the relaxation-based 10 0 50 100 150 graphs ordered by ratios 0.9 1 1.1 1.2 1.3 1.4 ratios between old and new multilevel solvers before final optimization after final optimization (a) l = 1 0 50 100 150 graphs ordered by ratios 0.9 1 1.1 1.2 1.3 1.4 ratios between old and new multilevel solvers before final optimization after final optimization (b) l = 2 Fig. 4.1 . R esults for the minimum 2-sum pr oblem. coarsening than for the M2SP . 0 50 100 150 graphs ordered by ratios 0.9 1 1.1 1.2 1.3 1.4 ratios between old and new multilevel solvers before final optimization after final optimization (a) l = 1 0 50 100 150 graphs ordered by ratios 0.9 1 1.1 1.2 1.3 1.4 ratios between old and new multilevel solvers before final optimization after final optimization (b) l = 2 Fig. 4.2 . R esults for the minimum 1-sum (line ar arr angement) pr oblem. Whic h graphs are most b eneficial? It is remark able that the most b eneficial graphs in our test set come from VLSI design and general optimization problems. W e kno w that these graphs are very irregular (compared, for example, with finite-elemen t graphs and with those that p ose 2D/3D geometry). Thus, w e may conclude that the algebraic couplings help to identify the w eakness of nonlo cal connections and prev ent them from b eing aggregated. In several examples, we ac hieved the b est known results with caliber 1, while using classical AMG-based approac hes they can b e achiev ed with bigger calib ers only . An algebraic coupling-based algorithm. W e hav e also tried a straightforw ard algorithm in whic h, during coarsening, the w eights of the graph are simply replaced b y their algebraic couplings. That is, in the if statement at the end of Algorithm 2 , only the first term is taken into accoun t (namely , ( P j ∈ ( C ∩ N ( i )) c ij / P j ∈ N ( i ) c ij ) ≤ Q ). Similarly , in Algorithm 3 , w ij (in the first if ) is replaced by c ij . W e present the comparison of the obtained simple algebraic couplings based coarsening scheme with 11 the mixed sc heme describ ed in Algorithms 2 and 3 in Figure 4.3 . The comparison was done for the M2SP including p ostpro cessing (of lo cal optimization) using the same exp erimen tal setup. The b old curve corresp onds to the ratios betw een the classical AMG-based results and the simple algebraic coupling-based coarsening sc heme. T o see the difference b et ween this algorithm and the more elab orate one, we add a copy of its results, that is, the b old curve from Figure 4.1 (a). The mixed version is clearly b etter: in ab out 25 more graphs the results are improv ed. The a verage impro vemen t w as 1 . 5%. 0 50 100 150 graphs ordered by ratios 0.9 0.95 1 1.05 1.1 1.15 1.2 ratios between old and new multilevel solvers Fig. 4.3 . Results for the minimum 2-sum pr oblem. Comparison of the algebr aic distanc e b ase d only and mixe d ful l r elaxation b ase d algorithms. 4.2. The minim um 2-partitioning problem. W e compared the relaxation- based coarsening and the classical AMG-based b y com bining t wo pack ages. The coarsening part was the same as in the minim um p -sum problems. The uncoarsening w as based on the Scotch pack age; details of its fastest v ersion can b e taken from [ 17 ]. The comparison of the relaxation-based and the AMG-based coarsenings with calib er 1 is presen ted in Figure 4.4 . The interpretation of x- and y-axes is similar to Figure 4.1 . Included are 15 graphs of differen t nature and size. The details regarding the numerical results can b e obtained from [ 39 ]. The four best ratios are obtained for graphs with p o wer-la w degree distributions. More results for the graph and h y- p ergraph partitioning problems are rep orted in [ 16 ]. Even though the algorithm used there only substitutes the original given couplings b y their algebraic couplings, it is already clear that b etter results are obse rv ed for most tested instances of b oth graphs and hypergraphs. 4.3. Running time. The implementation of stationary iterative pro cesses and their running time are w ell studied issues. These topics are b ey ond the scop e of this pap er; we refer the reader to tw o b o oks in which one can find discussions ab out sequen tial and parallel matrix-vector multiplications and general relaxations [ 25 , 26 ]. T ypical running time of an AMG-based framew ork for linear ordering problems on graphs can b e found in [ 40 , 41 , 42 ]. The introduction of the algebraic disctance did not increase significan tly those running time estimations. 5. Multiscale distance definition and hierarc hical organization. As men- tioned in the introduction, the algebraic distance defined ab o ve is only a crude lo c al 12 graphs ordered by ratios 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 ratios between old and new multilevel solvers Fig. 4.4 . R esults for the minimum 2-p artitioning pr oblem. distanc e , measuring meaningful relativ e distances only betw een neigh b oring no des while also detecting which no des should not b e considered as close neigh b ors. This fuzzy lo cal distance, which can be calculated rapidly , is all w e need for coarsening. A similar distance is then calculated at each coarser level, thus yielding a multiscale definition of distances through the entire graph, where at large distances one defines the distances only betw een (usually large) aggregates of no des, not b etw een any indi- vidual pair of distan t nodes. Suc h m ultiscale distances are not only far less exp ensive to calculate: w e next list several reasons wh y , in principle , distances in a general large graph should b e define d b etter in such a multiscale fashion. • A t large distances the detailed individual distances (the exact trav elling time from each house in Baltimore to each house in Boston, say) are usually not of interest. • The distance in a general graph is a fuzzy notion, whose definition is to a certain exten t arbitrary . The difference betw een the t wo distances of t wo neigh b oring no des from a third, far one is muc h less than the difference b e- t ween v arious, equally legitimate distance definitions, and also far less than the accuracy of the graph data (e.g., its edge w eights) and far less than the accuracy of solving the equations that define these distances. • The most important reason: A t differen t scales different factors should in principle en ter in to the distance definition. In particular, at increasingly larger distances, intrinsic prop erties of increasingly larger aggregates should pla y a progressively more important role. F or example, in image segmenta- tion, while at the finest level the “closeness” of t wo neighboring pixels (i.e., their c hance to b elong to the same segment) can b e defined b y their color similarit y , at larger scales the closeness of tw o neigh b oring patc hes should be defined in terms of the similarit y in their aver age color (whic h is differen t from the direct color similarity of neigh b oring pixels along the b oundary b e- t ween the patches) and also in terms of similarit y in v arious texture measure (color v ariances, shap e moments of subaggregates, av erage orien tation of fine em b edded edges, etc.) and other aggregativ e prop erties [ 44 ], [ 45 ]. Another example: in the problem of identifying clusters in a large set of points in R d , at the finest level the distance b etw een data points can simply b e their Euclidean distance, while at coarser levels the distance b et ween tw o aggre- 13 gates of p oin ts should also tak e into accoun t similarit y in terms of aggregativ e prop erties, suc h as density , orientation and dimensionality [ 32 ]. • The multiscale definition of distance also brings muc h needed flexibilit y into the wa y distances at one level are conv erted in to distances at coarser levels. F or example, in a graph whose finest level consists of face images and their similarit y scores, if at some coarse level node A is the union of tw o fine- lev el no des A 1 and A 2 , and node B is the union of B 1 and B 2 , then the coarse w eight w AB of the edge ( A, B ) can b e defined either as some aver age of w A 1 B 1 , w A 1 B 2 , w A 2 B 1 , and w A 2 B 2 , or alternativ ely as the maximum (or L p a verage with large p ) of those four weigh ts. The former choice (av erage) is more suitable if one wan ts to cluster faces ha ving a similar p ose , while the latter choice (max or L p ) is more suitable if we need clusters of images each b elonging to the same p erson (or, generally , when the clustering should b e based on transitiv e similarity). An ingenious rigorous definition of distances in a general graph, in tro duced in [ 18 ], is called diffusion distanc e . Denoting b y p ( t, y | x ) the probability of a random w alk on the graph starting at x to reach y after t steps, the diffusion distance b et ween t wo no des x i and x j is defined b y d ( x i , x j , t ) 2 = X y w ( y )[ p ( t, y | x i ) − p ( t, y | x j )] 2 , (5.1) with some suitable choice of the node w eights w . This is, in fact, a m ultiscale definition of distance, with the diffusion time t serving as the scaling parameter. And indeed the definition is used for hierarchical organizations of graphs (even though large- scale distances are still defined in detail for an y pair of nodes). The calculation of our “algebraic distance” can b e view ed as just a fast wa y to compute a crude appro ximation to diffusion distances at some small t . The essential practical point is that this crude and inexp ensiv e “algebraic dis- tance” is all one needs for solving graph problems by rep eated coarsening. The cal- culation of the diffusion map (the diffusion distances at v arious scales t ) for a large graph is, on the other hand, quite exp ensive, requiring computing (possibly many) eigen vectors of the graph Laplacian. The fast wa y to calculate them should inv olve using a m ultiscale algorithm such as AMG (which is lik ely to work well in those cases where hierarchical organizations of the graph is meaningful; the AMG solv er can, by the wa y , calculate many eigen vectors for nearly the same work of calculating only one [ 33 ]). How ever, instead of calculating the diffusion map and then use it for organizing the graph, the AMG structur e c an itself b e use d dir e ctly, and mor e efficiently for any such or ganization. Indeed, as p oin ted out in [ 9 ], the same coarsening pro cedures used by the AMG solv er can directly b e used for efficien t hierarchical organizations (such as multiscale clustering) of a graph (as in [ 32 ]) or for multiscale segmen tation of an image (as in [ 44 ], [ 45 ]). As exemplified in this article (and also in [ 41 ], [ 42 ]), this kind of pro cedures can also b e used for man y other types of graph problems, in particular, it can also be used for detecting small hidden cliques in random graphs [ 20 ]. Th us, for discrete graphs, and analogously also for related con tinuum field prob- lems, although the diffusion map is a useful theoretical concept, it is often not the most practical to ol. W e b eliev e this to b e true for most if not all spectral graph metho ds (using eigen vectors of the graph Laplacian): The same AMG structure that w ould rapidly calculate the eigenv ectors can b e b etter used to directly address the 14 problem at hand. As p ointed out in the discussion of multiscale distances, this can yield not just faster solutions but also, and more imp ortan t, b etter definitions and more tunable treatmen ts for many practical problems. 6. Conclusions. W e ha ve prop osed a new measure that quan tifies the “close- ness” betw een tw o no des in a given graph. The calculation of the measure is linear in the n umber of edges in the graph and in volv es just a small n umber of relaxation sw eeps. The calculated measure is all that is required for coarsening purp oses. A similar notion of distance is then calculated and used at each coarser lev el. W e demonstrate the use of this new measure for the minim um (1,2)-sum linear order- ing problem and for the minimum 2-partitioning problem. The improv ement in the results sho ws that this measure indeed detects the most imp ortan t couplings in the graph and helps in producing a b etter coarsening, while at the same time preven ting nonlo cal v ertices from b elonging to the same coarse aggregate. Ac knowledgmen ts. This w ork w as partially funded b y the CSCAPES institute, a DOE pro ject, and in part by DOE Con tract DE-A C02-06CH11357. W e express our gratitude to Dr. C ´ edric Chev alier for providing us with a mo dification of Scotc h soft ware and helping to design the minimum 2-partitioning problem exp erimen ts. REFERENCES [1] A. Ab ou-Rjeili and G. Karypis. Multilev el algorithms for partitioning p o wer-la w graphs. In IPDPS , 2006. [2] C. J. Alp ert, J.-H. Huang, and A. B. Kahng. Multilevel circuit partitioning. In Design Au- tomation Conferenc e , pages 530–533, 1997. [3] R. Banos, C. Gil, J. Ortega, and F.G. Mon toy a. Multilevel heuristic algorithm for graph partitioning. In Applic ations of Evolutionary Computing , pages 143–153, 2003. [4] S. T. Barnard and H. D. Simon. A fast m ultilevel implemen tation of recursiv e sp ectral bisection for partitioning unstructured problems. Concurr ency: Practic e and Experienc e , 6:101–107, 1994. [5] H. W. J. Bl¨ ote, J. R. Heringa, A. Hoogland, E.W. Meyer, and T. S. Smit. Monte Carlo renormalization of the 3D Ising mo del: Analyticity and conv ergence. Phys. Rev. L ett. , 76:2613, 1996. [6] A. Brandt. Multi-level adaptiv e solutions to b oundary-v alue problems. Math. Comp. , 31(138):333–390, April 1977. [7] A. Brandt. Algebraic m ultigrid theory: The symmetric case. Jornal of Appl. Math. Comp. , 19:23–56, 1986. [8] A. Brandt. General highly accurate algebraic coarsening. Ele ctronic T r ans. Num. Anal. 10 (2000) 1-20 , 2000. [9] A. Brandt. Multiscale scientific computation: Review 2001. In T. Barth, R. Haimes, and T. Chan, editors, Multisc ale and Multir esolution methods (Pro c e e ding of the Y osemite Educ ational Symp osium, Octob er 2000) . Springer-V erlag, 2001. [10] A. Brandt. Principles of systematic upscaling. In J. Fish, editor, Bridging the Sc ales in Science and Engineering . Oxford Universit y Press, 2008. [11] A. Brandt, S. McCormic k, and J. Ruge. Algebraic multigrid (AMG) for automatic multigrid solution with application to geodetic computations. Institute for Computational Studies, F ort Collins, Colorado, POB 1852, USA, 1982. [12] A. Brandt, S. McCormick, and J. Ruge. Algebraic m ultigrid (AMG) for sparse matrix equations. In D.J. Ev ans, editor, Sparsity and its Applic ations , pages 257–284. Cam bridge Universit y Press, Cambridge, 1984. [13] A. Brandt and D. Ron. Renormalization multigrid (RMG): Statistically optimal renormaliza- tion group flow and coarse-to-fine Monte Carlo acceleration. J. Stat. Physics , 102(231), 2001. [14] A. Brandt and D. Ron. Chapter 1 : Multigrid solvers and multilevel optimization strategies. In J. Cong and J. R. Shinnerl, editors, Multilevel Optimization and VLSICAD . Kluw er, 2003. 15 [15] T. N. Bui and B. R. Mo on. Genetic algorithm and graph partitioning. IEEE T rans. Comput. , 45(7):841–855, 1996. [16] Jie Chen and Ily a Safro. Algebraic distance on graphs. T echnical Report ANL/MCS-P1696- 1009, Argonne National Lab oratory . [17] C. Chev alier and I. Safro. Comparison of coarsening sc hemes for multilev el graph partitioning. In Pr o c e e dings of the 4th L e arning and Intel ligent Optimization c onfer enc e , 2009. [18] R. R. Coifman and S. Lafon. Diffusion maps. Applie d and Computational Harmonic Analysis , 21(1):5–30, 2006. [19] T. Da vis. Universit y of florida sparse matrix collection. NA Digest , 97(23), 1997. [20] Uriel F eige and Dorit Ron. Finding hidden cliques in linear time. Submitte d , 2010. [21] C. M. Fiduccia and R. M. Mattheyses. A linear-time heuristic for impro ving net work partitions. In Pr o c e e dings of the 19th Design Automation Confer enc e , pages 175–181. IEEE, 1982. [22] M. R. Garey and D. S. Johnson. Computers and Interactability. A Guide to the The ory of NP- Completeness . A Series of Bo oks in the Mathematical Sciences. F reemann And Company , 1979. [23] M. R. Garey , D. S. Johnson, and L. Sto ckmey er. Some simplified NP-complete problems. In Pr o c e e dings of the Sixth Annual ACM Symp osium on The ory of Somputing , pages 47–63. ACM Press, 1974. [24] A. George and A. Pothen. An analysis of sp ectral env elop e reduction via quadratic assignment problems. SIAM Journal on Matrix Analysis and Applic ations , 18(3):706–732, 1997. [25] Anan th Grama, George Karypis, Anshul Gupta, and Vipin Kumar. Intr o duction to Parallel Computing: Design and Analysis of Algorithms . Addison-W esley , 2003. [26] Mic hael T. Heath. Scientific Computing . McGraw-Hill, Inc., New Y ork, NY, USA, 2002. [27] B. Hendrickson and R. W. Leland. A multi-lev el algorithm for partitioning graphs. In Sup er- c omputing , 1995. [28] Y. F. Hu and J. A. Scott. A m ultilevel algorithm for wa vefron t reduction. SIAM J. Scientific Computing , 23:2000–031, 2001. [29] G. Karypis and V. Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. T echnical Rep ort 95-035, Univ ersity of Minnesota, June 1995. [30] G. Karypis and V. Kumar. Metis A Softwar e Package for Partitioning Unstructur e d Graphs, Partitioning Meshes, and Computing Fil l-R e ducing Or derings of Sparse Matrices . Univer- sity of Minnesota, Departmen t of Computer Science and Engineering, Arm y HPC Researc h Center, Minneap olis, MN 55455, U.S.A., 4 edition, September 1998. [31] B. W. Kernighan and S. Lin. An efficient heuristic pro cedure for partitioning graphs. BELL System T e chnic al Journal , pages 291–307, F ebruary 1970. [32] D. Kushnir, M. Galun, and A. Brandt. F ast multiscale clustering and manifold iden tification. Sp e cial Issue on Similarity-Base d Pattern R e c o gnition, Pattern R e c o gnition , 39:1876–1891, 2006. [33] D. Kushnir, M. Galun, and A. Brandt. Efficient multilevel eigensolvers with applications to data analysis tasks. IEEE T ransaction on Pattern Analysis and Machine Intel ligenc e (TP AMI-2008-08-0468) , 2008. [34] H. Mey erhenke, B. Monien, and T. Sauerwald. A new diffusion-based m ultilevel algorithm for computing graph partitions of v ery high quality . In Pr o c. of the 22nd International Parallel and Distributed Pr o c essing Symp osium, (IPDPS’08) . IEEE Computer So ciet y , 2008. [35] Scotch : Static mapping, graph partitioning, and sparse matrix blo ck ordering pack age. http: //www.labri.fr/ ~ pelegrin/scotch/ . [36] A. Pothen, H. D. Simon, and K.-P . Liou. Partitioning sparse matrices with eigenv ectors of graphs. SIAM Journal of Matrix Analysis , 11(3):430–452, July 1990. [37] D. Ron, S. Wishko-Stern, and A. Brandt. An algebraic multigrid based algorithm for bisection- ing general graphs. T echnical Rep ort MCS05-01, Departmen t of Computer Science and Applied Mathematics, The W eizmann Institute of Science, 2005. [38] J. Ruge and K. St ¨ ub en. Algebraic multigrid. In S. F. McCormic k, editor, Multigrid Metho ds , pages 73–130. SIAM, Philadelphia, 1987. [39] I. Safro. Homepage of our pro jects. http://www.mcs.anl.gov/˜safr o . [40] I. Safro, D. Ron, and A. Brandt. Graph minimum linear arrangemen t by multilev el w eighted edge contractions. Journal of Algorithms , 60(1):24–41, 2006. [41] I. Safro, D. Ron, and A. Brandt. Multilevel algorithm for the minim um 2-sum problem. Journal of Graph Algorithms and Applic ations , 10(2):237–258, 2006. [42] I. Safro, D. Ron, and A. Brandt. Multilevel algorithms for linear ordering problems. Journal of Experimental Algorithmics , 13:1.4–1.20, 2008. [43] E. Sharon, A. Brandt, and R. Basri. F ast multiscale image segmentation. In Pro c e e dings IEEE Confer enc e on Computer Vision and Pattern R ec o gnition , pages 70–77, 2000. 16 [44] E. Sharon, A. Brandt, and R. Basri. F ast multiscale image segmentation. In Pro c. IEEE Conf. on Computer Vision and Pattern R e c o gnition , pages 70–77, South Carolina, 2000. [45] E. Sharon, M. Galun, D. Sharon, R. Basri, and A. Brandt. Hierarch y and adaptivity in seg- menting visual scenes. Nature , 442:810–813, 2006. [46] C. W alshaw. Multilev el refinement for combinatorial optimisation problems. Annals Oper. R es. , 131:325–372, 2004. The submitted manuscript has b een created in part by UChicago Argonne, LLC, Op erator of Argonne National Laboratory (“Argonne”). Ar- gonne, a U.S. Department of Energy Office of Sci- ence lab oratory , is operated under Contract No. DE-AC02-06CH11357. The U.S. Gov ernment re- tains for itself, and others acting on its behalf, a paid-up nonexclusive, irrevocable worldwide li- cense in said article to reproduce, prepare deriv a- tive works, distribute copies to the public, and perform publicly and displa y publicly , by or on behalf of the Governmen t. 17