cs.CC 2017-11-17 0

A Spectral Approach to Analyzing Belief Propagation for 3-Coloring

Contributing to the rigorous understanding of BP, in this paper we relate the convergence of BP to spectral properties of the graph. This encompasses a result for random graphs with a ``planted'' solution; thus, we obtain the first rigorous result on…

Authors: ** - 원저자: (논문에 명시되지 않음 – 원문에서 확인 필요) **

A Spectral Approach to Analyzing Belief Propagation for 3-Coloring Amin Coja-Oghlan ∗ Elchanan Mossel † Dan V ilenchik ‡ October 25, 2018 Abstract Belief Propagation ( BP ) is a message-passing algorithm that computes the exact marginal distributions at ev ery verte x of a graphical model without cycles. While BP is designed to work correctly on trees, it is routinely applied to general graphical models that may contain cycles, in which case neither con v ergenc e, nor correctness in the case of con ver gence is guaranteed. Nonetheless, BP gained popu larity as it seems to remain effecti v e in many cases of interest, even when the underlying graph is “far” from being a tree. Ho we ver , the theoretical understand ing of BP (and its ne w relativ e Surv ey Propagation) when applied to CSPs is poor . Contributing to the rigorous understanding of BP , in this paper we relate the conv erg ence of BP to spectral prop- erties of the graph. This encompasses a result for random graphs with a “planted” solution; thus, we obtain the first rigorous result on BP for graph coloring in the case of a complex graphical structure (as opposed t o trees). In par- ticular , the analysis sho ws how Belief Propagation breaks the symmetry between the 3! po ssible permutations of the color classes. Keywords: Belief Pro pagation , Survey P ropag ation, grap h coloring, spectral algorithms. 1 Introd uction and Results 1.1 Message Passing Algorithms This p aper deals with a rigorou s analysis of the Belief Propagation (“BP” for sh ort) algo rithm on cer tain instances o f the 3-colo ring pr oblem. Originally BP was introdu ced by Pear l [14] as a message passing algorithm to compute the marginals at the vertices o f a pro bability distribution described by an acyclic “graphical model”, i.e ., a r epresentation of the distribution’ s dep endency structu re as an acyclic graph. Alth ough in the worst case BP will fail if the g raphical representatio n featur es cycles, various version of BP are in common use as heuristics in ar tificial intelligence and statistics, wh ere they frequen tly perform well empir ically as long as the und erlying mod el does at least not contain (many) “sh ort” cycles. Ho we ver , there is currently no gener al the ory th at could explain the empirical success of BP (with the notable exception of the use of BP in LDPC decoding [11, 12, 15]). A striking recent ap plication of BP is to instances of NP-hard con straint satisfaction problems such as 3- SA T or 3-color ing; this is the type of prob lems that we are dealing with in the present w ork. In this case the prim ary objecti ve is not to comp ute the margina ls of some distribution, b ut to constru ct a solu tion to the constraint satis faction problem. For example, BP can be used to (attem pt to ) com pute a prop er 3-coloring of a given grap h. Indeed , empirically BP (and its sibling Survey Propagation “SP”) seem s to perfo rm well on problem instances that are notoriou sly “har d” for other current algorithmic appro aches, including the case of spar se r andom graphs . ∗ Carne gie Mellon Uni versit y , Depart ment of Mathematical S cienc es. E mail: amincoja@andrew.cmu.edu . Supported by DFG CO 646. † U.C. B erke ley . E-mail: mossel @stat.berkeley. edu . Supported by a S loan fe llo wship in M athemati cs, by NSF Career aw ard DMS- 0548249, NSF grant DMS-0528488 and ONR grant N0014-07-1-05-0 6 ‡ T el-A viv Uni ve rsity . E-mail: vilenchi@post.tau.ac. il . 1 For instance, let G ( n, p ) be th e r andom g raph with vertex set V = { 1 , . . . , n } that is obtained by includ ing each possible ed ge with p robability 0 < p = p ( n ) < 1 indepen dently . Th us, th e expected d egree of any vertex in G ( n, p ) is ( n − 1) p ∼ np . Then th ere e xists a threshold τ = τ ( n ) such that for any ǫ > 0 the ran dom grap h G ( n, p ) is 3 -color able with p robab ility 1 − o (1) if np < (1 − ǫ ) τ , whereas G ( n, p ) is n ot 3 -colora ble if np > (1 + ǫ ) τ [1]. In fact, random graphs G ( n, p ) with a verage degree np just b elow τ were con sidered the example of “hard” instances of the 3-colorin g prob lem, until s tatistical physicists discovered that BP/SP can solve these gr aph problems efficiently in a r egime considered “h ard” for any pr eviously k nown algorithms ( possibly righ t up to the thre shold d ensity) [4, 6]. While th ere are exciting and d eep argumen ts from statistical physics that provide a plausible explanation of why these message p assing algorith ms succeed, these argumen ts are n on-rig orous, and indeed no math ematically rigorou s analysis is currently known. The difficulty in understanding the perfor mance of BP/SP on G ( n, p ) actually lies in two aspe cts. The first aspec t is the com binatorial structur e of the random g raph G ( n, p ) with r espect to the 3-colo ring prob lem, which is no t very well under stood. In fact, ev en the basic prob lem of obtaining the precise v alue of the thresho ld τ is one of the curr ent challenges in the theor y of rand om g raphs. Furth ermore, we lack a r igorou s under standing of the “solution space geometry ”, i.e., th e structure of the set of all proper 3-co lorings of a typical ran dom graph G ( n, p ) ( e.g., how m any proper 3- coloring s are there typ ically , and wh at is th e typ ical Hamming d istance between any two). But acc ording to the statistical physics analysis, the solution space geometry affects the behavior of BP significantly . The second aspect, which we focu s on in the present work, is the actual BP algo rithm: given a gr aph G , how/why does the BP algorith m “construct” a 3-colo ring? T hus far there has been no rigor ous analysis of BP th at applies to graph coloring instances except for graphs tha t are glob ally tree-like ( such as trees or f orests). Howe ver , it seems empirically that BP perf orms well on many g raphs that are just loca lly tree like (i.e., do not contain “shor t” cycles). Therefo re, in the present p aper our go al is to analyze BP rigorously on a class of graph s that may hav e a complex combinato rial structure globa lly , but that have a very simple solu tion space g eometry . Mo re precisely , we shall re late the suc cess o f BP to spectral pr operties of the a djacency matrix o f th e in put grap h. I n ad dition, we poin t ou t that the analysis comprises a natural rando m gr aph model (namely , a “planted solution” model). 1.2 Belief Propagation and Spectral T echniques The ma in c ontribution of th is pa per is a rigoro us analysis of BP f or 3- coloring . W e basically show that if a certain (simple) spectral heuristic for 3-coloring succeeds, then so does BP . Thu s, the result does not refer to a specific random graph model, b ut to a special class of graphs – namely graphs that satis fy a certain spectral condition. Mor e precisely , we say that a g raph G = ( V , E ) on n vertices is ( d, ǫ ) -r e gular if there exists a 3-coloring o f G with co lor classes V 1 , V 2 , V 3 such that the f ollowing is true. Let ~ 1 V i ∈ R V be the vector who se en tries eq ual 1 on coo rdinates v ∈ V i , and 0 on all other coordinate s; then R1. for all 1 < i < j < 3 the vector ~ 1 V i − ~ 1 V j is an eigenv ector of the adjacency matrix A ( G ) with e igen value − d , and R2. if ξ ⊥ ~ 1 V i for all i = 1 , 2 , 3 , then || A ( G ) ξ || ≤ ǫ d || ξ || . W e shall state a few elementary prope rties of ( d, ǫ ) -regu lar gr aphs in Pro position 11 b elow (assumin g tha t ǫ is suffi- ciently small – ǫ < 0 . 0 1 , say). For instance, we shall see th at ( d, ǫ ) -regu larity implies that each vertex v ∈ V i has precisely d neig hbors in e ach other c olor class V j ( i 6 = j ). Moreover , ( V 1 , V 2 , V 3 ) is the o nly 3 -colorin g of G (up to permutatio ns of the color classes, of cou rse), and for each pair i 6 = j the bip artite graph consisting of the V i - V j -edges is an expander . Furthermo re, if a graph G is ( d, ǫ ) - regular for an y ǫ < 0 . 0 1 , say , then the fo llowing sp ectral heuristic is easily seen to produc e a 3- coloring . 1. Compute a pair of perpendicu lar eigenvectors χ 1 , χ 2 ∈ R V of A ( G ) with eigen value − d . 2. Define an equ iv alence relation ≈ on V by letting v ≈ w iff χ i v = χ i w for i = 1 , 2 . Output the eq uiv alence classes of ≈ as a 3-coloring of G . 2 The equivalence class es of ≈ are precisely the three color classes V 1 , V 2 , V 3 . For if v , w belong to the same color class, then their entries in all three vectors ~ 1 V i − ~ 1 V j ( i < j ) coin cide; hence, as the space spanned by these vectors contains χ 1 , χ 2 , we hav e v ≈ w . Conv ersely , if v ≈ w , then the entries of v and w in all the vectors ~ 1 V i − ~ 1 V j coincide, because these vectors lie in the space spanned by χ 1 , χ 2 ; consequ ently , v, w belo ng to the same color class V k . The main result of th is paper is that BP can 3-color ( d, 0 . 01) -regu lar graph s in p olynom ial time, pr ovided that d is not too small and the numbe r of vertice s is sufficiently large. W e defer the descrip tion of the actual (ran domized, polyno mial time) BP colo ring algorithm BPCol , which the following theorem refers to, to Section 2. Theorem 1. Ther e exist c onstants d 0 , κ > 0 su ch that for each d ≥ d 0 ther e is a n umber n 0 = n 0 ( d ) so that the following holds. If G = ( V , E ) is a ( d, 0 . 01) -r e gular gr aph on n = | V | ≥ n 0 vertices, then with pr ob ability ≥ κn − 1 over the coin tosses of the algorithm, BPCol ( G ) outputs a pr oper 3-colo ring of G . Observe that Th eorem 1 d eals with “sparse” gr aphs, since th e lower bo und n 0 on the number of vertices depends on d . Th e proof yields an expon ential depend ence, i. e., n 0 = exp(Θ( d )) . Con versely , this means that the average degree of G is at most lo garithmic in n , wh ich is arguab ly the mo st relev ant r egime to an alyze BP (cf . Section 2) . Moreover , by app lying BPCol O ( n ) times in depend ently , the success pr obability can be boo sted to 1 − α for any α > 0 . Besides, ther e is an easy way to modify the (initializatio n step of) BPCol so that the success p robability of one iteration is at least κ (rather than κn − 1 ), cf. Remark 4 for details. Let us em phasize that the contribution o f Theo rem 1 is no t th at we can now 3-color a class of grap hs for which no efficient algorithms were previously known, as the aforemention ed spectra l heuristic 3-co lors ( d, 0 . 01) -regula r graphs in polynomial time. In stead, the new aspect is that we can show that the Belief Pr opaga tion algo rithm 3-co lors ( d, 0 . 01) -r egular instances, thu s shedding new light on this heuristic. Indeed , the proof of Theore m 1, wh ich we present in Section 3, sh ows that in a sense BP Col “ emulates” the spectral heuristic (a lthough no spectral techniques occur in the description of BPCol ). Thus, we establish a c onnection between spectral method s an d BP . Besides, we note that no “pur ely com binatorial” alg orithm (that av oids th e u se of ad vanced techniq ues such as Semidefin ite Programm ing or spectral meth ods) is known to 3-colo r ( d, 0 . 01) -regular graphs. T o illustra te Th eorem 1 , and to p rovide an example of ( d, 0 . 0 1) -regular g raphs, we point o ut that th e m ain result comprises a regular random graph model with a “planted” 3-coloring. Let G n,d, 3 be the rand om graph with vertex set V = { 1 , . . . , 3 n } obtained as follows. 1. Let V 1 , V 2 , V 3 be a random partition of V into three pair wise disjoint sets of equal size. 2. For any pair 1 < i < j < 3 ind ependen tly choose a d - regular bipartite g raph with vertex set V i ∪ · V j unifor mly at random. For a fix ed d we say that G n,d, 3 has a certain property P with hig h pr oba bility (“w .h.p”), if the probability that G n,d, 3 enjoys P tends to 1 as n → ∞ . Con cerning G n,d, 3 , Theorem 1 implies the following. Corollary 2 . Suppo se that d ≥ d 0 is fixed . W ith hig h pr obability a rand om graph G = G n,d, 3 has the following pr o perty: with p r ob ability ≥ κn − 1 over th e coin tosses of th e algorithm, BPCol ( G ) outputs a pr op er 3-coloring o f G . T o prove Corollary 2, we show that w .h.p. G n,d, 3 is ( d, 0 . 01) -r egular , cf. Section 4. 1.3 Related W ork Alon and Kahale [2] were the first to emp loy spectral techniq ues fo r 3-coloring sparse ran dom graphs. They present a spectral heuristic and sho w that this heuristic finds a 3-c oloring in the so-called “planted solution mo del”. This model is som ewhat more d ifficult to deal with algor ithmically than th e G n,d, 3 model th at we study in the pr esent work. F or while in the G n,d, 3 -model each vertex v ∈ V i has e xactly d neighbo rs in each of the other color classes V j 6 = V i , in the planted solution model o f Alon an d Kahale th e n umber of neighbor s of v ∈ V i in V j has a Po isson d istribution with mean d . In effect, the spectral algorithm in [2] is more sophisticated than the spectral heuristic from Sectio n 1 .2. In particular, the Alon- Kahale algorithm succeeds on ( d, 0 . 01) -regular graphs (and hence on G n,d, 3 w .h.p .). There are numero us papers on the performance of message passing algorithms for constraint satisfaction problems (e.g., Belief Propag ation/Survey Propagation) b y auth ors from the statistical p hysics com munity (cf. [4, 5, 10] and 3 P S f r a g r e p l a c e m e n t s w v N ( v ) \ { w } Figure 1: the BP equ ation. the references there in). While these papers provid e rather plausible (an d in sightful) explanations for the suc cess of message passing algo rithms o n pro blem instances such as random gra phs G n,p or random k -SA T formulae, the arguments (e.g., the rep lica or th e cavity method ) are mathem atically non-rig orous. T o the be st of o ur kn owledge, no connectio n betwe en spectral methods and BP has been established in the physics literature. Feige, Mossel, and V ilench ik [8] showed that the W ar ning Propagatio n (WP) algo rithm for 3- SA T converges in polyno mial time to a satisfy ing assignm ent on a m odel of random 3- SA T instan ces with a pla nted solutio n. Since the messages in WP are add itiv e in n ature, a nd n ot multip licati ve as in BP , the WP algorith m is con ceptually mu ch simpler . Moreover , on the model stud ied in [8] a fairly simple co mbinato rial algorithm (based on the “majority vote” algorithm ) is k nown to succe ed. By contrast, no purely combin atorial algorithm (that does not rely on spectral methods or semi-definite progra mming) is kno wn to 3 -color G n,d, 3 or e ven arbitrary ( d, 0 . 01) -regular instances. A very recent paper by Y amamoto and W atanabe [16] deals with a spectral ap proach to analyzing BP for the Min- imum Bis ection problem. Their work is similar to ou rs in that the y point out that a BP-related algorithm pseudo-bp emulates spectral m ethods. Howe ver , a significant d ifference is that ps eudo-bp is a simplified version of BP that is easier to analy ze, whereas in the present work we make a p oint of analy zing the BP algorithm for co loring as it is stated in [ 4] (cf. Remark 8 for mo re detailed comm ents). Nonetheless, an in teresting aspect o f [16] certainly is that this p aper shows that BP can b e ap plied to an actual o ptimization p roblem, r ather tha n to the p roblem of just finding any feasible solution (e.g., a k -co loring). The effectiveness of message passing algorithms fo r am plifying lo cal in formation in ord er to decod e codes close to chann el capacity was recently established in a number of pa pers, e.g. [11, 12, 15]. Our resu lts are similar in flavor , howe ver the an alysis provided here allows to recover a prop er 3 -colorin g of the e ntire graph, whereas in the rando m LDPC cod es setting, m essage p assing allows to r ecover only a 1 − o (1) fraction of the codeword correctly . In [12] it is shown that for the er asure channel, all bits may be recovered correctly using a message passing algorithm, howe ver in this case the m essage passing algorith m is of combina torial n ature (all messages are either 0 or 1 ) a nd the LDPC code is designed so that message passing works for it. 2 The Belief Propagation Algorithm for 3-Coloring Follo wing [4], in this section we will describ e the basic ideas behind the BP algorithm. Since BP is a heuristic based on non-r igorou s ideas (m ainly from artificial intelligence and/or statistical p hysics), the discussion of its main ideas will lack mathem atical rigor a bit; in fact, som e of th e assump tions that BP is b ased on ( e.g., “asy mptotic indepe ndence”) may seem ridiculous at first glance. Non etheless, as we poin ted ou t in the intr oduction , BP makes up for this by being very successful empiric ally . At the e nd of this sectio n, we will state the version of BP tha t we are go ing to work with precisely . The basic strategy behind th e BP algorithm for 3-colo ring is to perform a fixed point iteratio n for ce rtain “me s- sages”, starting f rom a suitable initial assign ment. In th e ca se of 3- coloring th e me ssages co rrespond to the edges o f the graph and to the three av ailable colors. M ore precisely , to each (un directed) edge { v , w } of the grap h G = ( V , E ) and eac h c olor a ∈ { 1 , 2 , 3 } we associate two messages η a v → w from v to w abo ut a , and η a w → v from w to v abo ut a ; in g eneral, we will have η a v → w 6 = η a w → v . Thu s, the me ssages are dir ected objects. Each o f these message s η a v → w is a number between 0 an d 1 , wh ich we inter pret as the “ probab ility” that vertex v takes the co lor a in the g raph obtain ed 4 from G by rem oving w . Here “p robab ility” refers to th e choice of a random (pro per) 3-color ing of G − w , while the graph G is considered fixed. (There is an obvious s ymmetry issue with this definition , which we will discuss shortly .) Having introduced the variables η a v → w , we can set up the Belief Pr opagation Equation s for coloring, which are t he basis of the BP algor ithm. The BP equation s reflect a relationship that the probabilities η a v → w should (appro ximately) satisfy under certain assumptions on the graph G , namely that η a v → w = Q u ∈ N ( v ) \ w 1 − η a u → v P 3 b =1 Q u ∈ N ( v ) \ w 1 − η b u → v (2.1) for all edges { v , w } of G and all a ∈ { 1 , 2 , 3 } (cf. Figur e 1). The idea b ehind (2.1) is that v takes color a in the grap h G − w iff none of its n eighbor s u ∈ N ( v ) \ w has color a in G − v . Furthermore, the proba bility of this ev ent (“no u has colo r a ”) is assumed to b e (asymp totically) equal to the pr oduct Q u ∈ N ( v ) \ w 1 − η a u → v of the individual probabilities; that is, the n eighbor s u 6 = w of v are assumed to b e asymptotically independe nt . Of cour se, this assumption does not h old fo r ar bitrary graphs G . Finally , the n umerato r on the r .h. s. of (2.1) is just a normalizing term, which ensures that P 3 a =1 η a v → w = 1 . The reason why in the above discussion we refer to the prob ability that v takes color a in the graph G − w obtained by r emoving w rath er than just to the probability that v takes color a in G is that in the latter case the neigh bors u ∈ N ( v ) would never b e (asym ptotically) indepen dent – not e ven if G is a tr ee. For in this case the presence of v – more p recisely , th e existence o f th e sho rt p ath ( u , v, u ′ ) fo r any two neighb ors u, u ′ ∈ N ( v ) of v – would r ender th e colors with in th e neig hborh ood N ( v ) heavily dep endent. Similarly , if G contain s triang les, so that for so me vertices v the n eighbo rhood N ( v ) is n ot an indepen dent set, then the indep endence assump tion that is im plicit in ( 2.1) will be vio lated. Non etheless, if G do es not feature (m any) sh ort cycles – say , all th e cycles are of len gth Ω(log | V | ) as | V | → ∞ – then the BP equations (2.1) may at least be asymp totically valid. The random graph model G n,d, 3 provides an example of graphs (essentially) without such short cycles. Now , the basic idea beh ind the BP algo rithm is the follo wing. W e start with a “ reasonable” initial assignment η a v → w (0) and use (2.1) to perfo rm a fixed point iteration by letting η a v → w ( l + 1) = Q u ∈ N ( v ) \{ w } 1 − η a u → v ( l ) P 3 b =1 Q u ∈ N ( v ) \{ w } 1 − η b u → v ( l ) (2.2) for all { v , w } ∈ E and a ∈ { 1 , 2 , 3 } . As soon a s some of th e values η a v → w ( l + 1) ar e strongly “biased” toward either 0 or 1 , we try to exploit this in formation to obtain a coloring. Before we state the BP algorithm p recisely , we need to discu ss a n importan t issue with the BP e quations (2.1). Namely , in the case of 3-co loring the set of all 3- coloring s is symmetric under permuting the color classes. Ther efore, if we actu ally define η a v → w to equa l the probability w .r .t. a rand om 3 -coloring of G − w , the n trivially η a v → w = 1 3 for all a, v , w . In fact, this trivial solu tion is actually a fixed p oint of (2.2). Hen ce, we need to “break symmetry”. In particular, it is not a good idea to choo se the initial assignment η a v → w (0) = 1 3 for all a, v , w . Th erefore, we do no t start from η a v → w (0) = 1 3 , b ut we assign to each η a v → w the value 1 3 plus a small random error δ . Th e hop e is that this random error will ca use the fixed poin t iterations (2.2) to co n verge to a non trivial fixed point (oth er th an η a v → w (0) = 1 3 for all a, v, w ), and that th is fixed poin t yields sufficient information to 3 -color G . For instance, if χ : V → { 1 , 2 , 3 } is a 3-color ing of G , then η a v → w =  1 if χ ( v ) = a 0 otherwise ( a = 1 , 2 , 3; { v , w } ∈ E ) is a fixed po int of (2 .2), an d clearly the 3 -colorin g χ ca n be read o ut of the above messages easily . The algorith m BPCol is shown in Fig. 2. Observe that Step 1 ensures that 3 X a =1 η a v → w (0) = 1 for all { v , w } ∈ E . (2.3) Remark 4. Theorem 1 states that the p robability (over th e random decisions in Step 1) that BPCo l yields a proper 3-color ing of its ( d, 0 . 01 ) -regular input graph is Ω( n − 1 ) . This can b e b oosted to Ω(1) by means of th e fo llowing 5 Algorithm 3. BPCol ( G ) Input: A grap h G = ( V , E ) . Outp ut: An assignment of colo rs to the vertices of G . 1. Let δ = exp( − log 3 n ) . For each v ∈ V pe rform the f ollowing ind epend ently: choose a ∈ { 1 , 2 , 3 } u niformly at r andom and assign η a v → w (0) = 1 3 + δ and η b v → w (0) = 1 3 − δ 2 for all b ∈ { 1 , 2 , 3 } \ { a } and w ∈ N ( v ) . 2. For l = 1 , . . . , l ∗ = ⌈ log 4 n ⌉ compute η a v → w ( l + 1) using (2 .2) f or all a , v , and w . 3. For each v ∈ V an d each a ∈ { 1 , 2 , 3 } compute β a v = | N ( v ) | − 1 P u ∈ N ( v ) 1 − η a u → v ( l ∗ ) . Assign to each v ∈ V a color a ∈ { 1 , 2 , 3 } such that β a v = max b ∈{ 1 , 2 , 3 } β b v . Figure 2: the algo rithm BPCol . slightly more caref ul initialization. Instead o f choosing a ra ndom a for each v ∈ V independ ently , we choose a random p ermutation σ o f V and let W a = { σ (( a − 1) n/ 3 + 1) , . . . , σ ( an / 3) } ( a = 1 , 2 , 3 ). Then, for each v ∈ W a we set η a v → w (0) = 1 3 + δ and η b v → w (0) = 1 3 − δ 2 ( b ∈ { 1 , 2 , 3 } \ { a } , w ∈ N ( v ) ) . The pro of o f Propo sition 13 below shows that this leads to a success probability of Ω(1) . Nonetheless, we chose to state BPCol with ind ependen t decisions in its initalization, because this appears more natural (and generic) to us. Remark 5. Althou gh in the above discussion of the BP equ ation (2. 2) we refer red to “local” proper ties (such as the absence of short cycles), such local pro perties will not o ccur explicitly in o ur analysis of BPC ol . In deed, relatin g BPCol to spectral graph pr operties, the analysis has a “glo bal” character . Non etheless, various local co nditions (e.g., a relatively small num ber of short c ycles) are implicit in the “global” assumption that the grap h G is ( d, 0 . 01 ) -regular (cf. Theore m 1) . For more backgrou nd on spectral vs. comb inatorial graph properties cf. C hung and Graham [7]. Remark 6. BPCol update s the messages η a v → w “in parallel”, i.e, the message s carry “time stamp s” (cf. (2 .2)). An alternative, equally common option would be “ serial” updates, e. g., b y choosing each time a random pair v , w of adjacent vertices along with a color a ∈ { 1 , 2 , 3 } and upd ating η a v → w via (2.1). Remark 7 . BPCol exploits the result of the fixed poin t iteration (2.2) in a more straig htforward fashion than the version of BP described in [4]. Name ly , after p erform ing a fixed point iteration of (2.2), the alg orithm in [4] does not assign co lors to all vertices ( as Step 3 of BPCol d oes), but only to a small fraction (the mo st d ecisiv e o nes with respect to th e calculated values). Then, the algor ithm perform s another fix ed poin t iteration, etc. The rea son is that in the rand om graph model con sidered in [4] typically the n umber of pr oper 3-colorin gs is exponential in the num ber of vertices, whereas ( d, 0 . 01) -regular graphs ha ve on ly one 3-coloring (up to permutation s of the colors). Remark 8. Let us discuss th e essential differences between BPCol for k = 2 and th e algorithm pseudo-bp analyzed in [16]. 1. In pseud o-bp the products in (2 .1) are taken over all n eighbo rs o f v , inc luding w . This appa rently minor modification has a major impac t on the analysis. For includin g w causes the messages η a v → w to be indepe ndent of w . Consequen tly , in pseudo-bp the messages at time l are 2 | V | -dimensional objects, whereas in the present work the dimension is 2 k | E | . 2. pseudo-bp actually works with th e logar ithms ln( η a v → w ) of th e messages instead of the original η a v → w . Of course, the e quation (2.1) can be phr ased in terms of ln( η a v → w ) as ln ( η a v → w ) = F (ln( η a u → v )) u ∈ N ( v ) for some function F . Now , in pseudo-bp this non- linear function F is replaced by a trun cated linear function ˆ F . 6 3 Pr oof of Theorem 1 3.1 Pr eliminaries and Notation Throu ghout this section, we let ǫ > 0 be a sufficiently small constan t (whose v alue will be determined implicitly i n the course of the proof ). Mo reover , we k eep the assumption s from Theorem 1. Thus, we let d > d 0 for a sufficiently lar ge constant d 0 ; in particular, we assum e that d 0 > e xp( ǫ − 2 ) . In addition, we assume that n > n 0 for some sufficiently large numb er n 0 = n 0 ( d ) , an d th at G = ( V , E ) is a ( d, 0 . 01) -regular graph o n n = | V | vertices. This is reflected by the use of asympto tic notation in the analy sis, which alw ays refers to n bein g suf ficiently large. Furthermo re, we let ( V 1 , V 2 , V 3 ) be a 3- coloring of G with r espect to which the condition s R1 and R2 fr om the definition of ( d, 0 . 01) -regularity hold. ( Actually a ( d, 0 . 01) -regular graph has a un ique 3-coloring up to per mutations of the color classes, but we will not use this f act.) The fo llowing easy obser vation will be used freque ntly . Lemma 9. Let i , j ∈ { 1 , 2 , 3 } , i 6 = j . Then in G each vertex v ∈ V i has p r ecisely d neighb ors in V j . Consequently , | N ( v ) | = 2 d . Pr o of. Assume w .l.o.g. th at i = 1 and j = 2 . By condition R1 ξ = ~ 1 V i − ~ 1 V j is an eig en vector of the adja- cency matrix A ( G ) = ( a vw ) v, w ∈ V with e igenv alue − d . Henc e, lettin g η = − dξ = A ( G ) ξ , we have − d = η v = − P w ∈ N ( v ) ∩ V j a vw = −| N ( v ) ∩ V j | . Follo wing [4], we will denote the elements ( v , w ) ∈ A as v → w . Furth ermore , we shall frequently work with the vector space R = R 3 ⊗ R A . Each elemen t Γ ∈ R has a uniq ue representation Γ =   1 0 0   ⊗ Γ 1 +   0 1 0   ⊗ Γ 2 +   0 0 1   ⊗ Γ 3 with Γ i = (Γ i v → w ) v → w ∈A ∈ R A ( i = 1 , 2 , 3 ). Hence, we shall denote such a vector as Γ = (Γ i v → w ) v → w ∈A ,i ∈{ 1 , 2 , 3 } . Semantically , o ne can think of Γ i v → w as the “message” that v sen ds to w abou t co lor i . Note that the messages η a v → w ( l ) defined from Section 2 constitute vectors η ( l ) = ( η a v → w ( l )) v → w ∈A ,a ∈ { 1 , 2 , 3 } ∈ R . W e will deno te th e scalar produ ct o f vectors ξ , η as h ξ , η i . Moreover , || ξ || = p h ξ , ξ i denotes the ℓ 2 -norm . In addition, if M : R n 1 → R n 2 is linear, then we let || M || = max ξ ∈ R n 1 , || ξ || =1 || M ξ || sign ify the op erator norm of M . Further, M T denotes the transp ose of M , i.e., the unique linea r oper ator R n 2 → R n 1 such that h M ξ , η i =  ξ , M T η  for all ξ ∈ R n 1 , η ∈ R n 2 . 3.2 Outline of the Analysis In order to an alyze BP Col , we shall relate the fixed po int iteration of (2.2) to the spectral colorin g algorithm from Section 1.2. Mo re precisely , we will ap proxim ate the fixed point iteratio n of th e non- linear operatio n (2.2) by a fixed point iteration for a lin ear operato r . On e o f the key in gredien ts in the analy sis is to sho w how symmetry is broken (i.e., conv ergence to the all- 1 3 fixed point is av oided). In deed, it m ay no t be clear a priori that this will happen at all, because the rand om bias generated in Step 1 of BPCol is u ncorre lated to the planted coloring . The analysis is based on the following crucial obser vation ( cf. Corollary 12 below): after a log arithmic numb er of itera tions, for all v ∈ V i , w ∈ V j , i 6 = j the me ssages η a v → w are do minated by eigenvectors of the lin ear operato r which we u se to app roximate (2.2). Furthermore, th ese eigenvectors mirr or the co loring ( V 1 , V 2 , V 3 ) an d are (alm ost) con stant on ev ery color class V i (with basically 0 , 1 , − 1 v alues on the different colo r classes). Hence, the (ran dom) initial bias gets amplified so that the planted 3-colo ring can eventually be read ou t of the messages. T o carry out this analysis precisely , we set ∆ a v → w ( l ) = η a v → w ( l ) − 1 3 . Moreover , we let B : R → R d enote the (non -linear) operator defin ed by ( B Γ) a v → w = − 1 3 + Q u ∈ N ( v ) \ w 1 − 3 2 Γ a u → v P 3 b =1 Q u ∈ N ( v ) \ w 1 − 3 2 Γ b u → v (Γ ∈ R ) . 7 Then (2.2) can be rephrased in terms of the vectors ∆( l ) = (∆ a v → w ( l )) v → w ∈A , a ∈{ 1 , 2 , 3 } ∈ R as ∆( l + 1 ) = B ∆( l ) . (3.1) W e shall see that we can approx imate the non- linear operator B in (3.1) by th e f ollowing lin ear operator B ′ if || ∆( l ) || ∞ is small; the oper ator B ′ maps a v ector Γ = (Γ a v → w ) a ∈{ 1 , 2 , 3 } ,v → w ∈A ∈ R to the v ector B ′ (Γ) = ( B ′ (Γ) a v → w ) a,v → w ∈ R with entr ies B ′ (Γ) a v → w = − 1 2 X u ∈ N ( v ) \ w Γ a u → v + 1 6 3 X b =1 X u ∈ N ( v ) \ w Γ b u → v . (3.2) Indeed , B ′ : R → R is ju st the total deriv ati ve of B a t 0 . W e define a sequen ce Ξ( l ) by letting Ξ(0) = ∆(0) and Ξ( l ) = B ′ l Ξ(0) for l ≥ 1 , thinking of Ξ( l ) as a “linear approx imation” to ∆( l ) . As a first step, we shall simplify the oper ator B ′ a little. Lemma 10. W e have ( B ′ (Ξ( l ))) a v → w = − 1 2 P u ∈ N ( v ) \ w Ξ a u → v ( l ) for a ll l ≥ 0 , v → w ∈ A , a ∈ { 1 , 2 , 3 } . Pr o of. Step 1 of BPCol ensur es that the initial vector satisfies 3 X b =1 Ξ b u → v (0) = 3 X b =1 ∆ b u → v (0) = 0 fo r all { u, v } ∈ E ( cf. (2.3) ) . Therefo re, by induction and by th e defin ition (3.2) of B ′ we see th at P 3 b =1 Ξ b u → v ( l ) = 0 for all l ≥ 0 . Co nsequently , P 3 b =1 P u ∈ N ( v ) \ w Ξ b u → v ( l ) = 0 for all l ≥ 0 , i.e., the second summand on the r .h .s. of (3.2) v anishes. Due to Lemma 10, we may just replace B ′ by the simpler linear operator L : R → R defined by ( L Γ) a v → w = − 1 2 X u ∈ N ( v ) \ w Γ a u → v ( v → w ∈ A , a ∈ { 1 , 2 , 3 } ) , (3.3) which satisfies Ξ( l ) = L l Ξ(0) = L l ∆(0) . (3.4) W e a lso note for future referenc e that 3 X a =1 Ξ a v → w ( l ) = 0 for all v → w ∈ A , l ≥ 0 , (3.5) because (2.3) entails that (3.5) is true for l = 0 , whence the definition (3.3) of L shows that (3.5) holds for all l > 0 . In order to prove Theor em 1 , we shall first analyze the sequence Ξ( l ) and then boun d t he error || Ξ( l ) − ∆( l ) || ∞ resulting from lin earization. T o study the sequence Ξ( l ) , we investigate the d ominant eigenv alues o f L and their correspo nding eigen vectors. M ore precisely , we shall see that our assumptio n on the spectrum of the adjacency matr ix A ( G ) implies th at the dominant eig en vectors of L mirror a 3-co loring of G . W e defer the p roof of the following propo sition to Section 3.3. Proposition 11. Let e a ij ∈ R be the vector with entries ( e a ij ) b v → w =  1 if b = a , v ∈ V i , and w ∈ N ( v ) ∩ V j , 0 otherwise ( v → w ∈ A , a, b, i, j ∈ { 1 , 2 , 3 } , i 6 = j ) . Mor eover , let E b e the s pace span ned by the 18 vectors e a ij ( a, i, j ∈ { 1 , 2 , 3 } , i 6 = j ). Then L operates on E as f ollows. 8 S1. Ther e ar e p r ecisely six line arly independ ent eigen vectors { ζ a 2 , ζ a 3 : a = 1 , 2 , 3 } with eigenvalue λ = d 4 + 1 4 √ d 2 − 8 d + 4 , which satisfy || ζ a 2 − e a 12 − e a 13 + e a 21 + e a 23 || ∞ ≤ 100 d − 1 , || ζ a 3 − e a 12 − e a 13 + e a 31 + e a 32 || ∞ ≤ 100 d − 1 . (3.6) These eigen vectors ar e symmetric with r espect to the colo rs a = 1 , 2 , 3 , i.e., for any two distinct a, b ∈ { 1 , 2 , 3 } and all v → w ∈ A we h ave ( ζ a j ) a v → w = ( ζ b j ) b v → w , and ( ζ a j ) b v → w = 0 . (3.7) In addition, || ζ 1 2 || = || ζ a j || fo r all j ∈ { 2 , 3 } , a ∈ { 1 , 2 , 3 } . (3.8) S2. The thr ee vectors e a = P i 6 = j e a ij with a = 1 , 2 , 3 are eigenvectors with eigen value 1 2 − d . S3. F or all ξ ∈ E such that ξ ⊥ { e a , ζ a j : a = 1 , 2 , 3 , j = 2 , 3 } we have || L ξ || ≤ 1 2 || ξ || . S4. Furthermor e, LE ⊂ E and L T E ⊂ E . F inally , we have S5. || L 2 ξ || ≤ 0 . 01 d 2 || ξ || for all ξ ⊥ E . The eigenvectors that we are mo stly interested in are ζ a 2 , ζ a 3 ( a = 1 , 2 , 3 ) as (3.6) shows that these vectors represent the coloring ( V 1 , V 2 , V 3 ) co mpletely . As a next step , we shall show that Ξ( l ) can b e app roximated well by a linear combinatio n of the vectors ζ a 2 , ζ a 3 , provided that l is sufficiently large. T o this end, let x a i = √ n · h ∆(0) , ζ a i i || ∆(0) || · || ζ a i || ( i = 2 , 3 , a = 1 , 2 , 3) (3.9) be the projec tion of the initial vector ∆(0) = Ξ(0) onto the eigenvector ζ a i ; we shall see below that the normaliza tion in (3 .9) en sures tha t x a i is b ound ed away from 0 . Furthermo re, recalling f rom ( 3.8) that || ζ a i || = || ζ 1 2 || for all i, a , we set ν = || ∆(0) || √ n || ζ 1 2 || . (3.10) Corollary 12. S uppo se that l ≥ L 1 = 2 ⌈ lo g n ⌉ , an d that Ξ(0) ⊥ e a for a = 1 , 2 , 3 . Then Ξ a v → w ( l ) = ν λ l 3 X a =1 3 X i =2 ( x a i + o (1)) ζ a i v → w for all a ∈ { 1 , 2 , 3 } and { v , w } ∈ E . Pr o of. Sin ce b y assumption the initial vector Ξ(0) is p erpend icular to e a for a = 1 , 2 , 3 and because e 1 , e 2 , e 3 are eigenv ectors of L b y S2 , we ha ve Ξ( l ) ⊥ e a . Ther efore, we can decompose Ξ( l ) as Ξ( l ) = ξ ( l ) + 3 X a =1 3 X i =2 z a i ( l ) ζ a i , wh ere ξ ( l ) ⊥ { e a , ζ a i : i ∈ { 2 , 3 } , a ∈ { 1 , 2 , 3 }} . (3.11) Thus, to prove the coro llary we need to compute the numbers z a i ( l ) and b ound || ξ ( l ) || ∞ . W ith resp ect to the coefficients z a i ( l ) , n ote that z a i ( l ) = λ l z a i (0) , because by S1 ζ a i is an eig en vector with eigen- value λ . Mo reover , z a i (0) = || ζ a i || − 2 h Ξ(0) , ζ a i i . Hen ce, (3.9) and (3.10) yield z a i (0) = x a i · ν . Th us, z a i ( l ) = λ l ν · x a i . (3.12) T o bound the “error term” || ξ ( l ) || ∞ , we note that S3 – S5 entail || L 2 γ || ≤ 0 . 01 d 2 || γ || ≤ (0 . 3 λ ) 2 || γ || fo r all γ ⊥ { e a , ζ a i : i ∈ { 2 , 3 } , a ∈ { 1 , 2 , 3 }} , (3.13) 9 provided that d ≥ d 0 for a large enough constant d 0 > 0 . Let k = ⌊ l / 2 ⌋ . Since ξ (2 k ) = L 2 k ξ (0) , ( 3.13) implies that || ξ (2 k ) || = || L 2 k ξ (0) || ≤ (0 . 3 λ ) 2 k || ξ (0) || ≤ (0 . 3 λ ) 2 k || Ξ(0) || . (3.14) Moreover , as l ≤ 2 k + 1 an d || L || ≤ d − 1 2 by Proposition 11, (3.14) yields || ξ ( l ) || ∞ ≤ || ξ ( l ) || ≤ d || ξ (2 k ) || ≤ d (0 . 3 λ ) l || Ξ(0) || . (3.15) Finally , if l ≥ L 1 , then d (0 . 3 λ ) l || Ξ(0) || = o ( λ l ν ) . Th us, the assertion follows from (3 .11), (3.12), and (3.15). While in the initial vector ∆(0) = Ξ(0) th e messages ar e co mpletely unco rrelated with the coloring ( V 1 , V 2 , V 3 ) , Corollary 12 entails tha t the domin ant con tribution to Ξ( L 1 ) com es fr om th e eigenvectors ζ a i , wh ich represent that coloring . T his implies that all vertices v in each class V a send essentially th e same messages to all oth er vertice s w ∈ V b about e ach of the colors 1 , 2 , 3 , and these messages ar e solely determine d by the initial projections x a i of ∆(0) on to ζ a i . Hence, after L 1 iterations the messages ar e essentially coherent a nd stron gly cor related to the p lanted coloring . Thu s, as a n ext step we an alyze th e d istribution of the projection s x a i . T o simplify the expression r esulting from Corollary 12, let y a 1 = x a 2 + x a 3 , y a 2 = − x a 2 , and y a 3 = − x a 3 . (3.16) Then (3.6) and Corollary 12 entail that for all v ∈ V i , all w ∈ N ( v ) , and l ≥ L 1 we hav e Ξ a v → w ( l ) = ( y a i + o (1)) · ν λ l . Of course, the numb ers y a i only depend on the initial vector ∆(0) . Therefo re, we say that ∆(0) is feasible if F1. ∆(0) ⊥ e a for a = 1 , 2 , 3 , and F2. for any pair a, b ∈ { 1 , 2 , 3 } , a 6 = b we have | y a a − 1 | < exp( − 1 /ǫ ) an d | y b a + 0 . 5 | < ex p( − 1 /ǫ ) . (3.17) Proposition 13. W ith pr obability Ω( n − 1 ) over the random bits used in Step 1 of BPCol ∆(0) is feasible. The elem entary (tho ugh tedio us) proo f of Proposition 1 3 can b e f ound in Section 3.4. Com bining Corollary 12 and Proposition 13, we conclude that with probability Ω( n − 1 ) (n amely , if ∆(0) is fea sible) we ha ve 0 . 49 ν λ l ≤ || Ξ( l ) || ∞ ≤ 1 . 1 ν λ l ( l ≥ L 1 ) . (3.18) Having o btained a sufficient understand ing o f the sequ ence Ξ( l ) , we will now show that these vectors provide a good approx imation to the vector s ∆( l ) , which we ar e ac tually interested in . The proo f of the following pro position can be found in Section 3.5. Proposition 14 . Sup pose that ∆(0) is feasible. Let L 2 > 0 be the maximum integer such that || Ξ( L 2 ) || ∞ ≤ ǫ . Then || Ξ( L 2 ) − ∆( L 2 ) || ∞ ≤ − log( ǫ ) · || Ξ( L 2 ) || 2 ∞ . Combining the inf ormation on the sequ ence Ξ( l ) provided by Corollary 1 2 and Pr oposition 13 with th e b ound on || Ξ( L 2 ) − ∆( L 2 ) || ∞ from Proposition 1 4, we can show that the messages obtained in the next o ne or two steps of the algorithm already represent the coloring rather well. T o be precise, let us call the vector η ( l ) pr oper if ∀ a ∈ { 1 , 2 , 3 } , b ∈ { 1 , 2 , 3 } \ { a } , v ∈ V a , w ∈ N ( v ) : η a v → w ( l ) ≥ 0 . 99 ∧ η b v → w ( l ) ≤ 0 . 01 . Proposition 15. If ∆(0) is feasible, then for either L 3 = L 2 + 1 or L 3 = L 2 + 2 the vector η ( L 3 ) is pr oper . The proof of Proposition 15 is the content of Section 3.6. Proposition 15 shows that the “rounding proce dure” in Step 3 of BPCol applied to the messages η ( L 3 ) would yield the coloring ( V 1 , V 2 , V 3 ) . Howe ver, BPC ol ac tually applies that round ing pr ocedure to η ( l ∗ ) , wh ere l ∗ > L 3 . Therefo re, in o rder to show that BPCol outputs a prope r 3- coloring , we ne ed to sh ow tha t these me ssages η ( l ∗ ) are proper, too. 10 Lemma 16. If η ( l ) is pr oper , then so is η ( l + 1 ) . Pr o of. L et v ∈ V a for some 1 ≤ a ≤ 3 , w ∈ N ( v ) , and { b, c } = { 1 , 2 , 3 } \ { a } . Since η ( l ) is proper, we hav e Y u ∈ V c ∩ N ( v ) \ w 1 − η a u → v ( l ) 1 − η b u → v ( l ) ≥ Y u ∈ V c ∩ N ( v ) \ w 1 − η a u → v ( l ) ≥ 0 . 99 2 d , (3.19) Y u ∈ V b ∩ N ( v ) \ w 1 − η a u → v ( l ) 1 − η b u → v ( l ) ≥  0 . 99 0 . 01  2 d − 1 = 99 2 d − 1 . (3.20) Consequently , the definition (2.2) of the sequen ce η ( l ) shows that η a v → w ( l + 1) η b v → w ( l + 1) = Y u ∈ V c ∩ N ( v ) \ w 1 − η a u → v ( l ) 1 − η b u → v ( l ) · Y u ∈ V b ∩ N ( v ) \ w 1 − η a u → v ( l ) 1 − η b u → v ( l ) ≥ 0 . 01 ·  (0 . 99) 2 0 . 01  2 d ≥ 0 . 01 · 90 2 d ≥ 1000 . (3.21) As the construction (2.2) of η ( l + 1) ensur es that η 1 v → w ( l + 1) + η 2 v → w ( l + 1) + η 3 v → w ( l + 1) = 1 , (3.21) entails that η a v → w ( l + 1) ≥ 0 . 99 and η b v → w ( l + 1) ≤ 0 . 01 , whence η ( l + 1) is pro per . Pr o of of Theor em 1. Pro position 13 states that ∆(0) is feasible with probab ility Ω( n − 1 ) . Th erefor e, to establish Theorem 1, we show t hat BPCol o utputs the coloring ( V 1 , V 2 , V 3 ) if ∆(0 ) is feasib le. Thus, assume th at ∆(0) is f easible a nd let L 2 be the max imum integer such that || Ξ( L 2 ) || ∞ ≤ ǫ . Then Corol- lary 12 implies that L 2 = Θ(log 3 n ) , because || Ξ( 0) || ∞ = δ = exp( − log 3 n ) , and the ℓ ∞ -norm of Ξ( l ) grows by a factor of λ in each iteration. T herefor e, Proposition 15 entails that η ( L 3 ) is proper for some L 3 = Θ(log 3 n ) . Thu s, by Lemma 16 the final η ( ℓ ∗ ) gener ated in Step 2 is proper, whence Step 3 of BPCol outputs the coloring V 1 , V 2 , V 3 . 3.3 Pr oof of Proposition 11 The op eration (3 .3) o f L is symmetric with respect to th e th ree co lors a = 1 , 2 , 3 . T herefor e, we shall represen t L as a tensor p roduc t of a 3 × 3 matrix and an opera tor that rep resents the gr aph G . T o this end , we define operato rs M : R A → R A and K : R A → R A by ( M Ξ) v → w = X u ∈ N ( v ) Ξ u → v , ( K Ξ) v → w = Ξ w → v (Ξ ∈ R A ) . (3.22) Thus, − 1 2 (( M − K )Ξ) v → w = − 1 2 X u ∈ N ( v ) \ w Ξ u → v , i.e., − 1 2 ( M − K ) represents the operation of L with respect to a single color a ∈ { 1 , 2 , 3 } . The refore, we can reph rase the definition (3.3) of L on th e space R = R 3 ⊗ R A as L = − 1 2   1 0 0 0 1 0 0 0 1   ⊗ ( M − K ) . (3.23) Hence, in order to understand L , we basically need to analyze M − K . For i, j ∈ { 1 , 2 , 3 } we defin e vectors e ij ∈ R A by letting ( e ij ) v → w =  1 if v ∈ V i , w ∈ V j , and w ∈ N ( v ) , 0 other wise. The fo llowing lemma sh ows that it makes sense to split th e an alysis of M − K into two parts: first w e shall analyze how M − K o perates on the space E 0 spanned by the vectors e ij ( 1 ≤ i, j ≤ 3 , i 6 = j ); then, we will study the operation of M − K o n E ⊥ 0 . 11 Lemma 17. If ξ ∈ E 0 , then M ξ , M T ξ , K ξ , K T ξ ∈ E 0 . Pr o of. L et i, j, k ∈ { 1 , 2 , 3 } be p airwise distinct. Since K e ij = e j i , we h av e KE 0 ⊂ E 0 . Moreover , K T = K . Furthermo re, by Lemm a 9 ( M e ij ) v → w = X u ∈ N ( v ) ( e ij ) u → v =  d if v ∈ V j , 0 other wise. (3.24) Hence, M e ij = d ( e j k + e j i ) , and thus ME 0 ⊂ E 0 . In add ition, the transpose of M is given by ( M T Ξ) v → w = X u ∈ N ( w ) Ξ w → u . Therefo re, ( M T e ij ) v → w = X u ∈ N ( w ) ( e ij ) w → u =  d if v ∈ V i , 0 o therwise. Consequently , M T e ij = d ( e ij + e ik ) , whence M T E 0 ⊂ E 0 . T o study the operation of M − K on E 0 , note that (3.24) implies that ( M − K ) e ij = de j k + ( d − 1) e j i , if i, j, k ∈ { 1 , 2 , 3 } are pairwise distinct. Ther efore, with respec t to the b asis e 12 , e 23 , e 21 , e 23 , e 31 , e 32 of E 0 , we can represent the operation of M − K on E 0 by the 6 × 6 matrix M =         0 0 d − 1 0 d 0 0 0 d 0 d − 1 0 d − 1 0 0 0 0 d d 0 0 0 0 d − 1 0 d − 1 0 d 0 0 0 d 0 d − 1 0 0         . Observe that M is not symmetric. Hence, a priori it is not clear that M is diag onalizable with real eigenv alues. Nev ertheless, a (very tedio us) direct computation yields the follo wing. Lemma 18. The 6 × 6 matrix M is diagonalizable an d has the non-zer o eig en values 1 , 2 d − 1 , Λ = − d 2 − √ d 2 − 8 d + 4 2 , Λ ′ = − d 2 + √ d 2 − 8 d + 4 2 . (3.25) The eigenspace w ith eigen value 2 d − 1 is spanned by ~ 1 . Mor eover , ther e ar e two mutually perpendicular eigenvector s ζ ′ 2 , ζ ′ 3 with eigen value Λ , w hich satisfy || ζ ′ 2 − (1 , 1 , − 1 , − 1 , 0 , 0) T || ∞ ≤ 10 d , || ζ ′ 3 − (1 , 1 , 0 , 0 , − 1 , − 1) T || ∞ ≤ 10 d and || ζ ′ 2 || = || ζ ′ 3 || . Since M describes the operation of M − K on the subspac e E 0 , Lemma 18 implies the following. Corollary 19. Restricted to th e sub space E 0 , the op erator M − K is diagon alizable with non-zer o eigen values 1 , 2 d − 1 , and Λ , Λ ′ as in (3.25). The vector e ∗ = P i 6 = j e ij spans the eigenspace of 2 d − 1 . Furthermore , ther e are two mutually perpendicu lar eigenvector s ζ 2 , ζ 3 with eigen value Λ , w hich satisfy || ζ 2 − ( e 12 + e 13 − e 21 − e 23 ) || ∞ ≤ 10 d , || ζ 3 − ( e 12 + e 13 − e 31 − e 32 ) || ∞ ≤ 10 d . 12 Corollary 19 describes the op eration of M − K on E 0 completely . Therefore, as a next step we shall an alyze how M − K o perates on E ⊥ 0 . M ore precisely , our goal is to show that restricted to E ⊥ 0 the nor m of M − K is sign ificantly smaller than Λ . T o this end, we observe that the operator K merely permutes the coordinate s. Consequently , || K || ≤ 1 . (3.26) T o to bound the norm of M on E ⊥ 0 , we consider thre e subspace s of E ⊥ 0 . The first subspace S con sists of all vectors ξ ∈ E ⊥ 0 such that the value ξ v → w only depends on the “start vertex” v ; in sym bols, S = { ξ ∈ E ⊥ 0 : ∀ v → w , v → u ∈ A : ξ v → w = ξ v → u } . If ξ ∈ S and v ∈ V , then we let ξ v → = ξ v → w for any w ∈ N ( v ) , i.e., ξ v → is the “outgoing value” of v . The second subspace T co nsists of all ξ ∈ E ⊥ 0 such that ξ u → v depend s only on the “target vertex” v , i.e. , T = { ξ ∈ E ⊥ 0 : ∀ u → v , w → v ∈ A : ξ u → v = ξ w → v } . For ξ ∈ T and v ∈ V we let ξ → v = ξ u → v for any u ∈ N ( v ) , i.e ., ξ → v signifies the “incomin g value” of v . Furthermo re, the third subspace U consists of all ξ such that for a ny vertex the sum of th e “in coming” v alues equals 0 : U =    ξ ∈ E ⊥ 0 : ∀ v ∈ V : X u ∈ N ( v ) ξ u → v = 0    . Lemma 20. 1. W e have U = K ern ( M ) ∩ E ⊥ 0 . 2. Mo r eover , if ξ ∈ T , then ( M ξ ) v → w = 2 dξ → v for all v → w ∈ A . In pa rticular , M ξ ∈ S . 3. F urthermor e, T ⊥ U , and E ⊥ 0 = T ⊕ U . Pr o of. T he first ass ertion follows immediately f rom the definition (3.22) of M . Mo reover , if ξ ∈ T , then ( M ξ ) v → w = P u ∈ N ( v ) ξ u → v = | N ( v ) | ξ → v = 2 dξ → v due to Lemma 9, whence 2. follows. Consequently , if ξ ∈ T an d η ∈ U , then h ξ , η i = X u → v ∈A ξ u → v η u → v = 2 d X v ∈ V ξ → v X u ∈ N ( v ) η u → v = 0 , whence T ⊥ U . Further more, for any γ ∈ E ⊥ 0 the vector η with entries η v → w = 1 2 d X u ∈ N ( w ) ξ u → w lies in T , because the sum on th e r .h.s. is indepe ndent of v . I n addition, ξ = γ − η satisfies X u ∈ N ( v ) ξ u → v =   X u ∈ N ( v ) γ u → v   − 2 dη → v = 0 fo r any v ∈ V , so that ξ ∈ U . Hence, any γ ∈ E ⊥ 0 can be written as γ = η + ξ with η ∈ T an d ξ ∈ U , i.e., E ⊥ 0 = T ⊕ U . By now we h av e all the prerequisites to analyze the operation of M on E ⊥ 0 . Lemma 21. If ξ ∈ E ⊥ 0 , then || M 2 ξ || ≤ 0 . 01 d 2 || ξ || . Pr o of. L et ξ ∈ E ⊥ 0 . By the th ird part of Lem ma 20 there is a d ecompo sition ξ = ξ T + ξ U such that ξ T ∈ T and ξ U ∈ U . Fu rthermo re, the first par t of Lemma 2 0 en tails that M ξ = M ξ T . Therefore, we may assum e witho ut loss of generality that ξ = ξ T ∈ T . He nce, the second part of of Lemma 20 implies that || ξ ′ || = 2 d || ξ || (3 .27) 13 and ξ ′ = M ξ ∈ S . Consequ ently , letting ξ ′′ = M ξ ′ = M 2 ξ , we ob tain ξ ′′ v → w = X u ∈ N ( v ) ξ ′ u → v = X u ∈ N ( v ) ξ ′ u → . (3.28) Since the r .h.s. of (3.2 8) is independen t of w , we co nclude ξ ′′ ∈ S . In order to bou nd || ξ ′′ || = || M 2 ξ || , we sh all express th e sum on the r .h.s. o f (3.28) in term s of th e ad jacency matrix A ( G ) . T o this end, con sider the tw o vectors η ′ = ( η ′ v ) v ∈ V ∈ R V with η ′ v = ξ ′ v → , η ′′ = ( η ′′ v ) v ∈ V ∈ R V with η ′′ v = ξ ′′ v → for all v ∈ V . Th en || ξ ′ || 2 = X v → w ∈A ξ ′ v → w 2 = 2 d X v ∈ V ξ ′ v → 2 = 2 d || η ′ || 2 , and analog ously (3.29) || ξ ′′ || 2 = 2 d || η ′′ || 2 . (3.30) Furthermo re, (3.28 ) implies that η ′′ v = P u ∈ N ( v ) η ′ u for all v ∈ V , i.e., η ′′ = A ( G ) η ′ . (3.31) Combining (3.27), (3.29), (3.30), and (3.31), we obtain || M 2 ξ || = || ξ ′′ || = 2 d || A ( G ) η ′ || || η ′ || · || ξ || . (3.32) Hence, we finally need to b ound || A ( G ) η ′ || . T o this end, we sh all employ ou r assumption that G is ( d, 0 . 01) - regular; na mely , conditio n R2 from the d efinition of ( d, 0 . 01) -regular ity entails tha t || A ( G ) ζ || ≤ 0 . 001 d || ζ || fo r all ζ ⊥ ~ 1 V 1 , ~ 1 V 2 , ~ 1 V 3 . Thu s, we need to sho w that η ′ ⊥ ~ 1 V i for i = 1 , 2 , 3 . Assumin g w .l.o.g. that i = 1 , we have D η ′ , ~ 1 V 1 E = X v ∈ V 1 ξ ′ v → = (2 d ) − 1 X v → w ∈A : v ∈ V 1 ξ ′ v → w = (2 d ) − 1 h ξ ′ , e 12 + e 13 i = (2 d ) − 1 hM ξ , e 12 + e 13 i = (2 d ) − 1  ξ , M T ( e 12 + e 13 )  . (3.33) Further, as M T ( e 12 + e 13 ) ∈ E 0 by Lemm a 17, while ξ ∈ E ⊥ 0 by ou r assumption, (3.3 3) implies that D η ′ , ~ 1 V 1 E = 0 . Consequently , we obtain that || A ( G ) η ′ || ≤ 0 . 0 01 d || η ′ || , when ce (3.32) yields the assertion. Pr o of of Pr oposition 11. Combining C orollary 19 with the tensor product representa tion (3.2 3) of L , we con clude that the six vectors ζ 1 j =   1 0 0   ⊗ ζ j , ζ 2 j =   0 1 0   ⊗ ζ j , ζ 3 j =   0 0 1   ⊗ ζ j ( j = 2 , 3 ) (3.34) are eig en vectors o f L with eigenv alue λ = − 1 2 Λ . In addition, the ten sor represen tation (3.34) of the vectors ζ a j immediately implies the symme try statem ent (3.7), while (3. 8) follows from Corollar y 19. Moreover , once more by Corollary 19 the three vectors e 1 =   1 0 0   ⊗ e ∗ , e 2 =   0 1 0   ⊗ e ∗ , e 3 =   0 0 1   ⊗ e ∗ are eigenv alues with eigen vector − 1 2 (2 d − 1 ) = 1 2 − d , and all other eigen values of L re stricted to E are ≤ 1 2 in absolute value. In addition , Lemma 1 7 shows in combin ation with (3.2 3) that LE , L T E ⊂ E . Finally , Lemma 21 implies in combinatio n with (3. 23) that || L 2 ξ || ≤ 0 . 01 d 2 || ξ || for all ξ ⊥ E . 14 3.4 Pr oof of Proposition 13 Before we get to the proof, let us briefly discuss why the assertion (i.e., Propo sition 13) is plausible. In fact, let us po int out that the vector ∆(0) is easily seen to satisfy F2 with pro bability Ω(1) . For each of the inner pro ducts h ∆(0) , ζ a i i is a sum of n inde penden t rand om v ariables, whence the central limit theorem implies that √ n || ∆(0) || − 1 || ζ a i || − 1 h ∆(0) , ζ a i i is asymptotically nor mal ( the factor √ n || ∆(0) || − 1 || ζ a i || − 1 , wh ich is ind ependen t o f the rand om vector ∆(0) , is needed to ensure that me an an d variance a re o f o rder Θ (1) ). In fact, since th e vector s ( ζ a i ) a =1 , 2 , 3; i =2 , 3 are mutually perpen dicular, the joint distribution of the rand om variables ( √ n || ∆(0) || − 1 || ζ a i || − 1 h ∆(0) , ζ a i i ) i =2 , 3; a =1 , 2 , 3 is asymptotically a (multiv ariate) Gaussian. Therefore, the probability that ∆(0) satisfies F2 is Ω(0) . Howe ver , once w e co ndition on ∆(0) satisfying F1 , the en tries o f ∆(0) are not independen t anymore, wh ence the above argumen t do es not yield a bo und on the probab ility th at ∆(0) satisfies both F 1 and F2 . Non etheless, th e depend ence of the entr ies of ∆(0) is weak enou gh to allow for an elemen tary direct analysis. W e begin with bounding the probab ility that ∆(0) satisfies F1 . T o this end, we define a partition ( W 1 , W 2 , W 3 ) of V by letting W i = { v ∈ V : ∆ i v → w = δ for all w ∈ N ( v ) } ; in other words, W i consists of all vertices for which the random number a chosen in Step 1 of BPCol was equ al to i . Lemma 22. The pr obability that ∆(0) satisfies F 1 is Ω( n − 1 ) . Pr o of. A sufficient con dition for ∆(0) to satisfy F1 is tha t W 1 = W 2 = W 3 = n 3 . Moreover, the total numb er of vectors that can be g enerated by Step 1 of BPCol eq uals 3 n , ou t of whic h  n n/ 3 n/ 3 n/ 3  yield W 1 = W 2 = W 3 = n 3 . Therefo re, t he assertion follows from Stirling’ s formula. In th e r emain der of this section we cond ition on the event that ∆(0 ) is such th at W 1 = W 2 = W 3 . Thus, ( W 1 , W 2 , W 3 ) , is just a rando m partition o f V into three c lasses of equal size, and for all v ∈ W i , all j ∈ { 1 , 2 , 3 } \ { i } , and all w ∈ N ( v ) we have ∆ i v → w = δ, ∆ j v → w = − δ 2 . Lemma 23. F or an y constant c 1 > 0 there e xists a consta nt c 2 > 0 such tha t the following holds. If ( s a i ) i,a =1 , 2 , 3 ar e inte gers of ab solute value | s a i | ≤ c 1 √ n such that P 3 a =1 s a j = P 3 i =1 s b i = 0 for all 1 ≤ b, j ≤ 3 , then P h ∀ 1 ≤ a, i ≤ 3 : | V a ∩ W i | = n 9 + s a i i ≥ c 2 n − 2 . Pr o of. T he sets W 1 , W 2 , W 3 are rando mly chosen mu tually d isjoint su bsets of V of cardinality n/ 3 each , wherea s V 1 , V 2 , V 3 are fixed s ubsets of V . Ther efore, the t otal number of ways to choose W 1 , W 2 , W 3 is giv en by the multino- mial coefficient  n n/ 3 ,n/ 3 ,n/ 3  ; by Stirling’ s fo rmula,  n n/ 3 , n / 3 , n/ 3  ≤ 10 n − 1 3 n . (3.35) Moreover , the number of ways to choose W 1 , W 2 , W 3 such that | V a ∩ W i | = s a i equals 3 Y a =1  n/ 3 n/ 9 + s a 1 , n/ 9 + s a 2 , n/ 9 + s a 3  (3.36) 15 (because the a ’th factor on the r .h.s. equals the nu mber o f ways to partition V a into three p ieces V a ∩ W 1 , V a ∩ W 2 , V a ∩ W 3 of the desired sizes). Comb ining (3.35) and (3.36) with Stirling’ s form ula, we get P h ∀ 1 ≤ a, i ≤ 3 : | V a ∩ W i | = n 9 + s a i i ≥ n ( n/ 3)! 3 10 · 3 n Q 1 ≤ i,a ≤ 3 ( n/ 9 + s a i )! ≥ n 5 / 2+ n 10 · (9e) n Q 1 ≤ i,a ≤ 3 ( n/ 9 + s a i )! . (3.37) Furthermo re, once mo re due to Stirling’ s form ula, ( n/ 9 + s a i )! ≤ exp( − n/ 9 − s a i )( n/ 9 + s a i ) n/ 9+ s a i √ n = exp( − n/ 9 − s a i )( n/ 9) n/ 9+ s a i (1 + 9 s a i /n ) n/ 9+ s a i √ n ≤ exp( − n/ 9 + 9 s a i 2 /n )( n/ 9) n/ 9+ s a i . ( 3.38) Since we are assuming that s a i ≤ c 1 √ n and P 3 i =1 s a i = 0 , (3.38) entails that Y 1 ≤ i,a ≤ 3 ( n/ 9 + s a i )! ≤ ( n/ 9e) n n 9 / 2 exp(9 X a,i s a i 2 /n ) ≤ c ′ 2 ( n/ 9e) n n 9 / 2 (3.39) for a bound ed num ber c ′ 2 that depend s only on c 1 . Fin ally , pluggin g (3.3 9) in to ( 3.37) and can celling, we obtain the assertion. Corollary 24. F or any two constan ts c 3 , β > 0 there exists a constant c 4 > 0 such that the following holds. If ( t a i ) i,a =1 , 2 , 3 ar e numbers of ab solute value | t a i | ≤ c 3 such that P 3 a =1 t a j = P 3 i =1 t b i = 0 for all 1 ≤ b, j ≤ 3 , then P h ∀ 1 ≤ a, i ≤ 3 : | n − 1 2 ( | V a ∩ W i | − n 9 ) − t a i | ≤ β i ≥ c 4 . Pr o of. L et S be the set of all tuples ( s a i ) a,i =1 , 2 , 3 of integers such that | n − 1 2 s b j − t b j | ≤ β , and P 3 a =1 s a j = P 3 i =1 s b i = 0 for all 1 ≤ b, j ≤ 3 . Then | S | ≥ β 4 n 2 / 32 . Mor eover , all ( s a i ) a,i =1 , 2 , 3 ∈ S satisfy | s b j | ≤ ( c 3 + 1) √ n ( 1 ≤ b, j ≤ 3 ). Therefo re, Lemma 23 (ap plied with c 1 = c 3 + 1 ) shows that P h ∀ a, i : | n − 1 2 ( | V a ∩ W i | − n 9 ) − t a i | ≤ β i ≥ X ( s a i ) ∈ S P h ∀ a, i : | V a ∩ W i | = n 9 + s a i i ≥ c 2 | S | n − 2 ≥ β 4 c 2 / 32 , as desired. Since th e vector ∆(0) just represents the p artition W 1 , W 2 , W 3 , an d the vectors P j 6 = i e a ij just re presents the col- oring V 1 , V 2 , V 3 , Corollary 24 easily implies a result on the joint distribution of the inner products D ∆(0) , P j 6 = i e a ij E . Corollary 25. F or any t wo constants c 5 , γ > 0 th er e e xists a constant c 6 > 0 such that the following is true. Supp ose that ( z a i ) 1 ≤ a,i ≤ 3 ar e numbers suc h that | z b j | ≤ c 5 and P 3 i =1 z b i = P 3 a =1 z a j = 0 for all 1 ≤ b, j ≤ 3 . Then P   ∀ a, i : | z a i − D ∆(0) , P j 6 = i e a ij E || ∆(0) || || ζ a i || · √ n | ≤ γ   ≥ c 6 . Pr o of. T he definition of η (0) in Step 1 of BPCol shows t hat ∆ a v → w (0) = η a v → w (0) − 1 3 =  δ if v ∈ W a , − δ / 2 o therwise. for all v → w ∈ A . (3.40) 16 Therefo re, || ∆(0) || = p 3 dn/ 2 · δ. (3.41) Moreover , by Proposition 11 there is a numb er 0 . 9 9 ≤ c 7 ≤ 1 . 0 1 such that || ζ a i || = c 7 || e a 12 + e a 13 − e a 21 − e a 23 || = 2 c 7 √ dn. (3.42) Furthermo re, using (3. 40), we can easily compute the scalar produc t D ∆(0) , P j 6 = i e a ij E ( 1 ≤ a, i ≤ 3 ): * ∆(0) , X j 6 = i e a ij + = X v → w ∈A : v ∈ V i ∆ a v → w (0) = | V i ∩ W a | · dδ − | V i \ W a | · dδ 2 = 3 dδ 2 ( | V i ∩ W a | − n/ 9) [because | V i | = | W a | = n/ 3 ] . (3.43) Combining (3.41), (3.42), and (3.43), we conclud e that f or a certain constant c 8 > 0 D ∆(0) , P j 6 = i e a ij E || ∆(0) || || ζ a i || · √ n = c 8 √ n · ( | V i ∩ W a | − n/ 9) . Therefo re, t he assertion follows from C orollary 24 by setting s a i = c − 1 4 √ n · z a i and β = γ /c 8 . Pr o of of Pr op osition 13. Let α = exp( − 1 /ǫ ) and ˆ x a i =  − 1 if a = i, 1 / 2 otherwise ( i = 2 , 3; a = 1 , 2 , 3) . (3.44) Then the definition s (3 .9) and (3.16) of the v ariables x a i and y a i entail that P [ ∀ a, i ∈ { 1 , 2 , 3 } , i 6 = a : | y a a − 1 | < α ∧ | y a i − 1 / 2 | < α ] ≥ P [ ∀ a, i ∈ { 1 , 2 , 3 } : | x a i − ˆ x a i | < α/ 2] . (3.45) Therefo re, we shall de riv e a lower bound on P [ ∀ a, i : | x a i − ˆ x a i | < α/ 2] . T o this end, let e a i = X j ∈{ 1 , 2 , 3 }\{ i } e a ij (1 ≤ a, i ≤ 3) , and let V ⊂ R A be the space spanned by these nine vectors. In addition, let q : R A → V be the orthog onal projection onto V . Since the constru ction of the initial vector ∆(0) in Step 1 of BPCol ensures that ∆(0) ∈ V , we have || ∆(0) || · || ζ a i || √ n · x a i = h ∆(0) , ζ a i i = h q ∆(0) , ζ a i i = h ∆(0) , q ζ a i i . Hence, instead of the v ectors ζ a i we may w ork with their projections q ζ a i onto V . Thus, let q a ij ∈ R be the coefficients such that q ζ a i = 3 X j =1 q a ij e a j ( i = 2 , 3 , a = 1 , 2 , 3 ) . Then by symme try we have q a ij = q b ij for all 1 ≤ a, b ≤ 3 ; theref ore, we will briefly write q ij instead o f q a ij . Furthermo re, (3.6 ) implies the boun ds 0 . 99 ≤ q 21 ≤ 1 . 0 1 , − 1 . 0 1 ≤ q 22 ≤ − 0 . 99 , − 0 . 01 ≤ q 23 ≤ 0 . 0 1 , (3.46) 0 . 99 ≤ q 31 ≤ 1 . 01 , − 0 . 01 ≤ q 32 ≤ − 0 . 0 1 , − 1 . 01 ≤ q 33 ≤ − 0 . 99 . (3.47) 17 As a consequ ence, the matrix Q =   q 21 q 22 q 23 q 31 q 32 q 33 1 1 1   is regular , and the re is a constant c 9 > 0 such that || Q − 1 || ≤ c 9 . Let   z a 1 z a 2 z a 3   = Q − 1   ˆ x a 2 ˆ x a 3 0   ( a = 1 , 2 , 3) . (3.48) Since || Q − 1 || ≤ c 9 and | x a i | ≤ 1 f or all a, i , we have | z a i | ≤ 5 c 9 ( 1 ≤ a, i ≤ 3 ) . (3.49) In addition, (3.44) and (3.48) imply that 3 X a =1   z a 1 z a 2 z a 3   = Q − 1   3 X a =1   x a 2 x a 3 0     = 0 , and (3.50) 3 X i =1 z b i = 0 (1 ≤ b ≤ 3) . (3.51) Combining (3.49)–(3.51), we see that ( z a i ) 1 ≤ a,i ≤ 3 satisfies the assumptions of Corollary 25, whence P  ∀ a, i : | z a i − h ∆(0) , e a i i || ∆(0) || || ζ a i || · √ n | ≤ α 2  ≥ c 6 (3.52) for some con stant c 2 > 0 . Fur thermore , if ∆(0) ∈ R A satisfies | z a i − h ∆(0) ,e a i i || ∆ (0) || || ζ a i || · √ n | ≤ α 2 , then ( 3.48) and the bound s (3.46)–(3.47) imp ly that | ˆ x a j − x a j | = | ˆ x a j −  ∆(0) , ζ a j  || ∆(0) || || ζ a j || · √ n | = | 3 X i =1 q j i z a i − h ∆(0) , e a i i || ∆(0) || || ζ a j || · √ n ! | ≤ α 2 3 X i =1 | q j i | ≤ 3 α 2 < α/ 2 ( j = 2 , 3 ; a = 1 , 2 , 3) . Therefo re, (3.52) yields P [ ∀ a, i : | x a i − ˆ x a i | < α/ 2] ≥ c 6 . Thu s, the assertion follows from (3.45) and Lemma 22. 3.5 Pr oof of Proposition 14 Our goal in this section is to boun d the error || ∆( l ) − Ξ( l ) || ∞ resulting from r eplacing the no n-linear operator B by the linear operato r L . Since ∆( l ) = B l ∆(0) and Ξ( l ) = L l Ξ(0) = L l ∆(0) by ( 3.4), the main difficulty of this analysis is to bou nd h ow err ors that w ere made early on in the sequ ence ( i.e., for “small” l ) am plify in the subsequen t iteratio ns. T o control this pheno menon , we shall procee d b y induction on l . W e begin with a simple lemma that bou nds the erro r occurring in a single itera tion. Recall tha t the constru ctions of Ξ( l ) and ∆( l ) ensure that P 3 a =1 Ξ a v → w ( l ) = P 3 a =1 ∆ a v → w ( l ) = 0 for all v → w ∈ A (cf. (2. 3) and (3.5)). Lemma 26. Sup pose that Γ sa tisfies P 3 a =1 Γ a v → w = 0 for all v → w ∈ A . If || Γ || ∞ < 0 . 001 d − 1 , th en || B Γ − L Γ || ∞ ≤ 100 d 2 || Γ || 2 ∞ . 18 Pr o of. W e employ the elementary inequalities exp( − x − x 2 ) ≤ 1 − x ≤ exp( − x ) ≤ 1 − x + x 2 ( | x | ≤ 0 . 1 ) . (3.5 3) Let v → w ∈ A , a ∈ { 1 , 2 , 3 } , and set Π b = Y u ∈ N ( v ) \ w 1 − 3 2 Γ b u → v ( b ∈ { 1 , 2 , 3 } ) . Moreover , let ˆ Γ = B Γ . Then we can rephra se t he definition (3.1) of B as ˆ Γ a v → w = − 1 3 + Π a P 3 b =1 Π b . (3.54) In order to p rove the lemm a, we shall bound the e rror term | Π b − (1 − 3 2 P u ∈ N ( v ) \ w Γ b u → v ) | . T o this end, n ote that b y (3. 53) ther e exist numbers 0 ≤ α b u → v ≤ 9 / 4 such that 1 − 3 2 Γ b u → v = exp( − 3 2 Γ b u → v − α b u → v Γ b 2 u → v ) . Hen ce, once more by (3.53) there is a number − 1 ≤ β b ≤ 1 such that Π b = exp   − X u ∈ N ( v ) \ w 3 2 Γ b u → v + α b u → v Γ b 2 u → v   = 1 − X u ∈ N ( v )  3 2 Γ b u → v + α b u → v Γ b 2 u → v  + β b   X u ∈ N ( v ) 3 2 Γ b u → v + α b u → v Γ b 2 u → v   2 = L b + E b , where we let (3.55) L b = 1 − X u ∈ N ( v ) 3 2 Γ b u → v , and E b = X u ∈ N ( v ) \ w α b u → v Γ b 2 u → v + β b   X u ∈ N ( v ) 3 2 Γ b u → v + α b u → v Γ b 2 u → v   2 . Further, since || Γ || ∞ ≤ 0 . 0 0 1 /d by assumptio n and | N ( v ) | = 2 d by Lemm a 9, we obtain the bound | E b | ≤ 10 d 2 || Γ || 2 ∞ ≤ 0 . 0 1 . (3.56) As P 3 b =1 L b = 3 du e to our assum ption that P 3 b =1 Γ b u → v = 0 fo r all u → v ∈ A , p lugging ( 3.55) into (3.54) yields ˆ Γ a v → w + 1 3 = L a + E a 3 + E 1 + E 2 + E 3 = L a 3 + 3 E a + L a ( E 1 + E 2 + E 3 ) 3(3 + E 1 + E 2 + E 3 ) . (3.57) Since | L a | ≤ 1 + 3 d || Γ || ∞ ≤ 2 , (3.56) and (3.57) yield that | 1 3 (1 − L a ) − ˆ Γ a v → w | ≤ 100 d 2 || Γ || 2 ∞ . (3.58) Finally , a glance at (3.3) reveals that ( L Γ) a v → w = 1 3 (1 − L a ) , and thus the assertion follows from (3.58). Lemma 2 6 a llows us to bound the e rror || ∆( l + 1) − Ξ( l + 1) || ∞ resulting f rom iter ation l + 1 in ter ms of the error || ∆( l ) − Ξ( l ) || ∞ from the previous i teration. In the sequ el we let C > 0 den ote a suf ficiently large constant. Lemma 27. S uppose that || ∆( l ) − Ξ( l ) || ∞ ≤ ( C d ) − 1 . Th en || ∆( l + 1 ) − Ξ( l + 1 ) || ∞ ≤ 2 C d 2 || Ξ( l ) || 2 ∞ + 4 d || ∆( l ) − Ξ( l ) || ∞ . 19 Pr o of. By Lem ma 26 and the definition (3.3) of L we have || ∆( l + 1) − Ξ( l + 1 ) || ∞ = || B ∆( l ) − L Ξ( l ) || ∞ ≤ || B ∆( l ) − L ∆( l ) || ∞ + || L ∆( l ) − L Ξ( l ) || ∞ ≤ C d 2 || ∆( l ) || 2 ∞ + 2 d || ∆( l ) − Ξ( l ) || ∞ . (3.59) Moreover , || ∆( l ) || ∞ ≤ || Ξ( l ) || ∞ + || Ξ( l ) − ∆( l ) || ∞ , whence (3.59) yields || ∆( l + 1) − Ξ( l + 1 ) || ∞ ≤ 2 C d 2  || Ξ( l ) || 2 ∞ + || Ξ( l ) − ∆( l ) || 2 ∞  + 2 d || ∆( l ) − Ξ( l ) || ∞ . This implies the assertion, because we are assuming that || ∆( l ) − Ξ( l ) || ∞ ≤ ( C d ) − 1 . Further, apply ing Lemma 27 L times recu rsiv ely , we obtain the follo wing bound. Corollary 28. S uppo se that || ∆( l ) − Ξ( l ) || ∞ ≤ ( C d ) − 1 for all l < L . Then || ∆( L ) − Ξ( L ) || ∞ ≤ 2 C d 2 L − 1 X j =1 (4 d ) j − 1 || Ξ( L − j ) || 2 ∞ + C d 2 (4 d ) L − 1 || ∆(0) || 2 ∞ . T o proceed, we need the follo wing (rough) absolute bound on the error || ∆( L ) − Ξ( L ) || ∞ . Lemma 29. If L ≤ lo g 2 n , then || ∆( L ) − Ξ( L ) || ∞ < ( C d ) − 1 . Pr o of. T he proo f is by induction on l . For L = 0 the ass ertion is tri vially true. Thu s, assume tha t || ∆( l ) − Ξ( l ) || ∞ < ( C d ) − 1 for all l < L ≤ log 2 n . Then Cor ollary 28 entails that || ∆( L ) − Ξ( L ) || ∞ ≤ 2 C d 2 L − 1 X j =1 (4 d ) j − 1 || Ξ( L − j ) || 2 ∞ + C d 2 (4 d ) L − 1 || ∆(0) || 2 ∞ . Further, the definition (3.3) of L shows that || Ξ( l ) || ∞ ≤ (2 d ) l || ∆(0) || ∞ = (2 d ) l δ. Hence, || ∆( L ) − Ξ( L ) || ∞ ≤ 4 C d 2 (2 d ) 2 L − 2 δ 2 + C d 2 (4 d ) L − 1 δ 2 . As δ ≤ exp( − log 3 n ) an d d = O (1) , the r .h .s. is o ( 1) as n → ∞ , and thus || ∆( L ) − Ξ( L ) || ∞ < ( C d ) − 1 , provided that n is sufficiently large. Lemma 30. Let L ∗ be the maximum inte ger such th at || Ξ( L ∗ ) || ∞ < ǫ .Then for all log 2 n ≤ L ≤ L ∗ we ha ve || Ξ( L ) − ∆( L ) || ∞ ≤ − log( ǫ ) · || Ξ( L ) || 2 ∞ . Pr o of. By the de finition (3.3) of L there are constants c 1 , c 2 > 0 such that || Ξ( l ) || ∞ ≤ (2 d ) l δ ( ∀ l ≤ c 2 log n ) , (3.60) || Ξ( l ) || ∞ ∈ h c − 1 1 λ l δ / √ dn, c 1 λ l δ / √ dn i ( ∀ l ≥ c 2 log n ) . (3.6 1) W e proceed inductively fo r log 2 n ≤ L ≤ L ∗ . Th us, assume that || Ξ( l ) − ∆( l ) || ∞ ≤ c 1 || Ξ( l ) || 2 ∞ for all lo g 2 n ≤ l < L . Since λ ≥ 0 . 1 d and || Ξ( L ) || ∞ < ǫ , this implies that || Ξ( l ) − ∆( l ) || ∞ ≤ ( C d ) − 1 for all log 2 n ≤ l < L. Furthermo re, || Ξ( l ) − ∆( l ) || ∞ < ( C d ) − 1 for all l < lo g 2 n by Lemma 29. Th erefore , we can apply Corollary 28 to obtain || Ξ( L ) − ∆( L ) || ∞ ≤ 2 C d 2 L − 1 X j =1 (4 d ) j − 1 || Ξ( L − j ) || 2 ∞ + C d 2 (4 d ) L − 1 || ∆(0) || 2 ∞ . (3.62) 20 Since L ≥ lo g 2 n and λ ≥ 0 . 1 d , ( 3.60) and ( 3.61) imp ly that th e sum o n the r .h.s. of (3.62) is d ominated by th e term for j = L − 1 . Hence, || Ξ( L ) − ∆( L ) || ∞ ≤ 4 C d 2 || Ξ( L − 1) || 2 ∞ + C d 2 (4 d ) L − 1 δ 2 ≤ c 3 d 2 δ 2  n − 1 λ 2 L − 2 + (4 d ) L − 1  ≤ 2 c 3 d 2 δ 2 λ 2 L − 2 n − 1 ≤ c 4 δ 2 λ 2 L n − 1 . (3.63) Combining (3.61) and (3 .63), we conclude that || Ξ( L ) − ∆( L ) || ∞ < − log( ǫ ) · || Ξ( L ) || 2 ∞ (provided that ǫ is chosen small enoug h). Finally , Proposition 14 follows from Lemma 30 directly . 3.6 Pr oof of Proposition 15 Let µ = ν λ L 2 . Then Coro llary 12 and Proposition 13 entail that (1 − ǫ 3 ) µ ≤ ∆ a v → w ( L 2 ) ≤ (1 + ǫ 3 ) µ if v ∈ V a and w ∈ N ( v ) , and (3.64) ( − 1 2 − ǫ 3 ) µ ≤ ∆ a v → w ( L 2 ) ≤ ( − 1 2 + ǫ 3 ) µ if v 6∈ V a and w ∈ N ( v ) . (3.65) T o prove P roposition 15, we c onsider two cases. The first case is that || ∆( L 2 ) || ∞ ≤ ( ǫd ) − 1 is “small”. Then it will take two mo re step s for the m essages to properly rep resent the c oloring ( V 1 , V 2 , V 3 ) , i.e. , L 3 = L 2 + 2 . By contrast, if || ∆( L 2 ) || ∞ > ( ǫd ) − 1 is “large”, we will just need one more step ( L 3 = L 2 + 1 ) . In both cases th e proo f is based on a direct analysis of the BP equation s (2.2 ). Lemma 31. If 0 . 0 1 ǫd − 1 ≤ || ∆( L 2 ) || ∞ ≤ ( ǫd ) − 1 , then η i u → v ( L 2 + 1) =  1 3 + (1 + γ ( u, v, i )) β if u ∈ V i , 1 3 − (1 + γ ( u, v , i )) β ′ otherwise, (3.66) wher e | γ ( u, v , i ) | ≤ ǫ 3 and β , β ′ > ǫ 2 . Pr o of. W e have η i v → w ( L 2 + 1) = Q u ∈ N ( v ) \ w 1 − 3 2 ∆ i u → v ( L 2 ) P 3 j =1 Q u ∈ N ( v ) \ w 1 − 3 2 ∆ j u → v ( L 2 ) = exp  − 3 2 P u ∈ N ( v ) \ w ∆ i u → v ( L 2 ) + O (∆ i u → v ( L 2 )) 2  P 3 j =1 exp  − 3 2 P u ∈ N ( v ) \ w ∆ j u → v ( L 2 ) + O (∆ j u → v ( L 2 )) 2  =   3 X j =1 exp   3 2 X u ∈ N ( v ) \ w ∆ i u → v ( L 2 ) − ∆ j u → v ( L 2 ) + O ( ǫd ) − 2     − 1 (3.67) Since for any v we have | N ( v ) | = 2 d , we can essentially neglect the O ( ǫd ) − 2 -term in (3 .67). More p recisely , for some − ǫ 2 ≤ γ 2 = γ 2 ( i, v , w ) ≤ ǫ 2 we hav e η i v → w ( L 2 + 1) = (1 + γ 2 )   3 X j =1 exp   3 2 X u ∈ N ( v ) \ w ∆ i u → v ( L 2 ) − ∆ j u → v ( L 2 )     − 1 . (3.68) T o analyze (3.68), assum e without loss of g enerality that v ∈ V 1 . Then (3.64) and ( 3.65) enta il that there is a number − ǫ 2 < γ 3 < ǫ 2 such that X u ∈ N ( v ) \ w ∆ 1 u → v ( L 2 ) − ∆ 2 u → v ( L 2 ) = −  3 2 + γ 3  dµ. 21 Consequently , η 1 v → w ( L 2 + 1) = (1 + γ 2 ) [1 + 2 exp ( − (3 / 2 + γ 3 ) dµ )] − 1 . Fin ally , since µ ≤ 2( ǫd ) − 1 , we obtain η 1 v → w ( L 2 + 1) = (1 + γ 4 )  1 + 2 ex p  − 3 2 dµ  − 1 (3.69) for some − 2 ǫ 2 ≤ γ 4 = γ 4 (1 , v , w ) ≤ 2 ǫ 2 . Now , assume that v ∈ V 2 . Then (3 .64) and (3.65) entail that there are numbers − ǫ 2 < γ 5 , γ 6 < ǫ 2 such that X u ∈ N ( v ) \ w ∆ 1 u → v ( L 2 ) − ∆ 3 u → v ( L 2 ) = γ 5 dµ, X u ∈ N ( v ) \ w ∆ 1 u → v ( L 2 ) − ∆ 2 u → v ( L 2 ) = (3 / 2 + γ 6 ) dµ. Therefo re, η 2 v → w ( L 2 + 1) = (1 + γ 4 )  2 + exp  3 2 dµ  − 1 (3.70) for some − 2 ǫ 2 ≤ γ 4 = γ 4 (2 , v , w ) ≤ 2 ǫ 2 . Com bining (3.69) and (3.70), we obtain the assertion. Corollary 32. Supp ose that. 0 . 01 ǫd − 1 ≤ || ∆( L 2 ) || ∞ ≤ ( ǫd ) − 1 . Then η a v → w ( L 2 + 2) ≥ 0 . 99 if v ∈ V a , and η a v → w ( L 2 + 2) ≤ 0 . 0 1 if v 6∈ V a . Pr o of. W e assume without loss of generality that a = 1 . Moreover, suppose that v ∈ V 1 . W e shall boun d the quotient η 1 v → w ( L 2 + 2) η 2 v → w ( L 2 + 2) = Q 2 · Q 3 , wh ere (3.71) Q j = Y u ∈ V j ∩ N ( v ) \ w 1 − η 1 u → v ( L 2 + 1) 1 − η 2 u → v ( L 2 + 1) for j = 2 , 3 , from below . Lemma 31 implies that for u ∈ V 3 1 − η 1 u → v ( L 2 + 1) 1 − η 2 u → v ( L 2 + 1) ≥ 2 / 3 + (1 − ǫ 3 ) β ′ 2 / 3 + (1 + ǫ 3 ) β ′ ≥ 1 + 3 ǫ 3 β ′ ≥ 1 − 6 ǫ 3 . Hence, Q 2 ≥ (1 − 6 ǫ 3 ) d . (3.72) Furthermo re, for u ∈ V 2 Lemma 31 entails that 1 − η 1 u → v ( L 2 + 1) 1 − η 2 u → v ( L 2 + 1) ≥ 2 / 3 + (1 − ǫ 3 ) β ′ 2 / 3 − (1 + ǫ 3 ) β = 1 + (1 − ǫ 3 )( β + β ′ ) 2 / 3 − (1 − ǫ 3 ) β ≥ 1 + 2 ǫ 2 . Consequently , Q 2 ≥ (1 + 2 ǫ 2 ) d − 1 . (3.73) Combining (3.72) and (3.73) and recalling that d ≫ ǫ − 2 , we obtain the assertion. Lemma 33. Sup pose that || ∆( L 2 ) || ∞ > ( ǫd ) − 1 . Th en η a v → w ( L 2 + 1) ≥ 0 . 9 9 if v ∈ V a , and η a v → w ( L 2 + 2) ≤ 0 . 0 1 if v 6∈ V a . Pr o of. Sin ce || ∆( L 2 ) || ∞ > ( ǫd ) − 1 , (3.64) and (3.65) yield µ ≥ (2 ǫd ) − 1 . (3.74) 22 W ithout loss of generality we may con sider a vertex v ∈ V 1 and a neighbor w ∈ N ( v ) . W e will prove that η 1 v → w ( L 2 + 1) /η 2 v → w ( L 2 + 1) > 1000 . Since P 3 j =1 η j v → w ( L 2 + 1) = 1 , this implies th e assertion . T o b ound the quotien t fro m below , we decompose η 1 v → w ( L 2 + 1) η 2 v → w ( L 2 + 1) = Q 2 · Q 3 , where (3.75) Q j = Y u ∈ V j ∩ N ( v ) \ w 1 − η 1 u → v ( L 2 ) 1 − η 2 u → v ( L 2 ) for j = 2 , 3 , W ith respect to Q 3 , (3.64) and (3.65) imply that for u ∈ V 3 1 − η 1 u → v ( L 2 ) 1 − η 2 u → v ( L 2 ) ≥ 2 / 3 + (1 / 2 − ǫ 3 ) µ ) 2 / 3 + (1 / 2 + ǫ 3 ) µ = 1 − 2 ǫ 3 µ 2 / 3 + (1 / 2 + ǫ 3 ) µ ≥ 1 − 3 ǫ 3 µ. Hence, Q 3 ≥ (1 − 3 ǫ 3 µ ) d . (3.76) Further, (3.6 4) and (3.65) yield that for u ∈ V 2 1 − η 1 u → v ( L 2 ) 1 − η 2 u → v ( L 2 ) ≥ 2 / 3 + (1 / 2 − ǫ 3 ) µ ) 2 / 3 − (1 − ǫ 3 ) µ = 1 + (3 / 2 − 2 ǫ 3 ) µ 2 / 3 + (1 / 2 − (1 − ǫ 3 )) µ ≥ 1 + 2 µ. Therefo re, Q 2 ≥ (1 + 2 µ ) d − 1 . (3.77) Thus, combin ing (3.7 4)–(3.77), we obtain η 1 v → w ( L 2 + 1) η 2 v → w ( L 2 + 1) = Q 2 · Q 3 ≥ (1 − 3 ǫ 3 µ ) d (1 + 2 µ ) d − 1 ≥ (1 + µ ) d − 1 ≥ 1000 , which implies the assertion. Finally , Proposition 15 is a direct conseque nce of Cor ollary 32 and Lemma 33. 4 Pr oof of Corollary 2 Thr o ughou t this section, we assume that d ≥ d 0 for a sufficiently lar ge constan t d 0 > 0 , and that n > n 0 = n 0 ( d ) for a lar ge enough n 0 . Set p = d/n . Let G = G n,d, 3 be a random graph with verte x set V = { 1 , . . . , 3 n } and “planted” 3-color ing V 1 , V 2 , V 3 . In ord er to analyze the adjacency A ( G ) , we shall employ t he following lemma, which follows imm ediately fro m the “conv erse expander mixing lemma” from [3]. Lemma 34. Let B = ( V ′ ∪ · V ′′ , E B ) be a bipa rtite d -r egular gr aph such that | V ′ | = | V ′′ | . A ssume that ∀ S ⊂ V ′ , T ⊂ V ′′ : | e B ( S, T ) − | S || T | p | ≤ d 0 . 51 p | S || T | , (4.1) wher e e B ( S, T ) is the nu mber of S - T - edges in B . Then the ad jacency matrix A ( B ) enjoys the pr o perty: for an y two vectors ξ , η ∈ R V ′ ∪ V ′′ such that both ξ , η are perpen dicular to ~ 1 V ′ and ~ 1 V ′′ the in equality h A ( B ) ξ , η i ≤ d 0 . 52 || ξ || || η || h olds. Moreover , the follo wing lem ma can be derived u sing standard techn iques fr om th e th eory of ran dom regular graphs [Chap - ter 9 9]. 23 Lemma 35. W .h.p. G h as the following pr operty . Let 1 ≤ i < j ≤ 3 . Then ∀ S ⊂ V i , T ⊂ V j : | e G ( S, T ) − | S || T | p | ≤ d 0 . 51 p | S || T | . Corollary 36. W .h.p. G is ( d, 0 . 01) -re g ular . Pr o of. L et A ( G ) = ( a v, w ) v, w ∈ V denote the adjacency matrix of G . Moreover , let a ij vw =  a vw if v , w ∈ V i ∪ V j , 0 other wise (1 ≤ i < j ≤ 3) . Then A ij = ( a ij vw ) v, w ∈ V is the adjacency matrix of the bipartite su bgraph of G ind uced on V i ∪ V j . Let E be th e subspace of R V spanned by the thr ee vectors ~ 1 V k ( k = 1 , 2 , 3 ). Combining Lemma 34 with L emma 35, we co nclude that w .h .p.  A ij ξ , η  ≤ d 0 . 52 || ξ || || η || fo r all ξ , η ⊥ E and any 1 ≤ i < j ≤ 3 . Sin ce A ( G ) = P 1 ≤ i

Original Paper

Loading high-quality paper...

Comments & Academic Discussion

Loading comments...

Leave a Comment