Error Correction Capability of Column-Weight-Three LDPC Codes

Error Correction Capability of Column-W eight-Three LDPC Codes Shashi Kiran Chilappagari, Student Member , IEEE, and Bane V asic, Senior Member , IEEE Abstract In this pap er , we in vestigate the er ror correctio n cap ability of colum n-weigh t-three LDPC codes whe n decoded using the Gallager A algorith m. W e prove that the n ecessary c ondition for a code to corr ect k ≥ 5 errors is to av oid cycles of length u p to 2 k in its T anner gr aph. As a consequen ce o f this result, we show that given any α > 0 , ∃ N such that ∀ n > N , no code in the ensemble of colu mn-weigh t-three co des can cor rect all αn or fewer e rrors. W e extend these resu lts to the bit flip ping algo rithm. Index T erms Low-density parity-chec k codes, Gallager A algor ithm, trapp ing sets, err or corr ection cap ability I . I N T RO D U C T I O N Gallager in [1] sh owe d t hat f or γ ≥ 3 and ρ > γ , there exist ( n, γ , ρ ) regular low-density parity-check (LDPC) codes for whi ch the bit error probabil ity tends t o zero asympt otically whenev er we o perate below the threshold. Richardson and Urbanke i n [2] derived the capacity of LDPC codes for various message passing algorithms and described density ev olution, a deterministic algorithm to compute thresholds. Zyablov and Pinsker [3] analyzed LDPC codes u nder a si mpler decoding algorith m known as the bit Manuscript received July 15, 2021. This work is funded by NSF under Grant CCF-0634969, ITR-0325979 and I NSIC-EHDR program. S. K. Chilappagari and B. V asic are wi th the D epartment of Electrical and Computer E ngineering, University of Arizona, Tucson , AZ, 85721 USA (e-mails: shashic@ece.arizona.edu, vasic@ece.arizon a.edu). flipping algorithm and showed that almost all the codes in the regular ensemble with γ ≥ 5 can correct a const ant fraction of worst case errors. Sipser and Spielman in [4] us ed expander graph ar guments to analyze bit flippi ng algorithm. Burshtein and Mill er in [5] applied expander based arguments to show that message passing algorithm s can also correct a fix ed fraction of worst case errors when the degree of each variable node is at least fi ve. Feldman et al. [6] showed that linear programmi ng decoder [7] is also capable of correcting a frac tion of errors. Recently , Burshtein in [8] sho wed that regular codes with variable nodes of degree four are capable of correcting a linear nu mber of errors u nder bit flipp ing algorithm. He als o showed tremendous im provement in the fraction of correctable errors when t he variable node degree is at least five . In this paper , we consider the error correction capability of the ensemble C n (3 , ρ > 3 ) of (3 , ρ ) re gular LDPC codes as defined i n [2] when decoded using the Gallager A algorithm . W e analyze decoding failures using the notion of trapping sets and prove that a code with girth g ≥ 10 cannot correct all g / 2 or fewer errors. Using this result, we prove that for any α > 0 , for sufficiently large bl ock l ength n , no code in the C n (3 , ρ ) ensemble can correct α fraction of errors. This resul t settles the prob lem of error correction capability of colum n-weight-three codes. The rest o f the paper is organized as foll ows. In Section II we establish the not ation and describ e th e Gallager A algorithm. W e t hen characterize the failures of the Gallager A decoder with th e help of fixed points. W e also introdu ce the no tions of trapping sets, failure sets and criti cal number . In Section III we in vestigate the relation between error correction capacity and girth of t he code. W e extend t he results to bit flipping algorithm in Section IV and conclude in Section V. I I . D E C O D I N G A L G O R I T H M S A N D T R A P P I N G S E T S A. Graphical Repr esentations of LDPC Codes LDPC codes [1] are a class of linear block codes which can be d efined by sparse bipartite graphs [9]. L et G be a bipartite graph wi th two s ets of nodes: n variable nodes and m check nodes. The check nodes (variable nodes) connected to a variable node (check node) are referre d to as its neighb ors. The degree of a n ode is t he numb er of its neighbors. This graph defines a linear block code C of length n and dimension at least n − m in the foll owing way: The n variable nodes are associated t o the n coordinates of codew ords. A vector v = ( v 1 , v 2 , . . . , v n ) is a code word if and only if for each check node, the s um of its n eighbors i s zero. Such a graphical representation of an LDPC code is called the T anner graph [10 ] of the code. The adj acency m atrix of G gives H , a parity check m atrix of C . An ( n, γ , ρ ) regular LDPC cod e has a T anner graph with n variable nodes each of degree γ (column weig ht) and nγ /ρ check nodes each of degree ρ (ro w weight). This code has l ength n and rate r ≥ 1 − γ /ρ [9]. In the rest of the paper we consider codes in the (3 , ρ ) , ρ > 3 , regular LDPC code ensemble. Note that t he column weight and row weight are als o referred to as left degree and right degree in literature. It shoul d also be noted that the T anner graph is not uni quely defined by t he code and when we say the T anner graph of an LDPC code, we only mean one possible graphi cal representati on. The girth g is the length of the shortest cycle in G . In this paper , • represents a var iable no de,  represents an even degree check node and  represents an odd degree check node. B. Har d Decision Decoding A lgorithms Gallager in [1] proposed t wo sim ple binary mess age passing algo rithms for decoding over the binary symmetric channel (BSC); Gall ager A and Gallager B. See [9] for a detailed description of Gallager B algorithm. For colum n-weight-three codes, which are t he main focus o f this paper , these two alg orithms are the same. Every round o f m essage p assing (it eration) starts with sending messages from variable nod es (first half of the iteration) and ends by sending messages from check nodes t o variable nodes (second half of the it eration). Let r , a binary n -tup le be the input to the decoder . Let ω j ( v , c ) denote t he message passed by a variable node v to its neighbori ng check node c in j th iteration and  j ( c, v ) denot e the message passed by a check node c t o its neighbo ring variable node v . Additionall y , let ω j ( v , : ) denote the s et of all messages from v , ω j ( v , : \ c ) denote the set of all messages from v except t o c , ω j ( : , c ) denote the set of all messages t o c . ω j ( : \ v , c ) ,  j ( c, : ) ,  j ( c, : \ v ) ,  j ( : , v ) and  j ( : \ c, v ) are defined simi larly . The Gallager A algorithm can be defined as foll ows. ω 1 ( v , c ) = r ( v ) ω j ( v , c ) =              1 , if  j − 1 ( : \ c, v ) = 1 0 , if  j − 1 ( : \ c, v ) = 0 r ( v ) , otherwise  j ( c, v ) =  X ω j ( : \ v , c )  mod 2 At th e end of each iteratio n, an estimate of each variable n ode is made based o n the incomin g messages and possibly the rece iv ed v alue. The decoded w ord at the e nd of j th iteration is denoted as r ( j ) . The decoder is run unti l a valid codew ord is found or a maximum nu mber of iterations M i s reached, whichever is earlier . The output of t he decoder is either a codew ord or r ( M ) . A Note on the Decision Rul e: Different rul es to es timate a var iable n ode after each it eration are poss ible and it i s likely that chang ing the rule after certain iterations may be beneficial. Howe ver , the analysis of var ious s cenarios is beyond th e scope of this paper . For column-weight-t hree codes o nly two rules are possible. • Decision Rule A: i f all incom ing messages t o a v ariable node from neighborin g checks are equal, set the variable n ode to th at value; else s et it t o recei ved value • Decision Rule B: set the v alue of a v ariable node to the majority of the incomi ng m essages; majority alwa ys exists since the column-weigh t is t hree W e adopt Decisi on Rule A throughout this paper . C. T rapping Sets of Gallager A A lgorithm W e now characterize failures of the Gallager A decoder using fixed p oints and trapping set s [11 ]. Consider an LDPC code of length n and let x be the bi nary vector which is the in put to the Gallager A decoder . Let S ( x ) be the support o f x . The support of x is defined as t he set of all positions i where x ( i ) 6 = 0 . W ithout loss of generality , we assume that the all zero code word is sent o ver BSC and that the inpu t to t he d ecoder is t he error vector . Hence, througho ut thi s paper a m essage of 1 is alternativ ely referred to as an incorrect message, a receiv ed value of 1 is referred to as an initi al error . Definition 1: [11] A decoder failure is said to h a ve occurred i f the o utput of the decoder is not equal to the t ransmitted codew ord. Definition 2: x is called a fixed p oint i f ω j ( v , c ) = x ( v ) , ∀ j > 0 That is, the m essage passed from variable nodes to check nodes along th e edges are t he same in e very iteration. Since the outgoing messages from var iable nodes are same in ev ery iteration, it follo ws t hat the incoming messages from check nodes to v ariable nodes are also same in e very it eration and so is the estimate of a variable after each iteration. In fact, the estim ate after each iteration coincides with t he recei ved value. It is clear from above definition that if the i nput to the decoder is a fixed point, t hen the output of the decoder is the same fixed poi nt. Definition 3: [12] Let x be a fixed point. Then S ( x ) is k nown as a t rapping set. A ( V , C ) trapping set T is a set of V variable n odes whose i nduced subgraph has C odd degree checks. Theor em 1: Let C be a code in th e ensem ble of (3 , ρ ) regular LDPC codes. Let T be a set consisting of V variable nodes wit h indu ced subgraph I . Let the checks in I be partition ed into two disjoint subs ets; O consistin g of checks with odd degree and E consisting of checks with even degree. Let |O | = C and |E | = S . T is a trapping s et iff : (a) Every variable node in I is con nected to at l east two checks in E and (b) No two checks o f O are connected t o a variable node outsid e I . Pr o of: See Ap pendix I W e no te t hat Theorem 1 is a consequence of Fact 3 in [11]. W e als o remark th at T heorem 1 can be extended to higher column weight codes but in this paper we restrict our att ention to column-weigh t-three codes. If the variable nodes corresponding to a t rapping set are in error , then a decoder failure occurs. Howev er , not all var iable nodes correspondi ng to trapping s et need t o be in error for a decoder failure t o occur . Definition 4: [12] T he m inimal number of variable nod es th at have to be initially in error for the decoder to end up i n the trappi ng set T wi ll be referred to as cri tical number for that trapping set. Definition 5: A set of variable nod es which if in error lead to a decoding failure is known as a f ailur e set . Remarks 1) T o “end up” in a trapping set T means t hat, after a possible finite numb er of iterations, the decoder will be in error , on at least on e variable node from T , at every iteration [11]. 2) The notion of a failure set is more fundamental than a trappi ng set. Howe ver , from the definition, we cannot derive necessary and sufficient condit ions for a set of variable n odes to form a failure set. 3) A trappin g set is a failure s et. Subsets of trappi ng sets can be failure s ets. More specifically , for a trapping set of si ze V , there exists at least one subs et of size equal to the critical number w hich is a failure set. 4) The critical nu mber of a trapping set is not fixed. It d epends on t he outsi de connections of checks in E . Howe ver , the m aximum value of crit ical numb er of a ( V , C ) t rapping set is V . Example 1: Fig.1(a) shows a s ubgraph induced b y a set of t hree variable nodes { v 1 , v 2 , v 3 } . If n o two odd degree check nodes from { c 4 , c 5 , c 6 } are connected to a variable outside th e subgraph, then by Theorem 1, Fig.1(a) is a (3 , 3) trapping set. On the ot her hand, if two od d degre e checks, say c 5 and c 6 , are connected t o another variable no de, s ay v 4 , the subgraph resembles Fig . 1(b). Assuming no ot her connections, Fig.1(b) is a ( 4 , 2) trappi ng set. W e m ake t he following observations: 1) The three variable nodes in a (3 , 3) trappi ng set form a six cycle. Howe ver , not all six cycles are (3 , 3) trapping sets. Apart from the subgraph induced by variable nod es, t he outside connecti ons should be known to determi ne whether a give n su bgraph is a trapping set or not. The (4 , 2) trapping set in Fig. 1(b) illus trates thi s point. 2) The critical number of a (3 , 3) t rapping set i s t hree. There exist (4 , 2) trapping sets with critical number t hree and it is hig hly un likely that a (4 , 2) trapping set d oes not contain a failure set of size three. Howe ver , we can s how by a coun terexample that this is in deed possi ble. 3) A ( V , C ) trappin g set is not u nique i.e., two trapping s ets with same V and C can hav e different underlying topologi cal structures (induced subgraphs). So, wh en we talk of a t rapping set, we refer to a specific topolog ical structure. In this paper , t he induced s ubgraph is assum ed to b e known from the context. 4) T o av oid a trappin g set in a code, it is suffi cient to av oid topological st ructures isomorphic to the subgraph induced by the trapping set. For example t o av oid (3 , 3) trapping sets of Fig.1(a), it is suffi cient t o av oi d six cycles. It is po ssible that a code has six cycles but no (3 , 3) trapping sets. In this case all six cycles are part of (4 , 2 ) or other trappi ng sets. v 1 v 2 c 1 2 c c 3 c 4 c 5 c 6 v 3 (a) v 3 c 1 c 2 c 3 c 4 c 6 c 7 v 4 v 2 c 5 v 1 (b) Fig. 1. Examples of trapping sets (a) a (3 , 3) trapping set (b) a (4 , 2) trapping set I I I . E R R O R C O R R E C T I O N C A P A B I L I T Y A N D G I RT H O F T H E C O D E Burshtein and Miller in [5] applied e xpander based arguments to m essage passing algorithms. They analyzed ensem bles o f irregular graphs and showed that i f the degree of each variable node is at least fi ve, then mess age passing algorit hms can correct a fraction of errors. Codes wi th colum n weight th ree and four cannot achiev e th e expansion requi red for these arguments. Recently , Burshtein in [8] de veloped a new technique to in vestigate the error correction capability of regular LDPC codes and showed that at suffi ciently large blo ck lengths, almo st all codes with colu mn weig ht four are also capable of correcting a fraction of errors under bit flipping algori thm. For col umn-weight-three codes he notes that such a result cannot be proved. This is because a non negligible fraction o f codes have parallel edges in t heir T anner graphs and su ch codes cannot correct a s ingle worst case error . In this paper we prove a stronger result by showing t hat for any given α > 0 , at suffic iently large bl ock lengths n , no code in the C n (3 , ρ ) ensemb le can correct all α n or fewe r errors under Gallager A algorith m and show that this ho lds for the b it flipping algo rithm also. Lemma 1: [8] A code whose T anner graph has parallel edges cannot correct a si ngle worst case error . Pr o of: See [8]. The proof i s for bit flipping algorit hm, but also appli es to Gall ager A algorithm . Lemma 2: Let C be an ( n, 3 , ρ ) regular LDPC code with girth g = 4 . Then C has at least on e failure set of size two or three. Pr o of: See Ap pendix II. Lemma 3: Let C be an ( n, 3 , ρ ) regular LDPC code wit h girth g = 6 . Then C has l east one failure set of size three or four . Pr o of: Since g = 6 , there is at least one six cycle. W ithout loss of generality , we assume that { v 1 , v 2 , v 3 } t ogether with the three ev en degree checks { c 1 , c 2 , c 3 } and the three odd degree checks { c 4 , c 5 , c 6 } form a six cycle as in Fig.1(a). If no two checks from { c 4 , c 5 , c 6 } are conn ected t o a var iable node, th en { v 1 , v 2 , v 3 } is a (3 , 3) trapping set and hence a failure set of size t hree. On the cont rary , assume that { v 1 , v 2 , v 3 } do not form a (3 , 3) trapping set. Then there exists v 4 which is connected to at least two checks from { c 4 , c 5 , c 6 } . If v 4 is conn ected to all the three checks, { v 1 , v 2 , v 3 , v 4 } is a codeword of weight four and it is easy t o see t hat { v 1 , v 2 , v 3 } is a failure set. Now ass ume that v 4 is connected to only two checks from { c 4 , c 5 , c 6 } . W ithout loss of generality , let the two checks be c 5 and c 6 . Let the th ird check connected to v 4 be c 7 as s hown in Fig. 1(b). If c 4 and c 7 are not conn ected to a common variable node then { v 1 , v 2 , v 3 , v 4 } is a (4 , 2) trapping set and hence a failure set of s ize four . If c 4 and c 7 are connected to say v 5 , we have two possibil ities: (a) The thi rd check is c 8 and (b) The th ird check of v 5 is c 2 (the thi rd check cannot be c 1 or c 3 as this would introduce a four cycle). W e claim that in both cases { v 1 , v 2 , v 3 , v 4 } is a failure set. The two cases are discuss ed b elow . Case (a): Let S ( r ) = { v 1 , v 2 , v 3 , v 4 } . ω 1 ( v , :) =        1 , v ∈ { v 1 , v 2 , v 3 , v 4 } 0 , otherwise The messages i n the second h alf of first iteration are,  1 ( c 1 , v ) =        1 , v ∈ { v 1 , v 2 } 0 , otherwise Similar equations h old for c 2 , c 3 , c 5 , c 6 . For c 4 we have  1 ( c 4 , v ) =        0 , v = v 1 1 , otherwise Similar equation s hold for c 7 . At the end of first iteration, we note that v 2 and v 3 recei ve all incorrect messages, v 1 , v 4 and v 5 recei ve two in correct messages and all other variable nodes receiv e at most one incorrect message. W e therefore have r (1) = r and S ( r (1) ) = { v 1 , v 2 , v 3 , v 4 } . The mess ages sent by variable nodes in the second i teration are, ω 2 ( v , :) = 1 , v ∈ { v 1 , v 2 , v 3 , v 4 } ω 2 ( v 5 , c 8 ) = 1 , ω 2 ( v 5 , { c 4 , c 7 } ) = 0 , ω 2 ( v , :) = 0 , v ∈ { v 1 , . . . , v n } \ { v 1 , v 2 , v 3 , v 4 , v 5 } . The mess ages passed in second half of second iteration are s ame as in second half of first iteration, except that  ( c 8 , : \ v 5 ) = 1 . At the end o f s econd i teration, we note that v 2 and v 3 recei ve all incorrect m essages, v 1 , v 4 and v 5 recei ve t wo incorrect messages and all oth er variable nodes recei ve at most o ne incorrect message. The situation is same as at the end of first i teration. The algorith m runs for M iterations and the decoder outputs r ( M ) = r which im plies t hat { v 1 , v 2 , v 3 , v 4 } is a failure set. Case (b): The proof is along the same li nes as for Case (a). The messages for first iteration are the same. The m essages in th e first half o f second iteration are, ω 2 ( v , :) = 1 , v ∈ { v 1 , v 2 , v 3 , v 4 } ω 2 ( v 5 , c 2 ) = 1 , ω 2 ( v 5 , { c 4 , c 7 } ) = 0 , ω 2 ( v , :) = 0 , v ∈ { v 1 , . . . , v n } \ { v 1 , v 2 , v 3 , v 4 , v 5 } . The mess ages passed in second half of second iteration are s ame as in second half of first iteration, except that  ( c 2 , : \{ v 2 , v 3 , v 5 } ) = 1 and  ( c 2 , { v 2 , v 3 , v 5 } ) = 0 . At the end of second iteration, v 1 , v 2 , v 3 , v 4 and v 5 recei ve two incorrect messages and all other variable nodes receiv e at m ost one incorrect message and hence r (2) = r . The messages p assed in first half of t hird iteratio n (and t herefore subsequ ent iterations ) are same as the messages passed in first half of second iteration. The algorithm runs for M iterations and the decoder outputs r ( M ) = r which im plies t hat { v 1 , v 2 , v 3 , v 4 } is a failure set. Lemma 4: Let C be an ( n, 3 , ρ ) regular LDPC code with girth g = 8 . Then C has at least on e failure set of size four or fiv e. Pr o of: See Ap pendix II. Remark: It might be possible that Lemm as 2 – 4 can be made stronger by further analy sis, i.e., it might be possibl e t o s how that a code with girth four has a failure set of size two, a code with girth six h as failure set of si ze three and a code with girth eight h as a failure set of size four . Howe ver , these weaker lemmas are sufficient t o establi sh the main th eorem. Lemma 5: Let C be an ( n, 3 , ρ ) regular LDPC code with girth g ≥ 10 . Then the s et of variable nodes { v 1 , v 2 , . . . , v g / 2 } in volve d i n the short est cycle is a trappi ng set of s ize g / 2 . Pr o of: Since C h as girth g , there is at least one cycle of length g . W ithout loss of generality , assum e that { v 1 , v 2 , . . . , v g / 2 } form a cycle of mini mum length as sh own in Fig.2. Let the ev en degree checks be E = { c 1 , c 2 , . . . , c g / 2 } and th e odd degree checks be O = { c g / 2+1 , c g / 2+2 , . . . , c g } . Note t hat each variable node is con nected to two checks from E and one check from O and c g / 2+ i is connected t o v i . W e claim that no two checks from O can be connected to a com mon var iable no de. The proof is b y contradiction . As sume c i and c j ( g / 2 + 1 ≤ i < j ≤ g ) are connected to a variable node v ij . Then { v i , . . . , v j , v ij } form a cycle of length 2( j − i + 2) and { v j , . . . v g / 2 , v 1 , . . . , v i , v ij } form a cycle of l ength 2( g / 2 − j + i + 2) . Since g ≥ 10 , min(2( j − i + 2) , 2( g / 2 − j + i + 2)) < g . This implies th at there is a cycle of length less than g , which is a contradicti on as th e girth of the graph is g . By Theorem 1, { v 1 , v 2 , . . . , v g / 2 } is a trapping set. v 1 v 2 v 3 v g/2 c 1 c 2 c 3 g/2−1 c c g/2 c g g/2+2 c c g/2+1 g/2+3 c Fig. 2. Illustration of a cycle of l ength g Cor oll ary 1: For a code to correct all k ≥ 5 or fewer errors, it i s necessary t o av oid all cycles up t o length 2 k . W e now st ate and prove the main theorem. Theor em 2: Consider the standard (3 , ρ ) re gular LDPC cod e ensemble. Let α > 0 . Let N be the smallest integer s atisfying αN > 2  log N log (2( ρ − 1)) + 1  αN ≥ 5 . Then, for n > N , no code in the C n (3 , ρ ) ensemble can correct all α n or fewer errors. Pr o of: First o bserve that for any n > N , we hav e αn > 2  log n log (2( ρ − 1)) + 1  . (1) From [Theorem C.1 [1]] and [Lem ma C.1 [1]], we ha ve the girt h g of any code in C n (3 , ρ ) is bound ed by g ≤ 4  log n log (2( ρ − 1)) + 1  (2) For n > N , Equations (1) and (2) im ply that for any code in the C n (3 , ρ ) ensemble, the girth i s bound ed by g < 2 αn. The result now follows from Corollary 1. I V . E X T E N S I O N T O T H E B I T F L I P P I N G A L G O R I T H M The b it flipping algorithm does not bel ong to the class of message passing algo rithms. Howe ver , the definitions from Section II and t he results from Section III can be generalized t o the parallel b it flipping algorithm [4]. W ithout lo ss of generality we assume th at the all zero codew ord is sent . W e begin with a fe w definitio ns. Definition 6: [4] A variable no de is said to be corrupt i f it is differ ent from i ts original s ent value. In our case, a v ariable no de is corrupt if it is 1 . A check node is said to be satisfied if it is con nected to e ven num ber of corrupt variables and unsati sfied o therwise. Definition 7: Let r be the input to the p arallel bit flipping decoder . S ( r ) is a trappin g set for bit flipping algorithm if t he set of corrupt variables after ev ery iteration is S ( r ) . Theor em 3: Let T be a set of variable nodes satis fying the con ditions of Theorem 1. Then T is a trapping set for the bi t flipping algorit hm. Pr o of: Let S ( r ) = T . Then T is the set of corrupt variable nodes. Observe that a var iable flips if i t is connected to at least two unsatisfied checks. Since no variable i s connected to two unsatisfied checks, the set of corrupt var iable nodes is u nchanged and by definition T i s a trapping set. W e note that Theorem 3 is also a consequ ence of Fact 3 from [11]. Cor oll ary 2: A trapping set for Gallager A is also a t rapping set for b it flipping algo rithm. It can b e shown that Lemmas 1 – 5 and Theorem 2 also hold for the bit flipping algorithm. V . C O N C L U S I O N In this paper we have in vestigated the error correction capabilit y of colum n-weight-three codes und er Gallager A and extended the results to bit flipping algo rithm. Future work inclu des in vestigat ion of suffi cient conditi ons to correct a given number o f errors for col umn-weight-three as well as higher column weight codes. A P P E N D I X I Pr o of of Theor em 1: Let r b e the input to the decoder with S ( r ) = T . Then, ω 1 ( v , :) =        1 , v ∈ T 0 , otherwise Let a check node c o ∈ O . Then,  1 ( c o , v ) =        0 , v ∈ T 1 , otherwise Let a check node c e ∈ E . Then,  1 ( c e , v ) =        1 , v ∈ T 0 , otherwise For any other check node c ,  1 ( c, v ) = 0 . By t he conditions of t he theorem, at the end of first iteration, any v ∈ T recei ves at least two 1 ’ s and any v / ∈ T receiv es at mo st one 1 . So, we h a ve ω 2 ( v , :) =        1 , v ∈ T 0 , otherwise By definition, T is a trapping set. T o see that the condi tions stated are necessary obs erve that for a va riable node t o send the same messages as in t he first iteration, it should receiv e at least t wo messages which coincide wit h the received value.  A P P E N D I X I I Pr o of of Lemma 2: Let { v 1 , v 2 } be the variable nodes that form a four cycle with even degree checks { c 1 , c 2 } and od d degree checks { c 3 , c 4 } . If c 3 and c 4 are not connected to a comm on variable node, then { v 1 , v 2 } is a (2 , 2 ) trapping s et and hence a failure set of si ze two. Now assum e that c 3 and c 4 are connected to a com mon v ariable node v 3 . Then, { v 1 , v 2 , v 3 } is a (3 , 1) trapping set and therefore a failure set of size three.  Pr o of of Lemma 4: Let T 1 = { v 1 , v 2 , v 3 , v 4 } be the variable nodes that form an eight cycle (see Fig.3(a)) . If no two checks from { c 5 , c 6 , c 7 , c 8 } are connected to a common variable node, t hen T 1 is a (4 , 4) trapping set and h ence a f ailure set of size four . On the ot her hand, if T 1 is not a trapping set, then there must be at least one variable n ode which is connected to t wo checks from { c 5 , c 6 , c 7 , c 8 } . Assu me that c 5 and c 7 are connected to v 5 and the third check of v 5 is c 9 (see Fig.3(b)). W e claim that T 2 = T 1 ∪ { v 5 } is a failure set. Let E and O b e as defined in Theorem 1. v 1 v 2 v 4 v 3 c 4 c 3 c 1 c 2 c 5 c 6 c 7 c 8 (a) v 3 v 4 v 5 c 2 c 3 c 4 v 1 v 2 c 1 c 6 c 5 c 9 c 8 c 7 (b) Fig. 3. Subgraphs isomorphic to (a) a (4 , 4) t rapping set (b) a (5 , 3) trapping set Case 1: No two checks from O = { c 6 , c 8 , c 9 } are conn ected t o a commo n variable node. Then T 2 is a (5 , 3) t rapping set and hence a failure s et of size five . Case 2: All the three checks i n O are connected to a common variable node, say v 6 . Then T 2 ∪ { v 6 } is a codeword of wei ght si x and it i s easy to s ee that T 2 is a failure set. Case 3: There are variable nodes connected to t wo checks from O . There can be at mos t two such var iable no des (if there are t hree such variable nodes, they will form a cycle of lengt h less than or equal to six violatin g the conditi on that the graph has girt h ei ght). Note that if S ( r ) = T 2 , the decoder has a chance of correcting only if a check node in E receiv es an incorrect m essage from a variable node ou tside T 2 in some j th iteration. W e n ow prove that thi s is not possibl e. Indeed i n the first iteration ω 1 ( v , :) =        1 , v ∈ T 2 0 , otherwise By similar ar guments as in the proof for Theorem 1, it can be seen that t he only check nodes which send incorrect messages t o var iable no des outside T 2 are c 6 , c 8 and c 9 . There are now t wo subcases. Subcase 1: There is one variable node connected to tw o checks from O . Let v 6 be connected to c 6 and c 8 . It can be seen t hat the thi rd check connected to v 6 cannot belon g to E as this would violate the girth conditi on. So, let t he t hird check be c 10 . In the first half of second iteration, we hav e ω 2 ( v , c ) =        1 , v ∈ T 2 or ( v , c ) = ( v 6 , c 10 ) 0 , otherwise The only check nodes which s end incorrect messages to variable no des outs ide T 2 , are c 6 , c 8 , c 9 and c 10 . The variable node v 6 is connected to c 6 and c 8 . If c 9 and c 10 are not connected to any commo n var iable node, we are don e. On the other hand, l et c 9 and c 10 be con nected t o a variable node, say v 7 . The third check of v 7 cannot be in E . Proceeding as in the case of proof for Lemm a 3, we can prove that T 2 is a failure set by observing th at there cannot be a variable node outside T 2 which send s an incorrect message to a check in E . Subcase 2: There are two variable nodes connected to two checks from O . Let c 6 and c 8 be connected to v 6 and c 6 and c 9 connected to v 7 . Proceeding as above, we can conclude that T 2 is a failure set. A C K N OW L E D G M E N T This work is fund ed by NSF under Grant CCF-0634969, ITR-032597 9 and INSIC-EHDR prog ram. The authors would l ike to thank Anant haraman K rishnan for il lustrations. R E F E R E N C E S [1] R. G. Gallager , L ow D ensity P arity Check Codes . Cambridge, MA: M.I.T . Press, 1963. [2] T . J. Ri chardson and R. Urbanke, “The capacity of low-den sity parity-check codes under message-passing decoding, ” IEE E T rans. Inform. Theory , vol. 47, no. 2, pp. 599–618, Feb . 2001. [3] V . V . Zyablo v and M. S. Pinsker , “Estimation of the error-correction complex ity for Gallager low-den sity codes, ” P r oblems of Information T ransmission , vol. 11, pp. 18–28, 1976. [4] M. Sipser and D. Spielman, “Expander codes, ” IEEE T rans. Inform. T heory , vo l. 42, no. 6, pp. 1710–1 722, Nov . 1996. [5] D. Burshtein and G. Miller , “Expander graph argume nts for message-passing algorithms, ” IEEE T rans. Inform. Theory , vol. 47, no. 2, pp. 782–79 0, Feb. 2001. [6] J. Feldman, T . Malkin, R. A. Servedio, C. Stein, and M. J. W ainwright, “LP decoding corrects a constant fraction of errors, ” I EEE T rans. Inform. Theory , vol. 53, no. 1, pp. 82–89, Jan. 2007. [7] J. Feldman, M. J. W ainwright, and D. R . Karger , “Using linear programming to decode binary linear codes, ” IEEE T rans. Inform. Theory , vol. 51, no. 3, pp. 954–972, March 2005. [8] D. Burshtein, “On the error correction of regu lar LDPC codes using the flipping algorithm, ” in International Symposium on Information Theory , June 24-29 2007, pp. 226–230 . [9] A. Shokrollahi, “ An i ntroduction to low-density parity-check codes, ” in T heor etical aspects of computer science: advance d lectur es . Ne w Y ork, NY , USA: Springer-V erlag New Y ork, Inc., 2002, pp. 175–197. [10] R. M. T anner, “ A recursi ve approach to low complexity codes, ” IEEE Tr ans. Inform. Theory , vol. 27, pp. 533–5 47, Sept. 1981. [11] T . J. Richardson, “Error floors of LDPC codes, ” in 41st Annual All erton Conf. on Communications, C ontr ol and C omputing , 2003, pp. 1426–1 435. [12] S. K. Chilappagari, S . S ankaranaraya nan, and B. V asic, “Error floors of LDP C codes on the binary symmetric channel, ” i n International Confer ence on Communications , vol. 3, June 11-15 2006, pp. 1089–109 4.