Constraint Complexity of Realizations of Linear Codes on Arbitrary Graphs

CONSTR AINT COMPLEX ITY OF R EALIZA TIONS OF LINE AR CODE S ON A RBITRARY GRAPHS N A VIN KASHY AP A B S T R A C T . A graphical realization of a linear code C consists of an assignment of the coordi- nates of C to the vertices of a graph, along wi th a specification of linear state spaces and linear “local constraint” codes to be associated with the edges and vertices, r especti vely , of the graph. The κ -comple xity of a graphical realization is defined to be the largest dimension of any of its local constraint codes. κ -complexity is a reasonable measure of the computational complexity of a sum- product decoding algorithm specified by a graphical realizati on. T he main focus of this paper is on the follo wing problem: gi ven a linea r code C and a grap h G , ho w small can the κ -complexity of a real- ization of C on G be? As useful tools for attacking this problem, we introduce the V ertex-Cut Bound, and the notion of “vc-treewidth” for a graph, which is closely related to the well-known graph- theoretic notion of treewidth. Using these tools, w e deriv e tight lower bounds on t he κ -complex ity of any realization of C on G . Our bounds enable us to conclude that good error-correcting codes can hav e low-comple xity realization s only on graphs wi th large vc-treewidth. Along the w ay , we also prov e the interesting result that the r atio of the κ -complexity of the best con ventional t rellis reali za- tion of a length- n code C to the κ -complexity of the best cycle-free realization of C gro ws at most logarithmically with codelength n . Such a logarithmic gro wth rate is, in fact, achie vable. 1. I N T R O D U C TI O N The stud y of graphical model s of codes and the associate d message-pass ing decoding alg orithms is a major focus of current research in coding theo ry . This is attrib utable to the fact that coding schemes u sing graph-b ased iterati ve decod ing strategie s — e.g . , turb o codes and lo w-density parity check (LDPC) codes — h av e lo w imple mentation comple xity , while the ir performan ce is cl ose to the optimum predicted by t heory . A uni fied trea tment o f gra phical mo dels and the associ ated de cod- ing algorithms be gan with the wo rk o f Wi ber g, Loeliger and K oetter [18],[1 9], and has sin ce been abstra cted and refined under the frame work of the gener alized distr ibu tiv e law [1], factor graph s [12], a nd normal reali zations [6],[7]. In f act, the s tudy of c ycle-free graphi cal models of c odes ( i.e . , models in which the unde rlying graphs are cycle-free ) can be tr aced back to the introductio n of the V iterbi decod ing alg orithm in the 1960’ s, w hich led to the study o f trellis representati ons of codes. A comprehen siv e account of the history and dev elopment of t rellis representa tions can be found in [17]. In this wo rk, w e will follo w the approach o f Forney [6],[7], and Hal ford an d Chug g [8] in studyi ng t he general “extracti ve” p roblem of constructin g lo w-complex ity g raphical models for a giv en linear code. Roughly speaking, a lo w-comple xity graphica l model is one that implies a lo w-comple xity decoding algorithm. In particula r , in this paper , we in vestig ate t he questio n o f ho w small the comple xity of an arb itrary graphical model for a giv en code can be. W e briefly in troduce gr aphical models h ere; a detailed description c an be found in Section 2. A graph decompositi on of a cod e C is a mapping of th e set of coordinate s of C to th e set of vertices of a graph. A graph deco mposition may be vie wed as an assignment of symbol va riables to the vertices of the graph . A graph decomposit ion can be extended to a graphical model which additiona lly Date : October 20, 2018. This work w as supported by a Discov ery Grant from the Natural Sciences and Enginee ring Research Council (NSERC ), Canada. The author is wi th the Department of Mathematics and Stati stics, Queen’ s Uni versity , Kingston, ON K7L 3N6, Canada. Email: nkas hyap@mast.que ensu.ca . 1 2 N . K A S H Y A P assign s state varia bles to the edges of the grap h, and specifies a local constra int code at each verte x of the graph. The full beha vior of the model is the set of all configu rations of symbol and state v ariables that satisfy all the local c onstrain ts. Such a mod el is c alled a grap hical real ization of C if the rest riction of the full beha vior to the set o f symbol v ariables is precise ly C . The realization is said to b e cycle-fre e if the underlying grap h in the model has no cyc les. A tr ellis representat ion of a code can be viewed as a c ycle-fre e realiz ation i n which the u nderlyin g graph is a simple path. In contra st, a tailbiting trellis representa tion [13],[14] is a graphic al re alizatio n in which the underl ying graph cons ists of a single cycle . W e will focus our attention on the case of realizatio ns of linear codes on connect ed graphs only . Indeed , ther e is no loss of g enerality in d oing so, si nce a linear c ode C has a realization on a graph G that is not connected if and only if C can be expressed as the direct sum of codes that may be indi vidually realized on the conne cted componen ts of G [6]. In this conte xt, we will refer to cyc le-free graphica l realizations simply as tree reali zations, as the underlying graph is a co nnected, cyc le-free graph, i.e. , a tree. It is by now well kno wn that any graphic al realizati on of a code specifies a canon ical iterati ve message- passing d ecoding algorithm, namely , the sum-prod uct alg orithm, on the underlyin g graph [1],[6],[12],[18]. When the u nderlyin g graph is a tree, the sum-p roduct algor ithm provid es an exa ct implementa tion of maximum-like lihood (ML) decodi ng. Even when th e underly ing graph contains cyc les, empirical evid ence suggests that, in many cases , the sum-product algorithm continue s to be a good appr oximation to M L decod ing. The computati onal complexity of the sum-pr oduct algor ithm assoc iated with a graph ical real- ization of a code is largely det ermined by the sizes of the local constrai nt codes in the reali zation. In Section 3 of this paper , we define vari ous measur es of “constraint comple xity” of a graphical realiza tion that ca n be used as estimates o f the computational comple xity of sum-p roduct decod ing. These complexi ty measur es may be viewed as generalizati ons of previ ously pro posed measu res of trellis compl exity [17],[14], and tree c omplexi ty [7],[8]. Howe ver , for the mos t part, we focus on the κ -comple xity of a graphical realization, which w e define to be the maximum of the dimensions of the local cons traint codes in the realization. In the restricte d co ntext of tree realiz ations, it h as pre viously been establishe d that certa in “min- imal” tree re alization s can be can onically d efined. Let the ter m tr ee decompositio n denote a g raph decompo sition in which the graph is a tre e. It is k nown t hat among all tree real izations of a code C that exten d a gi ven tree decompositio n, there is one that minimizes the dimension of the state space at each edge of the underlying tree, and this minimal tree realizat ion is unique [6]. It has furthe r been sho wn [11] that this unique minimal tree realization also minimizes (among all tree realiza tions e xtending the g iv en tree decompos itions) the dimension of the local constraint co de a t each ve rtex of the tree. In particu lar , it has the least κ -co mplexity among all such tree realization s. In contra st, there is very little kno wn about the general case of realizations of a code on an arbitra ry (n ot n ecessari ly cycl e-free) grap h. For instance , there appear to be no “ca nonical” minimal realiza tions that can be defined i n th is situa tion. The o nly sy stematic study in this direc tion remains that of K oetter and V ardy [13],[14], who studied minimal tailbiting trellis representati ons of codes, which, as alread y mentione d, are graphic al realizatio ns in which the und erlying graph consists of exa ctly one cycle. Beyond this basic (though by no means easy) case, there is little of interest in the l iterature on the comple xity of realization s of co des on arbitrary graph s, the notabl e excep tion to this bein g the work of Halford and Chugg [8]. In their work, Halford and C hugg lay the found ations for a systemat ic study of comple xity of graphi cal realizati ons. Their m ain result is the “Forest-In ducing Cut-Set Bound”, which giv es a lo wer bound on the constrain t complexity of a grap hical realizatio n in terms of its minimal tree comple xity . Howe ver , this bound does n ot appear be user-fr iendly in practic e. The main limitation of their approach is that th ey rely on t he Edge-Cut Bound o f W iber g e t al. [18],[19] to deriv e their C O N S T R A I N T C O M P L E X I T Y O F G R A P H I C A L R E A L I Z A T I O N S 3 results . While the Edge- Cut Bound has been put to good u se in the stu dy of “state compl exity ” of graphi cal realizations [5],[6], it is of limited value in the analy sis of constraint complexit y . The m ain aim of our paper is to present useful and tight lower bounds on the constraint com- ple xity of a g raphical realiza tion. Our bound s also provide con siderable insight into the pr oblem of finding low-compl exity grap hical realizat ions. T he fundamental tool in our a nalysis is the V erte x- Cut Bound, which we state and pro ve in Section 4. The V ertex-Cut Bound is a natural analogue of the E dge-Cut Bound, b ut as we shall see, it is mor e su itable f or use in the analysis of constr aint comple xity . In Section 5, we define a data structure called v erte x-cut tr ee that stores the information necessary about a g raph to ef fecti vely apply the V ertex-Cut Bou nd. V erte x-cut trees are simila r in structu re to the junctio n trees associ ated with belief propagatio n algor ithms [10],[1]. The v c-width of a vertex - cut tree i s a measure of the siz e of the v ertex- cut tree, and the vc- tr eewidt h of a grap h is the least vc-width among all it s verte x-cut trees. The vc-t ree width of a graph is very clo sely related to the notion of tree width of graphs much studied in graph theory [16],[3]. Using the V ert ex-Cut Bound a nd the notion of v ertex -cut trees, we deri ve, in Section 6, a s uite of lower bounds on the κ -complex ity of grap hical realizat ions of a linear co de C . W e sta te one of these bounds h ere as an illustrati ve ex ample. Let κ tree ( C ) be the lea st κ - complex ity among all t ree realiza tions of C . Consi der an arbitrary graph G , and let κ vc-tree ( G ) denote its vc-tree width. Then, the κ -comple xity of a ny realizat ion of C on G is bounded from belo w by the ratio κ tree ( C ) /κ vc-tree ( G ) . W e fu rther apply our methods to answer certai n questio ns raised in [11]. B orro w ing t erminology from [17], for a code C , let b ( C ) denote the least edge-comple xity of any trellis represent ation of C or any of its coordinate permut ations. In the langua ge of our paper , b ( C ) is th e le ast κ -co mplexi ty of any con vention al trellis realization of C . W e s ho w that for an y linea r code of leng th n , w e hav e b ( C ) /κ tree ( C ) = O (log 2 n ) , an d that th is is the bes t poss ible estimate of the ra tio, up to the con stant implicit in the O -notation. This is used t o ex tend a kno wn lower bo und [15] on b ( C ) in terms of the length n , dimensio n k and minimum dist ance d of C , to a lo wer bound on κ tree ( C ) . Our lower bound on κ tree ( C ) has an important implicati on. It sho ws that if C is a code family with the propert y that, for each C ∈ C , κ tree ( C ) is bound ed from abov e by a fi xed cons tant, then either the d imension or the minimu m di stance of the cod es in C gro ws sub -linearl y with code length. Thus, such code familie s are n ot goo d from a codin g-theore tic pers pecti ve. W e als o pro ve a slightly more gene ral result, which can be roughly interp reted as sayi ng that a good error -correcting code canno t ha ve a lo w -comple xity realiza tion on a graph with small vc-tree width. So, for good codes , if lo w-complexity gra phical realiza tions exist, the n the y must nece ssarily exist o n g raphs with lar ge vc-tree width. Some concludin g remarks are made in S ection 7, and an example in support of a state ment in Section 3 is gi ven in an appe ndix. 2. B AC K G R O U N D A N D N OT ATI O N In this section, we prov ide the necessa ry backgroun d, and define the notation we use in the paper . W e take F to be a n arbitrary fi nite field. Giv en a finite index set I , we hav e the vector space F I = { x = ( x i ∈ F , i ∈ I ) } . For x ∈ F I and J ⊆ I , the notation x | J will denote the pr ojectio n ( x i , i ∈ J ) . Also , for J ⊆ I , we will find it con venient to reserv e the use of J to denote the set { i ∈ I : i / ∈ J } . 2.1. Codes. A linear code over F , defined o n the index set I , is a sub space C ⊆ F I . In this paper , the terms “code ” and “linear code” will be used intercha ngeably to mean a linear code ov er an arbitra ry finite field F , unless explicit ly specified otherwise. The dime nsion, over F , of C will be denote d by dim( C ) . An [ n, k ] code is a code o f lengt h n and d imension k . If, addition ally , the co de has minimum distan ce d , then the code is an [ n, k , d ] code. 4 N . K A S H Y A P 1 2 3 4 5 6 7 8 9 10 11 F I G U R E 1 . A graph decomposit ion of I = { 1 , 2 , 3 , . . . , 11 } . Let J be a subset of the index set I . The pr ojection of C onto J is the code C | J = { c | J : c ∈ C } , which is a subsp ace of F J . W e wil l use C J to de note the cr oss-se ction of C consisting of all projec tions c | J of code words c ∈ C that satisfy c | J = 0 . T o be precise, C J = { c | J : c ∈ C , c | J = 0 } . Note that C J ⊆ C | J . Also, since C J is isomorphic to the kern el of the projection map π : C → C | J defined by π ( c ) = c | J , w e hav e that dim( C J ) = dim( C ) − dim( C | J ) . As a conseq uence, we see that if J ⊆ K ⊆ I , the n d im( C J ) ≤ dim( C K ) . If C 1 and C 2 are c odes ov er F defined o n mutuall y disjoint i ndex s ets I 1 and I 2 , re specti vely , th en their di r ect sum is t he code C = C 1 ⊕ C 2 defined on t he inde x set I 1 ∪ I 2 , su ch that C I 1 = C | I 1 = C 1 and C I 2 = C | I 2 = C 2 . T his definition naturally extends to mult iple codes ( or subspaces) C α , where α is a cod e id entifier th at ta kes value s in some set A . A gain, it must be assumed tha t the codes C α are defined on mutually disjo int inde x sets I α , α ∈ A . The dir ect sum in this situation is d enoted by L α ∈ A C α . 2.2. Graphs. In this paper , w e are primarily interested in graphs that are connecte d, so any un- qualif ed u se of the te rm “graph” should b e taken to mean “connecte d graph”. Let G = ( V , E ) be a graph, where V and E denote its ve rtex and ed ge s ets, r especti vely . T o re solve ambiguity , we will sometimes denote the vertex and edg e sets of G by V ( G ) and E ( G ) , respecti vely . G i ven a v ∈ V , the set of edges inci dent with v will be denot ed b y E ( v ) . For X ⊆ E , we define G \ X to be the subgrap h of G obtain ed by deleting all the edges in X . If X consists of a sing le edge e , then we will w rite G \ e instead of G \ { e } . If G \ X is disconn ected, then X is called an edge cut of G . Similarly , for W ⊆ V , w e define G − W to be the su bgraph of G obta ined by deleting all th e vertic es in W along with all inci dent edges. If W consists of a singl e ver tex v , then we will write G − v inste ad of G − { v } . If G − W is disconn ected, th en W is called a verte x c ut of G . A tre e is a connected graph wit hout cy cles. V ertices of de gree one i n a tree are called leav es , and all othe r vertic es are call ed internal nodes . N ote that an y subset of the edge s of a tree con stitutes an edge cut, an d any subset of interna l nodes constitut es a v ertex cut o f the tree. If e is an edge in a tree T , the n we will den ote by T ( e ) and T ( e ) the two components of T \ e . If v is a v ertex of degree δ in a tree T , then we will use T ( v ) 1 , T ( v ) 2 , . . . , T ( v ) δ to deno te the components of T − v . A path is a tree w ith exac tly two leav es (the end-points of the p ath). All internal nodes in a path ha ve de gree two. A simple cycle is a connected graph in which all verti ces ha ve degree two. An n -cycle , n ≥ 3 , is a simple c ycle with n ver tices. 2.3. Graphical Realiza tions of Codes. The de velop ment in this section is based on the e xpositi on of Forne y [6],[7]; see also [8],[11]. Let I be a finite inde x set. A graph decomposit ion of I is a pair ( G , ω ) , where G = ( V , E ) is a graph, and ω : I → V is an ind ex map ping . For a code C , we will usually write “graph dec omposition of C ” as s horthand for “ graph decomposition of th e index set of C ”. W e again wish to emphasize that, unless expl icitly stated otherwise, we will take G to be a connected graph. W hen G is a tree , ( G , ω ) will be called a t r ee decompo sition . Pictorially , a graph deco mposition ( G , ω ) is depicted as a graph with an addit ional feature: at each verte x v such that ω − 1 ( v ) is non-empty , we attach special “half- edges”, one for each inde x in ω − 1 ( v ) . Figure 1 depict s a graph decomposition of I = { 1 , 2 , 3 , . . . , 11 } . C O N S T R A I N T C O M P L E X I T Y O F G R A P H I C A L R E A L I Z A T I O N S 5 For a graph G = ( V , E ) , recall that E ( v ) , v ∈ V , denotes the set of edg es in cident with v in G . Consider a tuple of the form 1 ( G , ω , ( S e , e ∈ E ) , ( C v , v ∈ V )) , where • ( G , ω ) is a graph decomposition of I ; • for each e ∈ E , S e is a vecto r space over F called a state space ; • for each v ∈ V , C v is a subspace of F ω − 1 ( v ) ⊕  L e ∈ E ( v ) S e  , called a local constra int code , or simply , a local constra int . Such a tuple w ill be called a graphi cal model . A g raphical model in w hich the underlyin g graph G is a tre e will be called a tr ee model . The elements of an y state spa ce S e are calle d states . The index sets of t he stat e spaces S e , e ∈ E , are t aken to be mutually disj oint, and are also taken t o be d isjoint from the inde x set I correspon ding to the symb ol varia bles. A gl obal configu ration of a graphical model as above is an as signment of v alues to eac h of the symbol and state var iables. In other word s, it is a vector of the form (( x i ∈ F , i ∈ I ) , ( s e ∈ S e , e ∈ E )) . A global configurati on is said to be valid if it satisfies all the local constrain ts. Thus, (( x i ∈ F , i ∈ I ) , ( s e ∈ S e , e ∈ E )) is a va lid globa l configura tion if for each v ∈ V , (( x i , i ∈ ω − 1 ( v )) , ( s e , e ∈ E ( v )) ) ∈ C v . T he set of all v alid global configurations o f a graphical model is called the full behav ior of the model. Note that the full beha vior is a subs pace B ⊆ F I ⊕  L e ∈ E S e  . As usual, for J ⊆ I , B | J denote s the proje ction of B onto the index set J . For futu re con venie nce, we also define certai n other projecti ons of B . Let b = (( x i , i ∈ I ) , ( s e , e ∈ E )) be a global configuration in B . At any gi ven v ∈ V , the local configu ratio n of b at v is defined as b | v = (( x i , i ∈ ω − 1 ( v )) , ( s e , e ∈ E ( v ))) . The set of all loc al configu rations of B at v is th en define d as B | v = { b | v : b ∈ B } . By defini tion, B | v ⊆ C v . Similarly , giv en any e ∈ E , if b is a global configu ration as above , then we define b | e = s e ; we further define B | e = { b | e : b ∈ B } . Clearly , B | e is a subspac e of S e . A graphical mode l ( G , ω , ( S e , e ∈ E ) , ( C v , v ∈ V )) is defined to be essent ial if B | v = C v for all v ∈ V an d B | e = S e for all e ∈ E . When G is a tree, there is some re dundan cy in the abo ve definitio n, as the co ndition B | e = S e for a ll e ∈ E actual ly implies that B | v = C v for a ll v ∈ V [11, Lemma 2.2]. It is w orth n oting that , in an essential graph ical model, at an y edge e = { u, v } , the state space S e , may b e vie wed as a projection of ea ch o f the loc al co nstraint c odes C u and C v . Thus, for any e ∈ E , dim( S e ) ≤ dim( C v ) , where v is any ve rtex incid ent wit h e . An arbitr ary graphical model Γ with full behav ior B can alway s be “essen tialized” by simply replac ing each local constra int C v in Γ with the projec tion B | v , an d re placing each state spa ce S e with the projec tion B | e . The resultin g “e ssentiali zation” of Γ still has full beha vior B . An essentia l graphical model ( G , ω , ( S e , e ∈ E ) , ( C v , v ∈ V )) is defined to be a graphica l r ealization of a code C , or simpl y a r ealiza tion of C on G , if B | I = C . A graphica l realizatio n of C in which the underlyi ng graph G is a tree, is called a tr ee r ealizat ion of C . Our definition of a graphi cal (and tree) re alization dif fers slightly from the p rior definitio ns in [6],[7],[8],[11], in t hat we require the underly ing graphical model to be essential. A s exp lained abov e, any grap h model can be essen tialized, so there is no loss of generality in this definition. A gr aphical (resp. tree) r ealizatio n ( G , ω , ( S e , e ∈ E ) , ( C v , v ∈ V )) o f C is said to e xtend , or be an ext ension of, the graph (resp. tree) decompositio n ( G , ω ) of C . W e will denote by R ( C ; G , ω ) the set of all grap hical realization s of C that ex tend the graph decompositio n ( G , ω ) of C . No w , it is an easily verifiable fact that any tree decomposition of a code can always be exten ded to a tree realiza tion of the code [7],[11]. Such an extensio n is not unique in genera l, b ut we will descri be a canonica l “minimal” ext ension a little later . More gene rally , any graph decompo sition of a code C can always be extend ed to a graphic al realization of C , as we now exp lain. Let G = 1 In r eferring to a tuple of the f orm ( G , ω, ( S e , e ∈ E ) , ( C v , v ∈ V )) , we will implicitly assume that V and E den ote the verte x and edge sets, respectiv ely , of the graph G . 6 N . K A S H Y A P ( V , E ) be a connected graph, and suppose that ( G , ω ) is a graph deco mposition of C . T ake T to be any spanning tree of G , and let E T denote its edge set. T hen, ( T , ω ) is a tree dec omposition of C . As noted abov e, this tree decompositi on can be exten ded to a tree realiz ation ( T , ω , ( S e , e ∈ E T ) , ( C v , v ∈ V )) o f C . W e fur ther exten d this to a realization of C on G as follo w s. Define the state space s S e , e ∈ E , as S e = ( S e if e ∈ E T { 0 } if e ∈ E \ E T , and f or each v ∈ V , define the local constrai nt C v = C v ⊕ ( L e ∈ E ( v ) \ E T { 0 } ) . It sho uld be c lear that ( G , ω , ( S e , e ∈ E ) , ( C v , v ∈ V )) is a graphi cal r ealizatio n of C . Graphica l realization s of codes in which the underlyi ng graph is a path or a simple cycle ha ve recei ved cons iderable prior atte ntion in th e litera ture. Such realiz ations were c alled “con vention al state rea lization s” (when the underl ying graph is a pa th) and “tail-b iting state reali zations” (when the underlyin g graph is a simple cyc le) in [6]. W e will call them “trellis realizatio ns”. Briefly , a tr ellis r ealizatio n of a cod e C (defined on the index s et I ) is any e xtension of a grap h decomposi tion of C of th e for m ( G , ω ) , where G is either a path or a simple cy cle, and ω is a surjecti ve map ω : I → V ( G ) . A trellis rea lization in which the surj ecti ve m ap ω : I → V ( G ) is n ot in jecti ve (so tha t ω is not a bij ection), is usually call ed a sectiona lized tr ellis re alization . A con ventional trellis real ization is one in whic h the un derlying graph G is a path. When the underlying graph G is a simple c ycle, the t rellis realizat ion is said to be tailbiti ng . The th eory of con vention al trellis rea lizations is well establ ished; see, for example , [17]. O n the other hand, tailbiting trellis realizati ons are less w ell unders tood; the princi pal systematic study of these remains that of Koett er and V ardy [13],[14]. W e rema rk that o ur require ment that graphical realizatio ns hav e underly ing graph models that are essent ial correspon ds to the requir ement in [13],[14] that “linear trellises” be “reduced ”. 3. C O M P L E X I T Y M E A S U R ES F O R G R A P H I C A L R E A L I Z A T I O N S As ob served in [6], an y gra phical rea lization of a co de s pecifies a class of associated g raph-bas ed decod ing algorithms, namely , the sum-p roduct algorithm and its v ariants. T hus, ideally , any defi- nition o f a co mplexit y measure for a graphical realizat ion should try to capture the co mputation al comple xity of the associat ed decod ing algo rithms. T he analysis in [6 , Section V] shows that the computa tional complexi ty of the sum-product algorith m speci fied by a giv en grap hical realization of a code is determined in lar ge part by the cardin alities, or equiv alently dimension s, of the local constr aint codes in the realization . Thus, as a simple measure of the complexity of a graphical realiza tion, which roughly reflects the comple xity of sum-product decod ing, we w ill cons ider the maximum of the dimens ions of the local constraint codes in the realization . Let Γ = ( G , ω , ( C v , v ∈ V ) , ( S e , e ∈ E )) be a graphical realization of a code C . T he constr aint max-comple xity , or simply κ -comple xity , of Γ is defined to be κ (Γ) = max v ∈ V dim( C v ) . No w , recall that if ( G , ω ) is a gr aph decomposition of C , t hen R ( C ; G , ω ) denotes the se t of all graphica l realiza tions of C that ex tend ( G , ω ) . W e further define κ ( C ; G , ω ) = min Γ ∈ R ( C ; G ,ω ) κ (Γ) (1) W e add an other le vel of mini mization by defining, for a g iv en grap h G , the G -width of a code C to be κ ( C ; G ) = min ω κ ( C ; G , ω ) , (2) where the minimum is taken ove r all possible index mappings ω : I → V ( G ) , where I is the index set of C . T hus, the G -width of C is the least κ -comple xity of any realiza tion of C on G , and may be taken to b e a measu re o f the leas t co mputation al comple xity of any sum-prod uct-type decodi ng algori thm for C implemented on the graph G . C O N S T R A I N T C O M P L E X I T Y O F G R A P H I C A L R E A L I Z A T I O N S 7 A broade r optimizat ion problem of consid erable interest is the follo wing: gi ven a code C and a family of graphs G , identif y a G ∈ G on which C can be realize d w ith the least possible κ - comple xity . W e thus define κ ( C ; G ) = min G ∈ G κ ( C ; G ) . (3) T wo s pecial cases of this definition — tre ewidt h and pathwid th (or trellis-wid th) — are partic ularly of inter est. W e define tree width first, and pathwidth a little further below . If we let T denote the set of all trees, then κ ( C ; T ) is called the tre ewidth of the code C [11], which we will denote by κ tree ( C ) . The notion of tree w idth ( i.e. , minimal κ -complex ity among tree realiza tions) of a code w as first consider ed by Fo rney [7], an d an analogou s notion has been define d for matroi ds in [9]. The argu ments in [7, Section V] (and also in [9]) sho w that κ tree ( C ) can alwa ys be obta ined by m inimizing κ ( C ; T , ω ) ove r tree decompos itions ( T , ω ) in whic h T is a cubic tree ( i.e. , a tree in which all internal nodes hav e degree 3), and ω is a bijection between the index set of C and the set of lea ves of T . A complexi ty measure related to treewidth , termed minimal tree comple xity , was defined and studie d by Halford and C hugg [8]. Tr ee width, as we hav e defined abov e, i s an u pper bo und on the minimal tree compl exity of Halford and Chugg. The pathwidth , κ path ( C ) , of a code C is defined to be the qua ntity κ ( C ; P ) , w here P denotes the sub-f amily of T cons isting of all paths . W e will find it con venie nt to refer to tr ee decompos itions ( P , ω ) , with P ∈ P , as path deco mpositions . Thus, κ ( C ; P ) is the minimum va lue of κ ( C ; P , ω ) as ( P , ω ) range s ov er all path decompo sitions of C . In fac t, by the ar gument of [7, S ection V .B], the min imizing path dec ompositio n ( P, ω ) m ay be taken to be one in which th e index map ping ω is surjec tiv e. Thus, κ path ( C ) is the least κ - complex ity of any con vent ional trell is reali zation of C , a nd so we m ay al so call it the (con ventio nal) tre llis-width 2 of C . It is also kn own th at section alization canno t reduce the κ -comple xity 3 of a trellis realization [17, T heorem 6.3], and hence, κ path ( C ) is the minimum v alue o f κ ( C ; P , ω ) over all path de compositio ns ( P , ω ) in which the inde x mappin g ω is a bijection between the index set of C and the ver tices of P . Measures of co nstraint comple xity other than κ -c omplexi ty hav e been propo sed in the pr ev ious literat ure, especially in the conte xt of trellis realizat ions [17],[14]. The κ + -comple xity of a graph- ical real ization Γ = ( G , ω , ( C v , v ∈ V ) , ( S e , e ∈ E )) is defined to be κ + (Γ) = P v ∈ V dim( C v ) . Note that 1 | V | κ + (Γ) is the ave rage local constraint co de dimens ion in Γ . On th e oth er han d, it has been sugg ested [7] that the sum of the constrai nt code cardinaliti es “may be a better guide to de- coding complexity ” than κ -comple xity or κ + -comple xity . Thus, we define κ tot (Γ) = P v ∈ V | C v | = P v ∈ V | F | dim( C v ) . Analog ous to (1)–(3), we may d efine κ + ( C , G , ω ) , κ tot ( C ; G , ω ) , etc., b ut we will only touch upon thes e briefly in this paper . W e remark that while we ha ve used con straint code dimensions to define our comple xity mea- sures for graph ical realizations , one coul d also define measures of complexity based on state-spa ce dimensio ns. For e xample, we could define the state max-complex ity , σ (Γ) , of a graphical realiza - tion Γ to be the m aximum of the dimensions of the state spac es in Γ . Similarly , we may consider t he comple xity mea sures σ + (Γ) and σ tot (Γ) analo gous to κ + (Γ) and κ tot (Γ) . These measu res are espe- cially rele v ant and hav e been well studied in the context of trellis realizations ; again, see [17],[14]. Ho wev er , as noted by Forney [7 ], measures of state -space comple xity become less approp riate in the conte xt of realization s on arbitra ry g raphs or trees. For examp le, for any code C ⊆ F n , one can alw ays fi nd a tree on w hich C can be realized in such a way tha t all st ate-spac es ha ve dimension at most 1. T his w ould be the “st ar- shaped” tree T consis ting of n lea ves co nnected to a single interna l node v of de gree n . T ake ω to be any bijection be tween the inde x set of C and the lea ves of T ; set S e = F at each edge e of T ; and finally , tak e the local constraint code C v at the internal node to be C itself, and take the l ocal constraint codes at the lea ves to be [2 , 1] repeti tion codes. Clearl y , the 2 For this reason , what we have called κ path ( C ) here was called κ trellis ( C ) in [11]. 3 Our notion of κ -comple xity corresponds to t he notion of “edge-c omplexity” in [17]. 8 N . K A S H Y A P resulti ng tree model (after essentia lization ) is a tree realiza tion of C . Thus, it makes li ttle sense t o define a s tate-spa ce co mplexit y measure analog ous to tree width, unless we restri ct the kind of trees on which we are allo w ed to realize the gi ven code 4 . The astute reader may point out that in the tri vial tree realizat ion abov e, the sum of the state- space dimensio ns is non-tri vial, and so a state-sp ace analog ue to treewid th could potentia lly be defined in terms of σ + or σ tot . This may be tru e, but we do no t pursue this fur ther , s ince, as alre ady observ ed pre viously , complex ity measures ba sed on cons traint code di mensions are a better guide to dec oding comple xity . But, while on this topic, we mention in pas sing that for any t r ee rea lization Γ of a code C , it turns out that κ + (Γ) = dim( C ) + σ + (Γ) . (4) Thus, the pro blem of minimizing σ + (Γ) among tree realizatio ns Γ of a giv en c ode C is equiv alent to the probl em of minimizing κ + (Γ) . The ident ity in (4), which may be vie wed as a generalizati on of th e statemen t of Theorem 4.6 in [14] for con vention al t rellis realiza tions, w ill n ot be pro ved here as it w ould be an unnecessa ry de viation from the mai n line of our d ev elopment. It s uffice s to say that (4) follo w s from Theorem 3.4 in [11] by first veri fying that it indeed holds for any minimal tree realizatio n M ( C ; T , ω ) , and then observing that the dif ference between κ + (Γ) and σ + (Γ) is preser ved by the state-mer ging process mentioned in the statement of that theorem. Finally , we remark that the state max-comple xity of a graphical realization cannot excee d the constr aint m ax-comp lexity of the realiza tion. This is because, as observ ed in Section 2.3, in any essential graphical m odel ( G , ω , ( S e , e ∈ E ) , ( C v , v ∈ V )) , for each edge e ∈ E , we hav e dim( S e ) ≤ dim( C v ) , where v is any ve rtex incident with e . 3.1. Minimal Realiza tions. Giv en a code C and a tree decompos ition ( T , ω ) of C , there exists a tree realiza tion, ( T , ω , ( S ∗ e , e ∈ E ) , ( C ∗ v , v ∈ V )) , of C with the follo wing property [6],[7]: if ( T , ω , ( S e , e ∈ E ) , ( C v , v ∈ V )) is a tree realizatio n of C that exten ds ( T , ω ) , then for all e ∈ E , d im ( S ∗ e ) ≤ dim( S e ) . This minimal tree realization, w hich we hencefort h deno te by M ( C ; T , ω ) , is unique up to isomor- phism 5 . Constructi ons of M ( C ; T , ω ) can be found in [6],[7],[11]. It has fur ther been shown [1 1] that not only do es M ( C ; T , ω ) minimize (amon g realization s in R ( C ; T , ω ) ) the stat e sp ace dimen sion a t each edge of T , b ut it al so min imizes th e local cons traint code dimensi on a t each verte x of T . More precisely , M ( C ; T , ω ) also has the follo wing property: if ( T , ω , ( S e , e ∈ E ) , ( C v , v ∈ V )) is a tree realizatio n of C that exten ds ( T , ω ) , then for all v ∈ V , dim( C ∗ v ) ≤ dim( C v ) . Consequ ently , we hav e that κ ( C ; T , ω ) = κ ( M ( C ; T , ω )) , κ + ( C ; T , ω ) = κ + ( M ( C ; T , ω )) , and κ tot ( C ; T , ω ) = κ tot ( M ( C ; T , ω )) . The fact that M ( C ; T , ω ) minimizes local constrai nt code di- mension at each ve rtex of T w ill be cen tral to the deri v ation of our results in the sections to follo w . W e will henceforth consiste ntly use the notation S ∗ e and C ∗ v to denote state spaces and local constr aint codes in a minimal tree realiz ation M ( C ; T , ω ) . Exact expr essions for the dimensions of S ∗ e and C ∗ v in M ( C ; T , ω ) are known [6],[7]. Recall that for an edge e of T , w e denote by T ( e ) and T ( e ) the two compone nts of T \ e . Let us further define J ( e ) = ω − 1 ( V ( T ( e ) )) and J ( e ) = ω − 1 ( V ( T ( e ) )) . W e then ha ve dim( S ∗ e ) = dim( C ) − dim( C J ( e ) ) − dim( C J ( e ) ) . (5) 4 W e do get a reason able state-space analogue to tree width if we restrict the class of trees ove r which we attempt to minimize state-space comple xity to the class of cubic trees only; see [11]. 5 Graphical realizations ( G , ω , ( C v , v ∈ V ) , ( S e , e ∈ E )) and ( G , ω, ( C ′ v , v ∈ V ) , ( S ′ e , e ∈ E )) of a code C are said to be isomorphic if, for each v ∈ V , C v and C ′ v are isomorphic as vector spaces, and for each e ∈ E , S e and S ′ e are isomorphic as v ector spaces. W e do not distinguish between isomorphic graphical realizations. C O N S T R A I N T C O M P L E X I T Y O F G R A P H I C A L R E A L I Z A T I O N S 9 Next, consider an y vertex v in T . If v has de gree δ , then T − v has component s T ( v ) i , i = 1 , 2 , . . . , δ . Define J i = ω − 1 ( V ( T ( v ) i )) , for i = 1 , 2 , . . . , δ . Then [7, Theorem 1], dim( C ∗ v ) = dim( C ) − δ X i =1 dim( C J i ) . (6) In summary , the minimal tree r ealizatio n M ( C ; T , ω ) is an e xact solution to the problem o f dete r- mining the m inimum-comp lexity e xtension o f a tre e decompositio n ( T , ω ) of a code C . Moreov er , M ( C ; T , ω ) minimize s, among realizations in R ( C ; T , ω ) , any reason able measure o f complex ity , be it s tate-spac e complexity or co nstraint complexi ty . Unfortunatel y , when we mov e to real izations on graphs with cycles, there a ppear to be no “canonical ” minimal realizations w ith properti es sim- ilar to those of minimal tree realizati ons. In fact, if ( G , ω ) is a graph decompo sition of a code C , where G is a graph with cycles , there need not ev en be a realization Γ that simultaneo usly ac hie ves min Γ ∈ R ( C ; G ,ω ) κ (Γ) and min Γ ∈ R ( C ; G ,ω ) κ + (Γ) . An example of suc h a graph decompositi on is gi ven in Append ix A. Thus, gi ven a code C and a graph G containing c ycles, th e pr oblem of finding rea lizations of C on G with the lea st possib le κ -complex ity , κ + -comple xity , or κ tot -comple xity (within some int eresting sub-cl ass of realization s of C on G ) is much harder to solve than the corr espondin g problem for cyc le-free graph s. In the nex t section, we present a simple but v aluable tool that w ill en able us to deri ve no n-tri vial lower bounds on the constraint compl exity of real izations of a cod e on an arbitrary graph. These bounds could be used , for example, to determine whether or not the complex ity of a gi ven reali zation is close to the least possible. 4. C U T - S E T B O U N D S Let G = ( V , E ) be a connecte d graph. A partitio n ( V ′ , V ′′ ) of V is said to be separ ated by an edge cut X ⊆ E if, for each pair of vertices v ′ ∈ V ′ and v ′′ ∈ V ′′ , any path in G that joins v ′ to v ′′ passes thro ugh some edge e ∈ X . W e remark that if G \ X has more t han two co mponents, then there is more than one parti tion of V that is separated by X . The Edge- Cut Bound, stated bel ow , is a re sult of fundamen tal importance in the stu dy of graph i- cal realizat ions. This result was originall y obser ved by W iber g, Loelig er and Koet ter [18],[19], but the v ersion we giv e here is due to Forney [6, Coro llary 4.4]. Theor em 4.1 (The E dge-Cut Boun d) . Let Γ = ( G , ω , ( C v , v ∈ V ) , ( S e , e ∈ E )) be a r ealizatio n of a code C on a connected grap h G = ( V , E ) . If ( V ′ , V ′′ ) i s a pa rtition of V s epara ted by an ed ge cut X ⊆ E , then, defining J ′ = ω − 1 ( V ′ ) and J ′′ = ω − 1 ( V ′′ ) , we have X e ∈ X dim( S e ) ≥ dim( C ) − dim( C J ′ ) − d im( C J ′′ ) . The edge-cut b ound can be used to deri ve useful lower bounds on th e state-space c omplex ity of a graph ical rea lization; s ee, for example, [5]. T o deal with constrain t comple xity , ho wev er , we will need a close ly-relate d bound that uses vert ex cuts instead of edge cuts. W e introduce he re some terminology tha t w e w ill use to s tate our vertex -cut bo und. For v ∈ V , let N ( v ) = { u ∈ V : { u, v } ∈ E } denote the set of neigh bours of v in G . Further more, for W ⊆ V , define N ( W ) = S v ∈ W N ( v ) . Definition 4 .1. An or der ed collection ( V 0 , V 1 , . . . , V δ ) , δ ≥ 0 , of su bsets of V is said to be a star partiti on of V , if the V i ’ s form a p artition of V ( i.e. , t he V i ’ s ar e pai rwise disj oint, and S δ i =0 V i = V ), and for e ach i ∈ { 1 , 2 , . . . , δ } , we have N ( V i ) ⊆ V i ∪ V 0 . The definition has been worded so as to allo w some of the V i ’ s to be empty sets. Wh en V i is non-emp ty for at most one i ≥ 1 , a star partition is simply a partition . When at least two V i ’ s 10 N . K A S H Y A P . . . . . . . . . . . . . . . . . . . . . . . . v 0 v 1 v 2 v 3 v 4 G [ V 0 ] G [ V 1 ] G [ V 2 ] G [ V 3 ] G [ V 4 ] J 3 J 4 J 2 J 1 J 2 J 1 J 3 J 4 F I G U R E 2 . A depiction of the constructio n in the proof of the V ert ex-Cut Bound. G [ V i ] denotes the subgraph of G induced by the vertices in V i . other than V 0 are non-empt y , then a star partition is a partit ion that arises from a verte x cut of G , as we no w ex plain. For any i > j ≥ 1 , if V i and V j are both non -empty , then the abo ve definition simply says that an y pa th b etween a v erte x in V i and a vertex in V j must pass t hrough V 0 . Thus, if at least two V i ’ s other than V 0 are non-empt y , then V 0 is a verte x cut of G . Con versely , if V 0 is a ver tex c ut of G , and G 1 , G 2 , . . . , G δ , δ ≥ 2 , are th e (non-empty ) components of G − V 0 , th en, setting V i = V ( G i ) f or i = 1 , 2 , . . . , δ , we se e tha t ( V 0 , V 1 , . . . , V δ ) i s a star partition of V . The graph on the left in F igure 2, w hich depicts a typical sit uation cover ed b y the definition, sho uld als o explain the nomen clature. Theor em 4.2 (The V ertex -Cut Bound) . Let Γ = ( G , ω , ( C v , v ∈ V ) , ( S e , e ∈ E )) be a r ealiza tion of a code C on a connected gra ph G = ( V , E ) . If ( V 0 , V 1 , . . . , V δ ) is a star partitio n of V , then, definin g J i = ω − 1 ( V i ) for i = 1 , 2 , . . . , δ , we have X v ∈ V 0 dim( C v ) ≥ dim( C ) − δ X i =1 dim( C J i ) . Pr oof. Let B denote the full behavi our of Γ . If b = (( x i , i ∈ I ) , ( s e , e ∈ E )) is a global configu ra- tion in B , then giv en an X ⊆ E , we will use b | X to denote the projection ( s e , e ∈ X ) . W e further set B | X = { b | X : b ∈ B } . For i = 1 , 2 , . . . , δ , let X i be the set of edges of G with exactly on e end-p oint in V i , so that the other end- point is necessarily in V 0 . W e then define B | V i = { ( b | J i , b | X i ) : b ∈ B } , for i = 1 , 2 , . . . , δ . Furthermor e, s et J 0 = ω − 1 ( V 0 ) and X 0 = S δ i =1 X i , and define B | V 0 = { ( b | J 0 , b | X 0 ) : b ∈ B } . No w , consid er the “star -shaped” tree T = ( V T , E T ) con sisting of a single internal ve rtex v 0 of deg ree δ , w hose neigh bours v 1 , v 2 , . . . , v δ are all the lea ves of T . T hus, V T = { v 0 , v 1 , . . . , v δ } and E T = {{ v 0 , v i } : i = 1 , 2 , . . . , δ } . Define the inde x map ping α : I → V T as foll ows: α ( j ) = v i if f ω ( j ) ∈ V i . N ote th at, for i = 0 , 1 , 2 , . . . , δ , we hav e α − 1 ( V i ) = ω − 1 ( V i ) = J i . T he constructio n of the tree decomp osition ( T , α ) from ( G , ω ) is depi cted in Figure 2. C O N S T R A I N T C O M P L E X I T Y O F G R A P H I C A L R E A L I Z A T I O N S 1 1 W e next exte nd t he tree decompositio n ( T , α ) to a tree m odel b Γ = (( T , α, ( b S e , e ∈ E T ) , ( b C v , v ∈ V T )) by setting b C v i = B | V i for i = 0 , 1 , 2 , . . . , δ , and b S { v 0 ,v i } = B | X i for i = 1 , 2 , . . . , δ . From the f act that Γ is a realizat ion of C , it readily follows that b Γ is a tree realization of C . Recalling that the m inimal tree realizatio n M ( C ; T , α ) minimizes the local const raint code di- mension at each ve rtex of T , we obtain via (6), dim( b C v 0 ) ≥ dim( C ) − δ X i =1 dim( C J i ) . W e complet e the proof by observin g that dim( b C v 0 ) = dim( B | V 0 ) ≤ X v ∈ V 0 dim( B | v ) = X v ∈ V 0 dim( C v ) .  The follo wing useful corollary is an immediate consequenc e of the V erte x-Cut Bound. Cor ollary 4.3. Let ( G , ω ) be a graph decompos ition of a code C , wher e G is a connected gra ph. F or a verte x cut W of G , if G 1 , G 2 , . . . , G δ ar e the compon ents of G − W , then define λ ( W ) = dim( C ) − P δ i =1 dim( C J i ) , wher e J i = ω − 1 ( V ( G i )) for i = 1 , 2 , . . . , δ . Then, for an y r ealization Γ = ( G , ω , ( C v , v ∈ V ) , ( S e , e ∈ E )) of C that exte nds ( G , ω ) , we have X v ∈ W dim( C v ) ≥ λ ( W ) . In its most straig htforwa rd application, the V ertex-Cu t Bound, via the abov e corollary , can be used in c onjuncti on with constra ined optimiz ation technique s to find lo w er bound s on κ ( C ; G , ω ) , κ + ( C ; G , ω ) and κ tot ( C ; G , ω ) . Indeed, if ( G , ω ) is a grap h de compositio n of C , and W 1 , W 2 , . . . , W t are v ertex cuts of G , t hen, by Coro llary 4.3, t he dimens ions of the loca l constrai nt codes C v , v ∈ V , in any Γ ∈ R ( C ; G , ω ) must sati sfy P v ∈ W i dim( C v ) ≥ λ ( W i ) , i = 1 , 2 , . . . , t . Thus, for example, κ + ( C ; G , ω ) is lower bounded by the soluti on to the followin g linear pro gramming prob lem in the v ariables ξ v , v ∈ V : giv en a c ollectio n of v ertex cuts W 1 , W 2 , . . . , W t of G , minimize X v ∈ V ξ v , subjec t to X w ∈ W i ξ w ≥ λ ( W i ) , i = 1 , 2 , . . . , t. Ho wev er , we do not pursue this ang le any fu rther in thi s paper . Instead, w e will henceforth re- strict our at tention to the κ -comple xity measu re, for which w e will deriv e a suite of lo wer bounds, again based on the V ertex-Cut Bound, which unearth some interesting co nnection s with graph the- ory , and moreov er , are amenable to further mathematical analys is. The bo unds we deriv e rely on the noti on o f verte x-cut tr ees introduce d in the ne xt section. 5. V E RT E X - C U T T R E E S W e begin with a simple lemma, which play s a role in our definition of a verte x-cut tree belo w . Lemma 5 .1. Suppose that V 0 , V 1 , . . . , V δ ar e subsets of V such th at S δ i =0 V i = V . If, f or each pair of disti nct i ndices i, j , we have V i ∩ V j ⊆ V 0 , then ( V 0 , V 1 \ V 0 , . . . , V δ \ V 0 ) is a partition of V . Pr oof. It is evid ent that V 0 ∪ S δ i =1 ( V i \ V 0 ) = S δ i =0 V i = V . If i, j > 0 , i 6 = j , then ( V i \ V 0 ) ∩ ( V j \ V 0 ) = ( V i ∩ V j ) \ V 0 = ∅ , since V i ∩ V j ⊆ V 0 .  For the m ain definition of this sect ion, we introduce s ome con venien t notation, which will hence- forth be used consistentl y . If A and B are sets, and f : A → 2 B is a mapping from A to the power set of B , then f or an y X ⊆ A , we de fine f ( X ) = S x ∈ X f ( x ) . Also, reca ll that if z is a vertex of deg ree δ in a tree T , then the compon ents of T − z are deno ted by T ( z ) i , i = 1 , 2 , . . . , δ . 12 N . K A S H Y A P Definition 5 .1. Let G be a c onnected gra ph. A ver tex-cu t tree of G is a data structur e ( T , β ) , wher e T is a tr ee, and β : V ( T ) → 2 V ( G ) is a mapping with the following pr opertie s: (VC1) β ( V ( T )) = V ( G ) ; (VC2) fo r ea ch pair x, y ∈ V ( T ) , if z ∈ V ( T ) is any ver tex that lies on the unique path b etween x and y in T , then β ( x ) ∩ β ( y ) ⊆ β ( z ) ; (VC3) fo r eac h z ∈ V ( T ) , ( β ( z ) , V 1 , V 2 , . . . , V δ ) is a star par tition of V ( G ) , wher e δ is the de gr ee of z , and V i = β ( V ( T ( z ) i )) \ β ( z ) for i = 1 , 2 , . . . , δ . A verte x-cut tre e ( T , β ) in which T is a path is called a vertex -cut path . Note that, by Lemma 5.1 , condition s (VC1) and (VC2) in the abov e definition imply that, for each z ∈ V ( T ) , with δ and V i , i = 1 , 2 , . . . , δ , as in condit ion (VC 3), ( β ( z ) , V 1 , V 2 , . . . , V δ ) is a partiti on of V ( G ) . Thus, for ( T , β ) satisfy ing conditions (VC1) and (VC2), cond ition (VC3) is met if f, for each z ∈ V ( T ) , we hav e N ( V i ) ⊆ V i ∪ β ( z ) for all i ≥ 1 . T ri vial vertex -cut trees ( and paths) always exist for a graph G — gi ven any tree T , pick a verte x z 0 ∈ V ( T ) , and define a mapping β by setting β ( z 0 ) = V ( G ) , and β ( z ) = ∅ for z 6 = z 0 . This allo w s us to make the follo wing definition. Definition 5.2. Let G be a connected gr aph. The vc -width of a verte x-cut tr ee ( T , β ) of G is defi ned as max z ∈ V ( T ) | β ( z ) | , and is denote d by vc-wid th ( T , β ) . T he vc -tree width (r esp. vc-pathwid th ) of G is the least vc-width among all verte x-cut tre es (r esp. verte x-cut paths) of G , and is denoted by κ vc-tree ( G ) (r esp. κ vc-path ( G ) ). Thus, for any gra ph G , we hav e 0 < κ vc-tree ( G ) ≤ κ vc-path ( G ) ≤ | V ( G ) | . The vc-t ree width of an y tree is equal t o one. Indeed, if T is a tree, then ( T , β ) , defined by β ( z ) = { z } for a ll z ∈ V ( T ) , is a ver tex-cu t tree of T , with vc-width equal to one. Example 5.1. L et G be an n -cycle with v ertices v 0 , v 1 , . . . , v n − 1 , lab eled in cyclic or der . Let P be a path with n − 1 vertices, which in the lin ear or der defined by the pat h, ar e lab eled z 1 , z 2 , . . . , z n − 1 . T o be pr ecise , z 0 is on e of the two leav es, and for i = 1 , 2 , . . . , n − 1 , z i is ad jacent to z i − 1 in P . If we define the mapping β : V ( P ) → 2 V ( G ) as β ( z i ) = { v 0 , v i } for i = 1 , 2 , . . . , n − 1 , then ( P , β ) is a verte x-cut path of G , of vc-width two. It is not difficu lt to ver ify t hat G has no verte x-cut tre e of vc-width one , and hence, κ vc-tree ( G ) = κ vc-path ( G ) = 2 . Our definiti on of v erte x-cut trees ma y ap pear at fi rst to be an artificial con struct broug ht in so lely for th e purpose of findi ng applic ations for the V ertex -Cut Bound. Howe ver , this is f ar from bei ng the case. V ertex -cut tre es a re v ery clos ely rel ated to junction trees , commonl y as sociated wit h beli ef propag ation algorithms in Bayesian networks [10] and coding theory [ 1]. Another close relati ve of ver tex-cu t trees is the data struct ure known as tr ee decompos ition (of a gra ph) [16],[2],[3], which has recei ved considerab le atten tion in the graph theory and computer science literatures . Definition 5.3. A tree decompos ition o f a conn ected graph G is a d ata struc tur e ( T , β ) , wher e T is a tr ee, and β : V ( T ) → 2 V ( G ) is a mapping with the following pr opertie s: (T1) β ( V ( T )) = V ( G ) ; (T2) for eac h pai r x, y ∈ V ( T ) , if z ∈ V ( T ) is any ver tex that lies on the unique path b etween x and y in T , then β ( x ) ∩ β ( y ) ⊆ β ( z ) ; (T3) for each pair of adjacent vertice s u, v ∈ V ( G ) , ther e exis ts a z ∈ V ( T ) suc h that { u, v } ⊆ β ( z ) . A tr ee decomposition ( T , β ) in which T is a path is called a path decomposit ion . The width of a tree deco mposition as abov e is defined to be max z ∈ V ( T ) | β ( z ) | − 1 . The tr eewid th (resp. pathwidth ) of a graph G , denoted by κ tree ( G ) (resp. κ path ( G ) ), is the minimum among the widths of all its tree (resp. path) decompo sitions. N ote that if G has at least one non-loop edge, then, because of (T3), any tree decomposit ion of G m ust ha ve w idth at least one. Thus, for an y C O N S T R A I N T C O M P L E X I T Y O F G R A P H I C A L R E A L I Z A T I O N S 1 3 C D A B E G H F C A B D E C E B C B E G G F B G E H A B C D E F G H () (b) (a) F I G U R E 3 . (a) A graph G on eight ver tices. (b) A tree decompo sition of G with width 2, and vc-width 3. (c) A verte x-cut p ath of G of vc-width 3 that is not a path decompo sition of G . such graph G , we ha ve 0 < κ tree ( G ) ≤ κ path ( G ) ≤ | V ( G ) | − 1 . It is not hard to check that any tree with at least two vertices has tree w idth equa l to one 6 , and that if G is an n -cycle , then κ tree ( G ) = κ path ( G ) = 2 . Lemma 5.2. Any tr ee decompositio n of a connecte d gr aph G is also a verte x-cut tr ee of G . Hence , κ vc-tree ( G ) ≤ κ tree ( G ) + 1 , and κ vc-path ( G ) ≤ κ path ( G ) + 1 . Pr oof. Let ( T , β ) be a tree decompos ition of G . W e only hav e to sho w th at ( T , β ) sati sfies c ondition (VC3) of Definition 5.1 . Consid er an y z ∈ V ( T ) , with degre e δ , an d let V i , i = 1 , . . . , δ , b e de fined as in c ondition (VC 3). By virtue of (T1), (T2) and Lemm a 5.1, ( β ( z ) , V 1 , . . . , V δ ) is a par tition of V ( G ) . Thus, we must sho w t hat N ( V i ) ⊆ V i ∪ β ( z ) for i = 1 , . . . , δ . Suppose , to the contrar y , that there exi sts a v ∈ V i such that for some u ∈ N ( v ) , we hav e u / ∈ V i ∪ β ( z ) . T hus, by definition of V i , we hav e u / ∈ β ( V ( T ( z ) i )) ∪ β ( z ) , while v ∈ β ( x ) \ β ( z ) for s ome x ∈ V ( T ( z ) i ) . Now , by con dition (T 3), we ha ve { u, v } ⊆ β ( y ) for some y ∈ V ( T ) . In particu lar , v ∈ β ( x ) ∩ β ( y ) . O n the other ha nd, b y our ass umption on u , we must hav e y 6 = z and y / ∈ V ( T ( z ) i ) . Thus, y ∈ V ( T ( z ) j ) for some j 6 = i . T his means that the v erte x z lies on the uni que path in T joining x and y , and hence , by (T2), v ∈ β ( z ) , which contrad icts the existen ce of v as postul ated.  A graph G can h av e a v ertex -cut tree that is not a t ree decompositi on. Figure 3 sho ws an exam- ple of a verte x-cut path of a graph G that is not a path decompositi on of G . Also, the inequalit y κ vc-tree ( G ) ≤ κ tree ( G ) + 1 in Lemma 5.2 can hold wit h equality — for th e graph G in Figur e 3(a), it is possi ble to show that κ tree ( G ) = 2 , while κ vc-tree ( G ) = κ vc-path ( G ) = 3 . 6. L O W E R B O U N D S O N κ - C O M P L E X I T Y V ertex -cut trees allo w us to deriv e lower bounds on the κ -complex ity of a graphical realiza tion of a code, as we no w show . Let ( G , ω ) be a graph deco mposition of a co de C , and let ( T , β ) be a 6 W ithout t he ‘ − 1 ’ i n the definition of width of a tree decomposition, a tree with at least two vertices would hav e treewidth equal to two. 14 N . K A S H Y A P E B C A D G F H z ∗ F I G U R E 4 . T he mapping α : V ( T ) → 2 V ( G ) , with the v ertex z ∗ chosen as sho wn, constr ucted f rom the vertex -cut tree in F igure 3(b). ver tex-cu t tree of G . For each z ∈ V ( T ) , set J i = ω − 1 ( V i ) f or i = 1 , 2 , . . . , δ , where th e V i ’ s are as defined in condi tion (VC3) of Definition 5.1. Further define m ( z ) = dim( C ) − δ X i =1 dim( C J i ) . (7) No w , con sider any g raphical realiza tion, Γ = ( G , ω , ( S e , e ∈ E ) , ( C v , v ∈ V )) , of C that e xtend s ( G , ω ) . By the V er tex-Cut Bound, we ha ve, for each z ∈ V ( T ) , P v ∈ β ( z ) dim( C v ) ≥ m ( z ) . Since P v ∈ β ( z ) dim( C v ) ≤  max v ∈ V ( G ) dim( C v )  | β ( z ) | = κ ( Γ) | β ( z ) | , we obta in κ (Γ) | β ( z ) | ≥ m ( z ) . Maximizing ov er all z ∈ V ( T ) , we get κ (Γ) · vc-width ( T , β ) ≥ m ax z ∈ V ( T ) m ( z ) def = µ ( C ; ω , β ) (8) W e thus ha ve the follo wing p ropositi on. Pro position 6.1. Let ( G , ω ) be a gr aph deco mposition of a co de C , and let ( T , β ) be a vert ex- cut tr ee of G . Then, for any graph ical rea lization Γ ∈ R ( C ; G , ω ) , we have κ (Γ) ≥ µ ( C ; ω , β ) vc-width ( T , β ) . The lower bound in the ab ov e pr opositio n can be brought in to a form more con venie nt for fur ther analys is. T o do this, we will constru ct a tree decomposit ion ( T , γ ) of C such that µ ( C ; ω , β ) = κ ( C ; T , γ ) . So, once again, let ( G , ω ) be a gi ven graph decomposition of C , and ( T , β ) a giv en ver tex-cu t tree of G . W e will fi rst giv e a recip e for constructin g tree decompositio ns of C from ( G , ω ) and ( T , β ) . Then, we will sho w ho w the main ingredient of the r ecipe may be cho sen so that the resu lting tree decompositi on ( T , γ ) satisfies µ ( C ; ω , β ) = κ ( C ; T , γ ) . For any pair of verti ces x, y ∈ V ( T ) , let ( x, y ] denote the set of verti ces on the unique path between x and y in T , includin g y , b ut not includi ng x . Also, we will use β ( x, y ] , instead of the more cumbersome β (( x, y ]) , to deno te the se t S z ∈ ( x,y ] β ( z ) . Pick a n arbit rary ve rtex z ∗ in V ( T ) . Define the mapping α : V ( T ) → 2 V ( G ) as follo ws: α ( z ∗ ) = β ( z ∗ ) , and for z 6 = z ∗ , α ( z ) = β ( z ) \ β ( z , z ∗ ] . An example of such a mapping α constr ucted from the verte x-cut tree ( T , β ) in Figure 3(b) is depict ed in F igure 4. W e rec ord in the ne xt two lemmas some properties of the mapping α that we use in the sequ el. Recall our con ven tion that for X ⊆ V ( T ) , α ( X ) = S x ∈ X α ( x ) . C O N S T R A I N T C O M P L E X I T Y O F G R A P H I C A L R E A L I Z A T I O N S 1 5 z ∗ T ( z ) z F I G U R E 5 . A depict ion of the sit uation when z ∗ / ∈ V ( T ( z ) ) . D ashed ov als rep re- sent component s of T − z . Lemma 6.2. F or z ∈ V ( T ) , if T ( z ) is any compon ent of T − z , the n α ( V ( T ( z ) )) = ( β ( V ( T ( z ) )) if z ∗ ∈ V ( T ( z ) ) β ( V ( T ( z ) )) \ β ( z ) if z ∗ / ∈ V ( T ( z ) ) . Pr oof. Suppose first that z ∗ ∈ V ( T ( z ) ) , and set X = V ( T ( z ) ) \ { z ∗ } . It then follo ws from the definitio n of α that [ x ∈ X α ( x ) = [ x ∈ X β ( x ) ! \ β ( z ∗ ) . Therefore , α ( V ( T ( z ) )) = α ( X ) ∪ α ( z ∗ ) = ( β ( X ) \ β ( z ∗ )) ∪ β ( z ∗ ) = β ( V ( T ( z ) )) . No w , consider the case when z ∗ / ∈ V ( T ( z ) ) , a s depicted in F igure 5. In th is ca se, the definition of α implies that [ x ∈ V ( T ( z ) ) α ( x ) =   [ x ∈ V ( T ( z ) ) β ( x )   \ β [ z , z ∗ ] , where [ z, z ∗ ] denotes the set of verti ces on th e path b etween z and z ∗ in T , including both z and z ∗ . Note th at, as a con sequence of cond ition (VC2) of Definition 5.1, we ha ve, for an y x ∈ V ( T ( z ) ) , β ( x ) ∩ β [ z , z ∗ ] = β ( x ) ∩ β ( z ) . Hence,   [ x ∈ V ( T ( z ) ) β ( x )   \ β [ z , z ∗ ] =   [ x ∈ V ( T ( z ) ) β ( x )   \ β ( z ) , which pro ves the desired result .  Lemma 6.3. The sets α ( z ) , z ∈ V ( T ) , form a partition of V ( G ) . Pr oof. Let T ( z ∗ ) i , i = 1 , 2 , . . . , δ , denote the components of T − z ∗ . Observe that, by Lemma 6.2, [ z 6 = z ∗ α ( z ) = δ [ i =1 α ( V ( T ( z ∗ ) i )) = δ [ i =1 β ( V ( T ( z ∗ ) i )) \ β ( z ∗ ) =   [ z 6 = z ∗ β ( z )   \ β ( z ∗ ) . Therefore , S z ∈ V ( T ) α ( z ) = S z ∈ V ( T ) β ( z ) = V ( G ) , by condition (VC1) of Definition 5.1. Next, we want to sho w that α ( z ) ∩ α ( z ′ ) = ∅ for z 6 = z ′ . This is true by definition of α if either z = z ∗ or z ′ = z ∗ . So, we henceforth assume that z and z ′ are dist inct vertices in V ( T ) \ { z ∗ } . Suppose that there exist s a v ∈ α ( z ) ∩ α ( z ′ ) . W e then hav e 16 N . K A S H Y A P (1) v ∈ β ( z ) , but v / ∈ β ( y ) for any y ∈ ( z , z ∗ ] ; and (2) v ∈ β ( z ′ ) , b ut v / ∈ β ( y ) for any y ∈ ( z ′ , z ∗ ] . In particular , we see that v / ∈ β ( z ∗ ) . It also follo w s from (1) and (2) that z ′ / ∈ ( z , z ∗ ] , and z / ∈ ( z ′ , z ∗ ] , which toge ther imply that z ∗ lies on th e path between z and z ′ . Howe ver , by (VC2), this means that v ∈ β ( z ∗ ) , a contrad iction.  W e now constru ct a tree decomposi tion ( T , γ ) of the inde x set, I , of C , by defining an inde x mapping γ : I → V ( T ) as follo ws: for each i ∈ I , set γ ( i ) = z if ω ( i ) ∈ α ( z ) . In other words , for each z ∈ V ( T ) , γ − 1 ( z ) = ω − 1 ( α ( z )) . Since th e sets α ( z ) , z ∈ V ( T ) , form a partition of V ( G ) , the mappi ng γ is well-defined. Example 6.1. Sup pose that C is a co de of le ngth 10, defined on the inde x set I = { 1 , 2 , . . . , 10 } . F or the gra ph G shown in F igur e 3 (a), consider the graph decompo sition ( G , ω ) of C defined by ω (1) = ω (2) = A , ω (3) = ω (4) = ω (5) = B , ω (6) = ω (7) = E , ω (8) = F , and ω (9 ) = ω (10) = H . L et ( T , β ) be the verte x-cut tr ee of G in F igur e 3(b), fr om which we obtain the mapping α : V ( T ) → 2 V ( G ) depict ed in F igur e 4. Based on the last figur e, we w ill use the labels z ∗ , A , D , G , F and H to iden tify the verti ces of th e tr ee T . Then, the index map ping γ : I → V ( T ) is given by γ (1) = γ (2) = A , γ (3) = γ (4) = γ (5) = γ (6) = γ (7) = z ∗ , γ (8) = F , and γ (9) = γ (10) = H . Note that ou r co nstructio n of the tre e de compositio n ( T , γ ) depends o n th e ch oice of the v ertex z ∗ , via the mappin g α . U p t o this poin t, our choice of z ∗ was arbit rary . W e no w specify z ∗ ∈ V ( T ) to be such that m ( z ∗ ) = µ ( C ; ω , β ) . W e cl aim th at for the tree decomposition ( T , γ ) ari sing from such a choic e of z ∗ , we ha ve µ ( C ; ω , β ) = κ ( C ; T , γ ) . Pro position 6.4. G iven a gr aph de compositio n ( G , ω ) of a cod e C , and a verte x-cut tr ee ( T , β ) of G , ther e exis ts a tr ee decompositi on ( T , γ ) of C such that µ ( C ; ω , β ) = κ ( C ; T , γ ) . Pr oof. Pick a z ∗ ∈ V ( T ) such that m ( z ∗ ) = µ ( C ; ω , β ) , and const ruct the tree decompos ition ( T , γ ) as described above . Note that, by (6), κ ( C ; T , γ ) = κ ( M ( C ; T , γ )) = m ax z ∈ V ( T ) k ( z ) , where, for a verte x z ∈ V ( T ) of degree δ , k ( z ) def = dim( C ) − P δ i =1 dim( C K i ) , with K i = γ − 1  V ( T ( z ) i )  for i = 1 , 2 , . . . , δ . Recall from the definition of γ that γ − 1 ( z ) = ω − 1 ( α ( z )) for an y z ∈ V ( T ) . Hence, K i = ω − 1  α ( V ( T ( z ) i ))  , i = 1 , 2 , . . . , δ . T o prov e the propositio n, we show that k ( z ) ≤ m ( z ) for all z ∈ V ( T ) , with equality holding if z = z ∗ . F rom (7), we see that m ( z ) = dim( C ) − P δ i =1 dim( C J i ) , where, for i = 1 , 2 , . . . , δ . J i = ω − 1  β ( V ( T ( z ) i )) \ β ( z )  . W e must therefore sho w that, for each z ∈ V ( T ) , we hav e J i ⊆ K i for all i , which would p rov e that k ( z ) ≤ m ( z ) ; and furthermore, when z = z ∗ , we hav e J i = K i for all i , which would pro ve that k ( z ∗ ) = m ( z ∗ ) . So, co nsider any z ∈ V ( T ) . From Lemma 6.2, we see that i f T ( z ) i is any comp onent of T − z , then β ( V ( T ( z ) i )) \ β ( z ) ⊆ α ( V ( T ( z ) i )) (9) Hence, J i ⊆ K i . Moreo ver , when z = z ∗ , w e alway s ha ve z ∗ / ∈ V ( T ( z ) i ) , and so, again by Lemma 6.2, equa lity holds in (9), implying that J i = K i .  From Propositi ons 6.1 and 6.4, we obtain the follo wing theorem. Theor em 6.5. Let ( G , ω ) b e a grap h decompositio n of a co de C , and let ( T , β ) be a vert ex- cut tree of G . Then, ther e e xists a tr ee decompo sition ( T , γ ) of C such that, for any gra phical r ealization C O N S T R A I N T C O M P L E X I T Y O F G R A P H I C A L R E A L I Z AT I O NS 1 7 Γ ∈ R ( C ; G , ω ) , we have κ (Γ) ≥ κ ( C ; T , γ ) vc-width ( T , β ) . Hence , κ ( C ; G , ω ) ≥ κ ( C ; T , γ ) vc-width ( T ,β ) . The theore m a bov e is a fundamental result w ith se veral import ant consequenc es, some of which we pres ent here. The first o f these is a corollary that giv es a lo wer bound on the κ -c omplex ity of any realiza tion of a giv en code C on a graph G . Cor ollary 6.6. F or a code C and a connec ted graph G , we have κ ( C ; G ) ≥ κ tree ( C ) κ vc-tree ( G ) ≥ κ tree ( C ) κ tree ( G ) + 1 . Pr oof. Consider an arbi trary inde x mapping ω : I → V ( G ) , where I is the index set of C . Let ( T , β ) be an optimal verte x-cut tree of G , by which we m ean that vc-width ( T , β ) = κ vc-tree ( G ) . By Theorem 6.5, there ex ists a tree decompositio n ( T , γ ) of C such that κ ( C ; G , ω ) ≥ κ ( C ; T , γ ) vc-width ( T , β ) ≥ κ tree ( C ) κ vc-tree ( G ) . Minimizing ov er all possib le map pings ω : I → V ( G ) , we obtain κ ( C ; G ) ≥ κ tree ( C ) κ vc-tree ( G ) . From Lemma 5.2, we also ha ve κ vc-tree ( G ) ≤ κ tree ( G ) + 1 .  If, in the ab ove pro of, we take ( T , β ) to be an opt imal verte x-cut path instead, i.e. , tak e ( T , β ) to be a v ertex- cut p ath such that vc-width ( T , β ) = κ vc-path ( G ) , the n w e obt ain the next corollary . Cor ollary 6.7. F or a code C and a connec ted graph G , we have κ ( C ; G ) ≥ κ path ( C ) κ vc-path ( G ) ≥ κ path ( C ) κ path ( G ) + 1 . The (first) inequality abov e can be tight, in the sens e that there are examples of codes C and graphs G for which κ ( C ; G ) = ⌈ κ path ( C ) κ vc-path ( G ) ⌉ . Example 6.2. T ake C to be the [24 , 12 , 8] binary Golay code, and let G be the family of graphs consis ting of al l n -cycles. F r om Example 5.1, we know tha t κ vc-path ( G ) = 2 for any G ∈ G . It is also known that κ path ( C ) = 9 [7, Example 1 ] , [17, Section 5] . Hence , by the bound of Cor ollary 6.7, noting tha t κ ( C ; G ) must be a n inte ger , we ha ve κ ( C ; G ) ≥ 5 for an y G ∈ G . Thus, κ ( C ; G ) ≥ 5 . The tailbitin g tr ellis re alization of the G olay code given in [5] has κ -comple xity equal to 5, fr om which we co nclude that κ ( C ; G ) = 5 . Using Corollary 6 .7 as a starting point , we deriv e a lower bound on the treewidt h of an [ n, k , d ] linear code. For this, we will also need th e fol lowin g result [4, Theorem 7.1]: if G is a graph with tree w idth at most ℓ , then the pathwidth of G is at most ( ℓ + 1) log 2 | V ( G ) | . Pro position 6.8. F or an [ n, k , d ] linear code C , with n > 1 , κ tree ( C ) ≥ κ path ( C ) 3 + 2 log 2 ( n − 1) ≥ k ( d − 1) n (3 + 2 log 2 ( n − 1)) . 18 N . K A S H Y A P Pr oof. The second inequality abov e is due to the fact, shown in [15 ], that 7 for an [ n, k , d ] linear code C , κ path ( C ) ≥ k ( d − 1) /n . The first inequ ality is prov ed as follo ws. L et ( T , ω ) be an optimal tree decomposit ion of C , so that κ ( C ; T , ω ) = κ tree ( C ) . As mention ed in Section 3, ( T , ω ) may be ch osen so tha t T is a cu bic tree, a nd ω is a bijection bet ween the ind ex s et of C and the leav es of T . Thus, T is a cub ic tree with n lea ves, from which it f ollo ws that T has n − 2 intern al nodes. Hence, | V ( T ) | = 2 n − 2 . Since T has tree w idth equal to one, by the result from [4] quote d ea rlier , κ path ( T ) ≤ 2 log 2 (2 n − 2) . W e no w hav e, via Corollary 6.7, κ ( C ; T , ω ) ≥ κ ( C ; T ) ≥ κ path ( C ) κ path ( T ) + 1 ≥ κ path ( C ) 2 log 2 (2 n − 2) + 1 , which pro ves the propos ition.  In particul ar , Proposit ion 6.8 shows that for any line ar code C of length n , κ path ( C ) κ tree ( C ) = O (lo g 2 n ) . This estimate of the ratio κ path ( C ) κ tree ( C ) is the best possible 8 , up to the constan t impli cit in the O -no tation. Indeed , it was sho w n in [11 ] that a sequence of codes C ( i ) ( i = 1 , 2 , 3 , . . . ), with length n i = 12(2 i − 1) + 2 , κ tree ( C ( i ) ) = 2 and κ path ( C ( i ) ) ≥ 1 2 ( i + 3) , can be constructed o ver any finite field F . It is clear that κ path ( C ( i ) ) κ tree ( C ( i ) ) gro w s logarit hmically w ith code length n i . A less precise formulatio n of the second inequality in Proposition 6.8 is also instructi ve: there exi sts a constant c 0 > 0 such that, for any [ n, k , d ] linear code C , with n > 1 , κ tree ( C ) ≥ c 0 k d n log 2 n . (10) A note worthy implication of the abov e inequality is that code families of bounded treewidt h are not v ery good from a n error -correctin g perspecti ve. Gi ven a n inte ger t > 0 , deno te by TW ( t ) t he family of all codes o ver F o f tre ewid th at most t . A co de f amily C is called as ymptotical ly good if there e xists a sequence of [ n i , k i , d i ] codes C ( i ) ∈ C , with lim i n i = ∞ , such t hat lim inf i k i /n i and lim in f i d i /n i are b oth strictl y positi ve. The fol lo wing result, an eas y consequ ence of (10), res olves a conjec ture in [11]. Cor ollary 6.9. Let C ( i ) , i = 1 , 2 , 3 , be any seque nce of [ n i , k i , d i ] codes such that lim i →∞ κ tree ( C ( i ) ) log n i n i = 0 . Then, either lim i →∞ k i /n i = 0 or lim i →∞ d i /n i = 0 . In partic ular , fo r any t > 0 , the code family TW ( t ) is not asymptotic ally go od. In fa ct, a more gener al result is true. For a fi xed in teger ℓ > 0 , let G ℓ denote the f amily of all graphs w ith vc-tree width at m ost ℓ . In par ticular , note tha t, by Lemma 5.2, G ℓ contai ns all gr aphs with treewidt h at most ℓ − 1 . The n, for an [ n, k , d ] linear code C with n > 1 , we hav e, via (3), Corollary 6.6 and Propos ition 6.8, κ ( C ; G ℓ ) ≥ κ tree ( C ) ℓ ≥ k ( d − 1) ℓ n (3 + 2 log 2 ( n − 1)) . 7 The resu lt in [15] is only ex plicitly stated as a lower bound on t he st ate ma x-complexity of any con ventiona l trellis realization of C . Howe ver , the state ma x-complexity of a graphical realization can nev er exceed the constraint max- complex ity of the realization, as noted in Section 3. 8 It was conjectured in [11] that for codes C of length n , κ path ( C ) − κ tree ( C ) = O (log n ) , but we now do not belie ve this to be true. C O N S T R A I N T C O M P L E X I T Y O F G R A P H I C A L R E A L I Z AT I O NS 1 9 W e thus ha ve the follo wing c orollary to Propositio n 6.8, whic h extends Corollary 6.9. Cor ollary 6.10. Given an inte ger t > 0 , if C is a family of codes over F w ith the pr operty that κ ( C ; G ℓ ) ≤ t for all C ∈ C , th en C is not asympto tically good. A ro ugh interpret ation o f the las t result is that a “good” error- correctin g code c annot ha ve a lo w- comple xity realiz ation on a graph with small (vc-)tree width. In other words, codes that are good from an erro r-c orrecting standpoint can h av e lo w-complexit y realization s only on grap hs with lar ge (vc-)tr eewid th. 7. C O N C L U D I N G R E M A R K S In this paper , we demonstrated at length the use o f the V ertex-Cut Bound in fi nding lower bounds on th e κ -comple xity of graph ical realization s. As suggest ed at the end o f Section 4, a natu ral appli- cation of the V ertex-Cut Bound is to formulate constrain ed opt imization problems w hose solut ions are lo wer bounds to variou s meas ures of constraint complexity . T his method could be used, for in- stance , to determine lo w er bounds on the t rue computational compl exity of sum-p roduct de coding for a gi ven code. T his can be done by choosing a const raint complexity m easure that accurate ly reflects the cost of sum-produ ct decodin g. Such a complex ity measure can be chosen based on detaile d coun ts o f the n umber of a rithmetic operations re quired in th e sum-product al gorithm; see , for e xample, [1],[6],[7]. It is now probabl y fair to say that the main open proble m in the area of graphical realizati ons of codes is to expl icitly construct, for a giv en code C and an arbitra ry graph G , realizati ons of C on G whose con straint complex ity is within st riking distan ce of the lo wer bounds fou nd using the methods of this pap er . While we ha ve s hown that our b ounds can be tight for s pecific examples of codes C and graph s G , their tightness in the generic instance remains to be in vestigat ed. A PP E N D I X A. A N E X A M P L E W e pro vide here an ex ample of a cod e C and a graph deco mposition ( G , ω ) of C , for whic h there is no realiza tion Γ ∈ R ( C ; G , ω ) such that κ (Γ) = κ ( C ; G , ω ) and κ + (Γ) = κ + ( C ; G , ω ) . The exa mple we giv e is based on Example 3.1 in [14]. Consider the [11 , 3 , 3] binar y linear code C generated by the code words 00011100 000 , 00000 111000 and 001 010101 00 . W e tak e I = { 1 , 2 , 3 , . . . , 11 } to be the i ndex set of C , identifyi ng the inde x i with the i th coord inate of C . N o w , let G be the 11-cycle, and ω : I → V ( G ) the index mapping depict ed in F igure 1. Figure 6 sho ws a tailbiti ng trellis realizat ion, Γ 1 , of C that exte nds ( G , ω ) . Note that κ (Γ 1 ) = 2 , while κ + (Γ 1 ) = 14 . A second t ailbiting trellis re alizatio n, Γ 2 , in R ( C ; G , ω ) is sho wn in Fig ure 7, w ith κ (Γ 2 ) = 3 and κ + (Γ 2 ) = 13 . It can be shown (using ar guments similar to those giv en in E xample 3.1 in [1 4]) that κ (Γ 1 ) = κ ( C ; G , ω ) , while κ + (Γ 2 ) = κ + ( C ; G , ω ) , and that there is no realization Γ ∈ R ( C ; G , ω ) that simultaneousl y achie ves κ ( C ; G , ω ) and κ + ( C ; G , ω ) . R EF E R E N C E S [1] S. M. Aji an d R. J. McEliece, “The generalized distri buti ve law , ” IEEE T rans. Inform. Theo ry , vol. 46, no. 2, pp. 325–34 3, 2000. [2] S. Arnbor g, D.G. Co rneil and A. Proskuro wski, “Complex ity of findin g embe ddings in a k -tr ee, ” SIAM J . Alg. Disc. Meth. , vol. 8, pp. 277 –284, 1987. [3] H.L . Bodlaender , “ A tourist guide through treewidth, ” Act a Cybernetica , v ol. 11, pp. 1–23, 1993. [4] H.L . Bodlaender , T . Kloks, “Ef ficient an d constructiv e algo rithms fo r the pa thwidth and treewidth of graphs, ” J . Algorithms , vol. 21, no . 2, pp. 358–402, 1996. [5] A.R. Calderbank, G.D. Forney Jr ., and A. V ardy , “Minimal tail-biti ng trel lises: The Golay code and more, ” IEEE T rans. Inform. Theory , vol. 45, pp. 14 35–1455, July 1999. [6] G.D. Forne y Jr ., “Codes on graphs: normal r ealizations, ” IEEE Tr ans. Inform. Theory , vol. 47, no. 2, pp. 520–548, Feb . 2001. [7] G.D. Forney Jr ., “Codes on graphs: constraint complex ity of cycle-free realizations of li near codes, ” IEEE Tr ans. Inform. Theory , vol. 49, no. 7, pp . 1597–1 610, July 2003. 20 N . K A S H Y A P 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 [3,1] [3,1] [3,1] [3,1] [3,1] [3,1] [3,1] [3,2] [3,2] [3,2] [3,1] (a) (b) F I G U R E 6. A tailbiting trellis realization for an [11 , 3 , 3] binary linear code. (a) Tr ellis diagram. (b) Depiction of trellis realizati on sho w ing the length and dimensio n of t he local constraint codes; all state spaces hav e dimension 1. (a) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 (b) [3,1] [3,1] [3,0] [3,0] [4,2] [5,2] [5,3] [5,2] [4,2] [3,0] [3,0] 0 0 1 2 2 2 2 1 0 0 0 F I G U R E 7. A tailbiting trellis realization for an [11 , 3 , 3] binary linear code. (a) Tr ellis diagram. (b) Depiction of trellis realizati on sho w ing the length and dimensio n of t he local constraint codes, and the dimensions of the state spaces. [8] T .R. Halford and K.M. Chugg, “The extraction and complexity limits of graphical models for linear codes, ” IE EE T rans. Inform. Theory , to appear . [9] P . Hlin ˇ en ´ y an d G. Whittl e, “Matroid tree-width, ” E ur op. J. Combin. , vo l. 27, pp. 1117–1128, 2006. [10] F .V . Jensen, An Introd uction to Bayesian Networks , Springer , New Y ork, 1996. [11] N. Kashya p, “On minimal tree realizations of linear codes, ” su bmitted to IEE E T rans. Inform. Theory . ArXi v e-print 0711.138 3 [12] F .R. Kschischang, B.J. Frey and H.-A. Loeliger , “Factor graphs and the sum-product algorithm, ” IEEE T rans. Inform. Theory , vol. 47, no. 2, pp . 498–51 9, F eb . 2001. [13] R. K oetter and A. V ardy , “On the theory of li near trellises, ” in Information, Coding and Mathematics , M. Blaum, P .G. Farrell an d H.C. A. v an T ilborg, eds., Kluwer , Boston, Mass., May 2002, pp. 323–354. [14] R. K oetter and A. V ardy , “The structure of tail-biting t rellises: minimalit y and basic principles, ” IEEE Tr ans. Inform. Theory , vol. 49, no. 9, pp. 2081 –2105, Sept. 2003. [15] A. Lafourcad e and A. V ardy , “ Asymptotically goo d cod es have infinite trellis c omplexity , ” IEEE. Tr ans. Inform. Theory , vol. 41, no. 2, pp. 555– 559, March 199 5. C O N S T R A I N T C O M P L E X I T Y O F G R A P H I C A L R E A L I Z AT I O NS 2 1 [16] N. Robertson and P .D. Se ymour , “Graph minors. I. Excluding a forest, ” J. Combin. Theory , Ser . B , vol. 35, pp. 39–61, 1983. [17] A. V ardy , “T rellis Structure of Codes, ” in Handbook of C oding Theory , R. Br ualdi, C. Huffman and V . Pl ess, Eds., Amsterdam, The Netherlands: Elsev ier , 1998. [18] N. Wiber g, C odes and Decoding on General Graphs , Ph.D. t hesis, Link ¨ oping Univ ersity , Link ¨ oping, Sweden, 1996. [19] N. W iberg. H. -A. Loe liger and R. K oetter, “Codes an d iterativ e decoding on general grap hs, ” Eur o. T rans. T elecom- mun. , vol. 6, pp. 513 –525, S ept./Oct. 1995.