The Serializability of Network Codes

The Serializabilit y of Net w ork Co des Anna Blasiak ∗ Rob ert Klein b erg † Abstract Net work coding theory studies the transmission of information in netw orks whose vertices ma y p erform non trivial enco ding and deco ding op erations on data as it passes through the net work. The main approach to deciding the feasibility of netw ork co ding problems aims to reduce the problem to optimization o ver a p olytop e of “entropic vectors” sub ject to constrain ts imp osed by the net w ork structure. In the case of directed acyclic graphs, these constrain ts are completely understo od, but for general graphs the problem of enumerating them remains op en: it is not kno wn how to classify the constraints implied b y a prop ert y that we call serializability , whic h refers to the absence of parado xical circular dep endencies in a netw ork code. In this work w e initiate the ﬁrst systematic study of the constraints imposed on a net work co de b y serializability . W e ﬁnd that serializabilit y cannot be detected solely b y ev aluating the Shannon en tropy of edge sets in the graph, but nevertheless, w e give a polynomial-time algorithm that decides the serializabilit y of a net work code. W e deﬁne a certiﬁcate of non-serializability , called an information vortex , that pla ys a role in the theory of serializability comparable to the role of fractional cuts in multicommodity ﬂow theory , including a type of min-max relation. Finally , we study the serializability deﬁcit of a net work code, deﬁned as the minimum num b er of extra bits that m ust b e sen t in order to make it serializable. F or linear co des, we show that it is NP-hard to appro ximate this parameter within a constan t factor, and w e demonstrate some surprising facts ab out the behavior of this parameter under parallel comp osition of codes. 1 Departmen t of Computer Science, Cornell Universit y , Ithaca NY 14853. E-mail: ablasiak@cs.cornell.edu . Supp orted an b y an NDSEG Graduate F ellowship, an A T&T Labs Graduate F ellowship, and an NSF Graduate F ellowship. 2 Departmen t of Computer Science, Cornell Universit y , Ithaca NY 14853. E-mail: rdk@cs.cornell.edu . Supp orted by NSF gran t CCF-0729102, a Microsoft Research New F aculty F ellowship, and an Alfred P . Sloan F oundation F ello wship. 1 In tro duction Net work coding theory studies the transmission of information in net works whose v ertices ma y p erform non trivial enco ding and deco ding operations on data as it passes through the net work. More sp eciﬁcally , a net w ork co de consists of a net work with sp eciﬁed sender and receiver edges and co ding functions on each edge. The classic deﬁnition of a netw ork co de requires that each vertex can compute the message on every outgoing edge from the messages receiv ed on its incoming edges, and that each receiv er is sent the message it requires. In directed acyclic graphs, a netw ork co de that satisﬁes these requiremen ts sp eciﬁes a v alid communication proto col. How ev er, in graphs with cycles this need not be the case; the deﬁnition do es not preclude the p ossibilit y of cyclic dep endencies among co ding functions. Therefore, in graphs with cycles we also require that a net work co de is serializable , meaning it correctly summarizes a communication proto col in which sym b ols are transmitted on edges ov er time, and eac h sym b ol transmitted b y a vertex is computed without kno wledge of information it will receiv e in the future. The presen t pap er is devoted to the study of characterizing the constraint of serializability . Motiv ation. The cen tral question in the area of netw ork coding is to determine the amount by whic h co ding can increase the rate of information ﬂo w as compared to transferring information without co ding. Crucial to answering this question is developing to ols to ﬁnd upper bounds for the net work co ding rate. The question of serializability must b e considered in order to determine tigh t upp er b ounds on net work co des in cyclic graphs. Determining tight upp er b ounds is esp ecially relev ant to one of the most imp ortan t op en problems in net work co ding, the undir e cte d k -p airs c onje ctur e , whic h states that in undirected graphs with k sender-receiver pairs, co ding cannot increase the maximum rate of information ﬂow; that is, the netw ork co ding rate is the same as the multicommodity ﬂo w rate. Apart from its in trinsic interest, the conjecture also has imp ortan t complexit y-theoretic implications: for example, if true, it implies an aﬃrmative answer to a 20- y ear-old conjecture regarding the I/O complexit y of matrix transp osition [1]. Almost all eﬀorts to pro duce upp er b ounds on the net work co ding rate ha ve fo cused on the follo wing construction. W e regard eac h edge of the net w ork as deﬁning a random v ariable on a probabilit y space and then asso ciate eac h set of edges with the Shannon en tropy of the join t distribution of their random v ariables. This giv es us a vector of non-negativ e num b ers, one for each edge set, called the entr opic ve ctor of the net w ork co de. The closure of the set of en tropic v ectors of net work co des forms a con v ex set, and net w ork co ding problems can b e expressed as optimization problems ov er this set [14]. In muc h previous work, tight upp er b ounds hav e been constructed b y com bining the constraints that deﬁne this con vex set. How ever, this technique is limited b ecause there is no kno wn description of all these constrain ts. There are t wo t yp es of constraints: the purely information-theoretic ones (i.e., those that hold univ ersally for all n -tuples of random v ariables, regardless of their in terpretation as co ding functions on edges of a net work) and the constrain ts derived from the com binatorial structure of the net work. The former type of constrain ts include the so-called Shannon and non-Shannon inequalities, and are currently a topic of intense in vestigation [3, 4, 13, 12, 17]. The latter type of constraints — namely , those determined b y the netw ork structure — are trivial to characterize in the case of directed acyclic graphs: if one imp oses a constraint that the en tropy of eac h no de’s incoming edge set equals the entrop y of all of its incoming and outgoing edges, then these constraints together with the purely information-theoretic ones imply all other constraints resulting from the net work structure [16]. Ho wev er, in graphs with cycles there are additional constraints determined from the net work structure. In a series of w ork on ﬁnding net work co ding upper bounds, large classes of information inequal- ities in graphs with cycles w ere discov ered indep endently b y Jain et al. [8], Kramer and Sa v ari [10], 1 and Harvey et al. [6]. These go b y the names crypto ine quality , PdE b ound , and informational dominanc e b ound , resp ectiv ely . In v arious forms, all of them describ e a situation in whic h the information on one set of edges completely determines the information on another set of edges. A more general necessary condition for serializability w as presented in a recen t pap er by Harvey et al. [7]; we will henceforth refer to this information inequality as the Chicken and Egg inequality; see Theorem 3.1. Though in all the previous w ork the inequalities used were suﬃcient to prov e the needed b ounds on the sp eciﬁc graphs analyzed in the pap er, no one has ask ed if this set of inequalities provides a complete characterization of serializabilit y . This inspires the following nat- ural questions: Are the information inequalities given in previous work suﬃcien t to characterize serializabilit y? Do es there exist a set of information theoretic inequalities that gives a suﬃcient condition for serializability? Is there a ﬁnite set of information theoretic inequalities implied by serializabilit y? Is there an y “nice” condition that is necessary and suﬃcient serializability? W e see these questions not only as interesting for a general understanding of netw ork co ding, but also a k ey ingredient to even tually dev eloping algorithms and upp er b ounds for netw ork co ding in general graphs. Our contributions. Our w ork is the ﬁrst systematic study of criteria for serializability of net- w ork co des. W e ﬁnd that serializability cannot b e detected solely from the entropic v ector of the net work code; a counter-example is giv en in Section 3. This leads us to fo cus the pap er on t w o in- dep enden t, but dual, questions: Is there any eﬃciently veriﬁable necessary and suﬃcien t condition for serializability? What is the complete set of entrop y inequalities implied by serializabilit y? W e answer the ﬁrst question in the aﬃrmative in Section 4 by providing an algorithm to decide whether a co de is serializable. The running time of this algorithm is polynomial in the cardinalities of the edge alphabets, and it is p olynomial in their dimensions in the case of linear netw ork co des. W e answer the second question for the 2-cycle in Section 3, giving four inequalities derived from the net work structure, and sho wing that any entropic vector satisfying those inequalities as w ell as Shannon’s inequalities can b e realized by a serializable net work code. (Though structurally simple, the 2-cycle graph has been an imp ortant source of inspiration for information inequalities in prior w ork, including the crypto inequality [8], the informational dominance bound [6], and the Chick en and Egg inequality [7].) Disapp ointingly , we do not know if this result extends b eyon d the 2-cycle. Bey ond providing an algorithm for deciding if a netw ork co de is serializable, our work provides imp ortan t insigh ts into the prop ert y of serializabilit y . In Section 4 we deﬁne a certiﬁcate that we call an information vortex that is a necessary and suﬃcien t condition for non-serializability . F or linear net work co des, an information v ortex consists of linear subspaces of the dual of the message space. F or general net work codes, it consists of Boolean subalgebras of the pow er set of the message set. W e prov e a n umber of theorems ab out information v ortices that suggest their role in the theory of net work co ding ma y b e similar to the role of fractional cuts in net work ﬂo w theory . In particular, w e pro ve a t yp e of min-max relation betw een serializable co des and information vortices: under a suitable deﬁnition of serializable r estriction it holds that every net work co de has a unique maximal serializable restriction, a unique minimal information v ortex, and these t wo ob jects coincide. Finally , motiv ated b y examples in which non-serializable co des, whose co ding functions ha ve dimension n , can b e serialized b y adding a single bit, we consider the idea of a netw ork co de being “close” to serializable. W e formalize this by studying a parameter w e call the serializability deﬁcit of a net work co de, deﬁned as the minimum num b er of extra bits that must b e sent in order to mak e it serializable. F or linear co des, we sho w that it is NP-hard to appro ximate this parameter within a constant factor. W e also demonstrate, p erhaps surprisingly , that the serialization deﬁcit ma y b eha ve subadditiv ely under parallel comp osition: when executing t wo indep enden t copies of a net work co de, the serialization deﬁcit may scale up b y a factor less than tw o. In fact, for ev ery 2 δ > 0 there is a netw ork co de Φ and a p ositiv e integer n such that the serialization deﬁcit of Φ gro ws by a factor less than δ n when executing n indep endent copies of Φ. Despite these examples, w e are able to pro ve that for an y non-serializable linear code Φ there is a constant c Φ suc h that the serialization deﬁcit of n independent executions of Φ is at least c Φ n . The concept of an information v ortex is crucial to our results on serializability deﬁcit. Related w ork. F or a general in tro duction to netw ork coding w e refer the reader to [11, 15]. There is a standard deﬁnition of netw ork co des in directed acyclic graphs (Deﬁnition 2.1 b elo w) but in man y pap ers on graphs with cycles the deﬁnition is either not explicit (e.g. [2]) or is restricted to sp ecial classes of co des (e.g. [5]). Precise and general deﬁnitions of netw ork codes in graphs with cycles app ear in [9, 1, 6] and the equiv alence of these deﬁnitions (mo dulo some diﬀering assumptions about no des’ memory) is prov en in [11]. The deﬁnition for serializabilit y that w e set forth in Section 2 was used, but nev er formally deﬁned, in [7]. In its essence it is the same as the “graph ov er time” deﬁnition given in [11] but requires less cumbersome notation. 2 Deﬁnitions W e deﬁne a netw ork co de to op erate on a directed m ultigraph we call a sour c e d gr aph , denoted G = ( V , E , S ) . 1 S is a set of sp ecial edges, called sour c es or sour c e e dges , that hav e a head but no tail. W e denote a source with head s b y an ordered pair ( • , s ). Elemen ts of E ∪ S are called e dges and elements of E are called or dinary e dges . F or a vertex v , w e let In ( v ) = { ( u, v ) ∈ E } b e the set of edges whose head is v . F or an edge e = ( u, v ) ∈ E , w e also use In ( e ) = In ( u ) to denote the set of inc oming e dges to e . A netw ork co de in a sourced graph speciﬁes a protocol for communicating sym b ols on error-free c hannels corresponding to the graph’s ordinary edges, giv en the tuple of messages that originate at the source edges. Deﬁnition 2.1 A network c o de is sp eciﬁed b y a 4-tuple Φ = ( G, M , { Σ e } e ∈ E ∪ S , { f e } e ∈ E ∪ S ) where G = ( V , E , S ) is a sourced graph, M is a set whose elemen ts are called message-tuples , and for all edges e ∈ E ∪ S, Σ e is a set called the alphab et of e and f e : M → Σ e is a function called the c o ding function of e . If e is an edge and e 1 , . . . , e k are the elemen ts of In ( e ) then the v alue of the co ding function f e m ust b e completely determined b y the v alues of f e 1 , . . . , f e k . In other words, there m ust exist a function g e : Q k i =1 Σ e i → Σ e suc h that for all m ∈ M , f e ( m ) = g e ( f e 1 ( m ) , . . . , f e k ( m )) . In graphs with cycles a co de can hav e cyclic dep endencies so Deﬁnition 2.1 do es not suﬃce to characterize the notion of a v alid netw ork co de. W e must imp ose a further constraint that we call serializability , which requires that the netw ork code summarizes a complete execution of a comm unication proto col in whic h every bit transmitted by a vertex dep ends only on bits that it has already received. Belo w w e deﬁne serializabilit y formally using a deﬁnition implicit in [7]. Deﬁnition 2.2 A net work co de Φ is serializable if for all e ∈ E there exists a set of alphab ets Σ (1 ...k ) e = n Σ (1) e , Σ (2) e , . . . , Σ ( k ) e o and a set of functions f (1 ...k ) e = n f (1) e , f (2) e , . . . , f ( k ) e o suc h that 1. f ( i ) e : M → Σ ( i ) e , 2. ∀ m 1 , m 2 ∈ M , if f e ( m 1 ) = f e ( m 2 ), then ∀ i, f ( i ) e ( m 1 ) = f ( i ) e ( m 2 ), 1 In prior work it is customary for the underlying netw ork to also ha ve a sp ecial set of receiving edges. Sp ecifying a sp ecial set of receivers is irrelev ant in our work, so w e omit them for conv enience, but ev erything w e do can be easily extended to include receiv ers. 3 3. ∀ m 1 , m 2 ∈ M , if f e ( m 1 ) 6 = f e ( m 2 ), then ∃ i, f ( i ) e ( m 1 ) 6 = f ( i ) e ( m 2 ), and 4. ∀ m ∈ M , e ∈ E , j ∈ { 1 . . . k } there is some function h ( j ) e suc h that f ( j ) e ( m ) = h ( j ) e   Y ˆ e ∈ In ( e ) f (1 ..j − 1) ˆ e   . 2 W e call suc h a Σ (1 ...k ) e , f (1 ...k ) e a serialization of Φ. The function f ( i ) e describ es the information sen t on edge e at time step i . Item 2 requires that together the functions f (1 ..k ) e send no more information than f e and Item 3 requires that f (1 ..k ) e sends at least as muc h information as f e . Item 4 requires that we can compute f ( j ) e giv en the information sent on all of e ’s incoming edges at previous time steps. In working with net work codes, we will o ccasionally w ant to compare tw o netw ork co des Φ , Φ 0 suc h that Φ 0 “transmits all the information that is transmitted by Φ.” In this case, w e sa y that Φ 0 is an extension of Φ, and Φ is a restriction of Φ 0 . Deﬁnition 2.3 Supp ose that Φ = ( G, M , { Σ e } , { f e } ) and Φ 0 = ( G, M , { Σ 0 e } , { f 0 e } ) are tw o net- w ork co des with the same sourced graph G and the same message set M . W e say that Φ is a restriction of Φ 0 , and Φ 0 is an extension of Φ, if it is the case that for every m ∈ M and e ∈ E , the v alue of f 0 e ( m ) completely determines the v alue of f e ( m ); in other w ords, f e = g e ◦ f 0 e for some function g e : Σ 0 e → Σ e . The en tropic v ector of a net work co de giv es a non-negativ e v alue for eac h subset of a netw ork co de. The v alue of an edge set F is the Shannon en tropy of the joint distribution of the random v ariables asso ciated with eac h element of F , as is formalized in the following deﬁnition. Deﬁnition 2.4 Giv en a netw ork code Φ = ( G , M , { Σ e } , { f e } ) , G = ( V , E , S ), the entr opic ve ctor of Φ has co ordinates H ( F ) deﬁned for eac h edge set F = { e 1 , . . . , e j } ⊆ E ∪ S by: H ( F ) = H ( e 1 e 2 . . . e j ) = X x 1 ∈ Σ e 1 ,x 2 ∈ Σ e 2 ,...,x j ∈ Σ e j − p ( x 1 , x 2 , . . . , x j ) log( p ( x 1 , x 2 , . . . , x j )) , where the probabilities are computed assuming a uniform distribution o v er M . 3 Serializabilit y and En tropy Inequalities Constrain ts imp osed on the entropic vector alone suﬃce to characterize serializability for DA Gs, but, the addition of one cycle causes the en topic v ector to b e an insuﬃcient characterization. W e sho w that the en tropic vector is not enough to determine serializability even on the 2-cycle by giving a serializable and non-serializable co de with the same entropic v ector. The tw o co des illustrated in Figure 1 apply to the message tuple ( X , Y , Z ), where X , Y , Z are uniformly distributed random v ariable o ver F 2 . It is easy to c hec k that the entrop y of every subset of corresponding source and edge functions is the same, and thus the codes ha ve the same en tropic v ector. The co de in Figure 1(a) is clearly serializable: at time step one we send X on ( u, v ) and Y on ( v , u ); then Y on ( u, v ) and X on ( v , u ). On the other hand, the co de in Figure 1(b) is not serializable b ecause, informally , to send X + Y on the top edge requires that w e already sen t X + Y 2 Throughout this pap er, when the op erator Q is applied to functions rather then sets w e mean it to denote the op eration of forming an ordered tuple from an indexed list of elements. 4 (X,Y) (X,Z) (Y ,Z) (X,Y) u v (a) Serializable (X+Y ,Z) (X,Z) (Y ,Z) (X+Y ,Z) u v (b) Not Serializable Figure 1: Two net work codes with the same en tropy function. on the bottom edge, and vice v ersa. A formal pro of that the co de in Figure 2(b) is not serializable can b e obtained by applying the characterization of serializability in Theorem 4.6. In order to use entrop y inequalities to give tigh t upp er b ounds on net work co ding rates, w e need an enumeration of the complete set of entrop y inequalities implied b y serializability , i.e. a list of necessary and suﬃcient conditions for a vector V to b e the entropic vector of a serializable co de. (Note that it need not be the case that every co de whose entropic v ector is V must be serializable.) F or the 2-cycle we can enumerate the complete set of inequalities. In particular, we giv e four inequalities that must hold for any serializable co de on the 2-cycle: t wo are a result of downstr e amness whic h is a condition that m ust hold for all graphs (it sa ys that the entrop y of the incoming edges of a vertex must b e at least as muc h as the entrop y of the incoming and outgoing edges together), the third is the Chicken and Egg inequalit y due to [7], and the fourth is a new inequalit y that we call the gr e e dy inequalit y . It is equiv alent to b eing able to complete the ﬁrst iteration of our greedy algorithm in Section 4. W e sho w that these four inequalities together with Shannon’s inequalities are the only inequalities implied b y serializabilit y , in the following sense: Theorem 3.1 Given a rational-v alued entropic vector, V , of a 2-cycle on no des u, v , with source x into no de u , source y in to no de v , and edges a = ( u, v ) and b = ( v , u ), there exists a se- rializable co de that realizes cV , for some constant c , if and only if V satisﬁes Shannon’s in- equalities, do wnstreamness ( H ( abx ) = H ( bx ), H ( aby ) = H ( ay )), the Chick en and Egg inequal- it y ( H ( ab ) ≥ H ( abx ) − H ( x ) + H ( aby ) − H ( y )), and the greedy inequalit y ( H ( a ) + H ( b ) > H ( ax ) − H ( x ) + H ( by ) − H ( y ) when H ( a ) + H ( b ) 6 = 0). Multiplication of the v ector b y a constan t c is a natural relaxation b ecause the theorem b ecomes oblivious to the base of the logarithm w e use to compute the Shannon entrop y . The proof of Theorem 3.1 in v olves considering four cases corresponding to the relationship b et w een H ( a ) and H ( ax ) , H ( ay ) , H ( x ) , H ( y ) and b etw een H ( b ) and H ( bx ) , H ( by ) H ( x ) , H ( y ). Eac h case requires a distinctly diﬀerent co ding function to realize the en tropic vector. All the co ding functions are relativ ely simple, in volving only sending uncoded bits, and the X OR of t wo bits. Most of the w ork is limiting the v alues of co ordinates of the entropic vector based on the inequalities that the entropic v ector satisﬁes. The pro of of Theorem 3.1 is provided in App endix A. The big open question left from this work is whether w e can ﬁnd a complete set of constrain ts on the entropic v ector implied b y the serializability of co des on arbitrary graphs. W e curren tly do not kno w of any pro cedure for pro ducing such a list of inequalities. Even if we had a conjecture for such a list, showing that it is complete is likely to b e quite hard. If w e hav e more than three sources, just determining the p ossible dep endencies b et ween sources is diﬃcult b ecause they are sub ject to non-Shannon information inequalities. 5 4 A Characterization of Serializabilit y 4.1 Linear Co des A c haracterization of serializabilit y for linear net work co des is simpler than the general case b ecause it relies on more standard algebraic to ols. Accordingly , w e treat this case ﬁrst b efore mo ving on to the general case. Throughout this section we use V ∗ to denote the dual of a vector space ov er a ﬁeld F , and f ∗ to denote the adjoint of a linear transformation f . F or an edge e with alphab et Σ e and co ding function f e : M → Σ e , we use T e to denote the linear subspace f ∗ e (Σ ∗ e ) ⊆ M ∗ . Though it is impossible to characterize the serializabilit y of a net work co de in terms of its en tropic vector, computationally there is a straigh tforward solution. In p olynomial time w e can either determine a serialization for a co de or show that no serialization exists using the obvious algorithm: try to serialize the co de b y “sending new information when possible.” When w e can no longer send any new information along any edge w e terminate. If we ha ve sen t all the information required along eac h edge, then the greedy algorithm ﬁnds a serialization; otherwise, w e sho w that no serialization exists b y presen ting a succinct certiﬁcate of non-serializabilit y . Though our algorithm is straigh tforw ard, we b eliev e that the c hange in mindset from characterizing co des in terms of the en tropic v ector is an important one, and that our certiﬁcate of non-serializability (see Deﬁnition 4.1) furnishes an eﬀective tool for addressing other questions about serializability , as w e shall see in later sections. Giv en a netw ork co de Φ = ( G , M , { Σ e } , { f e } ), with co ding functions ov er the ﬁeld F , our greedy algorithm (pseudo code, LinSerialize (Φ) , is giv en in App endix B.1), constructs a set of edge functions f (1 ..k ) e and alphab ets Σ (1 ..k ) e for each edge. These ob jects are constructed iterativ ely , deﬁning the edge alphab ets Σ ( i ) e and co ding functions f ( i ) e in the i th iteration. Throughout this pro cess, w e maintain a pair of linear subspaces A e , B e ⊆ M ∗ for eac h edge e = ( u, v ) of G . A e is the linear span 3 of all the messages transmitted on e so far, and B e is intersection of T e with the linear span of all the messages transmitted to u so far. (In other w ords, B e spans all the messages that could currently b e sen t on e without receiving an y additional messages at u .) In the i th iteration, if there exists an edge e 0 suc h that B e 0 con tains a dual vector x e 0 that do es not b elong to A e 0 , then w e create co ding function f ( i ) e for all e . The co ding function of f ( i ) e 0 is set to b e x e 0 and its alphab et is set to b e F . F or all other edges we set f ( i ) e = 0. This pro cess contin ues un til B e = A e for every e . A t that p oin t, we rep ort that the code is serializable if and only if A e = T e for all e . At the end, the algorithm returns the functions f (1 ..k ) e and the alphab ets Σ (1 ..k ) e , where k is the num b er of iterations of the algorithm, as well as the subspaces { A e } . If the co de w as not serializable, then { A e } is interpreted as a certiﬁcate of non-serializability (a “non-trivial information vortex”) as explained b elo w. LinSerialize (Φ) runs in time p olynomial in the size of the co ding functions of Φ. In ev ery iteration of the while lo op we increase the dimension of some A e b y one. A e is initialized with dimension zero and can hav e dimension at most dim( T e ). Therefore, the algorithm go es through at most P e ∈ E dim( T e ) iterations of the while lo op. Additionally , eac h iteration of the while lo op, aside from constan t time assignmen ts, computes only in tersections and spans of v ector spaces, all of which can b e done in p olynomial time. T o prov e the algorithm’s correctness, we deﬁne the following certiﬁcate of non-serializability . Deﬁnition 4.1 An information vortex (IV) of a net work co de consists of a linear subspace W e ⊆ M ∗ for each edge e , such that: 1. F or a source edge s , W s = T s . 3 If { V i : i ∈ I } is a collection of linear subspaces of a vector space V , their linear span is the minimal linear subspace con taining the union S i ∈I V i . W e denote the linear span by + i ∈I V i . 6 2. F or every other edge e , W e = T e ∩  + e 0 ∈ In ( e ) W e 0  . An information vortex is nontrivial if W e 6 = T e for some edge e . W e think of W e as the information that we can send o v er e giv en that its incoming edges, e 0 ∈ I n ( e ), can send W e 0 . In our analysis of the greedy algorithm, we show that the messages the greedy algorithm succeeds in sending (i.e., the linear subspaces { A e } ) form an IV and it is non-trivial if and only if the code isn’t serializable. The follo wing theorem shows the relationship b et w een IVs, serialization, and the greedy algo- rithm. The pro of can b e found in App endix B.1. Theorem 4.2 F or a net work code Φ = ( G , M , { Σ e } , { f e } ), the following are equiv alent: 1. Φ is not serializable 2. LinSerialize (Φ) returns { A e } s.t. ∃ e, A e 6 = T e 3. Φ has a non-trivial information vortex In Section 5 and Section 6 w e will see that information v ortices pro vide a concise w a y for pro ving the non-serializabilit y of a net work code. Moreo ver, the notion of an information v ortex was critical to our discov ery of the result in Section 6. 4.2 General co des Our characterization theorem extends to the case of general netw ork co des, provided that we generalize the greedy algorithm and the deﬁnition of information v ortex appropriately . The message space M is no longer a vector space, so instead of deﬁning information vortices using the vector space M ∗ of all linear functions on M , w e use the Bo olean algebra 2 M of all binary-v alued functions on M . W e b egin b y recalling some notions from the theory of Bo olean algebras. Deﬁnition 4.3 Let S b e a set. The Bo olean algebra 2 S is the algebra consisting of all { 0 , 1 } - v alued functions on S , under and ( ∧ ), or ( ∨ ), and not ( ¬ ). If f : S → T is a function, then the Bo ole an algebr a gener ate d by f , denoted b y h f i , is the subalgebra of 2 S consisting of all functions b ◦ f , where b is a { 0 , 1 } -v alued function on T . If A 1 , A 2 are subalgebras of a Bo olean algebra A, their in tersection A 1 ∩ A 2 is a subalgebra as w ell. Their union is not, but it generates a subalgebra that we will denote by A 1 + A 2 . If S is a ﬁnite set and A ⊆ 2 S is a Boolean subalgebra, then there is an equiv alence relation on S deﬁned b y setting x ∼ y if and only if b ( x ) = b ( y ) for all b ∈ A . The equiv alence classes of this relation are called the atoms of A , and w e denote the set of atoms by A t( A ). There is a canonical function f A : S → At( A ) that maps each elemen t to its equiv alence class. Note that A = h f A i . The relev ance of Boolean subalgebras to netw ork co ding is as follo ws. A subalgebra A ⊆ 2 M is a set of binary-v alued functions, and can b e in terpreted as describing the complete state of kno wledge of a part y that kno ws the v alue of each of these functions but no others. In particular, if a sender kno ws the v alue of f ( m ) for some function f : M → T , then the binary-v alued messages this sender can transmit given its current state of kno wledge correspond precisely to the elements of h f i . This observ ation supplies the ra w materials for our deﬁnition of the greedy algorithm for general net work co des, whic h we denote by GenSerialize (Φ). As b efore, the edge alphab ets and co ding functions are constructed iteratively , with Σ ( i ) e and f ( i ) e deﬁned in the i th iteration of the main loop. Throughout this pro cess, we maintain a pair of Bo olean subalgebras A e , B e ⊆ 2 M for each edge e = ( u, v ) of G . A e is generated by all the messages transmitted on e so far, and B e is in tersection of h f e i with the subalgebra generated b y all messages transmitted to u so far. (In other words, B e spans all the binary-v alued messages that 7 could currently b e sen t on e without receiving any additional messages at u .) In the i th iteration, if there exists an edge e 0 suc h that B e 0 con tains a binary function x e 0 6∈ A e 0 , then w e create a binary-v alued co ding function f ( i ) e for all e , which is set to b e x e 0 if e = e 0 and the constan t function 0 if e 6 = e 0 . This pro cess contin ues un til B e = A e for every e . A t that p oint, we rep ort that the co de is serializable if and only if A e = h f e i for all e . A t the end, the algorithm returns the functions f (1 ..k ) e and the alphabets Σ (1 ..k ) e , where k is the n um b er of iterations of the algorithm, as w ell as the subspaces { A e } . The pseudo co de for this algorithm GenSerialize (Φ) is presented in App endix C. If Φ has ﬁnite alphab ets, then GenSerialize (Φ) must terminate b ecause the total n umber of atoms in all the Bo olean algebras A e ( e ∈ E ) is strictly increasing in eac h iteration of the main lo op, so P e ∈ E | Σ e | is an upp er b ound on the total n umber of lo op iterations. In implemen ting the algorithm, eac h of the Bo olean algebras can b e represen ted as a partition of M into atoms, and all of the op erations the algorithm p erforms on Bo olean algebras can be implemented in p olynomial time in this representation. Thus, the running time of GenSerialize (Φ) is p olynomial in P e ∈ E | Σ e | . In ligh t of the algorithm’s termination condition, the following deﬁnition is natural. Deﬁnition 4.4 If G = ( V , E , S ) is a sourced graph, a gener alize d information vortex (GIV) in a net work co de Φ = ( G, M , { Σ e } , { f e } ) is an assignmen t of Bo olean subalgebras A e ⊆ 2 M to every e ∈ E ∪ S , satisfying: 1. A s = h f s i for all s ∈ S ; 2. A e =  + ˆ e ∈ In ( u ) A ˆ e  ∩ h f e i for all e = ( u, v ) ∈ E . A GIV is nontrivial if A e 6 = h f e i for some e ∈ E . A tuple of Bo olean subalgebras Γ = ( A e ) e ∈ E ∪ S is a semi-vortex if it satisﬁes (1) but only satisﬁes one-sided con tainment in (2), i.e., 3. A e ⊆  + ˆ e ∈ In ( u ) A ˆ e  ∩ h f e i for all e = ( u, v ) ∈ E . If Γ = ( A e ) and Υ = ( A 0 e ) are semi-vortices, w e sa y that Γ is contained in Υ if A e ⊆ A 0 e for all e . In App endix C we pro ve a series of statemen ts (Lemmas C.2-C.5) sho wing that: • Semi-v ortices are in one-to-one corresp ondence with restrictions of Φ. The corresp ondence maps a semi-vortex ( A e ) e ∈ E ∪ S to the netw ork co de with edge alphab ets At( A e ) and co ding functions given b y the canonical maps M → A t( A e ) deﬁned in Deﬁnition 4.3. • There is a set of semi-vortices corresponding to serializable restrictions of Φ under this cor- resp ondence. They can b e thought of as representing p artial serializations of Φ. • There is a set of semi-vortices corresp onding to GIV’s of Φ . These can be thought of as c ertiﬁc ates of infe asibility for serializing Φ. • GenSerialize (Φ) computes a semi-v ortex Γ which is both a GIV and a partial serialization. These lemmas combine to yield a “min-max theorem” showin g that the every netw ork co de has a maximal serializable restriction that coincides with its minimal GIV, as well as an analogue of Theorem 4.2; pro ofs of both theorems are in Appendix C. Theorem 4.5 In the ordering of semi-v ortices b y con tainmen t, the ones corresp onding to partial serializations ha v e a maximal elemen t and the GIV’s hav e a minimal elemen t. These maximal and minimal elements coincide, and they are b oth equal to the semi-v ortex Γ = ( A e ) e ∈ E ∪ S computed b y GenSerialize (Φ). Theorem 4.6 F or a net work code Φ with ﬁnite alphabets, the following are equiv alent. 1. Φ is serializable. 2. GenSerialize (Φ) outputs { A e } e ∈ E s.t. ∀ e, A e = h f e i . 3. Φ has no nontrivial GIV. 8 5 The Serializabilit y Deﬁcit of Linear Net w ork Co des The min-max relationship betw een serializable restrictions and information vortices (Theorem 4.5) is reminiscen t of classical results lik e the max-ﬂo w min-cut theorem. Ho wev er, there is an important diﬀerence: one can use the minim um cut in a net work to detect how far a netw ork ﬂow problem is from feasibilit y , i.e. the minim um amount by which edge capacities would need to increase in order to mak e the problem feasible. In this section, w e will see that determining ho w far a netw ork co de is from serializability is more subtle: tw o net work co des can b e quite similar-lo oking, with similar-lo oking minimal information v ortices, y et one of them can be serialized b y sending only one extra bit while the other requires man y more bits to b e sent. W e b egin with an example to illustrate this point. The codes in Figure 2 apply to the message tuple ( X 1 , . . . , X n , Y 1 , . . . , Y n ) where X i , Y i are indep endent, uniformly distributed random v ariables o ver F 2 . The co des in Figures 2(a) and 2(b) are almost identical; the only diﬀerence is that the co de in Figure 2(a) has one extra bit along the top edge. The code in Figure 2(a) is serializable: transmit X 1 along ( u, v ), then X 1 + Y 1 on edge ( v , u ), then X 2 + Y 1 on ( u, v ), ... , X n + Y n on ( v , u ), and ﬁnally X 1 + Y n on ( u, v ). On the other hand, the co de in Figure 2(b) is not serializable, whic h can b e seen by applying our greedy algorithm. u v (X 1 ,X 2 ,...,X n ) (Y 1 ,Y 2 ,...,Y n ) (X 1 ,X 2 +Y 1 ,X 3 +Y 2 ,...,X 1 +Y n ) (X 1 +Y 1 ,X 2 +Y 2 ,...,X n +Y n ) (a) Serializable u v (X 1 ,X 2 ,...,X n ) (Y 1 ,Y 2 ,...,Y n ) (X 2 +Y 1 ,X 3 +Y 2 ,...,X 1 +Y n ) (X 1 +Y 1 ,X 2 +Y 2 ,...,X n +Y n ) (b) Not Serializable Figure 2: Two almost identical net w ork codes. Th us, the co de in Figure 2(b) is v ery close to serializable because w e can consider an extension of the co de in whic h we add one bit 4 to the edge ( u, v ) to obtain the co de in Figure 2(a) that is serializable. On the other hand, there are similar co des that are v ery far from b eing serializable. If w e consider the co de with the same sources and f ( u,v ) = f ( v ,u ) = Q n i =1 X i + Y i , its edge alphab ets ha ve the same size and its minimal information vortex is iden tical, y et any serializable extension requires adding n bits. T o completely characterize serializability we would like to b e able to separate co des that are close to serializable from those that are far. This motiv ates the following deﬁnition. Deﬁnition 5.1 F or a net work code Φ = ( G , M , { Σ e } , { f e } ) and an extension Φ 0 = ( G 0 , M , { Σ 0 e } , { f 0 e } ), the gap of Φ 0 , deﬁned by γ (Φ 0 ) = P e ∈ E log 2 | Σ 0 e | − log 2 | Σ e | , represen ts the combined num b er of extra bits transmitted on all edges in Φ as compared to Φ. The serializability deﬁcit of Φ, denoted b y SD (Φ), is deﬁned to b e the minimum of γ (Φ 0 ) ov er all serializable extensions Φ 0 of Φ. The line ar serializability deﬁcit of a linear co de Φ, denoted LSD (Φ), is the minim um of γ (Φ 0 ) ov er all linear serializable extensions Φ 0 . Unfortunately , determining the serialization deﬁcit is m uch more diﬃcult than simply deter- mining serializability . Theorem 5.2 Giv en a linear net work co de Φ, it is NP-hard to approximate the size of the minimal linear serializable extension of Φ. Moreo ver, there is a linear netw ork co de Φ and a p ositiv e in teger n such that LSD (Φ n ) / ( n LSD (Φ)) < O ( 1 log 2 ( n ) ). 4 In this section, for simplicit y , w e refer to one scalar-v alued linear function on an F -vector space as a “bit” ev en if | F | > 2. 9 Both statements in the theorem follo w directly from the following lemma. Lemma 5.3 Given a hitting set instance ( N , S ) with universe N , | N | = n , and subsets S ⊆ 2 N , an optimal integral solution k , and an optimal fractional solution z 1 q , z 2 q , .., z n q , with P n i =1 z i p 1 = p q , in p olynomial time w e can construct a linear netw ork code suc h that LSD (Φ) = k , but LSD (Φ q ) ≤ p . Pr o of sketch. The full pro of of the Lemma is in App endix B.1. s w n u n v n p n w 1 u 1 v 1 p 1 w i u i v i p i Y 1 Y i Y n X Figure 3: The reduction from Hitting Set Here, due to space limitations, w e merely sk etch the main ideas. The graph used in the reduc- tion is illustrated pictorially in Figure 3. Given a hitting set instance ( N , S ) we create a net- w ork co de with one source for eac h i ∈ N (source message denoted by ~ Y i ) and a sup er- source s (source message denoted ( ~ X 1 , ~ X 2 , . . . , ~ X n )). The sym b ols ~ X i , ~ Y i don’t refer to bits, but ac- tually to blo c ks of n i bits, where n i is the num- b er of sets in S containing i ; each of the bits in ~ X i or ~ Y i corresp onds to one of the sets that i b elongs to. F or eac h i ∈ N w e use a gadget consisting of a 2-cycle on v ertices u i , w i , with ~ Y i feeding in to w i and ~ X i feeding from the sup er- source s in to u i . The edges betw een u i and w i are a copy of the gadget in Figure 2. W e exploit the fact that sending one extra bit in this gadget allo ws the information v ortex in the gadget to “unra vel”, leading to transmission of all the bits enco ded on the edges of the 2-cycle. The 2-cycle ( u i , w i ) participates in a larger 4-vertex gadget { u i , w i , p i , v i } corresp onding to the elemen t i . The role of v i is to participate in “set gadgets”, where the gadget corresp onding to a set A consists of a bidirected clique on all the vertices { v j | j ∈ A } . The role of p i is less imp ortan t; it plays a necessary part in disseminating bits to lefto ver parts of the netw ork after the “imp ortan t” parts hav e b een serialized. If there is a hitting set of size k then w e send one bit on eac h of the 2-cycles ( u i , w i ) corresp onding to elements i in the hitting set. This “unlocks” the bits that were lo c ked up in those 2-cycles, whic h allo ws a suﬃcien t amount of information to ﬂow in to the set gadgets that they b ecome serialized. The vertices p j are then used for disseminating the remaining bits to the un used 2-cycles ( u j , w j ) where j did not b elong to the hitting set. T o pro ve, conv ersely , that a serializabilit y deﬁcit of at most k implies that there is a hitting set of size k , we mak e use of the fact that the net work co de constructed by our reduction has a large n umber of information v ortices, one for eac h pair consisting of an elemen t of N and a se t S that it b elongs to. If C is the set of all i suc h that an extra bit is transmitted somewhere in the 4-v ertex gadget for i , and C fails to contain an element of some set A , then this in turn implies that one of the aforemen tioned information vortices remains an information vortex in the extension of the co de. Thus, C m ust be a hitting set. The more diﬃcult step in proving Lemma 5.3 lies in sho wing that fractional solutions of the hitting set problem can b e transformed into eﬃcient serializable extensions of Φ q . F or this, w e mak e use of the fact that the edge alphab ets in Φ q can be regarded as Σ e ⊗ F q . and their duals can b e regarded as Σ ∗ e ⊗ ( F ∗ ) q . If | F | is large enough, then the uniform matroid U q ,p is represen table as a set { t 1 , . . . , t p } of p v ectors in ( F ∗ ) q . F or each of the “fractional elements” z i /q in the fractional hitting set, w e send z i bits of the form x ⊗ t in the extension of Φ q , where t is one of the elemen ts of our matroid representation of U q ,p in ( F ∗ ) q and x is the bit that w e w ould ha ve sent for eleme n t i in the hitting set reduction describ ed t wo paragraphs earlier. The fact that P i z i = p implies that 10 ev ery element of the matroid representation is used exactly once in this construction. The fact that w e ha ve a fractional set co v er implies that in each set gadget, w e receiv e extra bits corresp onding to q distinct elemen ts of S . Since these elements are a basis for ( F ∗ ) q , it is then p ossible to sho w that they combine to allo w a serialization of all the “missing bits” in that gadget, and from there w e ﬁnish serializing the entire net work co de Φ q as b efore. 6 Asymptotic Serializabilit y The results in the previous section show ed b oth that there are non-serializable co des with large edge alphab ets that b ecome serializable by adding only one bit (example in Figure 2) and that the serialization deﬁcit can b eha ve sub-additively when w e take the n -fold cartesian product of Φ (Theorem 5.2). This prompts the in vestigation of whether there exists a co de that isn’t serializable, but the n -fold parallel rep etition of the co de can b e serialized b y extending it by only a constant n umber of bits, and th us it is essen tially indistinguishable from serializable. W e formalize this idea with the following deﬁnition. Deﬁnition 6.1 A net work co de Φ is asymptotic al ly serializable if lim n →∞ 1 n LSD (Φ n ) / LSD (Φ) = 0 where Φ n is n -fold cartesian pro duct of Φ with cartesian pro duct deﬁne in the ob vious wa y . If one is using a netw ork code to transmit inﬁnite streams of data by c hopping each stream up into a sequence of ﬁnite blo c ks and applying the sp eciﬁed co ding functions to eac h blo c k, then an asymptotically serializable net work co de is almost as go od as a serializable one, since it can b e serialized by adding a side c hannel of arbitrarily small bit-rate to eac h edge of the netw ork. Despite indications to the contrary in Section 5, w e sho w that any non-serializable linear co de is not asymptotically serializable via the follo wing theorem. Theorem 6.2 F or a linear netw ork co de Φ = ( G , M , { Σ e } , { f e } ) ov er a ﬁeld F , then LSD (Φ n ) ≥ cn where c is a constant dependent on Φ. The pro of of the theorem considers the alphab ets of the n -fold pro duct of Φ as elemen ts of a tensor pro duct space. Using this mac hinery , we sho w that information v ortices in the graph are preserv ed if w e don’t increase the amount of information w e send do wn some edge by order n bits. More sp eciﬁcally , if { W e } is a non-trivial information vortex in Φ, and e is an edge suc h that dim( W e ) < dim( T e ) = m , then if w e add some edge function f to every edge in the graph, the information v ortex remains non-trivial as long as the dimension of f is less than mn . A complete pro of is provided in App endix E. References [1] Micah Adler, Nic holas J. A. Harv ey , Kamal Jain, Rob ert Klein b erg, and April Rasala Lehman. On the capacity of information net works. In Pr o c. of the 17th ACM-SIAM Symp osium on Discr ete Algorithms (SODA) , pages 241–250, 2005. [2] Rudolf Ahlswede, Ning Cai, Shuo-Y en Rob ert Li, and Raymond W. Y eung. Netw ork informa- tion ﬂow. IEEE T r ansactions on Information The ory , 46(4):1204–1216, 000. [3] T. H. Chan and Ra ymond W. Y eung. On a relation b etw een information inequalities and group theory . IEEE T r ansactions on Information The ory , 48:1992–1995, 2002. [4] Randall Dougherty , Chris F reiling, and Ken Zeger. Six new non-shannon information inequal- ities. In Pr o c. 2006 International Symp osium on Information The ory (ISIT) , pages 233–236, 2006. 11 [5] Elona Erez and Meir F eder. Eﬃcien t net work co des for cyclic netw orks. In Pr o c. 2005 Inter- national Symp osium on Information The ory (ISIT) , pages 1982–1986, 2005. [6] Nic holas J. A. Harv ey , Rob ert Klein b erg, and April Rasala Lehman. On the capacity of information netw orks. IEEE T r ansactions on Information The ory , 52(6):2345–2364, 2006. [7] Nic holas J.A. Harvey , Rob ert Kleinberg, Chandra Nair, and Y unnan W u. A “chic ken & egg” net work co ding problem. In Pr o c. 2007 IEEE International Symp osium on Information The ory (ISIT) , pages 131–135, 2007. [8] Kamal Jain, Vija y V azirani, Ra ymond W. Y eung, and Gideon Y uv al. On the capacit y of m ultiple unicast sessions in undirected graphs. In Pr o c. 2005 IEEE International Symp osium on Information The ory (ISIT) , 2005. [9] Ralf Ko etter and Muriel Medard. An algebraic approach to net work co ding. IEEE/ACM T r ansactions on Networking , 11(5):782–795, 2003. [10] Gerhard Kramer and Serap Sav ari. Edge-cut b ounds on netw ork co ding rates. Journal of Network and Systems Management , 14(1):49–67, 2006. [11] April Rasala Lehman. Network Co ding . PhD thesis, MIT, 2005. [12] Konstan tin Mak arychev, Y uri Mak arychev, Andrei Romashc henko, and Nik olai V ereshchagin. A new class of non shannon type inequalities for entropies. Communic ations in Information and Systems , 2(2):147–166, 2002. [13] F ran ti ˇ sek Mat ´ u ˘ s. Inﬁnitely man y information inequalities. In Pr o c. 2007 International Sym- p osium on Information The ory (ISIT) , pages 41–44, 2007. [14] Ra ymond W. Y eung. A First Course in Information The ory . Springer, 2002. [15] Ra ymond W. Y eung, Sh uo-Y en Rob ert Li, Ning Cai, and Zhen Zhang. Network Co ding The ory . No w Publishers, 2006. [16] Ra ymond W. Y eung and Zhen Zhang. Distributed source co ding for satellite comm unication. IEEE T r ansactions on Information The ory , 45(4):1111–1120, 1999. [17] Zhen Zhang and Ra ymond W. Y eung. On characterization of en tropy function via information inequalities. IEEE T r ansactions on Information The ory , 44:1440–1452, 1998. A Pro of of Theorem 3.1 Theorem A.1 (Theorem 3.1 restated) Given a rational-v alued en tropic vector, V , of a 2-cycle on no des u, v , with source x in to no de u , source y into node v , and edges a = ( u, v ) and b = ( v , u ), there exists a serializable co de that realizes cV , for some constant c , if and only if V satisﬁes Shannon’s inequalities, do wnstreamness ( H ( abx ) = H ( bx ), H ( aby ) = H ( ay )), the Chick en and Egg inequalit y ( H ( ab ) ≥ H ( abx ) − H ( x ) + H ( aby ) − H ( y )), and the Greedy inequality ( H ( a ) + H ( b ) > H ( ax ) − H ( x ) + H ( by ) − H ( y ) when H ( a ) + H ( b ) 6 = 0). Throughout this pro of it will often b e con v enient to refer to the conditional entrop y of tw o sets of edges. Deﬁnition A.2 F or tw o subsets of edges F = { e 1 , e 2 , . . . , e j } and F 0 = { e 0 1 , e 0 2 , . . . , e 0 k } , the conditional en tropy of F given F 0 , denoted H ( F | F 0 ) = H ( e 1 e 2 . . . e j | e 0 1 e 0 2 . . . e 0 k ) = H ( F F 0 ) − H ( F 0 ). 12 W e ﬁrst show that all the inequalities are necessary . Shannon’s inequalities m ust hold for the en tropic vector of any set of random v ariables. Downstreamness (term coined by [11]) was shown to b e necessary ev en for DA GS b y Y eung and Zhang [16]. Harv ey et al. [7] show ed that the Chick en and Egg inequality is necessary . Th us, it remains to sho w that our greedy inequalit y is a necessary condition for serializability . Lemma A.3 The inequalit y H ( a ) + H ( b ) > H ( ax ) − H ( x ) + H ( by ) − H ( y ) = H ( a | x ) + H ( b | y ) holds for any serializable co de on a 2-cycle when H ( a ) + H ( b ) > 0. Pr o of. Suppose there is a serializable co de suc h that H ( a ) + H ( b ) ≤ H ( a | x ) + H ( b | y ) and H ( a ) + H ( b ) > 0. Because conditioning reduces entrop y , H ( a ) ≥ H ( a | x ), and lik ewise H ( b ) ≥ H ( b | y ). These three inequalities together imply that H ( a ) = H ( a | x ) and H ( b ) = H ( b | y ). It follo ws from the deﬁnition of serializabilit y and H ( a ) + H ( b ) > 0 that there exists a non-zero f ( i ) a or f ( i ) b . Let i ∗ b e the smallest suc h i and let f ( i ∗ ) a b e the asso ciated non-zero co ding function (the choice of a is without loss of generality). W e can rewrite H ( a | x ) as H ( f ( i ∗ ) a | x ) + H ( a | f ( i ∗ ) a x ). H ( f ( i ∗ ) a | x ) = 0 b ecause i ∗ is the smallest suc h i implies that f ( i ∗ ) a is computed soley from x . But, this giv es us that H ( a ) = H ( a | f ( i ∗ ) a x ), which is a contradiction to f ( i ) a non-zero. T o prov e the other direction of Theorem 3.1 we will use a case based analysis, but ﬁrst w e make a few observ ations to bound the cases we need to consider. Observ ation A.4 The follo wing ten v alues completely determine the entropic vector of the 2- cycle: I ( x ; y ) , H ( x | y ) , H ( y | x ) , H ( a | x ) , H ( b | x ) , H ( a | y ) , H ( b | y ) , H ( a ) , H ( b ) , H ( ab ). Pr o of. Due to downstreamness and Shannon’s inequalities the following equations hold: H ( xy ) = I ( x ; y ) + H ( x | y ) + H ( y | x ) , H ( axy ) = H ( bxy ) = H ( abxy ) = H ( xy ) , H ( y ) = I ( x ; y ) + H ( y | x ) , H ( x ) = I ( x ; y ) + H ( x | y ) , H ( ax ) = H ( a | x ) + H ( x ) , H ( aby ) = H ( ay ) = H ( a | y ) + H ( y ) , H ( abx ) = H ( bx ) = H ( b | x ) + H ( x ) , H ( by ) = H ( b | y ) + H ( y ). This implies that the v alue of all 15 non-zero elements of the entropic v ector are determined b y the 10. Observ ation A.5 H ( b | x ) ≥ H ( a | x ) Pr o of. H ( bx ) = H ( abx ) ≥ H ( ax ) b y downstreamness and then monotonicity . Observ ation A.6 H ( a | y ) ≥ H ( b | y ) Pr o of. P arallel to proof of observ ation A.5 Observ ation A.7 max( H ( a | x ) , H ( a | y )) ≤ H ( a ) ≤ H ( a | x ) + H ( a | y ) + I ( x ; y ) Pr o of. H ( a ) ≤ H ( a | x ) + H ( a | y ) + I ( x ; y ): Apply submo dularit y on ax and ay to get H ( ax ) + H ( ay ) ≥ H ( axy ) + H ( a ) = H ( xy ) + H ( a ), then subtract H ( x ) + H ( y ) from b oth sides. H ( a ) ≥ max( H ( a | x ) , H ( a | y )) : H ( a ) ≥ H ( a | x ) and H ( a ) ≥ H ( a | y ) b ecause conditioning reduces en tropy . Observ ation A.8 max( H ( b | x ) , H ( b | y )) ≤ H ( b ) ≤ H ( b | x ) + H ( b | y ) + I ( x ; y ) Pr o of. P arallel to proof of observ ation A.7 Observ ation A.9 H ( b | x ) + H ( a | y ) ≤ H ( ab ) ≤ H ( b | x ) + H ( a | y ) + I ( x ; y ) 13 Pr o of. H ( ab ) ≥ H ( b | x ) + H ( a | y ) b y the c hick en and egg inequalit y . H ( ab ) ≤ H ( b | x ) + H ( a | y ) + I ( x ; y ): by submo dularit y on ay and bx : H ( ay ) + H ( bx ) = H ( aby ) + H ( abx ) ≥ H ( abxy ) + H ( ab ). ⇒ H ( ab ) ≤ H ( bx ) + H ( ay ) − H ( xy ). No w, w e come to our case analysis for proving the forw ard direction of Theorem 3.1. W e ﬁrst m ultiply our en tropic v ector by the least common denominator so that all the elements of the vector are integer. W e sho w that w e can ﬁnd a co de that realizes this integer v alued entropic v ector. Let X 1 , ...X H ( x | y ) , Z 1 , ...Z I ( x ; y ) b e random v ariables originating at source x, and let Y 1 , ..., Y H ( y | x ) , Z 1 , ..., Z I ( x ; y ) b e random v ariables originating at source y , where X i , Y j , Z k are indep enden t for all i, j, k . W e split up the pro of in to 4 cases. Case 1 corresp onds to when H ( a ) is greater than H ( a | x ) + H ( a | y ) and H ( b ) is greater than H ( b | x ) + H ( b | y ). Case 4 takes care of the instances when b oth H ( a ) is less than H ( a | x ) + H ( a | y ) and H ( b ) is less than H ( b | x ) + H ( b | y ). Cases 2 and 3 are symmetric corresp onding to when exactly one of H ( a ) and H ( b ) is greater than the sum of the conditional en tropy on x and y . Cases 1,2 (or 3),4 corresp ond to distinctly diﬀeren t co ding functions on edges a and b . Case 1 has the simplest co des - we send bits unco ded with the exception of p ossibly X ORing X and Z or Y and Z . In cases 2 and 3 we need to X OR bits of X , Y on one edge, and in case 4 w e need to XOR bits of X, Y on both edges in a manner similar to the example in Figure 2(b). Case 1: H ( a ) = H ( a | x ) + H ( a | y ) + f , f ≥ 0 and note f ≤ I ( x ; y ) b y Observ ation A.7. H ( b ) = H ( b | x ) + H ( b | y ) + g , g ≥ 0 and note g ≤ I ( x ; y ) b y Observ ation A.8. H ( ab ) = H ( b | x ) + H ( a | y ) + h , and note 0 ≤ h ≤ I ( x ; y ) by Observ ation A.9. Observ ation A.10 h ≤ H ( a | x ) + H ( b | y ) + f + g Pr o of. Implied b y submodularity on a and b . Observ ation A.11 h ≥ max( f , g ) Pr o of. H ( x | a ) ≥ H ( x | ab ) Conditioning reduces entrop y H ( ax ) − H ( a ) ≥ H ( abx ) − H ( ab ) H ( ab ) − H ( bx ) − H ( a | y ) + H ( x ) ≥ H ( a ) − H ( ax ) − H ( a | y ) + H ( x ) H ( ab ) − H ( b | x ) − H ( a | y ) ≥ H ( a ) − H ( a | x ) − H ( a | y ) h ≥ f The pro of that h ≥ g is similar. W e claim that the following co de realizes the entropic v ector and is serializable: F or notational con venience let Z 0 1 = Z f +1 , Z 0 2 = Z f +2 , ..., Z 0 h − f − g = Z h − g . Any Z 0 i with i > h − f − g w e will tak e to b e 0. f a = X 1 , . . . , X H ( a | y ) , Y 1 + Z 0 1 , . . . , Y H ( a | x ) + Z 0 H ( a | x ) , Z 1 , . . . Z f f b = X 1 + Z 0 H ( a | x )+1 , . . . , X H ( b | y ) + Z 0 H ( a | x )+ H ( b | y ) , Y 1 , . . . , Y H ( b | x ) , Z h − g − 1 , . . . , Z h 14 This is a v alid co de b ecause H ( x | y ) ≥ H ( a | y ) ≥ H ( b | y ), H ( y | x ) ≥ H ( b | x ) ≥ H ( a | x ), h ≤ I ( x ; y ), h ≤ H ( a | x ) + H ( b | y ) + f + g , and h ≥ max( f , g ). It is easy to chec k that this co de realizes the en tropic v ector. It is serializable b ecause H ( a | y ) ≥ H ( b | y ), H ( b | x ) ≥ H ( a | x ) and b oth sources kno w Z . Case 2: H ( a ) = H ( a | x ) + H ( a | y ) − f , f ≥ 0 and note f ≤ min( H ( a | x ) , H ( a | y )) by Observ ation A.7. H ( b ) = H ( b | x ) + H ( b | y ) + g , g ≥ 0 and note g ≤ I ( x ; y ) b y Observ ation A.8. H ( ab ) = H ( b | x ) + H ( a | y ) + h , and note 0 ≤ h ≤ I ( x ; y ) by Observ ation A.9. Observ ation A.12 h ≤ ( H ( a | x ) − f ) + H ( b | y ) + g W e claim that the following co de realizes the entropic v ector and is serializable: An y Z i with i > h we will tak e to b e 0. f a = X 1 + Y 1 , X 2 + Y 2 , . . . , X f + Y f , X f +1 , . . . , X H ( a | y ) , Y f +1 + Z g +1 , . . . , Y H ( a | x ) + Z g + H ( a | x ) − f f b = X 1 + Z g + H ( a | x ) − f +1 , . . . , X H ( b | y ) + Z g + H ( a | x ) − f + H ( b | y ) , Y 1 , . . . , Y H ( b | x ) , Z 1 , . . . , Z g This is a v alid co de for the same reasons as Case 1, and also b ecause h ≤ ( H ( a | x ) − f ) + H ( b | y ) + g , and f ≤ H ( a | x ) and f ≤ H ( a | y ). It is easy to c heck that this co de realizes the en tropic vector; here it is imp ortan t that g ≤ h which is true b y the argument from Observ ation A.11. It is serial- izable b ecause we can send Y 1 , ..., Y H ( b | x ) along edge b , then b ecause H ( b | x ) ≥ H ( b | y ) w e can send ev erything along edge a , and then b ecause H ( a | y ) ≥ H ( a | x ) we can send all the X s and Z s on edge b . Case 3: H ( a ) = H ( a | x ) + H ( a | y ) + f , f ≥ 0 and note f ≤ I ( x ; y ) b y Observ ation A.7. H ( b ) = H ( b | x ) + H ( b | y ) − g , g ≥ 0 and note g ≤ min( H ( b | x ) , H ( b | y )) by Observ ation A.8. H ( ab ) = H ( b | x ) + H ( a | y ) + h , and note 0 ≤ h ≤ I ( x ; y ) by Observ ation A.9. Symmetric to Case 2. Case 4: H ( a ) = H ( a | x ) + H ( a | y ) − f , f ≥ 0 and note f ≤ min( H ( a | x ) , H ( a | y )) by Observ ation A.7. H ( b ) = H ( b | x ) + H ( b | y ) − g , g ≥ 0 and note g ≤ min( H ( b | x ) , H ( b | y )) by Observ ation A.8. H ( ab ) = H ( b | x ) + H ( a | y ) + h , and note 0 ≤ h ≤ I ( x ; y ) by Observ ation A.9. Applying the inequality H ( a ) + H ( b ) > H ( a | x ) + H ( b | y ), together with the fact that H ( a ) ≥ H ( a | x ) and H ( b ) ≥ H ( b | x ) implies that at least one of H ( a ) > H ( a | x ), H ( b ) > H ( b | y ) holds. Or, written in terms of f , g this means that at least one of f < H ( a | y ), g < H ( b | x ) holds. Observ ation A.13 h ≤ ( H ( a | x ) − f ) + ( H ( b | y ) − g ) Case 4a: f < H ( a | y ) W e claim that the following co de realizes the entropic v ector and is serializable: An y Z i with i > h we will tak e to b e 0. f a = X 2 + Y 1 , X 3 + Y 2 , . . . , X f +1 + Y f , X 1 , X f +2 , . . . , X H ( a | y ) , Y f +1 + Z 1 , . . . , Y H ( a | x ) + Z H ( a | x ) − f f b = X 1 + Y 1 , X 2 + Y 2 , . . . , X g + Y g , X g +1 + Z H ( a | x ) − f +1 , . . . , X H ( b | y ) + Z H ( a | x ) − f + H ( b | y ) − g , Y g +1 , . . . , Y H ( b | x ) 15 This is a v alid co de b ecause f +1 ≤ H ( a | y ), h ≤ ( H ( a | x ) − f )+( H ( b | y ) − g ), f ≤ min( H ( a | x ) , H ( a | y )) and g ≤ min( H ( b | x ) , H ( b | y )). It is easy to c heck that this co de realizes the entropic vector. T o sho w it is serializable, we ﬁrst consider the case when f ≤ g : we can send X 1 along edge a ; then X 1 + Y 1 along edge b ; then X 2 + Y 1 along edge a ; . . . ; then X f +1 + Y f , X f +2 , . . . , X H ( a | y ) along edge a ; then b ecause H ( a | y ) ≥ H ( b | y ), we can send then ev erything along edge b ; and then since H ( b | x ) ≥ H ( a | x ) we can complete the transmission for edge a . The case for f > g is very similar. Case 4b: g < H ( b | x ) This case is similar, but we switc h the roles of edge a and edge b . B Pro ofs omitted from Section 4 B.1 Linear co des Algorithm 1 Greedy Algorithm for Linear Codes LinSerialize (Φ) 1: / ∗ Φ = ( G , M , { Σ e } , { f e } ) , G = ( V , E , S ) is a network c o de with c o ding functions over ﬁeld F . We c onstruct Σ (1 ..k ) e and f (1 ..k ) e . ∗ / 2: A e ← 0 for all e ∈ E . / ∗ A e ⊆ T e r epr esents the information we have sent over e dge e ∗ / 3: A s ← T s for all s ∈ S . 4: B e ← T e ∩  + s ∈ In ( e ) A s  for all e ∈ E . / ∗ B e ⊆ T e r epr esents the information that the tail of e knows ab out T e ∗ / 5: i = 1 6: while ∃ e = ( u, v ) in G such that A e 6 = B e do 7: Let x e b e an y vector in B e that do esn’t lie in A e 8: Σ ( i ) e ← F , f ( i ) e ← x e 9: A e ← A e + h x e i 10: ∀ e 0 ∈ E , e 0 6 = e, Σ ( i ) e 0 ← 0, f ( i ) e 0 ← 0 11: ∀ e 0 = ( v , · ) ∈ E , B e 0 ← T e 0 ∩ ( B e 0 + { x e } ) / ∗ No de v “le arns” x e ∗ / 12: i + + 13: end while Theorem B.1 (Theorem 4.2 restated) F or a netw ork code Φ = ( G , M , { Σ e } , { f e } ), the following are equiv alent: 1. Φ is not serializable 2. LinSerialize (Φ) returns { A e } s.t. ∃ e, A e 6 = T e 3. Φ has a non-trivial information vortex Pr o of. ¬ 2 ⇒ ¬ 1 If LinSerialize (Φ) returns { A e } s.t. ∀ e, A e = T e then Φ is serializable: W e show that the f (1 ..k ) e , Σ (1 ..k ) e created by LinSerialize (Φ) satisfy the conditions in Deﬁnition 2.2: 1. f ( i ) e : M → Σ ( i ) e b y construction. 2. The non-zero functions f ( i ) e form a basis for T e . Because linear maps are indiﬀerent to the c hoice of basis, if f e ( m 1 ) = f e ( m 2 ) then in any basis, each co ordinate of f e ( m 1 ) equals the corresp onding co ordinate of f e ( m 2 ), and thus f ( i ) e ( m 1 ) = f ( i ) e ( m 2 ) for all i . 16 3. If f e ( m 1 ) 6 = f e ( m 2 ) then for an y basis we choose to represent f e , the v alues f e ( m 1 ) , f e ( m 2 ) will diﬀer in at least one coordinate, and thus ∃ i, f ( i ) e ( m 2 ) 6 = f ( i ) e ( m 2 ). 4. When we assign a function f ( i ) e = x e w e ha ve that x e is in B e whic h guaran tees it is computable from information already sen t to the tail of e . 2 ⇒ 3 If LinSerialize (Φ) returns { A e } s.t. ∃ e A e 6 = T e then Φ has a non-trivial IV. W e claim the the vector spaces { A e } returned b y LinSerialize (Φ) form a non-trivial IV. { A e } is non-trivial b y h yp othesis, so it remains to sho w it is an I V . { A e } satisﬁes prop ert y (1): F or each S ∈ S , A S = T S b y construction (Line 3 of LinSerialize (Φ) ). { A e } satisﬁes prop ert y (2): By induction on our algorithm, B e is exactly T e ∩  + e 0 ∈ In ( e ) A e 0  . At termination, B e = A e for all e ∈ E . So, we ha ve that A e = T e ∩  + e 0 ∈ In ( e ) A e 0  . 3 ⇒ 1 If Φ has a non-trivial IV then it isn’t serializable. Supp ose for con tradiction that Φ = ( G , M , { Σ e } , { f e } ) , G = ( V , E , S ) is serializable. Let f (1 ..k ) e and Σ (1 ..k ) e satisfy the conditions of deﬁnition 2.2. Let { W e } b e a non-trivial IV for Φ. W e sa y that a function f ( j ) e has property P if there ∃ m 1 , m 2 ∈ M suc h that f ( j ) e ( m 1 ) 6 = f ( j ) e ( m 2 ) and m 1 , m 2 ∈ W ⊥ e . There must b e suc h a function since our IV is non-trivial and Σ (1 ..k ) e , f (1 ..k ) e is a serialization of Φ. Let i ∗ b e the smallest i such that any function satisﬁes prop ert y P and suppose f i ∗ e ∗ satisﬁes P with messages m ∗ 1 , m ∗ 2 . By deﬁnition, W e ∗ = T e ∗ ∩  + e 0 ∈ In ( e ∗ ) W e 0  , so m ∗ 1 , m ∗ 2 ∈ W ⊥ e ∗ implies that for all e 0 ∈ In ( e ∗ ), m ∗ 1 , m ∗ 2 ∈ W ⊥ e 0 . But, f i ∗ e ∗ can distinguish b et w een m ∗ 1 , m ∗ 2 so at least one of e 0 ∈ In ( e ∗ ) must also b e able to distinguish betw een m ∗ 1 , m ∗ 2 at a time b efor e i ∗ . Therefore, there exists some f i 0 e 0 , i 0 < i ∗ that satisﬁes prop erty P , a contradiction to the fact that i ∗ w as the smallest such i . C Pro ofs omitted from Section 4.2 The following lemma is standard; for completeness, we pro vide a proof here. Lemma C.1 Supp ose f 1 : S → T 1 and f 2 : S → T 2 are tw o functions on a set S . 1. h f 2 i ⊆ h f 1 i if and only if there exists a function g : T 1 → T 2 suc h that f 2 = g ◦ f 1 . 2. Suppose S is ﬁnite. If f 1 × f 2 denotes the function S → T 1 × T 2 deﬁned by x 7→ ( f 1 ( x ) , f 2 ( x )), then h f 1 i + h f 2 i = h f 1 × f 2 i . Pr o of. A Bo olean subalgebra of 2 S can b e equiv alen tly describ ed as a collection of subsets of S , closed under union, intersection, and complementation, by equating a { 0 , 1 } -v alued function b with the set b − 1 (1) . In this pro of we adopt the “collection of subsets” deﬁnition of a Boolean subalgebra of 2 S , since it is more con v enient. Note that under this interpretation, if f : S → T is an y function then h f i consists of all subsets of the form f − 1 ( U ) , U ⊆ T . If f 2 = g ◦ f 1 for some g , then every set of the form f − 1 2 ( U ) can b e expressed as f − 1 1 ( g − 1 ( U )) whic h sho ws that h f 2 i ⊆ h f 1 i . Conv ersely , if h f 2 i ⊆ h f 1 i then for ev ery u ∈ T 2 the set f − 1 2 ( { u } ) ∈ h f 2 i b elongs to h f 1 i , i.e. it can b e expressed as f − 1 1 ( V u ) for some set V u ⊆ T 1 . The sets f − 1 1 ( V u ) are disjoin t as u ranges ov er the elements of T 2 so the sets V u themselv es m ust b e disjoin t. Deﬁne g ( v ) = u if v ∈ V u for some u ∈ T 2 , and deﬁne g ( v ) to b e an arbitrary elemen t of T 2 otherwise. F or an y x ∈ S, if u = f 2 ( x ) then x ∈ f − 1 2 ( u ) = f − 1 1 ( V u ), whic h implies that g ( f 1 ( x )) = u. Hence f 2 = g ◦ f 1 as desired. T o prov e (2) w e argue as follo ws. Clearly h f 1 i , h f 2 i ⊆ h f 1 × f 2 i , so h f 1 i + h f 2 i ⊆ h f 1 × f 2 i as w ell. F or the reverse inclusion, note that every element of h f 1 × f 2 i can b e expressed as a ﬁnite 17 union of sets of the form ( f 1 × f 2 ) − 1 ( t 1 , t 2 ). Ev ery suc h set can b e expressed as f − 1 1 ( t 1 ) ∩ f − 1 2 ( t 2 ), whic h pro v es that it b elongs to h f 1 i + h f 2 i . Algorithm 2 Greedy algorithm for general net w ork codes GenSerialize (Φ) 1: / ∗ Φ = ( G , M , { Σ e } , { f e } ) , G = ( V , E , S ) is a network c o de. ∗ / 2: / ∗ We c onstruct Σ (1 ..k ) e and f (1 ..k ) e . ∗ / 3: A e ← 0 for all e ∈ E . / ∗ A e ⊆ h f e i r epr esents the information we have sent over e dge e ∗ / 4: A s ← h f s i for all s ∈ S . 5: B e ← h f e i ∩  + s ∈ In ( e ) A s  for all e ∈ E . 6: / ∗ B e ⊆ h f e i r epr esents the information that the tail of e knows ab out f e ∗ / 7: i = 1 8: while ∃ e = ( u, v ) in G such that A e 6 = B e do 9: Let x e b e an y binary-v alued function in B e \ A e . 10: Σ ( i ) e ← { 0 , 1 } , f ( i ) e ← x e 11: A e ← A e + h x e i 12: ∀ e 0 ∈ E , e 0 6 = e, Σ ( i ) e 0 ← { 0 } , f ( i ) e 0 ← 0 13: ∀ e 0 = ( v , · ) ∈ E , B e 0 ← h f e 0 i ∩ ( B e 0 + h x e i ) / ∗ No de v “le arns” x e ∗ / 14: i ← i + 1 15: end while Lemma C.2 F or a given netw ork co de Φ, restrictions Φ 0 of Φ are in one-to-one corresp ondence with semi-v ortices Γ. The corresp ondence maps Γ to the netw ork co de Φ 0 [Γ] whose alphabets are Σ 0 e = A t( A e ) and whose co ding functions are the functions f 0 e = f A e deﬁned in Deﬁnition 4.3. The inv erse corresp ondence maps Φ 0 to the unique semi-vortex Γ[Φ 0 ] satisfying A r = h f 0 r i for all r ∈ E ∪ S. Pr o of. Suppose Γ = ( A r ) is a semi-v ortex and Φ 0 [Γ] is deﬁned as stated, with co ding functions f 0 e = f A e . F or all e = ( u, v ) ∈ E , let In ( e ) = { e 1 , . . . , e k } and let f i = f 0 e i . The relation A e =  + k i =1 A e i  ∩ h f e i implies that A e ⊆ + k i =1 A e i = + k i =1 h f i i = h ( f 1 , . . . , f k ) i , where the last equation follows from Lemma C.1. Since h f A e i = A e ⊆ h ( f 1 , . . . , f k ) i , we can apply Lemma C.1 again to conclude that f A e = g ◦ ( f 1 , . . . , f k ) for some g . Thus Φ 0 [Γ] is a netw ork co de. T o pro ve that it is a restriction of Φ, w e use the containmen t A e ⊆ h f e i for every edge e ∈ E ∪ S , together with Lemma C.1, to construct the functions g e : Σ e → Σ 0 e required b y the deﬁnition of a restriction of Φ . Lemma C.3 If Φ 0 is a restriction of Φ, and Φ 0 is serializable, then Γ[Φ 0 ] is contained in every GIV of Φ. Pr o of. Suppose Φ 0 is a serializable restriction of Φ, with serialization consisting of alphab ets Σ ( i ) e and co ding functions f ( i ) e . Supp ose now that Γ = { A e } e ∈ E ∪ S is an y GIV of Φ. First, w e claim h f ( i ) e i ⊆ A e for ev ery edge e . T o prov e the claim w e use induction on i . The claim is clearly true when i = 0. Otherwise, 18 let e 1 , . . . , e r b e the edges in In ( e ). By Lemma C.1, the existence of a function h ( i ) e suc h that f ( i ) e ( m ) = h ( i ) e  Q r j =1 f 1 ..i − 1 e j  implies the ﬁrst of the following con tainments: h f ( i ) e i ⊆ + r j =1 + i − 1 ` =1 h f ( ` ) e j i ⊆ + r j =1 A e j . (1) The second con tainment in (1) follo ws from our induction h yp othesis. Now, prop ert y 2 of a serial- ization implies that h f ( i ) e i ⊆ h f 0 e i . Combining this with (1) w e obtain h f ( i ) e i ⊆  + r j =1 A e j  ∩ h f 0 e i = A e , (2) as desired. If Γ[Φ 0 ] is not con tained in Γ, then there exists an edge e of G such that h f 0 e i 6⊆ A e . (3) Prop ert y 2 of a serialization implies the existence of a function H : Σ 0 e → Q k i =1 Σ ( i ) e suc h that H ( f 0 e ( m )) = ( f (1) e ( m ) , . . . , f ( k ) e ( m )) for all m ∈ M . Prop ert y 3 implies that H is one-to-one, hence it has a left in verse: a function G : Q k i =1 Σ ( i ) e → Σ 0 e suc h that G ◦ H is the identit y . Letting F = Q k i =1 f ( i ) e , the deﬁnition of H implies that F = H ◦ f 0 e , whence f 0 e = G ◦ F . Applying Lemma C.1 once more, h f 0 e i ⊆ h F i = + k i =1 h f ( i ) e i , and the right side is con tained in A e b y (2). This con tradicts (3), which completes the argumen t. Lemma C.4 At the start of any iteration of the main lo op of GenSerialize (Φ) the following in v ariants hold. 1. A e = h f (1) e , . . . , f ( i − 1) e i for all e ∈ E . 2. B e = h f e i ∩  + ˆ e ∈ In ( u ) A ˆ e  for all e ∈ E . 3. The collection of subalgebras Γ = { A e } e ∈ E ∪ S constitutes a semi-vortex. 4. Φ 0 [Γ] is a serializable restriction of Φ. Pr o of. The ﬁrst three inv ariants can be v eriﬁed by a trivial induction on the n um b er of loop itera- tions. W e claim that Φ 0 [Γ] is serializable, and in fact that the co ding functions { f ( j ) e } constructed in the preceding iterations of the main lo op constitute a serialization of Φ 0 [Γ] . F or prop ert y 1 of a serialization, there is nothing to chec k. T o prov e prop ert y 2, observe that f ( j ) e ∈ A e = h f 0 e i , whic h implies by Lemma C.1 that f ( j ) e = b ◦ f 0 e for some binary-v alued function b on Σ 0 e . If f 0 e ( m 1 ) = f 0 e ( m 2 ) then b ( f 0 e ( m 1 )) = b ( f 0 e ( m 2 )), which establishes prop erty 2. T o prov e prop ert y 3, observ e that A e = h f 0 e i is generated b y the functions f (1 ..i − 1) e , so if f 0 e ( m 1 ) 6 = f 0 e ( m 2 ) then there is some j ≤ i − 1 suc h that f ( j ) e ( m 1 ) 6 = f ( j ) e ( m 2 ) . Finally , prop ert y 4 follows from the structure of the algorithm itself. Either f ( j ) e is the constant function 0, in which case there is nothing to pro ve, or f ( j ) e is equal to the function x e c hosen in line 9 of the j th lo op iteration of GenSerialize (Φ). In that case x e b elonged to the Bo olean algebra B e at the start of that lo op iteration, whic h means x e ∈ h f e i ∩  + ˆ e ∈ In ( u ) A ˆ e  ⊆ + ˆ e ∈ In ( u ) A ˆ e = + ˆ e ∈ In ( u )  + 1 ≤ ` 0 to b e c hosen later. Rather than en umer- ate our edge set E and the co ding functions on each edge, w e will just sp ecify the co ding functions for each edge in E . If a function f e is not sp eciﬁed, then e is not in E . W e also show the netw ork co ding instance pictorially in Figure 3. W e use Q denote the n -fold cartesien product, so Q n i =1 X i is synonomous with the ordered n -tuple ( X 1 , X 2 , ...X n ). The co ding functions are as follows: f ( s,u i ) = n i Y k =1 X A i ( k ) i , ∀ i ∈ N f ( s,v i ) = Y j ∈ N : j 6 = i n j Y k =1 X A j ( k ) j ∀ i ∈ N f ( s,p i ) = n i X k =2 X A i ( k ) i , ∀ i ∈ N f ( v i ,v j ) = Y A ∈ S ( i ) ∩ S ( j ) X k ∈ A X A k , ∀ i, j ∈ N f ( w i ,v i ) = n i Y k =1 X A i ( k ) i , ∀ i ∈ N f ( v i ,p i ) = n i X k =1 X A i ( k ) i , ∀ i ∈ N f ( p i ,w i ) = X A i (1) i , ∀ i ∈ N f ( w i ,u i ) = n i Y k =1 X A i ( k ) i + Y A i ( k ) i , ∀ i ∈ N f ( u i ,w i ) = n i Y k =1 X A i ( k +1 mo d n i ) i + Y A i ( k ) i , ∀ i ∈ N Pr o of of Part 1 of L emma 5.3. Given a hitting set instance ( N , S ) we create the net work co de Φ using the Reduction D.1. W e show that ( N , S ) has a hitting set of size k if and only if LSD (Φ) ≤ k . ( ⇒ ) Supp ose ( N , S ) has a hitting set of size k . W e show that LSD (Φ) ≤ k . Let C b e a hitting set of size k . Consider adding bit X A c (1) c for c ∈ C to edge ( u c , w c ). This allo ws us to serialize all bits in the following stages, implicitly w e are deﬁning f (1 ..k ) e and Σ (1 ..k ) e for all e ∈ E : 1. F or all c ∈ C , we can serialize all bits on edges ( w c , u c ) and ( u c , w c ): w c learns X A c (1) c , so it can send bit X A c (1) c + Y A c (1) c to u c . No w, this allows X A c (2) c + Y A c (1) c to b e sen t on ( u c , w c ), and we con tinue in this wa y un til all bits serialized on these t wo edges. 2. F or all c ∈ C , send f ( w c ,v c ) = Q n c k =1 X A c ( k ) c on ( w c , v c ). 3. F or all i ∈ N , send f ( s,v i ) Q j ∈ N : j 6 = i Q n j k =1 X A j ( k ) j on ( s, v i ) 4. F or every set A ∈ S there is an elemen t c ∈ A ∩ C b ecause C is a hitting set. Thus, there is a v ertex in V , v c that kno ws X A c . v c can therefore send bit t ( A ) = P a ∈ A X A a to all v a , a ∈ A, a 6 = c . No w ev ery v a kno ws t ( A ) and can send it along ( v a , v a 0 ) for all a 0 ∈ A, a 0 6 = a . This serializes all bits on edges betw een vertices in V . 5. No w ev ery vertex v i kno ws ev ery bit X : it receiv ed all but Q n i k =1 X A i ( k ) i in step 3, and determined Q n i k =1 X A i ( k ) i in step 4. So, w e can send f ( v i ,p i ) = P n i k =1 X A i ( k ) i on edge ( v i , p i ). 6. A t p i w e can add the code P n i k =2 X A i ( k ) i from ( s, p i ) to f ( v i ,p i ) to obtain X A i (1) i and send it on ( p i , w i ). 21 7. No w, for all i ∈ N − C , we can serialize ( w i , u i ) and ( u i , w i ) as we did in step 1. ( ⇐ ) Supp ose that LSD (Φ) ≤ k . W e sho w that ( N , S ) has a hitting set of size k . Consider the partition of E in to sets E ( i ) ∀ i ∈ N suc h that E ( i ) = { ( · , · i ) | i ∈ N } , that is e ∈ E ( i ) if and only if the tail of e is indexed by i . Lemma D.2 F or a set A ∈ S supp ose no bits are added to any edge in S i ∈ A E ( i ), then the bit t ( A ) = P a ∈ A X A a on ( v a , v a 0 ), ∀ a, a 0 ∈ A cannot b e serialized. Lemma D.2 implies that for ev ery set A ∈ S , at least one bit must b e added on an edge in S i ∈ A E ( i ) to serialize Φ. In particular, if Φ 0 is a minimal serializable extension of Φ and let C = { i | Φ 0 sends at least one additional than Φ on some edge in E ( i ) } then C is a hitting set for ( N , S ). And | C | ≤ LSD (Φ) ≤ k . Pr o of of L emma D.2. Let V ( A ) = S a ∈ A v a . An y edge e going into any no de in V ( A ) m ust send f e b ecause no bits are added on an y of these edges. In any serialization, it m ust be that for some v a ∈ V ( A ), bit t ( A ) is sent on ( v a , v a 0 ) for some a 0 b efore v a receiv es t ( A ) from any v a 00 ∈ V ( A ). Without loss of generality , suppose this v ertex is v i , i ∈ A . Consider the subgraph induced b y v ertices indexed b y i . There is an information v ortex on this subgraph with W ( s,v i ) = T ( s,v i ) , W ( s,u i ) = T ( s,u i ) , and W e = 0 for all other edges in the subgraph. T o c heck this one simply has to verify that for edges e out of v i T e ∩ T ( s,v i ) = 0, and similarly for u i . This implies that this subgraph is not serializable. W e don’t add an y bits to the subgraph b y h yp othesis, so to “destroy” this IV, and serialize the subgraph w e need W ( v j ,v i ) + W ( s,v i ) to ha v e a non-zero in tersection with T ( v i , · ) . But this is a con tradiction to our choice of i . Pr o of of Part 2 of L emma 5.3. Given a hitting set instance ( N , S ) we create the net work co de Φ using the Reduction D.1. If | F | is large enough then one can sho w using facts from linear algebra (or matroid theory) that the subset T =         1 x 1 1 . . . x q − 1 1    ,    1 x 1 2 . . . x q − 1 2    , . . . ,    1 x 1 p . . . x q − 1 p         ⊂ F q for { x 1 , . . . , x p } distinct elements in F has the prop erty that any q element subset forms a basis for F q ; in other w ords, T is a realization of the uniform matroid U p,q o ver F . P artition the p vectors of T in to | N | subsets T 1 , . . . , T | N | suc h that T i con tains z i v ectors, note that P i ∈ N z i = p makes this v alid. W e now consider Φ q . Here, we will regard the edge alphab et for edge e as a v ector in Σ e ⊗ F q . The tensor product space allo ws us to consider q copies of Σ e on each edge e without ﬁxing a basis. W e claim that the extenstion of Φ q in which we transmit the extra bits Q t ∈ T i X A i (1) i ⊗ t on edge ( u i , w i ) for all i ∈ N is serializable. Observ ation D.3 T ransmitting Q t ∈ T i X A i (1) i ⊗ t along edge ( u i , w i ) allows no de w i to learn Q t ∈ T i X A i ( k ) i ⊗ t for all k ∈ { 1 . . . n i } . Pr o of. W e saw in the pro of of the forward direction of the part 1 of Lemma 5.3 that transmitting X A i (1) i along edge ( u i , w i ) in Φ implies no de w i can learn X A i ( k ) i for all k ∈ { 1 . . . n i } . This implies that in the q -fold rep etition, transmitting X A i (1) i ⊗ t along edge ( u i , w i ) allows no de w i to learn X A i ( k ) i ⊗ t for all k ∈ { 1 . . . n i } . Observ ation D.4 If Q t ∈ T a X A a ⊗ t can be transmitted along edge ( w a , v a ) for all a ∈ A then we can transmit σ ( A ) = P a ∈ A X A a ⊗ F q on all edges ( v a , v a 0 ), a, a 0 ∈ A . 22 Pr o of. Q j ∈ N : j 6 = a Q n j k =1 X A j ( k ) j ⊗ F q can b e transmitted along ( s a , v a ) for all a ∈ A , and that, together with Q t ∈ T a X A a ⊗ t transmitted along ( w a , v a ), allows no de v a , for all a ∈ A , to compute α ( a ) = Q t ∈ T a P a ∈ A X A a ⊗ t . No w, ﬁx i ∈ A . Send α ( a ) along ( v a , v i ) for all a ∈ A, a 6 = i . Eac h message in the tuple α ( a ) is a linear com bination of P a ∈ A X A a ⊗ F q and is hence a legal message on all edges ( v a , v i ). After sending these messages, node v i kno ws P a ∈ A X A a ⊗ t for P a ∈ A z a distinct v ectors t . Our z a ’s form a feasible fractional hitting set, thus P a ∈ A z a q ≥ 1, and P a ∈ A z a ≥ q . Any q -elemen t subset of T forms a basis of F q , and so v i can determine σ ( A ). W e can then send σ ( A ) on edges ( v i , v a ) , a ∈ A and then ( v a , v a 0 ) for all a 0 ∈ A, a 0 6 = a . Observ ation D.3 and D.4 together imply that for all sets A ∈ S we can send σ ( A ) on the clique formed by v a , a ∈ A . A simple argument identitical to the last steps in the forw ard direction of the pro of of part 1 of Lemma 5.3 imply that we can serialize the rest of the co ding functions. E Pro ofs omitted from Section 6 The pro ofs in this section rely on knowledge of tensor pro ducts. W e include a brief tutorial here for conv enience. E.1 T ensor pro ducts If V , W are any t wo vector spaces with bases { e V i } i ∈I and { e W j } j ∈J , resp ectively , their tensor pro duct is a v ector space V ⊗ W with a basis indexed by I × J . The basis vector corresp onding to an elemen t ( i, j ) in the index set will b e denoted by e V i ⊗ e W j . F or any t wo v ectors v = P i a i e V i in V and w = P j b j e W j in W , their tensor pr o duct is the v ector v ⊗ w = X i X j a i b j e V i ⊗ e W j in V ⊗ W . Lemma E.1 If { x i } i ∈I and { y j } j ∈J are bases of V , W, resp ectiv ely , then { x i ⊗ y j } ( i,j ) ∈I ×J is a basis of V ⊗ W . Pr o of. It suﬃces to prov e the lemma when y j = e W j for all j ∈ J . If the lemma holds in this case, then by symmetry it also holds when x i = e V i for all i ∈ I , and then the general case of the lemma follows by applying these tw o special cases in succession: ﬁrst c hanging the basis of V , then c hanging the basis of W . T o pro ve that B = { x i ⊗ e W j } ( i,j ) ∈I ×J is a basis of V ⊗ W , it suﬃces to prov e that every vector of the form e V i ⊗ e W j can b e written as a linear combination of elemen ts of B . By the assumption that { x i } i ∈I is a basis of V , w e know that e V i = P k ∈ K a k x k for some ﬁnite subset K ⊆ I and scalars ( a k ) k ∈ K . Now it follo ws that e V i ⊗ e W j = P k ∈ K a k  x k ⊗ e W j  , as desired. E.2 Basis and Rank Deﬁnition E.2 If V is a v ector space with basis B , and W = { W i } i ∈I is a collection of linear subspaces, we sa y that W is B -c omp atible if W i ∩ B is a basis of W i , for all i ∈ I . W e sa y that W is b asis-c omp atible if there exists a basis B for V such that W is B -compatible. 23 Lemma E.3 If V is a v ector space and W is a basis-compatible collection of linear subspaces, then W can b e enlarged to a basis-compatible collection of linear subspaces ¯ W that forms a Boolean algebra under + and ∩ . In particular, an y three subspaces X , Y , Z ∈ W satisfy: ( X ∩ Y ) + Z = ( X + Z ) ∩ ( Y + Z ) ( X + Y ) ∩ Z = ( X ∩ Z ) + ( Y ∩ Z ) . Pr o of. Simply let ¯ W b e the set of all linear subspaces of W ⊆ V suc h that W ∩ B is a basis of W . Lemma E.4 If V is a v ector space and W is a basis-compatible collection of linear subspaces, then W ∪ { V } ∪ { 0 } is also a basis-compatible collection of linear subspaces. Pr o of. The pro of is a trivial consequence of the deﬁnition of basis-compatible. Lemma E.5 If V is a vector space and X , Y are any tw o linear subspaces, then W = { X, Y } is basis-compatible. Pr o of. Let B X Y b e an y basis of X ∩ Y , let B X b e an y basis of X con taining B X Y , and let B Y b e an y basis of Y containing B X Y . All the vectors in B X ∪ B Y are linearly indep enden t, b ecause if v is an y v ector that can be expressed as a linear combination of elements of B Y \ B X and as a linear com bination of elemen ts of B X , then v must b elong to b oth X and Y , hence v ∈ X ∩ Y . But the only elemen t of X ∩ Y that can be expressed as a linear com bination of elements of B Y \ B X is the zero v ector, b ecause B X Y is disjoin t from B Y \ B X , and these tw o sets together constitute a basis of Y . Hence B X ∪ B Y can b e extended to a basis B of V , and then W = { X, Y } is B -compatible. If A, B are subspaces of vector spaces V , W , resp ectiv ely , then A ⊗ B is deﬁned to b e the linear subspace of V ⊗ W consisting of all linear combinations of v ectors in the set { a ⊗ b : a ∈ A, b ∈ B } . If V is a collection of linear subspaces of V and W is a collection of linear subspaces of W , then V ⊗ W denotes the collection of all linear subspaces A ⊗ B ⊆ V ⊗ W such that A ∈ V and B ∈ W . Lemma E.6 If V is a basis-compatible collection of linear subspaces of V and W is a basis- compatible collection of linear subspaces of W then V ⊗ W is a basis-compatible collection of linear subspaces of V ⊗ W . Pr o of. If B , B 0 are bases of V , W , resp ectiv ely , suc h that V is B -compatible and W is B 0 -compatible, then V ⊗ W is ( B × B 0 )-compatible. Corollary E.7 If X , Y are subspaces of a vector space V and Z is a subspace of another v ector space W , then [( X ⊗ W ) + ( Y ⊗ W )] ∩ ( V ⊗ Z ) = [( X ⊗ W ) + ( V ⊗ Z )] ∩ [( Y ⊗ W ) + ( V ⊗ Z )] . Pr o of. By Lemmas E.4 and E.5, we kno w that V = { X , Y , V } is basis-compatible in V and W = { Z, W } is basis-compatible in W . Hence V ⊗ W is basis-compatible in V ⊗ W. The corollary no w follo ws b y applying Lemma E.3. Deﬁnition E.8 If V , W are any vector spaces, a r ank-one elemen t of V ⊗ W is an element that can be expressed in the form v ⊗ w for some v ∈ V , w ∈ W . The r ank of an elemen t x ∈ V ⊗ W is the minim um v alue of r suc h that x can b e expressed as a linear com bination of r rank-one elements of V ⊗ W. (If x = 0 then its rank is deﬁned to b e 0.) 24 Lemma E.9 If V , W are ﬁnite-dimensional vector spaces and x ∈ V ⊗ W then the rank of x is b ounded ab o ve b y min { dim( V ) , dim( W ) } . Pr o of. Let n = dim( V ) , m = dim( W ) . W e will assume without loss of generality that n ≤ m and pro ve that the rank of x is at most n . Let { e V i } and { e W j } b e bases of V , W, resp ectiv ely . W e may express x as a linear combination x = n X i =1 m X j =1 a ij e V i ⊗ e W j . F or 1 ≤ i ≤ n let y i = P m j =1 a ij e W j . Then x = n X i =1 e V i ⊗ y i , and this expresses x as a sum of n rank-one elements, implying that the rank of x is at most n . Lemma E.10 If V , W are ﬁnite-dimensional vector spaces of dimension n, m , and P ⊆ V ⊗ W is a linear subspace of dimension p , then there exists a pn -dimensional linear subspace Q ⊆ W suc h that P ⊆ V ⊗ Q. Pr o of. Let { x 1 , . . . , x p } b e a basis of P . By Lemma E.9, w e can write each x k in the form x k = n X i =1 v ki ⊗ w ki , for some v ectors v ki (1 ≤ k ≤ p, 1 ≤ i ≤ n ) in V and w ki (1 ≤ k ≤ p, 1 ≤ i ≤ n ) in W . The v ectors { w ki } span a subspace of W of dimension at most pn . T aking Q to b e an y pn -dimensional subspace of W containing { w ki } , we see that P ⊆ V ⊗ Q as desired. E.3 Non-serializabilit y Theorem E.11 (Theorem 6.2 restated) F or a linear netw ork code Φ = ( G , M , { Σ e } , { f e } ) o ver a ﬁeld F , then LSD (Φ n ) ≥ cn where c is a constant dep enden t on Φ. Pr o of. If a linear netw ork co de has an information vortex { W e } e ∈ E then { W n e } e ∈ E constitutes an information v ortex in the pro duct of n copies of the netw ork co de. Using the fact that V n = V ⊗ F n for every v ector space V , we ma y rewrite the information vortex as { W e ⊗ F n } . No w supp ose that T is a linear subspace of M ∗ ⊗ F n of dimension p . (Think of T as a set of bits that are added to Φ to get an exten tion of φ .) Using Lemma E.10, there is a subspace Q ⊆ F n of dimension pd (where d = dim( M ∗ )) such that T ⊆ M ∗ ⊗ Q. Deﬁne a new family of subspaces X e = ( W e ⊗ F n ) + ( M ∗ ⊗ Q ) . These subspaces constitute an information vortex. T o verify this, w e m ust chec k that ( W e ⊗ F n ) + ( M ∗ ⊗ Q ) = [( T e ⊗ F n ) + ( M ∗ ⊗ Q )] ∩  + e 0 ∈ In( e ) (( W e 0 ⊗ F n ) + ( M ∗ ⊗ Q ))  . (4) Because { W e ⊗ F n } is an information vortex, w e hav e W e ⊗ F n = ( T e ⊗ F n ) ∩  + e 0 ∈ In( e ) W e 0 ⊗ F n  . 25 Equation (4) no w follows by applying Corollary E.7 with X = T e , Y = + e 0 ∈ In( e ) W e 0 , Z = Q. Th us the collection of subspaces { X e } constitutes an information v ortex in the setting where T added to every edge of the net work co de. If the net work co de is serializable in this setting, then { X e } m ust b e a trivial information vortex, implying that X e = T e ⊗ F n for ev ery edge e . Because { W e } is non-trivial, we kno w there is at least one edge e suc h that dim( W e ) < dim( T e ) . Let m = dim( T e ) , k = dim( W e ) . The dimension of W e ⊗ F n is k n. The dimension of M ∗ ⊗ Q is pd 2 . Hence the dimension of X e is b ounded ab o ve b y k n + pd 2 . On the other hand, if the information v ortex { X e } is trivial, then X e = T e ⊗ F n implying that dim( X e ) = mn. Thus LSD (Φ) ≥ p ≥ m − k d 2 · n. 26

The Serializability of Network Codes

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment