Source Coding for a Simple Network with Receiver Side Information

Source Coding for a Simple Network wi th Recei v er Side Information R. T imo 1 , 2 , A. Grant 3 , T . Chan 3 and G. Kramer 4 1 Departmen t of Engineer ing, the Au stralian National University , Canberr a, A CT , Australia. 2 Networked Systems, NI CT A, Canberr a Research Labora tory , A CT , Australia. 3 The I nstitute f or T elecommunications Research, the University of South Australia, Adela ide, SA, Australia. 4 Bell Laborator ies, Alcatel-Lucent, Mu rray Hill, NJ, USA. roy .tim o@anu.ed u.au, { alex.gr ant, terence.ch an } @unisa.ed u.au, gk r@research.b ell-labs.com. Abstract — W e consider the problem of source coding with recei ver side in fo rmation for the simple network p roposed by R. Gray and A. W yner in 1974. In this n etwork, a transmitter must reliably transport the output of two correlated informa tion sources to two re ceiv ers usin g three noiseless channels: a pub lic channel which connects th e transmitter to b oth rec eiv ers, and two private channels which connect the transmitter directly to each recei ver . W e extend Gray and W yner’ s o riginal problem by permitting side in fo rmation to b e present at each receiver . W e derive inner and outer bound s for the achi ev able rate region and, fo r th ree special cases, we show that the outer bound is tight. I . I N T RO D U C T I O N The field of network source cod ing is cen tered on the following problem: given a noiseless communica tions network and a set of information sources, what is th e b est way to compress the outpu t of each source for efficient an d r eliable transportatio n over the network ? A solution to this type of problem nee ds to r emove any temporal redu ndancy in each source, exploit any statistical correlatio ns between dif ferent sources and op timize th e use of limited ch annel capacities. In network source co ding, a code is a collection of rules that define ho w th e output o f each source is to be compr essed, transported over the n etwork and reconstructed . A code is said to be r eliable if the output of each source can be reconstruc ted without error at each of its intend ed destinations. The performan ce of a reliable code is measured by the ra tes at which it sends data ov er eac h channel; an optim al cod e will send data at th e smallest rates and thereby consume the lea st network capacity . An ordered collection o f rates (one for eac h channel) is said to be a chievable if th ere exists a reliable co de which operates at these rates. The set of all achiev able rates R is called the achievable rate re g ion of the network, and its lower boun dary R pr ovides a perfor mance benchma rk for the compariso n of reliable co des. NICT A is funded by the Australian Go ve rnment as repre sented by the Departmen t of Broadband, Communicat ions and the Digital E conomy and the Australia n Research Counci l through the ICT Centre of Excellen ce program. The work presented in this paper was undertak en by R. Timo w hile on visit at the Institute for T el ecommunica tions Research, the Uni versi ty of South Australia , and Bell Laboratori es, Alcatel-Luc ent. R. Timo’ s trav el was funded by student trav el schola rships from NICT A, ARC Communicat ions Research Networ k (AC oRN), and the Uni ve rsity of South Australia. The achiev able rate region R is known for a small ad-ho c collection of networks; for mo st “real world” networks, R is un known [1] . W ith the exception of [2 ], achiev able rate regions have been stud ied o n a network- by-n etwork basis; researchers h ave designed an d studied simple networks which isolate particular prob lems of interest. T wo notable examples are: th e separate coding of corr elated sou rces [ 3], an d the sharing of a finite capacity ch annel between multip le users [4]. It is ho ped that solutions to these simple n etworks will yield practical and efficient co des fo r larger n etworks. Discrete Memory less Source { ( X i , Y i ) } Encoder Channel 1 Channel 0 Channel 2 Decoder Decoder { b X i } { b Y i } { U i } { V i } X -Receiv er Y -Recei v er Fig. 1. Figure shows the netw ork source codin g problem proposed by R. Gray and A. W yner [5]. The transmitt er is connected to two recei v ers via three noisel ess channels. The sequences { X i } and { Y i } are to be encode d at the transmitter , transpo rted ov er the network and decode d at the x and y -recei v ers respect i ve ly . In this paper , we study an extension of this problem where “side informati on” { U i } and { V i } are present at each recei ver . These additi onal information sources are marked with dashed lines in the figure. W e study the achiev ab le rate r egion R of the network shown in Figure 1. A transmitter m ust transpor t the o utput of two co rrelated sou rces to two receivers using th ree n oiseless channels: a pu blic ch annel wh ich co nnects the transmitter to both rec eiv er s, and two priv a te chan nels which connect the transmitter directly to each receiver . The achie vable rate region R of this n etwork was found by R. Gra y and A. W yn er [5 ] in 19 74 . They showed that an o ptimal code shou ld endeavor to use the public channel to transpor t inf ormation common to both sources. As we will see, the in tuition of this solu tion is lost when side infor mation is intro duced a t e ach receiver; in particular, it is no t clear how one sh ould decom pose the outp ut of ea ch sou rce f or transmission over the three c hannels. An outline of the p aper is as fo llows. T o fix ideas, we briefly revie w [5] in Section II . In Section III, we for mally define R for the network with side infor mation. In Sections IV and V, we derive outer and inner bou nds for R respectively . In Section VI , we ascerta in R f or one sour ce, a degraded network and a co mplemen tary delivery network resp ectiv ely . Finally , we conclu de the paper in Section VII. I I . T H E G R A Y - W Y N E R P RO B L E M Consider the network (without receiv er side infor mation) shown in Figure 1. W e denote the capacities (in bits p er second) of channels 0 , 1 and 2 by C 0 , C 1 and C 2 respectively . Finally , let X and Y be finite alph abets, and let X n and Y n denote th eir respectiv e n -fold c artesian p rodu ct spaces. Suppose { ( X i , Y i ) } , { ( X i , Y i ); i = 1 , 2 , . . . } is a sequence of independent an d id entically distributed (i.i.d. ) X × Y valued rand om v ar iables emitted by a discrete mem- oryless sou rce Q X Y ( x, y ) = Prob [ X = x, Y = y ] . Suppose further that the ran dom sequence { ( X i , Y i ) } appears at th e transmitter at the rate o f one per second . It is desired that the transmitter deli vers a reliable rep roductio n { b X i } , { b X i ; i = 1 , 2 , . . . } of the sequen ce { X i } to the x -r eceiver , and a reliab le reprod uction { b Y i } , { b Y i ; i = 1 , 2 , . . . } o f th e seq uence { Y i } to the y -re ceiv e r . Assuming n o delay constraints and unlimited computational p ower at the transmitter an d receivers, the ma in prob lem is to ascertain wh ich chann el capacity triples ( C 0 , C 1 , C 2 ) are both necessary and suf ficien t fo r e ach sequence to be reliably transp orted to its inten ded destination . W e assume the classic n -blo ck source coding model where the sequence { ( X i , Y i ) } is parsed and tran sported over the network in m essage block s of leng th n (for some large in teger n ). Let ( X n , Y n ) = ( X 1 , Y 1 ) , ( X 2 , Y 2 ) , . . . , ( X n , Y n ) d enote the message at the transmitter, and let b X n = b X 1 , b X 2 , . . . , b X n and b Y n = b Y 1 , b Y 2 , . . . , b Y n denote the re constructed messages at the x and y -rece iv er s respectively . For each i = 0 , 1 , 2 , let M i = { 1 , 2 , . . . , | M i |} be a finite index set for use on channel i . A network so urce code is a collection of m appings ( e ( n ) , d ( n ) x , d ( n ) y ) , wher e e ( n ) : X n × Y n → M 0 × M 1 × M 2 is th e enco der at the tran smitter; d ( n ) x : M 0 × M 1 → X n is the decod er at the x -recei ver; and d ( n ) y : M 0 × M 2 → Y n is the decod er at the y -receiver . The transmitter encodes the pair ( X n , Y n ) with thr ee indices ( M 0 , M 1 , M 2 ) = e ( n ) ( X n , Y n ) which are sent over chan nels 0 , 1 and 2 respectively . After receiving indices M 0 and M 1 , the x -receiver recon structs b X n = d ( n ) x ( M 0 , M 1 ) . Similarly , after r eceiving indices M 0 and M 2 , the y -receiver reconstructs b Y n = d ( n ) y ( M 0 , M 2 ) . An error is said to occur if e ither b X n 6 = X n or b Y n 6 = Y n , and the code is said to oper ate a t a rate of (1 / n ) log 2 | M i | bits per source symb ol on channel i (for i = 0 , 1 , 2 ) . A triple of rates ( R 0 , R 1 , R 2 ) is said to b e ac hiev able if ther e exists a sequence o f co des { ( e ( n ) , d ( n ) x , d ( n ) y ); n = 1 , 2 , . . . } suc h tha t th e probability o f error appr oaches zero and (1 /n ) log | M i | app roaches R i (for i = 0 , 1 , 2 ) as n go es to infinity . Let R GW denote the set of all ac hiev able rate triple s. I t can be shown th at R GW is a closed co n vex subset of Euclid ean three space, which is completely defined by its lower boundary R GW [5]: R GW ,  ( R 0 , R 1 , R 2 ) ∈ R GW : ( b R 0 , b R 1 , b R 2 ) ∈ R GW , b R i ≤ R i ( i = 0 , 1 , 2) → b R i = R i ( i = 0 , 1 , 2)  . Giv en Q X Y and a network with capa city trip le ( C 0 , C 1 , C 2 ) , the sequ ences { X i } and { Y i } may be reliably re con- structed at the x and y -receivers r espectively if and o nly if ( C 0 , C 1 , C 2 ) lies ab ove R GW ; thu s, R GW defines exactly those ca pacity triples which are b oth n ecessary and sufficient for reliable com municatio n. Gray and W yner [5] showed that to achiev e rates ( R 0 , R 1 , R 2 ) which lie on the lower bou ndary R GW , the ca- pacity of chan nel 0 should be prioritized fo r use by inf ormation common to bo th { X i } and { Y i } . Specifically , they designed a coding scheme which used an auxiliary rando m variable W to rep resent the information transpo rted over ch annel 0 , and they showed any ( R 0 , R 1 , R 2 ) ∈ R GW may be achieved by optimizing over the choice of W . The formal description of R GW in terms of W is as follow . Let W be a finite alp habet of cardinality | W | ≤ | X || Y | + 2 , and let P GW denote the family of p robab ility function s on W × X × Y su ch that P w p ( w, x, y ) = Q X Y ( x, y ) . Now , for each p ∈ P GW , let R ( p ) GW ,    ( R 0 , R 1 , R 2 ) : R 0 ≥ I p ( X, Y ; W ) R 1 ≥ H p ( X | W ) R 2 ≥ H p ( Y | W )    , where I p ( · ; · ) denotes mutual information and H p ( ·|· ) d enotes condition al entropy (with respect to p ). Lemma 1 : [5, Thm. 4] The ac hiev able r ate r egion R GW of th e Gray-W yner Network is g iv en by R GW =   [ p ∈ P GW R ( p ) GW   c , where ( · ) c denotes the set closur e operatio n. It follows from Lemma 1 that R GW is comp letely described by a sing le coding sche me which makes use o f an au xiliary random variable W . As we will see, this coding schem e extends, in a natural w a y , to the network with side information . Unfortu nately , however , this extension do es not appear to completely describe the correspo nding rate region. I I I . E X T E N S I O N T O T H E S I D E I N F O R M AT I O N C A S E Suppose X , Y , U and V ar e finite sets, and le t X n , Y n , U n and V n denote th eir re spectiv e n -fold cartesian pro duct spaces. Sup pose furth er tha t { ( X i , Y i , U i , V i ) } is a sequenc e of i.i.d. X × Y × U × V valued random variables emitted by a discrete memo ryless source Q X Y U V ( x, y , u, v ) = Prob  X = x, Y = y, U = u, V = v  . Finally , for each i = 0 , 1 , 2 , let M i = { 1 , 2 , . . . , | M i |} be a finite index set for ch annel i . As before, a source code is a collection of mapp ings ( e ( n ) , d ( n ) x , d ( n ) y ) , where e ( n ) : X n × Y n → M 0 × M 1 × M 2 is the encoder at the transmitter; d ( n ) x : M 0 × M 1 × U n → X n is the decoder at the x - receiver; and d ( n ) y : M 0 × M 2 × V n → Y n is the decoder at th e y -receiver . Th e transmitter encodes the pair ( X n , Y n ) with ind ices ( M 0 , M 1 , M 2 ) = e ( n ) ( X n , Y n ) which are sent ov e r c hannels 0 , 1 an d 2 respectively . After receiving indices M 0 and M 1 as-well-as side information U n , the x -r eceiv e r reco nstructs b X n = d ( n ) x ( M 0 , M 1 , U n ) . Similarly , after receiving M 0 , M 2 and V n , the y -receiver reconstruc ts b Y n = d ( n ) y ( M 0 , M 2 , V n ) . An erro r occurs if either b X n 6 = X n or b Y n 6 = Y n . Let P e,x , Pro b [ b X n 6 = X n ] , P e,y , Prob [ b Y n 6 = Y n ] and P e , max { P e,x , P e,y } . Definition 1 (A chievable Rate): A rate triple ( R 0 , R 1 , R 2 ) is said to be ach iev able if, for arbitrar y ǫ > 0 and suf- ficiently large n , there exists a cod e ( e ( n ) , d ( n ) x , d ( n ) y ) with parameters ( n, | M 0 | , | M 1 | , | M 2 | , P e ) suc h that P e ≤ ǫ and (1 /n ) lo g | M i | ≤ R i + ǫ for all i = 0 , 1 , 2 . W e let R d enote the set of all achiev able r ate triples. I V . A N O U T E R B O U N D Suppose W is a finite set o f ca rdinality | W | ≤ | X || Y | + 3 and P is the family of pro bability function s on W × X × Y × U × V such that p ( w , x, y , u, v ) = p ( w | x, y ) p ( x, y , u , v ) and Q X Y U V ( x, y , u, v ) = X w ∈ W p ( w, x, y , u, v ) for all p ∈ P . Now , for each p ∈ P let R ( p ) out = n ( R 0 , R 1 , R 2 ) : R 0 ≥ max  I p ( X, Y ; W | U ) , I p ( X, Y ; W | V )  R 0 + R 1 ≥ max  I p ( X, Y ; W | U ) , I p ( X, Y ; W | V )  + H p ( X | W , U ) , R 0 + R 2 ≥ max  I p ( X, Y ; W | U ) , I p ( X, Y ; W | V )  + H p ( Y | W , V ) .            Theor e m 1 (Outer Bound) : If ( R 0 , R 1 , R 2 ) is an achiev- able rate triple, then there exists a p ∈ P such that ( R 0 , R 1 , R 2 ) ∈ R ( p ) out . A. P r oof Outline: Theo r em 1 W e show: if { ( e ( n ) , d ( n ) x , d ( n ) y ) } is a sequ ence of cod es where P e → 0 a s n → ∞ , then there exists a p ∈ P such that ((1 /n ) lo g | M 0 | , (1 /n ) log | M 1 | , (1 /n ) log | M 2 |  ∈ R ( p ) out . Suppose ( e ( n ) , d ( n ) x , d ( n ) y ) is a cod e with ( M 0 , M 1 , M 2 ) = e ( n ) ( X n , Y n ) , b X n = d ( n ) x ( M 0 , M 1 , U n ) and b Y n = d ( n ) y ( M 0 , M 2 , V n ) , then log | M 0 | ≥ H ( M 0 | U n ) ≥ I ( X n , Y n ; M 0 | U n ) = n X i =1 I ( X i , Y i ; M 0 , X i − 1 1 , X i − 1 1 , U i − 1 1 , U n i +1 | U i ) (1) ≥ n X i =1 I ( X i , Y i ; M 0 | U i ) = n X i =1 I ( X i , Y i ; W i | U i ) , (2) where (1) follows beca use { ( X i , Y i , U i , V i ) } is drawn in an i.i.d. fashion and ( 2) follows by setting W i = M 0 . Similarly , log | M 0 | ≥ n X i =1 I ( X i , Y i ; W i | V i ) . (3) On ap plying Fano’ s Inequality [ 6, Pg . 37 ] we get H ( X n | M 0 , M 1 , U n ) ≤ H ( X n | b X n ) ≤ nδ ( P e , n ) , (4) where δ ( P e , n ) , (1 /n ) + P e log | X || Y | . Similar ly , we also have that H ( Y n | M 0 , M 1 , V n ) ≤ nδ ( P e , n ) . Now con sider the series of Shannon (in)equa lities (5) throug h (12). Note, (7) follows because { ( X i , Y i , U i , V ) } is drawn in an i.i.d . fashion and (11) fo llows since M 0  ( X i , Y i )  ( U i , V i ) form s a Markov Chain and (4). From (1 0) and (12), it respectively follows that 1 n  log | M 0 | + log | M 1 |  ≥ 1 n n X i =1 h I ( X i , Y i ; W i | U i ) + H ( X i | W i , U i ) i − δ ( P e , n ) , and 1 n  log | M 0 | + log | M 1 |  ≥ 1 n n X i =1 h I ( X i , Y i ; W i | V i ) + H ( X i | W i , U i ) i − δ ( P e , n ) . Note, (1 /n )[log | M 0 | + log | M 2 | ] may be bound in a similar manner . Following the time sharing principle given in [5, Pg. 1709], we may now co nstruct a p ∈ P suc h that each inequality in the theo rem holds as n → ∞ and P e → 0 . Finally , we m ay bou nd the cardin ality o f the au xiliary random variable W using th e su pport lemma o f Ahlswede and K ¨ orner [ 7, L emma 3]. V . A N I N N E R B O U N D A n atural extension of th e code p roposed by Gray and W yner [5] y ields the following inner bound for R . Let W and P be defined as in Section III. For p ∈ P , let R ( p ) in = n ( R 0 , R 1 , R 2 ) : R 0 ≥ max  I p ( X, Y ; W | U ) , I p ( X, Y ; W | V )  R 1 ≥ H p ( X | W , U ) , R 2 ≥ H p ( Y | W , V ) .    , and R in =  ∪ p ∈ P R ( p ) in  c . Theor e m 2: R ⊇ R in . Remark 1 : If U = V , then R = R in . Remark 2 : Suppose ( X , Y )  U  V forms a Markov chain. It can be shown that a sum rate R 0 + R 1 + R 2 is achievable if and only if R 0 + R 1 + R 2 ≥ H ( Y | V ) + H ( X | Y , U ) . (See [8] for the sp ecial case wh ere V = con stant.) W e may set W = Y in Th eorem 2 to achieve this sum rate. Remark 3 : Suppose X = Y . Sgarro [9] showed that the sum rate R 0 + R 1 + R 2 is achievable if a nd on ly if R 0 + R 1 + log | M 0 | + lo g | M 1 | ≥ H ( M 0 , M 1 ) = H ( M 0 , M 1 | U n ) + I ( M 0 , M 1 ; U n ) (5) ≥ I ( X n , Y n ; M 0 , M 1 | U n ) + I ( M 0 , M 1 ; U n ) (6) = n X i =1  I ( X i , Y i ; M 0 , M 1 , X i − 1 1 , Y i − 1 1 , U i − 1 1 , U n i +1 | U i ) + I ( U i ; M 0 , M 1 , U i − 1 1 )  (7) ≥ n X i =1  I ( X i , Y i ; M 0 , M 1 , U i − 1 1 , U n i +1 | U i ) + I ( U i ; M 0 )  (8) = n X i =1  I ( X i , Y i ; M 0 | U i ) + I ( X i , Y i ; M 1 , U i − 1 1 , U n i +1 | M 0 , U i ) + I ( U i ; M 0 )  (9) ≥ n X i =1  I ( X i , Y i ; M 0 | U i ) + I ( X i ; M 1 , U i − 1 1 , U n i +1 | M 0 , U i ) + I ( U i ; M 0 )  (10) = n X i =1  I ( X i , Y i ; M 0 | V i ) + H ( X i | M 0 , U i ) − nδ ( P e , n )  (11) = n X i =1  I ( X i , Y i ; W i | V i ) + H ( X i | W i , U i ) − nδ ( P e , n )  (12) R 2 ≥ max { H ( X | U ) , H ( X | V ) } . W e may set W = X = Y in Theorem 2 to achieve th is su m rate . Remark 4 : Suppose U = Y and V = X . W yner et. al. [4] showed that the sum rate R 0 + R 1 + R 2 is achiev able if and only if R 0 + R 1 + R 2 ≥ max { H ( X | Y ) , H ( Y | X ) } . W e may set W = ( X , Y ) in Theorem 2 to achieve this sum rate. Remark 5 : The code, which yields the ach iev ability of R in , is essentially a version o f Heegard and Berger’ s “triple rate split code” given in [1 0, Thm. 2]. Indee d, we note that the problem of minimizing the sum rate R 0 + R 1 + R 2 is a special case of the two receiver gener alized Kaspi- Heegard-Berger problem [10, Sec. VII]. A. P r oof Outline: Theo r em 2 1) Code Constructio n: Suppose p ∈ P . Let R ′ 0 , R ′ 1 and R ′ 2 be non-n egati ve integers whose v alu es will be chosen later . Generate 2 nR ′ 0 indepen dent w -codewords o f leng th n by cho osing symbols i.i.d. from W accordin g to p W (the W - marginal o f p ). Label the re sulting code book with the in dex m ′ 0 : C W , { w n ( m ′ 0 ) : 1 ≤ m ′ 0 ≤ 2 nR ′ 0 } . Similarly , gener ate 2 nR ′ 1 and 2 nR ′ 2 indepen dent x an d y - codewords u sing p X and p Y respectively: C X , { x n ( m ′ 1 ) : 1 ≤ m ′ 1 ≤ 2 nR ′ 1 } , and C Y , { y n ( m ′ 2 ) : 1 ≤ m ′ 2 ≤ 2 nR ′ 2 } . Uniformly at ran dom assign to e ach w n ∈ C W a “bin label” from the set M 0 = { 1 , 2 , . . . , 2 ⌊ nR 0 ⌋ } , and let h W : C W → M 0 denote the indu ced mapping. Let B W ( m 0 ) den ote th e set of w -codewords with bin label m 0 : B W ( m 0 ) , { w n ∈ C W : h W ( w n ) = m 0 } , and let B W denote the collection of all w - bins. In the same way , assign one of 2 ⌊ nR 1 ⌋ and 2 ⌊ nR 2 ⌋ bin labels to each x a nd y - codeword, and define h X , h Y , B X and B Y . 2) Encoding : The enco der assumes the messages x n , y n , u n and v n emitted by the source are ǫ -stro ng joint typ ical; that is, ( x n , y n , u n , v n ) ∈ A ∗ ( n ) ǫ ( p X Y U V ) . Let E 1 denote the ev en t where this assum ption is false. Then [6 , Lem . 10 .6.1] Pr  E 1  ≤ ǫ 1 ( n, X × Y × U × V ) , (13) where ǫ 1 ( n, X × Y × U × V ) → 0 in n for fixed ǫ > 0 . The transmitter looks for a w n ( m ′ 0 ) ∈ C W which is ǫ -strong joint typical with ( x n , y n ) . If two-o r-more such co dew o rds exist, the tra nsmitter selects the codew o rd with the smallest index. I f no such co dew o rd exists, a n erro r is declared and the transmitter arb itrarily selects some w n e ( m ′ 0 ) ∈ C W . Let E 2 denote th is er ror e vent. Then [ 6, L em. 1 0.6.2 ], Pr  E 2  ≤ e − “ 2 nR ′ 0 2 − n ( I ( X,Y ; W )+ ǫ 2 ) ” , (14) where ǫ 2 → 0 as ǫ → 0 and n → ∞ . W e assume R ′ 0 ≥ I ( X, Y ; W ) + ǫ 2 , so that Pr [ E 2 ] → 0 as ǫ → 0 an d n → ∞ . After the transmitter selects w n ( m ′ 0 ) ∈ C W it sends the index m 0 = h W ( w n ( m ′ 0 )) o n ch annel 0 . The tr ansmitter loo ks fo r a x n ( m ′ 1 ) ∈ C X such th at x n ( m ′ 1 ) = x n . I f two-o r-more such codewords exist, the transmitter selects th e codeword with the smallest index. If no such codeword exists, an er ror is declar ed and the tra nsmitter arbitrarily selects some x n e ( m ′ 1 ) ∈ C X . Let E 3 ,x denote th is error event. Then, Pr  E 3 ,x  ≤ e − “ 2 nR ′ 1 2 − n ( H ( X )+ ǫ 3 ,x ) ” , (15) where ǫ 3 ,x → 0 a s ǫ → 0 an d n → ∞ . Cho ose R ′ 0 ≥ H ( X ) + ǫ 3 ,x arbitrarily , so that Pr [ E 3 ,x ] → 0 as ǫ → 0 and n → ∞ . The transmitter en code y n is a similar fashion, and sends m 1 = h X ( x n ( m ′ 1 )) and m 2 = h Y ( y n ( m ′ 2 )) on chann els 1 and 2 respectively . 3) Deco ding: Given m 0 and u n , the X -receiver looks for a uniqu e b w n ∈ B W ( m 0 ) which is jo intly typical with u n . If no such c odeword can be foun d, an error is declared and the decoder arbitrarily selects some b w n e ∈ B W ( m 0 ) . Let • E 4 ,x : the codeword w n ( m ′ 0 ) chosen b y th e transmitter is not jo intly ty pical w ith u n , and • E 5 ,x : th ere are two-or -more w -codewords in B W ( m 0 ) which are jointly typical with u n . Consider E 4 ,x . Since W  ( X , Y )  U forms a Markov Chain u nder p , we have that [6 , Lem . 15 .8.1] Pr  E 4 ,x  ≤ ǫ 4 ,x , (16) where ǫ 4 ,x → 0 as n → ∞ . Now consider E 5 ,x . W e have that u n ∈ A ∗ ( n ) ǫ ( P U ) . As before, the proba bility that a rando mly g enerated w -cod ew o rd is jointly typical with u n is upp er b ound by 2 − n ( I ( W ; U )+ ǫ 5 ,x ) , where ǫ 5 ,x → 0 as n → ∞ . Moreover , the numb er of codewords in each bin is at m ost 2 n ( R ′ 0 − R 0 ) + ǫ 5 ′ ,x , where ǫ 5 ′ ,x → 0 as n → ∞ [ 11, Pg. 276 6]. Hence, Pr  E 5 ,x  ≤ 2 − n ( R 0 − R ′ 0 + I ( W ; U ) − ǫ 5 ,x ) + ǫ 5 ′ ,x . W e need R 0 − R ′ 0 + I ( W ; U ) − ǫ 5 ,x ≥ 0 , so that Pr  E 5 ,x  → 0 as n → ∞ . This req uires R 0 ≥ R ′ 0 − I ( W ; U ) + ǫ 5 ,x ≥ I ( X, Y ; W ) − I ( W ; U ) + ǫ 2 + ǫ 5 ,x (17) = I ( X, Y , U ; W ) − I ( W ; U ) + ǫ 2 + ǫ 5 ,x (18) = I ( X, Y ; W | U ) + ǫ 2 + ǫ 5 ,x , (19) where (17) follows becau se we selected R ′ 0 ≥ I ( X , Y ; W ) + ǫ 2 , (18) f ollows because W  ( X , Y )  U forms a Ma rkov Chain, and (1 9) follows from the chain rule for mutual informa tion. Similarly , the y -receiver will correctly find a w - codeword with high pr obability if R 0 ≥ I ( X, Y ; W | V ) + ǫ 2 + ǫ 5 ,y , where ǫ 5 ,y → 0 as ǫ → 0 and n → 0 . Giv en b w n , m 1 and u n , the X -r eceiv er looks fo r a un ique b x n ∈ B X ( m 1 ) whic h is jointly typical with b w n and u n . I f there exists two-or - more su ch codewords, an error is declared and the d ecoder arb itrarily selects some b x n e ∈ B X ( m 1 ) . Let E 6 ,x denote th is er ror e vent. It fo llows that Pr  E 6 ,x  ≤ 2 − n ( R 1 − R ′ 1 + I ( X ; W,U ) − ǫ 6 ,x ) + ǫ 6 ′ ,x , (20) where ǫ 6 ,x → 0 and ǫ 6 ′ ,x → 0 as ǫ → 0 and n → ∞ . If R 1 ≥ H ( X | W , U ) + ǫ 6 ,x it follows fro m (2 0) that Pr [ E 6 ,x ] → 0 as n → ∞ . Similar ly , the y -receiver will correctly find b y n with high prob ability if R 2 ≥ H ( Y | W, V ) + ǫ 6 ,y , where ǫ 6 ,y → 0 as ǫ → 0 and n → 0 . V I . T H R E E S I M P L E N E T W O R K S A. T wo Descriptions of R when X = Y Let P and R ( p ) out be defined as in Section I V. Theor e m 3: If X = Y , then R =  ∪ p ∈ P R ( p ) out  c . Now suppose A and B are finite sets of cardinalities | A | ≤ | X | + 1 and | B | ≤ | X | + 1 . Let P ∗ denote the family o f probab ility functions on A × B × X × U × V such th at p ( a, b, x, u, v ) = p ( a, b | x ) p ( x, u, v ) and Q X U V ( x, u, v ) = X ( a,b ) ∈ A × B p ( a, b, x, u, v ) . For each p ∈ P ∗ , let R ( p ) ∗ = n ( R 0 , R 1 , R 2 ) : R 0 ≥ max  H p ( X | A, U ) , H p ( X | B , V )  R 1 ≥ I p ( X ; A | U ) , R 2 ≥ I p ( X ; B | V ) .    . Theor e m 4: If X = Y , then R =  ∪ p ∈ P ∗ R ( p ) ∗  c . B. R for a T yp e of Degr a ded Network Let P and R ( p ) out be defined as in Section I V. Theor e m 5: If Y = ( X , Z ) and ( X , Z )  U  V form s a Markov Chain, then R =  ∪ p ∈ P R ( p ) out  c . C. R for a Comp lementary Delivery Network Let P and R ( p ) out be defined as in Section I V. Theor e m 6: If U = Y and V = X , then R =  ∪ p ∈ P R ( p ) out  c . Now suppose A and B are finite sets of cardinalities | A | ≤ | X || Y | + 1 an d | B | ≤ | X || Y | + 1 . Let P ∗∗ denote the set of p robab ility functions on A × B × X × Y such that Q X Y ( x, y ) = X ( a,b ) ∈ A × B p ( a, b, x, y ) is tru e fo r all ( x, y ) and p ∈ P ∗∗ . For each p ∈ P ∗∗ , let R ( p ) ∗∗ = n ( R 0 , R 1 , R 2 ) : R 0 ≥ max  H p ( X | A, Y ) , H p ( Y | B , X )  R 1 ≥ I p ( X ; A | Y ) , R 2 ≥ I p ( Y ; B | X ) .    . Theor e m 7: If U = Y and V = X , then R =  ∪ p ∈ P ∗∗ R ( p ) ∗∗  c . V I I . C O N C L U S I O N W e inv e stigated the achiev ab le rate region R of a simple network with side information present at e ach receiver . Ou r first theo rem g av e an outer bo und which , for three simple networks, was sho wn to be equ al to R . Our second result gave an in ner boun d which was obtained via an extension of the co ding the orem given by Gray and W y ner [ 5]. R E F E R E N C E S [1] M. Effro s , “Networ k Source Codi ng: A Perspect i ve, ” IE EE Inform. Theory Society Newslett er , vol. 57, no. 4, pp. 15–23, December 2007. [2] I. Csiszar and J. K ¨ orner , “T owards a General Theory of Source Net- works, ” IEEE T ransactio ns on Informati on Theory , vol. 26, no. 2, pp. 155–165, March 1980. [3] D. Slepian and J. W olf, “Noiseless Coding of Correlate d Sources, ” IEEE T ransacti ons on Information Theory , vol. 19, no. 4, pp. 471–4 80, July 1973. [4] A. W yner , J. W olf, and F . Wil lems, “Communicatin g Via Processing Broadca s t Satelli te, ” IEEE T ransacti ons on Information T heory , vol. 48, no. 6, pp. 1243–1249, June 2002. [5] R. Gray and A. W yner , “Source Coding for a Simple Network, ” B ell System T echni cal J ournal , v ol. 53, no. 9, pp. 1681–1721, Nov . 1974. [6] T . Cove r and J. Thomas, Elements of Information Theory , 2nd ed. W iley , 2006. [7] R. F . Ahlswede and J. K ¨ orner , “Source Coding with Side Information and a Con verse for Degraded Broadcast Channels, ” IEEE T ransactions on Informatio n Theory , vol. 21, no. 6, pp. 629–637, Nove m ber 1975. [8] R. Timo, A. Grant, and L. Hanlen, “Source Coding for a Noiseless Broadca s t Channel with Parti al Recei ver Side Information, ” in Pro- ceedi ngs IEEE Australian Communicat ions Theory W orkshop, AusCTW , Adelaid e, Australia, February 2007. [9] A. Sgarro, “Source Coding with Side Information at Sev eral Decode rs, ” IEEE T ransactions on Informati on T heory , vol. 23, no. 2, pp. 179–182, March 1977. [10] C. Heega rd and T . Berge r, “Rate Distortion when Side Information May Be Absent, ” IE EE T ransaction s on Information Theory , vol. 31, no. 6, pp. 727–734, Nov ember 1985. [11] M. Gastpar , “The Wyner Ziv Problem With Multip le Sources, ” IEEE T ransacti ons on Information Theory , vol. 50, no. 11, pp. 2762–276 8, Nov ember 2004.