On the inner and outer bounds for 2-receiver discrete memoryless broadcast channels

1 On the inner a nd outer bounds f or 2-recei v er discrete memoryle ss broadcast channels Chandra Nair , CUHK and V incent W ang Zizhou , CUHK Abstract — W e study the best known general inner bound[1] and outer bound[2] fo r the capacity region of the two user discrete mem ory less channel. W e prov e that a seemingly stro nger outer bound is identical to a weaker form of the outer boun d that was also presented in [2]. W e are able to further expr ess the best outer bound in a fo rm that is computable, i.e. th ere a re bounds on the cardin alities of t he auxili ary random variables. The inner and outer bounds coincide for all ch annels fo r which the capacity re gion is known and i t is not kn own whether the regions described by th ese bounds are same or different. W e present a channel, where assuming a certain conjecture backed by simulations and partial theoretical re sults, one can show that the bounds are different. I . I N T R O D U C T I O N In [3], Cover in troduce d the no tion of a br oadcast chann el throug h which one sender transmits informa tion to two or more receivers. For the pur pose of this pap er we fo cus our attention on b roadcast channels with precisely two r eceiv ers. Definition: A br oad cast channe l (BC) consists of an inp ut alphabet X and o utput alph abets Y 1 and Y 2 and a probability transition function p ( y 1 , y 2 | x ) . A ((2 nR 1 , 2 nR 2 ) , n ) code for a broa dcast c hannel consists of an encoder x n : 2 nR 1 × 2 nR 2 → X n , and two decod ers ˆ W 1 : Y n 1 → 2 nR 1 ˆ W 2 : Y n 2 → 2 nR 2 . The pro bability of error P ( n ) e is defined to b e the pro bability that the decoded message is not equal to the transmitted message, i.e., P ( n ) e = P  { ˆ W 1 ( Y n 1 ) 6 = W 1 } ∪ { ˆ W 2 ( Y n 2 ) 6 = W 2 }  where the message is assumed to be unif ormly distrib uted over 2 nR 1 × 2 nR 2 . A rate p air ( R 1 , R 2 ) is said to be a chievable for the broad- cast ch annel if th ere exists a sequen ce of ((2 nR 1 , 2 nR 2 ) , n ) codes with P ( n ) e → 0 . The capacity r e gion of the broa dcast channel with is the closure of th e set of achievable rates. The capacity r e gion of the two u ser discr e te memoryless chan nel is unkno wn. The cap acity region is known for lots o f special cases such as d egraded, less noisy , m ore capable, determin istic, semi- deterministic, etc. - see [4] and the ref erences therein. General inner a nd o uter b ounds for the two-user discrete memory less broadc ast channe l have also been known in liter- ature. Her e we state th e b est known inner and ou ter bou nds for the region from the litera ture. Bound 1: [M ¨ arton ’ 79] Th e f ollowing rate p airs ar e ach iev- able: R 1 ≤ I ( U, W ; Y 1 ) R 2 ≤ I ( V , W ; Y 2 ) R 1 + R 2 ≤ min { I ( W ; Y 1 ) , I ( W ; Y 2 ) } + I ( U ; Y 1 | W ) + I ( V ; Y 2 | W ) − I ( U ; V | W ) for any p ( u, v , w , x ) suc h that ( U, V , W ) → X → ( Y 1 , Y 2 ) form a Markov chain. Bound 2: [N air-El Gamal ’07 ] The r egion R defined by the union over the rate pairs satisfying R 1 ≤ I ( U, W ; Y 1 ) R 2 ≤ I ( V , W ; Y 2 ) R 1 + R 2 ≤ min { I ( U, W ; Y 1 ) + I ( V ; Y 2 | U, W ) , I ( V , W ; Y 2 ) + I ( U ; Y 1 | V , W ) } over all p ( u ) p ( v ) p ( w, x | u, v ) such that ( U, V , W ) → X → ( Y 1 , Y 2 ) fo rm a M arkov chain forms an outer bo und to th e capacity region. Remark 1: Both the boun ds are tight for all the special classes of two-user broad cast channels f or whic h the capacity region is kn own. Howe ver , since the bo unds are difficult to ev alu ate in general it is not known whether the tightness of these bound s is specific to the scenario s or wh ether they coincide yielding the ca pacity region. A possibly weaker for m of the outer b ound was also presented in [2] by rem oving th e indep endence between U and V . Under th is relaxation we have the following: Bound 3: [N air-El Gamal ’07 ] Th e region R 1 defined by the union over the rate pairs satisfying R 1 ≤ I ( U ; Y 1 ) R 2 ≤ I ( V ; Y 2 ) R 1 + R 2 ≤ min { I ( U ; Y 1 ) + I ( V ; Y 2 | U ) , I ( V ; Y 2 ) + I ( U ; Y 1 | V ) } over all p ( u, v , x ) such that ( U, V ) → X → ( Y 1 , Y 2 ) fo rm a Markov chain constitutes an outer bou nd to the capacity region. One of the main r esults of th e p aper is the following: The r egions described by Boun ds 2 a nd 3 a r e iden tical. The organization of the paper is a s follows. In Section II we show that the region s described by Bound 2 and Bound 3 2 are the same. W e also pr esent a dif ferent representation of the the bound which allows us to have bo unds o n the ca rdinalities of the au xiliary rand om variables. In Section III we study the binary skew-symmetric ch annel [5] an d conjectu re th at the inner and o uter bounds are different for this chann el. I I . O N E V A L UAT I O N O F T H E O U T E R B O U N D A. Id entity of the bo unds Theor em 1: Th e r egions R and R 1 coincide, i.e. R = R 1 . Pr oof: Clearly , by setting U ′ = ( U, W ) an d V ′ = ( V , W ) , we have th at R ⊆ R 1 . T herefor e it suf fices to show that R 1 ⊆ R . The idea of the p roof 1 is a s f ollows: Giv en a ( U, V ) we will prod uce a ( U ∗ , V ∗ , W ∗ ) with U ∗ , V ∗ being indepen dent such that I ( U ; Y 1 ) = I ( U ∗ , W ∗ ; Y 1 ) I ( V ; Y 2 ) = I ( V ∗ , W ∗ ; Y 2 ) I ( U ; Y 1 | V ) = I ( U ∗ ; Y 1 | V ∗ , W ∗ ) (1) I ( V ; Y 2 | U ) = I ( V ∗ ; Y 2 | U ∗ , W ∗ ) . Let ( U, V , X ) be a triple such that ( U, V ) → X → ( Y 1 , Y 2 ) form a Markov chain . L et V = { 0 , 1 , ..., m − 1 } . Define n ew random variables U ∗ , V ∗ , W ∗ and a distribution p ( u ∗ , v ∗ , w ∗ , x ) accor ding to P( U ∗ = u, V ∗ = i , W ∗ = j, X = x ) = 1 m P( U = u, V = ( i + j ) m , X = x ) , where ( · ) m denotes the mo d oper ation. It is straigh tforward to check the following: P( U ∗ = u , V ∗ = i ) = 1 m P( U = u ) and hence in depend ent , P( U ∗ = u, W ∗ = i , X = x ) = 1 m P( U = u, X = x ) , P( V ∗ = i , W ∗ = j, X = x ) = 1 m P( V = ( i + j ) m , X = x ) . From the above it follows in a straig htforward manner th at (1) hold s and thus completes the pr oof. B. An a lternate characterization W e reprodu ce so me of the arguments in [2] to express the Bound 3 in an alternate manner to aid its ev alua tion. Lemma 1: The region R 1 is eq uiv ale nt to the follo wing region, R 2 , defined b y the union of rate pairs satisfying R 1 ≤ I ( U ; Y 1 ) R 2 ≤ I ( V ; Y 2 ) R 1 + R 2 ≤ min { I ( U ; Y 1 ) + I ( X ; Y 2 | U ) , I ( V ; Y 2 ) + I ( X ; Y 1 | V ) } over all p ( u , v , x ) such that ( U, V ) → X → ( Y 1 , Y 2 ) 1 The idea of the construc tion is motiv ated in part by a s imilar construction (2) original ly appe aring in [2] and al so from a c on versation with Prof. Hajek about the tight ness of Bound 2 for the deterministi c broadcast chan nel. Pr oof: Since ( U, V ) → X → ( Y 1 , Y 2 ) form a Markov chain, we hav e I ( U ; Y 1 | V ) ≤ I ( X ; Y 1 | V ) and I ( V ; Y 2 | U ) ≤ I ( X ; Y 2 | U ) . Therefo re it is c lear that R 1 ⊆ R 2 . Hence it suffices to sh ow that R 2 ⊆ R 1 . Let l den ote the size o f X . Given ( U, V , X ) it was shown in [ 2] that fo r the fo llowing triple ( U ∗ , V ∗ , X ) , having cardi- nalities l kU k , l kV k , l respectively , defined accor ding to P( U ∗ = u i , V ∗ = v j ) = 1 l P( U = u, V = v , X = ( i − j ) l ) , P( X ∗ = k | U ∗ = u i , V ∗ = v j ) =  1 if k = ( i − j ) l 0 otherwise, (2) one obtain s I ( U ; Y 1 ) = I ( U ∗ ; Y 1 ) I ( V ; Y 2 ) = I ( V ∗ ; Y 2 ) (3) I ( X ; Y 1 | V ) = I ( X ; Y 1 | V ∗ ) = I ( U ∗ ; Y 1 | V ∗ ) I ( X ; Y 2 | U ) = I ( X ; Y 2 | U ∗ ) = I ( V ∗ ; Y 2 | U ∗ ) . Thus R 2 ⊆ R 1 . C. Car dinality bou nds Using the streng thened Carath ´ eodo ry theorem by Fenchel and Egg leston [6] it can be r eadily shown that for any choice o f the auxiliar y rand om variable U , th ere e xists a random variable U 1 with car dinality bo unded by kX k + 1 such that I ( U ; Y 1 ) = I ( U 1 ; Y 1 ) , I ( X ; Y 2 | U ) = I ( X ; Y 2 | U 1 ) an d preserves the d istribution p ( X ) . Similarly on e can find a V 1 with cardinality bounded by kX k + 1 such that I ( V ; Y 2 ) = I ( V 1 ; Y 2 ) , I ( X ; Y 1 | V ) = I ( X ; Y 1 | V 1 ) and preserves the distri- bution p ( X ) . Since both U 1 and V 1 share the sam e distribution p ( X ) one can create a triple ( U 1 , V 1 , X ) (f or e.g. by generating U 1 and V 1 condition ally indepen dent o f X ) . Thus one can assume with out loss of generality th at the cardinalities of U , V in Lemma 1 a re bound ed by k X k + 1 each. D. An o uter bou nd formu lation that can b e evaluated Putting all of these togeth er we h av e th e following ch arac- terization of the Bou nd 2. Bound 4: Th e region R co nsists of the union of rate pairs satisfying R 1 ≤ I ( U ; Y 1 ) R 2 ≤ I ( V ; Y 2 ) R 1 + R 2 ≤ min { I ( U ; Y 1 ) + I ( X ; Y 2 | U ) , I ( V ; Y 2 ) + I ( X ; Y 1 | V ) } over a ll p ( u , v , x ) such that ( U, V ) → X → ( Y 1 , Y 2 ) and constitutes the Bou nd 2. Further, on e can assume th at kU k , kV k ≤ kX k + 1 . Alternately one can also use construction (2) to restrict X to be a determ inistic functio n of U, V wh ile relax ing the cardinalities to kU k , kV k ≤ kX k ( kX k + 1) . 3 I I I . T H E B I N A RY S K E W - S Y M M E T R I C C H A N N E L A. On evalua ting M ¨ arton inner bou nd W e consider the f ollowing ch annel [ 5] called the Binary ske w-symmetric channel, BSSC. For ease we restrict ourselves to the case p = 1 2 . Z Y P S f r a g r e p l a c e m e n t s X Y 1 Y 2 p p 1 − p 1 − p 0 0 0 1 1 1 Fig. 1. Binary Sk e w Symmetric Channel Remark 2: The channel, BSSC, has already appeared in a couple of instances to p roduce the fo llowing surprising results: • In [5] BSSC was used to show that u sing the auxiliar y random variable W in the Cover - van der M eulen achiev- able region, even in the absenc e of rate R 0 (commo n informa tion), enha nced the achiev able r egion. • In [2] BSSC was used to show that an ou ter bound to 2-user bro adcast chann el by K o rner and M ¨ arton [1] was not tight and that the region prescribe d by Theor em 2 was strictly containe d in side the Korner-M ¨ arto n region. Backed b y numerica l simulation s we make the following conjecture abou t th e BSSC with p = 1 2 . Conjectur e 1: L et ( U, V ) be auxiliary ran dom variables such that ( U, V ) → X → ( Y 1 , Y 2 ) form a Mar kov chain. Then the following holds: I ( U ; Y 1 ) + I ( V ; Y 2 ) − I ( U ; V ) ≤ max { I ( X ; Y 1 ) , I ( X ; Y 2 ) } . Remark 3: It is easy to see that this conjecture implies that Marton’ s b ound witho ut the ra ndom variable W reduces to the time-division r egion. When U and V a re indep endent, this co njecture has b een established in the appendix o f [5]. In this paper, we shall establish the validity of the con jecture for some range s of P( X = 0) . By symmetry of BSSC the maximum of the t erm I ( U ; Y 1 ) + I ( V ; Y 2 ) − I ( U ; V ) is same for P( X = 0) = η an d P( X = 0) = 1 − η and hence it suffices to consider η in the range 0 ≤ η ≤ 1 2 . Observe th at I ( U ; Y 1 ) + I ( V ; Y 2 ) − I ( U ; V ) , ≤ I ( V ; Y 2 ) + I ( U ; Y 1 , V ) − I ( U ; V ) , = I ( V ; Y 2 ) + I ( U ; Y 1 | V ) , ≤ I ( V ; Y 2 ) + I ( X ; Y 1 | V ) , = I ( X ; Y 1 ) + I ( V ; Y 2 ) − I ( V ; Y 1 ) . Figure 2 plots H ( Y 1 ) − H ( Y 2 ) an d the line 2 η − 1 as a function of P( X = 0) = η . Let f ( η ) = H ( η 2 ) − H ( 1 − η 2 ) , 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 Fig. 2. The pl ot of the func tion f ( η ) = H ( η 2 ) − H ( 1 − η 2 ) . where H ( · ) d enotes th e binary entr opy functio n. Then it is easy to see that f ( η ) is concave in 0 ≤ η ≤ 1 2 and conve x in the remaining region, 1 2 ≤ η ≤ 1 . Suppose that P ( X = 0) = η and we seek the V that maximizes I ( V ; Y 2 ) − I ( V ; Y 1 ) subje ct to V → X → ( Y 1 , Y 2 ) being Markov and P( X = 0) = η . Then it is not diffi cult to see that the optima l choice would be to set V = Φ ( the trivial random variable) fo r all η ≤ η 0 = 1 5 where η 0 is the unique point in [0 , 1 2 ] at which the line joining ( η 0 , f ( η 0 )) to th e po int (1 , 1) is a tangent to the cur ve f ( η ) . Lemma 2: Let P( X = 0) = η ≤ η 0 where η 0 = 1 5 is the unique solution of the e quation f ′ ( η ) = 1 − f ( η ) 1 − η . or in other w ords the point at which the line joining ( η 0 , f ( η 0 )) to the po int (1 , 1) is a tangent to the cur ve f ( η ) . Then for all V → X → ( Y 1 , Y 2 ) we ha ve I ( V ; Y 2 ) ≤ I ( V ; Y 1 ) . Pr oof: Define g ( η ) as follows: g ( η ) = ( f ( η ) 0 ≤ η ≤ η 0 1 − η 1 − η 0 f ( η 0 ) + η − η 0 1 − η 0 f (1) η 0 ≤ η ≤ 1 . Observe that g ( η ) is concave and th at f ( η ) ≤ g ( η ) , 0 ≤ η ≤ 1 . Let P( V = i ) = v i and P( X = 0 | V = i ) = α i . W e ha ve P i v i α i = η . Observe that we have the following, I ( V ; Y 2 ) − I ( V ; Y 1 ) = H ( Y 1 | V ) − H ( Y 2 | V ) − ( H ( Y 1 ) − H ( Y 2 )) = X i v i f ( α i ) − f ( η ) ≤ X i v i g ( α i ) − f ( η ) ( a ) ≤ g ( X i v i α i ) − f ( η ) = g ( η ) − f ( η ) = 0 as 0 ≤ η ≤ η 0 . Here ( a ) follows fro m the con cavity o f g ( η ) . This comp letes the proof of Lem ma 2. 4 This implies that for η ≤ η 0 = 1 5 , we have I ( U ; Y 1 ) + I ( V ; Y 2 ) − I ( U ; V ) , ≤ I ( X ; Y 1 ) + I ( V ; Y 2 ) − I ( V ; Y 1 ) , ≤ I ( X ; Y 1 ) + 0 , = I ( X ; Y 1 ) . Further u sing the symmetry o f BSSC and th e fact that the maximum o f I ( U ; Y 1 ) + I ( V ; Y 2 ) − I ( U ; V ) is same fo r P( X = 0) = η or 1 − η , we ha ve the following result. Lemma 3: Conjecture 1 is tr ue a s lo ng as max { P( X = 0) , P( X = 1) } ≤ η 0 = 1 5 . Assuming Con jecture 1 is true we can now analyze the sum rate of the Marton in ner boun d with the ran dom variable W . Theorem 1 implies R 1 + R 2 ≤ min { I ( W ; Y 1 ) , I ( W ; Y 2 ) } + I ( U ; Y 1 | W ) + I ( V ; Y 2 | W ) − I ( U ; V | W ) . Let W 0 = { w : P( X = 0 | W = w ) ≤ 0 . 5 } a nd W 1 = { w : P( X = 0 | W = w ) > 0 . 5 } . Let T be a function of W defined by T = ( 0 if w ∈ W 0 1 if w ∈ W 1 . W e h av e the following b ound on the sum rate R 1 + R 2 ≤ min { I ( W , T ; Y 1 ) , I ( W , T ; Y 2 ) } + I ( U ; Y 1 | W , T ) + I ( V ; Y 2 | W , T ) − I ( U ; V | W, T ) ( a ) ≤ min { I ( W , T ; Y 1 ) , I ( W , T ; Y 2 ) } + P( T = 0 ) I ( X ; Y 1 | W , T = 0 ) + P( T = 1 ) I ( X ; Y 2 | W , T = 1 ) ( b ) ≤ min { I ( T ; Y 1 ) , I ( T ; Y 2 ) } + P( T = 0 ) I ( X ; Y 1 | T = 0) + P( T = 1 ) I ( X ; Y 2 | T = 1) . Here ( a ) fo llows from Conjecture 1 an d ( b ) follows from th e fact that P( T = 1) I ( W ; Y 1 | T = 1) ≤ P( T = 1) I ( W ; Y 1 | T = 1) , P( T = 0) I ( W ; Y 2 | T = 0) ≤ P( T = 0) I ( W ; Y 1 | T = 0) . In [2] the boun d o n sum rate, min { I ( T ; Y 1 ) , I ( T ; Y 2 ) } + P( T = 0) I ( X ; Y 1 | T = 0) + P( T = 1) I ( X ; Y 2 | T = 1) has been studied and th e maximum was evaluated as ≈ 0 . 3 616 . This cou ld also b e inferr ed from [5] and the ev aluation of the Cover - van-der-Meulen region for this ch annel. Thus assuming Conjectur e 1 we have that the sum r ate of the M ¨ arton in ner bou nd is b ounde d by 0 . 361 6 ... (corr ect to 4 decimal places). B. Eva luating outer bou nd - BSSC In [2] the sum rate o f the pairs ( R 1 , R 2 ) descr ibed by Bound 4 was e valuated a nd it was sh own tha t the maximum sum ra te was bo unded by 0 . 3711 .. (correct to 4 de cimal places). Thus we h ave tha t the r egion described b y Bo und 2 is strictly lar ger than that described by Boun d 1(assuming Conjectur e 1) and thus the inn er and outer bou nds differ fo r BSSC. I V . C O N C L U S I O N In this paper, we study the inner and ou ter bound s f or the 2-u ser discrete m emoryless bro adcast chann el. W e p rove that f or the purp ose of evaluating the ou ter bo und the region described by a weaker version ( which is easier to ev alu ate) indeed coincides with a stro nger version. The bounds matched for all the special classes of chann els for which the capacity was k nown. It is n ot known if the bound s were inhere ntly different o r not. W e then stud ied th e bound s fo r the particu lar case of the binary ske w symmetric channel (BS SC). W e present a conjecture that, if proved, would establish that the in ner and the outer bound s are indeed not tight for BSSC. Numerical simu lations also in dicate that th e bound s differ for BSSC. This d efinitely indicates that one o f the bo unds o r p ossibly both are weak . W e hav e d emonstrated tha t resolving the capacity r egion for the BSSC would d efinitely give a strong hint on the cap acity region of the br oadcast chann el f or two users. A C K N O W L E D G M E N T S The au thors would like to acknowledge Prof. Bruce Hajek for very stimulating discussions durin g his visit to CUHK. The authors would also like to a cknowledge some valuable sugges- tions and stimu lating exchanges on th e bro adcast channel and on BSSC by Pro f. Abbas El Gam al. R E F E R E N C E S [1] K. Marton, “ A coding theorem for the discrete memoryl ess broadcast channe l, ” IEEE Tr ans. Info. Theory , vol. IT -25, pp. 306–311, May , 1979. [2] C. Na ir a nd A. El Gamal, “ An ou ter bound to the capacit y region of the broadca st channel, ” IEEE T rans. Info. T heory , vol. IT -53, pp. 350–355, January , 2007. [3] T . Cover , “Broadca st channel s, ” IE EE T rans. Info. Theory , vol. IT -18, pp. 2–14, January , 1972. [4] ——, “Comments on broadcast channe ls, ” IEEE T rans. Info. Theory , vol. IT -44, pp. 2524–2530, Octo ber , 1998. [5] B. Hajek and M. Pursley , “Eva luation of an achie v able rate region for the broadcast channel, ” IEEE T rans. Info. Theory , v ol. IT -25, pp. 36–46, January , 1979. [6] I. Csiz ´ ar and J. K ¨ or ner , “Broadcast channels with confidenti al messages, ” IEEE T rans. Info. Theory , vol. IT -24, pp. 339–348, May , 1978.