Capacity Bounds for Broadcast Channels with Confidential Messages

1 Capacity Bounds for Broadcast Channels with Confidential Messages Jin Xu, Y i Cao, and Biao Chen Abstract In this paper , we study capac ity bounds f or discrete memory less bro adcast ch annels with c onfidential messages. T wo p riv ate messages as well as a common message are transmitted; the common m essage is to be dec oded by both receivers, while e ach private message is on ly for its in tended receiver . In addition, each pri vate message is to be kep t secret from the uninten ded r eceiver whe re secrecy is measured by equiv o cation. W e propo se bo th in ner and outer bo unds to the rate equiv o cation regio n f or b roadca st channels with confiden tial messages. T he p roposed inner bou nd generalize s Csisz ´ ar and K ¨ orner’ s ra te equiv o cation region fo r bro adcast ch annels with a single con fidential message, Liu et al ’ s ach iev able rate region for broa dcast chann els with per fect secrecy , Marto n’ s and Ge l’fand and Pinsker’ s achiev able rate region for g eneral broad cast channels. Ou r pro posed outer bo unds, togeth er with the inner b ound , he lps establish the rate e quiv o cation region of several c lasses o f discrete m emoryless b roadcast chann els with confidential messages, inclu ding less no isy , deterministic, and sem i-determin istic channels. Furthermo re, specializing to the general b roadcast ch annel by removing the con fidentiality constraint, our p roposed outer b ounds r educe to new capacity outer boun ds for the discrete memory broad cast channe l. I . I N T RO D U C T I O N W ith the increasingly widespread wir eless devices and services , the deman d for reliable and secure communications is becoming more ur gent due to the broadca st nature of wireless communication. Existing systems typica lly rely on key-based enc ryption schemes : the intended transce i ver pair s hare a priv ate key which is unknown to any uninten ded users. Ass uming idea l transmission of enc rypted messa ges, Shannon in his 1949 landmark paper [1] proved, using information theo retic ar gumen t, a surprising result: sec urity is guaranteed only if the ke y size is at least as long a s the so urce messag e. While this establishes p rov a ble security of the so-ca lled one-time pad system, the exce ssive req uirement o n the key s ize e ssentially forebodes a negativ e result: any key-based encryption scheme is almost always not prov a bly secure as the key size requiremen t forbids dyn amic key exchang e. This resu lt moti vates many secure communication s cheme where p rov a ble s ecurity is sac rificed in fav or of computational security; The authors are with Syracuse Univ ersit y , Department of E lectrical Enginee r ing and Computer Science, Syracuse, NY 13244. Email: j xu11@syr .edu, ycao01@sy r.edu , bichen@syr .edu. October 27, 2018 DRAFT 2 howe ver , this n otion of s ecurity relies on unproven intractability hypotheses. For instan ce, the security of RSA [2] is based on the unproven difficulty of factoring lar g e integers. W yner in his se minal work in 1975 [3] demonstrated that, for n oisy channe ls, p rov ab le se cure commu- nication (in the s ame sense as that of Shanno n) can be a chieved by exploring information theoretic limits at the phys ical laye r . W yne r introduced the so-called wiretap ch annel which is in essen ce a degraded broadcas t c hannel and cha racterized its capacity-se crecy tradeoff. It was shown that, through the use of stochas tic enc oding, pe rfect sec recy is pos sible in the abse nce of a secret key . Later , Csisz ´ ar and K ¨ orner generalized W yner’ s res ult [4] by cons idering a non-degraded discrete me moryless b roadcas t chann el (DMBC) with a single co nfidential message for one of the users and a common me ssage for both users. Following the ap proach o f [3] and [4], information-theoretic limits of sec ret commun ications for several diff e rent wireless networks have bee n in ves tigated, inc luding multi-user systems with confi dential messag es [5]–[12], secret c ommunication over fading chann els [13], [14 ] and MIMO wiretap channels [15]–[17]. In this work, w e ge neralize Csisz ´ ar and K ¨ o rner’ s model b y c onsidering discrete memoryless broa dcast channe ls where both receivers h ave their own pri vate message s a s well as a commo n mess age to d ecode . W e refer to this model a s simply DMBC with tw o c onfidential messa ges (DMBC-2CM). T he DMBC- 2CM model was first s tudied b y Liu, Ma ric, Spasojevic, and Y ates [9], [18] where, in the ab sence of a common messag e, the authors imposed the perfect secrecy con straint an d obtaine d inner and outer bounds for the perfect secrecy c apacity region. In this paper , we study capacity bounds to the rate equi vocation region for the general DMBC-2CM. Our model gen eralizes that of [18] by including a common messa ge. More importantly , we d o not impose the perfect sec recy c onstraint and stud y instead the general trade-off a mong rates for reliable co mmunication and s ecrecy for c onfiden tial messages . Study of this gene ral model a llo ws us to unify many existing results. Both inne r and outer bou nds are prop osed for the ge neral DMBC-2CM. The prop osed achiev a ble rate region gene ralizes Csisz ´ ar and K ¨ orner’ s ca pacity rate region in [4] where only a sing le confidential messag e is to be c ommunicated, Liu et a l ’ s achievable rate region un der pe rfect s ecrecy constraint [18], and Marton and Gel’f a nd-Pinsker’ s achiev ab le rate region for ge neral b roadcast chan nels [19], [20]. The proposed outer bounds to the rate equiv oc ation region of a DMBC-2CM also encompa ss existing o uter bounds for various special cases of the D MBC-2CM. In particular , it redu ces to Csisz ´ ar and K ¨ orne r’ s rate equiv ocation region for DMBC with on ly one confid ential mes sage and Liu et la ’ s oute r bound to the capacity region with perfect s ecrecy . Th e prop osed inner a nd o uter b ounds coincide with each other for the les s no isy , deterministic, and s emi-deterministic DMBC-2CM, which settle the rate e quiv oca tion region for these chan nels. Furthermore, in the absen ce of secrecy c onstraints, our prop osed o uter bo unds specialize to new outer bo unds to the ca pacity region of the ge neral DMBC. Comparison with existing outer bound s in [19], [21]–[23] will be disc ussed . The rest of the pa per is or ga nized a s follows. In Section II, we gi ve the channe l mo del a nd review October 27, 2018 DRAFT 3 relev ant existing results. In Section III, we pres ent an achiev a ble rate equiv oca tion region for our chan nel model an d show that it coinc ides with various existing results unde r respectiv e conditions. In s ection IV , we present outer bo unds to the rate eq uiv ocation region of DMBC-2CM. W e prove that the outer bound is tight for the less no isy , deterministic, and semi-deterministic DMBC-2CM. W e also discuss the induced outer bound to the ge neral DMBC and its subs et relations with existing capacity outer bounds. Finally , we conc lude in Se ction V . I I . P R O B L E M F O R M U L AT I O N A N D P R E V I O U S R E S U L T S A. Pr o blem Statement A d iscrete me moryless broad cast channe l with con fidential messa ges K is a quad ruple ( X , p, Y 1 , Y 2 ) , where X is the finite input alph abet s et, Y 1 and Y 2 are two finite ou tput alpha bet sets, a nd p is the channe l transition probab ility p ( y 1 , y 2 | x ) . W e assume that the channels are memoryless, i.e., p ( y 1 , y 2 | x ) = n Y i =1 p ( y 1 i , y 2 i | x i ) (1) where, x = ( x 1 , · · · , x n ) ∈ X n , (2) y 1 = ( y 11 , · · · , y 1 n ) ∈ Y n 1 (3) y 2 = ( y 21 , · · · , y 2 n ) ∈ Y n 2 (4) Let M 0 = { 1 , 2 , · · · , M 0 } be the common message se t, M 1 = { 1 , 2 , · · · , M 1 } and M 2 = { 1 , 2 , · · · , M 2 } be user 1 a nd u ser 2’ s pri vate message se ts, and W 0 , W 1 , W 2 are the respectiv e message variables on the sets M 0 , M 1 , M 2 . W e ass ume stoch astic e ncoding as randomization may increa se secrecy [4]. A stochas tic encod er f with block length n for K is sp ecified by f ( x | w 1 , w 2 , w 0 ) , where x ∈ X n , w 1 ∈ M 1 , w 2 ∈ M 2 , w 0 ∈ M 0 and X x f ( x | w 1 , w 2 , w 0 ) = 1 . (5) Here f ( x | w 1 , w 2 , w 0 ) is the probability that the me ssage triple ( w 1 , w 2 , w 0 ) is encoded as the channe l input x . Our model in volves two decode rs, i.e., a pair of mappings ϕ 1 : Y n 1 → M 1 × M 0 , ϕ 2 : Y n 2 → M 2 × M 0 . The av erage probabilities of de coding error of this channel are defined as P ( n ) e, 1 △ = 1 M 1 M 2 M 0 X w 1 ,w 2 ,w 0 P ( { ϕ 1 ( y 1 ) 6 = ( w 1 , w 0 ) }| ( w 1 , w 2 , w 0 ) sent ) , (6) P ( n ) e, 2 △ = 1 M 1 M 2 M 0 X w 1 ,w 2 ,w 0 P ( { ϕ 2 ( y 2 ) 6 = ( w 2 , w 0 ) }| ( w 1 , w 2 , w 0 ) sent ) . (7) October 27, 2018 DRAFT 4 A rate quintuple ( R 1 , R 2 , R 0 , R e 1 , R e 2 ) is s aid to be ach iev able if there exist messa ge se ts M 1 , M 2 , M 0 and e ncode r -de coders ( f , ϕ 1 , ϕ 2 ) such that P n e, 1 → 0 and P n e, 2 → 0 , where for a = 0 , 1 , 2 lim n →∞ 1 n log ||M a || = R a (8) lim n →∞ 1 n H ( W 1 | Y 2 ) ≥ R e 1 (9) lim n →∞ 1 n H ( W 2 | Y 1 ) ≥ R e 2 (10) The rate eq uiv ocation region of the DMBC-2CM is the closure of union of all achievable rate quintuples ( R 0 , R 1 , R 2 , R e 1 , R e 2 ) . Our objec ti ve in this pap er is to ob tain meaningful bound s to the rate e quiv oca tion region for DMBC-2CM. The DMBC-2CM mode l is illustrated in Fig. 1. W e n ote tha t in the absen ce o f W 2 , the model reduces to Csisz ´ ar a nd K ¨ orne r’ s mod el with on ly one c onfiden tial mes sage [4]. On the other h and, in the abse nce of con fidentiality constraints (i.e., H ( W 1 | Y 2 ) and H ( W 2 | Y 1 ) ), ou r model red uces to the cla ssical DMBC with two priv a te messages and one common messag e. P S f r a g r e p l a c e m e n t s Encode r Channel 1 Channel 2 Decode r 1 Decode r 2 f ( x | W 0 W 1 W 2 ) W 0 W 1 W 2 W 0 p ( y 1 | x ) p ( y 2 | x ) ϕ 1 ϕ 2 ( ˆ W 1 , ˆ W 0 ) H ( W 2 | y 1 ) ( ˆ W 2 , ˆ W 0 ) H ( W 1 | y 2 ) Fig. 1. Broadcast channel with two confidential messages W 1 , W 2 and one common message W 0 Before proc eeding, we introduce the following definitions. L et Z = ( U, V 1 , V 2 , X , Y 1 , Y 2 ) b e a set o f random variables such that X ∈ X , Y 1 ∈ Y 1 , Y 2 ∈ Y 2 , and the corresp onding p ( y 1 , y 2 | x ) is the chann el transition probability of the DMBC-2CM. Define • Q 1 to be the set of Z whose joint distribution factors as p ( u, v 1 , v 2 , x, y 1 , y 2 ) = p ( u, v 1 , v 2 ) p ( x | u, v 1 , v 2 ) p ( y 1 , y 2 | x ) . Thus any Z ∈ Q 1 satisfies the Markov chain condition U V 1 V 2 → X → Y 1 Y 2 . • Q 2 to be the set of Z whose joint distribution factors as p ( u, v 1 , v 2 , x, y 1 , y 2 ) = p ( u ) p ( v 1 , v 2 | u ) p ( x | v 1 , v 2 ) p ( y 1 , y 2 | x ); Thus any Z ∈ Q 2 satisfies the Markov chain condition U → V 1 V 2 → X → Y 1 Y 2 . • Q 3 to be the set of Z whose joint distribution factors as p ( u, v 1 , v 2 , x, y 1 , y 2 ) = p ( v 1 ) p ( v 2 ) p ( u | v 1 , v 2 ) p ( x | u, v 1 , v 2 ) p ( y 1 , y 2 | x ) . Q 3 results in the same Markov chain as Q 1 except that V 1 and V 2 are independe nt of each other . Clearly , Q 2 ⊆ Q 1 and Q 3 ⊆ Q 1 . October 27, 2018 DRAFT 5 B. Related W o rk In the section, we revie w several existing resu lts related to the p resent work. Csisz ´ ar and K ¨ orner characterized the rate equiv oc ation region [4] for b roadcast c hanne l with a commo n messag e for both users and a s ingle confid ential messa ge intended for o ne of the two users. W ithout loss of generality (WLOG), we assume W 2 is absen t from our mode l. T he result is summarized below . Pr opos ition 1: [4, Theorem 1] The ra te eq ui vocation region R C K for a DMBC with on e common messag e for both rec eiv e rs and a single co nfidential mess age for the first recei ver is a closed conv ex set consisting of tho se triples ( R 1 , R e , R 0 ) for wh ich there exist rando m v a riables U → V → X → Y 1 Y 2 such that 0 ≤ R e ≤ R 1 (11) R e ≤ I ( V ; Y 1 | U ) − I ( V ; Y 2 | U ) (12) R 1 + R 0 ≤ I ( V ; Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (13) R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (14) W e note that the Markov chain condition in Proposition 1 can be relaxed, a s stated below . Lemma 1: De fine R ′ C K to be the c on vex closure of rate triples ( R 1 , R e , R 0 ) that s atisfy (11)-(14) where the random variables follow the Markov ch ain: U V → X → Y 1 Y 2 , the n R C K = R ′ C K (15) Pr oof: R C K ⊆ R ′ C K follo ws trivially from the fact t h at U → V → X → Y 1 Y 2 implies U V → X → Y 1 Y 2 . T o p rove R ′ C K ⊆ R C K , as sume ( R 1 , R e , R 0 ) ∈ R ′ C K for s ome U V → X → Y 1 Y 2 . Defin e U ′ = U and V ′ = U V , one can verify easily that ( R 1 , R e , R 0 ) satisfie s (11)-(14) for U ′ → V ′ → X → Y 1 Y 2 , i.e., ( R 1 , R e , R 0 ) ∈ R C K . Recently , Liu et al p roposed an inner bound and an outer bound to the capacity region for broadca st channe ls with perfect-secrecy cons traint on the confid ential messa ges [9], [18]. The model in [9], [18] is in essen ce a DMBC-2CM without the common messag e. In their model, each user has its own confiden tial messag e that is to be completely protected from the other user . The propose d a chiev ab le region and outer bound are giv e n in Propo sitions 2 a nd 3, respec ti vely . Pr opos ition 2: [18, The orem 4] Let R LM S Y − I denote the union of all ( R 1 , R 2 ) satisfying 0 ≤ R 1 ≤ I ( V 1 ; Y 1 | U ) − I ( V 1 ; Y 2 | V 2 U ) − I ( V 1 ; V 2 | U ) 0 ≤ R 2 ≤ I ( V 2 ; Y 2 | U ) − I ( V 2 ; Y 1 | V 1 U ) − I ( V 1 ; V 2 | U ) (16) over all random variables ( U, V 1 , V 2 , X , Y 1 , Y 2 ) ∈ Q 2 . Any rate pair ( R 1 , R 2 ) ∈ R LM S Y − I is achiev a ble for DMBC-2CM without common mess age and with pe rfect secrecy for the co nfidential messag es, i.e ., R 0 = 0 , R 1 = R e 1 , and R 2 = R e 2 . October 27, 2018 DRAFT 6 Pr opos ition 3: [18, Theorem 3 ] An ou ter bo und to the ca pacity region for the DMBC-2CM with perfect secrecy c onstraint is the s et of all ( R 1 , R 2 ) satisfying 0 ≤ R 1 ≤ min[ I ( V 1 ; Y 1 | U ) − I ( V 1 ; Y 2 | U ) , I ( V 1 ; Y 1 | V 2 U ) − I ( V 1 ; Y 2 | V 2 U )] (17) 0 ≤ R 2 ≤ min[ I ( V 2 ; Y 2 | U ) − I ( V 2 ; Y 1 | U ) , I ( V 2 ; Y 2 | V 1 U ) − I ( V 2 ; Y 1 | V 1 U )] . (18) for some ( U, V 1 , V 2 , X , Y 1 , Y 2 ) ∈ Q 2 . W e denote by R LM S Y − O this outer bound. In the absenc e of se crecy cons traint, the pres ent mod el reduc es to the DMBC first introduced by Cover [24]. The capacity region for a DMBC is only known for s ome special c ases (see [25] and references therein). The best ac hiev able region for gen eral DMBC is gi ven by Gel’fand and Pinsker in [20] which reduces to Marton’ s ach iev able region [19, Theorem 2] for DMBC in the absenc e of c ommon messag e. Capacity region outer bou nds inc lude K ¨ orne r an d Marton’ s ou ter bound [19 , Theore m 5], Liang and Kramer’ s outer boun d [22], [26], Nair a nd El Ga mal’ s outer bound [21], [27 ], a nd a rec ently propo sed outer bound by Liang, Kramer and Shamai (Shitz) [23]. Marton in 1979 considered DMBC in the absenc e of commo n mes sage an d propose d the follo wing achiev a ble rate region [19]. Pr opos ition 4: [19, The orem 2] Let R M be the union of non-negative rate pairs ( R 1 , R 2 ) satisfying R 1 , R 2 ≥ 0 and R 1 ≤ I ( U V 1 ; Y 1 ) (19) R 2 ≤ I ( U V 2 ; Y 2 ) (20) R 1 + R 2 ≤ min { I ( U ; Y 1 ) , I ( U ; Y 2 ) } + I ( V 1 ; Y 1 | U ) + I ( V 2 ; Y 2 | U ) − I ( V 1 ; V 2 | U ) (21) for some ( U, V 1 , V 2 , X , Y 1 , Y 2 ) ∈ Q 1 . Th en R M is an achiev a ble rate region for the DMBC wit h out common messa ge. Gel’fand an d Pinsker gene ralized Ma rton’ s model by considering DMBC with c ommon information. The achiev a ble rate region they propo sed [20] is summa rized below . Pr opos ition 5: [20, Theorem 1] Let R GP be the union o f non-negati ve rate triples ( R 0 , R 1 , R 2 ) satisfying R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (22) R 1 + R 0 ≤ I ( V 1 ; Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (23) R 2 + R 0 ≤ I ( V 2 ; Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (24) R 1 + R 2 + R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] + I ( V 1 ; Y 1 | U ) + I ( V 2 ; Y 2 | U ) − I ( V 1 ; V 2 | U ) (25) for some ( U, V 1 , V 2 , X , Y 1 , Y 2 ) ∈ Q 1 . Then R GP is an ach iev able rate region for the DMBC. W e comment he re that in the absenc e o f common message, R GP can be shown to b e equiv alent to R M [20]. Furthermore, an equivalent definition of R GP can be obtained by restricting Z ∈ Q 2 instead of Q 1 , i.e., October 27, 2018 DRAFT 7 Lemma 2: De fine R ′ GP to be the union of non-negativ e rate triples ( R 0 , R 1 , R 2 ) satisfying (22)-(25 ) with Z ∈ Q 2 , the n R GP = R ′ GP (26) The proof is similar to tha t for Le mma 1 an d is skippe d. Similarly , R M can be eq uiv alently defined using Z ∈ Q 2 . An earlier outer b ound by K ¨ orner and Marton [19 , Theorem 5 ] for the capacity region of DMBC is subsume d by several recen t outer bound s. On e of the recent o uter bounds was proposed by Liang a nd Kramer [22], [26, Theorem 6], as summarized in Proposition 6. Pr opos ition 6: If ( R 0 , R 1 , R 2 ) is achiev a ble, then there exists Z ∈ Q 1 and R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] , (27) R 0 + R 1 ≤ I ( V 1 , U ; Y 1 ) , (28) R 0 + R 2 ≤ I ( V 2 , U ; Y 2 ) , (29) R 0 + R 1 + R 2 ≤ I ( X ; Y 2 | V 1 U ) + I ( V 1 ; Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] , (30) R 0 + R 1 + R 2 ≤ I ( X ; Y 1 | V 2 U ) + I ( V 2 ; Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] . (31) W e den ote this outer bound as R LK , i.e., R LK is the un ion of non -negati ve rate triples ( R 0 , R 1 , R 2 ) satisfying (27)-(31) ov er Z ∈ Q 1 . Furthermore, we can a lso re strict the Mark ov chain co ndition to be Z ∈ Q 2 , i.e., Lemma 3: De fine R ′ LK to be the conv ex c losure o f union o f n on-negativ e rate triples ( R 0 , R 1 , R 2 ) satisfying (27)-(31) with Z ∈ Q 2 , then R LK = R ′ LK (32) In [21 , T heorem 2.1], anothe r outer b ound to the capacity region o f the general DMBC was gi ven b y Nair a nd El Gamal, a s summarized in Propo sition 7. This ou ter boun d was shown to be strictly tighter than the K ¨ orner and Marton outer bound [19, Theorem 5]. Pr opos ition 7: If ( R 0 , R 1 , R 2 ) is achiev a ble, then there exists Z ∈ Q 3 and R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] , (33) R 0 + R 1 ≤ I ( V 1 U ; Y 1 ) , (34) R 0 + R 2 ≤ I ( V 2 U ; Y 2 ) , (35) R 0 + R 1 + R 2 ≤ I ( V 2 ; Y 2 | V 1 U ) + I ( V 1 U ; Y 1 ) , (36) R 0 + R 1 + R 2 ≤ I ( V 1 ; Y 1 | V 2 U ) + I ( V 2 U ; Y 2 ) . (37) W e denote by R N E this new outer bound, i.e ., R N E is the u nion o f n en-negativ e rate triples ( R 0 , R 1 , R 2 ) satisfying (33)-(37) over Z ∈ Q 3 . The most rece nt outer bound to the capac ity region for DMBC was proposed by Liang , Kramer , and Shamai (Shitz) [23]: October 27, 2018 DRAFT 8 Pr opos ition 8: If ( R 0 , R 1 , R 2 ) is achiev able, then there exist r a ndom v ariables ( W 0 , W 1 , W 2 , V 1 , V 2 , X , Y 1 , Y 2 ) whose joint distrib ution factors as p ( w 0 ) p ( w 1 ) p ( w 2 ) p ( v 1 , v 2 | w 0 , w 1 , w 2 ) p ( x | v 1 , v 2 , w 0 , w 1 , w 2 ) p ( y 1 , y 2 | x ) (38) such that, 0 ≤ R 0 ≤ min[ I ( W 0 ; Y 1 | V 1 ) , I ( W 0 ; Y 2 | V 2 )] (39) R 1 ≤ I ( W 1 ; Y 1 | V 1 ) (40) R 2 ≤ I ( W 2 ; Y 2 | V 2 ) (41) R 0 + R 1 ≤ min[ I ( W 0 W 1 ; Y 1 | V 1 ) , I ( W 1 ; Y 1 | W 0 V 1 V 2 ) + I ( W 0 V 1 ; Y 2 | V 2 )] (42) R 0 + R 2 ≤ min[ I ( W 0 W 2 ; Y 2 | V 2 ) , I ( W 2 ; Y 2 | W 0 V 1 V 2 ) + I ( W 0 V 2 ; Y 1 | V 1 )] (43) R 0 + R 1 + R 2 ≤ I ( W 1 ; Y 1 | W 0 W 2 V 1 V 2 ) + I ( W 0 W 2 V 1 ; Y 2 | V 2 ) (44) R 0 + R 1 + R 2 ≤ I ( W 2 ; Y 2 | W 0 W 1 V 1 V 2 ) + I ( W 0 W 1 V 2 ; Y 1 | V 1 ) (45) R 0 + R 1 + R 2 ≤ I ( W 1 ; Y 1 | W 0 W 2 V 1 V 2 ) + I ( W 2 ; Y 2 | W 0 V 1 V 2 ) + I ( W 0 V 1 V 2 ; Y 1 ) (46) R 0 + R 1 + R 2 ≤ I ( W 2 ; Y 2 | W 0 W 1 V 1 V 2 ) + I ( W 1 ; Y 1 | W 0 V 1 V 2 ) + I ( W 0 V 1 V 2 ; Y 2 ) , (47) where X is a de terministic fun ction of ( W 0 , W 1 , W 2 , V 1 , V 2 ) , and W 0 , W 1 , W 2 are u niformly dis trib uted . W e refer to this new ou ter bound as R LK S . I I I . A N A C H I E V A B L E R AT E E Q U I VO C A T I O N R E G I O N Our proposed ac hiev able rate e quiv oca tion region for DMBC-2CM is giv e n in Theo rem 1. The cod ing scheme comb ines binning, superpo sition c oding, an d rate splitting. For the rate constraints, the binning approach in [28] is suppleme nted with su perposition coding to accommod ate the common mes sage. An additional b inning is introduced for c onfiden tiality o f pri vate me ssage s. W e n ote tha t this doub le binn ing technique has be en us ed b y various a uthors for c ommunication in volving confide ntial mess ages (see , e.g ., [18], [29]). Dif ferent from that of [18], we make expli cit use of rate splitt ing for the two priv ate messages in order to boost the rates R 1 and R 2 . W e note that this ra te splitting was implicitly used in [4] (specifica lly , proo f o f Lemma 3 in [4]). T o be precise , we split t h e pri vate message W 1 ∈ { 1 , ·· · , 2 nR 1 } into W 11 ∈ { 1 , ·· · , 2 nR 11 } and W 10 ∈ { 1 , · · · , 2 nR 10 } , and W 2 ∈ { 1 , · · · , 2 nR 2 } into W 22 ∈ { 1 , · · · , 2 nR 22 } and W 20 ∈ { 1 , · · · , 2 nR 20 } , respectively . W 11 and W 22 are on ly to be dec oded by intend ed receivers while W 10 and W 20 are to be decode d by both receivers. No tice tha t this rate s plitting is typica lly used in interference cha nnels to achieve a la r g er rate region as it enables interference cance llation a t the rec eiv e rs. It is clea r that this rate splitting is prohibited if pe rfect s ecrecy is required as in [18]. No w , we combine ( W 10 , W 20 , W 0 ) tog ether into a single auxiliary variable U . The messages W 11 and W 22 are represente d by auxiliary variables V 1 and V 2 respectively . October 27, 2018 DRAFT 9 The achiev a ble rate equiv ocation for a DMBC-2CM is formally s tated below . Theorem 1: Let R I be the union of all non-negativ e rate q uintuple ( R 1 , R 2 , R 0 , R e 1 , R e 2 ) satisfying R e 1 ≤ R 1 (48) R e 2 ≤ R 2 (49) R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (50) R 1 + R 0 ≤ I ( V 1 ; Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (51) R 2 + R 0 ≤ I ( V 2 ; Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (52) R 1 + R 2 + R 0 ≤ I ( V 1 ; Y 1 | U ) + I ( V 2 ; Y 2 | U ) − I ( V 1 ; V 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (53) R e 1 ≤ I ( V 1 ; Y 1 | U ) − I ( V 1 ; Y 2 V 2 | U ) (54) R e 2 ≤ I ( V 2 ; Y 2 | U ) − I ( V 2 ; Y 1 V 1 | U ) (55) over all ( U, V 1 , V 2 , X , Y 1 , Y 2 ) ∈ Q 2 . Then R I is an achiev a ble rate region for the DMBC-2CM. Pr oof: See Appendix I. Remark 1: The region R I remains the same if we replace Q 2 with Q 1 . Formally , Pr opos ition 9: Define R ′ I to be the union of all n on-negativ e rate quintuple ( R 1 , R 2 , R 0 , R e 1 , R e 2 ) satisfying (48)-(55) over Z ∈ Q 1 , the n R I = R ′ I (56) Pr oof: The fact tha t R I ⊆ R ′ I follo ws tri vially from Q 2 ⊆ Q 1 . W e now show R ′ I ⊆ R I . Assume ( R 1 , R 2 , R 0 , R e 1 , R e 2 ) ∈ R ′ I , i.e., there exists ( U, V 1 , V 2 , X , Y 1 , Y 2 ) ∈ Q 1 such that ( R 1 , R 2 , R 0 , R e 1 , R e 2 ) satisfie s (48)-(55 ). The proo f is comp leted by defin ining U ′ = U , V ′ 1 = U V 1 , and V ′ 2 = U V 2 and observe that the same ( R 1 , R 2 , R 0 , R e 1 , R e 2 ) satisfies (48) -(55 ) for ( U ′ , V ′ 1 , V ′ 2 , X , Y 1 , Y 2 ) ∈ Q 2 . This achievable rate equiv ocation region unifies many existing results which we enu merate be low . A. Csisz ´ ar an d K ¨ orner’ s re gion In [4], Csisz ´ a r a nd K ¨ orner characterized the rate equ i vocation region for broadca st ch annels wit h a single confiden tial messa ge a nd a common message . By setting R 2 = 0 an d R e 2 = 0 in Theore m 1, it is easy to see R I reduces to Csisz ´ ar and K ¨ orner’ s capac ity region R C K described in P roposition 1. B. Liu et al’ s re gion In [18], Liu et al prop osed a n a chiev a ble rate region for broad cast channe l with confi dential mess ages where there are two pri vate me ssage and no common mess age. In addition, the pri vate mess ages are to be perfectly protected from the unintende d receivers. By se tting R 1 = R e 1 , R 2 = R e 2 and R 0 = 0 in The orem 1, one ca n easily chec k that R I reduces to Liu et al ’ s achiev ab le rate region R LM S Y − I described in Proposition 2. October 27, 2018 DRAFT 10 C. Gel’fand and Pinsker ’ s re gion In [20 ], Ge l’fand and Pinks er ge neralized Marton’ s res ult by proposing an achievable rate region for broadcas t chann els with co mmon me ssage . If we remove the s ecrecy co nstraints in our model by setting R e 1 = 0 and R e 2 = 0 in The orem 1, we ob tain an achievable rate region for the general DMBC, d enoted by ˆ R , with the exac t expressions in (22)-(25) with U → ( V 1 , V 2 ) → X → ( Y 1 , Y 2 ) . From Propos ition 5 and Lemma 2, ˆ R = R GP . Remark 2: The proo fs in [19], [20] both use a corner po int approach . A binning a pproach was use d in [28] to prove a wea kened version of [19, The orem 2]. The proof introduced in the prese nt pape r , by stripping o ut all c onfiden tiality c onstraints, provides a new way to prove the gene ral achiev a ble rate region of DMBC [20, The orem 1] [19, Theorem 2] along the line of [28]. I V . O U T E R B O U N D S Define R O 1 to b e the union, over all Z ∈ Q 1 , of non-negativ e rate quintuple ( R 0 , R 1 , R 2 , R e 1 , R e 2 ) satisfying R e 1 ≤ R 1 (57) R e 2 ≤ R 2 (58) R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (59) R 0 + R 1 ≤ I ( V 1 ; Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (60) R 0 + R 2 ≤ I ( V 2 ; Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (61) R 0 + R 1 + R 2 ≤ I ( V 2 ; Y 2 | V 1 U ) + I ( V 1 ; Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (62) R 0 + R 1 + R 2 ≤ I ( V 1 ; Y 1 | V 2 U ) + I ( V 2 ; Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (63) R e 1 ≤ min[ I ( V 1 ; Y 1 | U ) − I ( V 1 ; Y 2 | U ) , I ( V 1 ; Y 1 | V 2 U ) − I ( V 1 ; Y 2 | V 2 U )] (64) R e 2 ≤ min[ I ( V 2 ; Y 2 | U ) − I ( V 2 ; Y 1 | U ) , I ( V 2 ; Y 2 | V 1 U ) − I ( V 2 ; Y 1 | V 1 U )] . (65) Similarly , defin e R O 2 and R O 3 in exactly the same f a shion except with Q 1 replaced by Q 2 and Q 3 , respectively . W e h av e Theorem 2: R O 1 , R O 2 , and R O 3 are all outer bounds to the rate equiv oca tion region of the DMBC- 2CM. Pr oof: The proof that R O 2 and R O 3 are o uter bounds is gi ven in Appendix II. Tha t R O 1 is an outer bound follo ws directly from Proposition 1 0. Pr opos ition 10: R O 3 ⊆ R O 1 = R O 2 . (66) Proposition 10 can be es tablished by s imple algeb ra whose proof is skipped. While R O 3 subsume s b oth R O 1 and R O 2 , the latter expressions are often easier to use in es tablishing ca pacity resu lts or comparing October 27, 2018 DRAFT 11 with existing boun ds. For example, it is s traightforward to show that R O 2 is tight for Csisz ´ ar and K ¨ orner’ s model [4], i.e., DMBC with only one confiden tial mes sage. Below , we disc uss v arious implications of Theorem 2. A. The rate equivoca tion re gion of less noisy DMBC-2CM For the DMBC d efined in Sec tion II-A, channel 1 is said to be less n oisy tha n ch annel 2 [30] if for ev ery V → X → Y 1 Y 2 , I ( V ; Y 1 ) ≥ I ( V ; Y 2 ) . (67) Furthermore, for every U → V → X → Y 1 Y 2 , the above less noisy cond ition also implies I ( V ; Y 1 | U ) ≥ I ( V ; Y 2 | U ) . (68) Using Theorems 1 a nd 2, we ca n establish the rate eq uiv oca tion region for less noisy DMBC-2CM as in Theorem 3. Theorem 3: If cha nnel 1 is less no isy than c hannel 2, then the rate equiv o cation region for this less noisy DMBC-2CM is the set of all non-negativ e ( R 0 , R 1 , R 2 , R e 1 , R e 2 ) satisfying R e 1 ≤ R 1 (69) R 0 + R 2 ≤ I ( U ; Y 2 ) (70) R 0 + R 1 + R 2 ≤ I ( V ; Y 1 | U ) + I ( U ; Y 2 ) (71) R e 1 ≤ I ( V ; Y 1 | U ) − I ( V ; Y 2 | U ) (72) R e 2 = 0 , (73) for some ( U, V , X, Y 1 , Y 2 ) such that U → V → X → Y 1 Y 2 . Pr oof: The a chiev a bility is estab lished by se tting V 2 = const in Theorem 1 and using Eqs. (67) and (68). T o prove the con verse, we need to show that for any rate quintuple satisfying Eqs . (57)-(65 ) in The orem 2, we can find ( U ′ , V ′ , X , Y 1 , Y 2 ) su ch that U ′ → V ′ → X → Y 1 Y 2 and (69)-(73) are satisfied. This can be ac complished using simple algebra and by d efining U ′ = U V 2 and V ′ = V 1 where ( U, V 1 , V 2 , X , Y 1 , Y 2 ) ∈ Q 2 are the variables use d in Th eorem 2. Remark 3 : Th e fact that R e 2 = 0 is a direct conseq uence of the les s no isy assump tion: receiver 1 can always dec ode anything that receiver 2 can decode. B. The rate equivoca tion re gion of semi-deterministic DMBC-2CM Theorem 2 also allows us to establish the rate equiv oc ation region o f the semi-deterministic DMBC- 2CM. WLOG, let chan nel 1 be deterministic. October 27, 2018 DRAFT 12 Theorem 4: If p ( y 1 | x ) is a (0 , 1) matrix, the n the rate equiv oc ation region for this DMBC-2CM, denoted by R sd , is the set of all n on-negativ e ( R 0 , R 1 , R 2 , R e 1 , R e 2 ) satisfying R e 1 ≤ R 1 (74) R e 2 ≤ R 2 (75) R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (76) R 0 + R 1 ≤ H ( Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (77) R 0 + R 2 ≤ I ( V 2 ; Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (78) R 0 + R 1 + R 2 ≤ H ( Y 1 | V 2 U ) + I ( V 2 ; Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (79) R e 1 ≤ H ( Y 1 | Y 2 V 2 U ) (80) R e 2 ≤ I ( V 2 ; Y 2 | U ) − I ( V 2 ; Y 1 | U ) , (81) for some ( U, Y 1 , V 2 , X , Y 1 , Y 2 ) ∈ Q 2 . Pr oof: The direct part of this theorem follo ws tri vially from The orem 1 b y setting V 1 = Y 1 . The proof is therefore complete by s howing R S D − O 2 ⊆ R sd , whe re R S D − O 2 is the ou ter bou nd R O 2 specializing to the s emi-deterministic DMBC-2CM. That is, for any Z ∈ Q 2 and ( R 0 , R 1 , R 2 , R e 1 , R e 2 ) satisfying (57)-(65), we need to show that ( R 0 , R 1 , R 2 , R e 1 , R e 2 ) also satisfie s (74)-(81). W e note tha t Eqs. (74)-(76), (78), and (81) can be tri v ially e stablished. That the sum-rate b ound Eq. (77) is satisfied follo ws easily from the fact H ( Y 1 | U ) ≥ I ( V 1 ; Y 1 | U ) . (82) The sum-rate bound for R 0 + R 1 + R 2 in Eq. (62) and (63) can be re-written as R 0 + R 1 + R 2 ≤ min[ I ( V 2 ; Y 2 | V 1 U ) + I ( V 1 ; Y 1 | U ) , I ( V 1 ; Y 1 | V 2 U ) + I ( V 2 ; Y 2 | U )] (83) + m in[ I ( U ; Y 1 ) , I ( U ; Y 2 )] . (84) Thus (79) is satisfied since H ( Y 1 | V 2 , U ) + I ( V 2 ; Y 2 | U ) ≥ I ( V 1 ; Y 1 | V 2 U ) + I ( V 2 ; Y 2 | U ) . (85) For Eq. (80), we on ly n eed to show (cf. (64)) H ( Y 1 | Y 2 V 2 U ) ≥ I ( V 1 ; Y 1 | V 2 U ) − I ( V 1 ; Y 2 | V 2 U ) . (86 ) W e have H ( Y 1 | Y 2 V 2 U ) ≥ I ( V 1 ; Y 1 | Y 2 V 2 U ) (87) = I ( V 1 ; Y 1 Y 2 | V 2 U ) − I ( V 1 ; Y 2 | V 2 U ) (88) ≥ I ( V 1 ; Y 1 | V 2 U ) − I ( V 1 ; Y 2 | V 2 U ) . (89) October 27, 2018 DRAFT 13 The proof of Theorem 4 is therefore complete. Similarly , the rate equ i vocation region of deterministic DMBC-2CM can be establishe d as follows. Pr opos ition 11: If p ( y 1 | x ) and p ( y 2 | x ) are both (0 , 1) ma trices, then the rate e quiv oca tion region for this deterministic DMBC-2CM is the set of all ( R 0 , R 1 , R 2 , R e 1 , R e 2 ) satisfying 0 ≤ R e 1 ≤ R 1 (90) 0 ≤ R e 2 ≤ R 2 (91) 0 ≤ R 0 ≤ min [ I ( U ; Y 1 ) , I ( U ; Y 2 )] (92) R 0 + R 1 ≤ H ( Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (93) R 0 + R 2 ≤ I ( Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (94) R 0 + R 1 + R 2 ≤ H ( Y 1 Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (95) R e 1 ≤ H ( Y 1 | Y 2 U ) (96) R e 2 ≤ H ( Y 2 | Y 1 U ) , (97) for some ( U, Y 1 , Y 2 , X , Y 1 , Y 2 ) ∈ Q 2 . C. Outer boun d for DMBC -2CM with perfect secrecy By setting R 0 = 0 , R e 1 = R 1 and R e 2 = R 2 in Th eorem 2 , we obtain outer bounds for DMBC-2CM with p erfect se crecy , deno ted respec ti vely b y R P S − O 1 , R P S − O 2 , and R P S − O 3 for Z ∈ Q 1 , Z ∈ Q 2 , and Z ∈ Q 3 . Clearly , R P S − O 1 = R P S − O 2 ⊇ R P S − O 3 (98) In addition, from Proposition 3, we have R P S − O 2 = R LM S Y − O . (99) i.e., R P S − O 2 coincides with Liu et al ’ s outer bo und in Propos ition 3 . Finally , all thes e outer b ounds are tight for the semi-deterministic DMBC-2CM with perfect secrecy . D. New outer bounds for the general DMBC Specializing Theo rem 2 to the general DMBC, i.e, s etting R e 1 = R e 2 = 0 , we obtain the follo w ing outer bound s for the general DMBC. October 27, 2018 DRAFT 14 Theorem 5: For a ny Z ∈ Q 1 , let S B C ( Z ) be the se t of all ( R 0 , R 1 , R 2 ) of n on-negativ e numbers satisfying R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (100) R 0 + R 1 ≤ I ( V 1 ; Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (101) R 0 + R 2 ≤ I ( V 2 ; Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (102) R 0 + R 1 + R 2 ≤ I ( V 2 ; Y 2 | V 1 U ) + I ( V 1 ; Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (103) R 0 + R 1 + R 2 ≤ I ( V 1 ; Y 1 | V 2 U ) + I ( V 2 ; Y 2 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] . (104) Then R B C − O 1 = [ Z ∈Q 1 S B C ( Z ) (105) constitutes an outer bound to the capacity region for the DMBC. One can establish in a similar fashion two o ther ou ter bounds for the g eneral DMBC, denoted by R B C − O 2 and R B C − O 3 , by replacing Q 1 in Th eorem 5 with Q 2 and Q 3 , respec ti vely . Similar to Proposition 10, we have R B C − O 3 ⊆ R B C − O 1 = R B C − O 2 . (106) Remark 4: It is interesting to o bserve that the ineq ualities of our o uter bound R B C are a ll identical to those of the existing inne r b ound [20], d escribed in Proposition 5, except for the boun d on R 0 + R 1 + R 2 , for which there is a gap of γ = min[ I ( V 1 ; V 2 | Y 1 , U ) , I ( V 1 ; V 2 | Y 2 , U )] . (107) Remark 5: It is easy to show that R B C − O 2 subsume s the outer bound in [22, Theore m 6 ] since I ( V 1 ; Y 1 | V 2 U ) ≤ I ( X ; Y 1 | V 2 U ) , (108) I ( V 2 ; Y 2 | V 1 U ) ≤ I ( X ; Y 2 | V 1 U ) . (109) Remark 6: The new outer boun d R B C − O 3 is also a sub set of the outer bou nd propos ed in [21, Th eorem 2.1], as described in Proposition 7. More precisely , we h av e Pr opos ition 12: R B C − O 3 ⊆ R N E , whe re the equa lity holds wh en 1) R 0 = 0 ; o r 2) R 1 = 0 ; or 3) R 2 = 0 . Pr oof: See Appendix III. Remark 7 : Note tha t the con ditions in Propos ition 12 are only su f fi cient c onditions, i.e., there may b e other instances when the two b ounds are equiv a lent. It is also poss ible that R B C − O 3 = R N E though we have n ot been succes sful in proving (or disapproving) it. October 27, 2018 DRAFT 15 Remark 8: One ca n easily verify that the outer bound proposed in [23], R LK S in Proposition 8, subsume s all the above outer boun ds. T o summarize , we have R LK S ⊆ R B C − O 3 ⊆ ( R LK R N E (110) It remains unknown if any of the a bove the subse t relations can be strict or not. The fact that R LK S subsume s existing o uter bounds ca n be attrib u ted to the way auxiliary random variables are defined in [23]. By further splitti n g a uxiliary rando m variables a nd isolating thos e corre- sponding to the messag e variables, one can keep the terms in the rate uppe r bounds which a re otherwise dropped if only three a uxiliary variables are u sed as in Theorem 2 or [21]. Finally , we rema rk that the approach in [23] ca n be adopted to the problem in volving sec recy c onstraint in a straightforward man ner to obtain a new outer bound to the rate equivocation region for DMBC-2CM. V . C O N C L U S I O N W e proposed inner and outer bo unds for the rate equiv oc ation region of discrete memoryles s b roadcas t channe ls with two confide ntial messag es (DMBC-2CM). The propose d inner bound combines superpo - sition, rate splitting, and doub le binning and unifies existing k nown res ults for broadca st chann els with or without c onfiden tial messag es. These include Csisz ´ a r and K ¨ orner’ s capacity rate region for b roadcas t channe l with s ingle p ri vate message [4], Liu e t al ’ s rate region for broadcast channe l with perfect s ecrecy [18], Marton and Gel’f a nd-Pinsker’ s a chiev ab le rate region for ge neral broa dcast channe ls [19], [20]. The proposed outer bo unds also generalize s everal existing resu lts. In addition, the p roposed inn er and outer bounds settle the rate equi vocation region of less noisy , deterministic, and semi-deterministic DMBC- 2CM. In the absen ce of the equi vocation constraints, the proposed outer bound s reduce to oute r bounds for the general b roadcas t channe l. General subse t relations with other known outer bounds were established. V I . A C K N O W L E D G M E N T The autho rs would like to thank Dr . Gerhard Kramer for b ringing to our attention reference [23] an d for many he lpful discussion s. A P P E N D I X I P RO O F F O R T H E O R E M 1 W e prove that if ( R 0 , R 1 , R 2 , R e 1 , R e 2 ) is a chiev a ble, then it must sa tisfy Eqs. (48)-(55) in Theorem 1 for some ( U, V 1 , V 2 , X , Y 1 , Y 2 ) ∈ Q 2 . W e first prove the c ase when R 1 ≥ R e 1 = I ( V 1 ; Y 1 | U ) − I ( V 1 ; Y 2 V 2 | U ) ≥ 0 , (111) R 2 ≥ R e 2 = I ( V 2 ; Y 2 | U ) − I ( V 2 ; Y 1 V 1 | U ) ≥ 0 . (112) October 27, 2018 DRAFT 16 Rate splitting, as desc ribed in Sec tion III g i ves rise to the follo wing fi ve message variables: W 0 ∈  1 , 2 , · · · , 2 nR 0  W 10 ∈  1 , 2 , · · · , 2 nR 10  W 11 ∈  1 , 2 , · · · , 2 nR 11  W 20 ∈  1 , 2 , · · · , 2 nR 20  W 22 ∈  1 , 2 , · · · , 2 nR 22  where R 10 + R 11 = R 1 and R 20 + R 22 = R 2 . W e remark here that (111) and (112) c ombined with the rate splitting and the fact that W 10 and W 20 are deco ded by both rece i vers ens ures tha t, R 11 ≥ R e 1 = I ( V 1 ; Y 1 | U ) − I ( V 1 ; Y 2 V 2 | U ) ≥ 0 , (113) R 22 ≥ R e 2 = I ( V 2 ; Y 2 | U ) − I ( V 2 ; Y 1 V 1 | U ) ≥ 0 . (114) A ux iliary Co debook Gener ation: Fix p ( u ) , p ( v 1 | u ) , p ( v 2 | u ) a nd p ( x | v 1 , v 2 ) . F or arbitrary ǫ 1 > 0 , Define L 11 = I ( V 1 ; Y 1 | U ) − I ( V 1 ; Y 2 V 2 | U ) , (115) L 12 = I ( V 1 ; Y 2 | V 2 U ) , (116) L 21 = I ( V 2 ; Y 2 | U ) − I ( V 2 ; Y 1 V 1 | U ) , (117) L 22 = I ( V 2 ; Y 1 | V 1 U ) , (118) L 3 = I ( V 1 ; V 2 | U ) − ǫ 1 . (119) Note that L 11 + L 12 + L 3 = I ( V 1 ; Y 1 | U ) − ǫ 1 , (120) L 21 + L 22 + L 3 = I ( V 2 ; Y 2 | U ) − ǫ 1 . (121) • Generate 2 n ( R 10 + R 20 + R 0 ) independ ent and identically distrib uted (i.i.d.) cod ew o rds u ( k ) , with k ∈ { 1 , · · · , 2 n ( R 10 + R 20 + R 0 ) } , according to Q n t =1 p ( u t ) . • For each codeword u ( k ) , generate 2 n ( L 11 + L 12 + L 3 ) i.i.d. code words v 1 ( i, i ′ , i ′′ ) , wi th i ∈ { 1 , · · · , 2 nL 11 } , i ′ ∈ { 1 , · · · , 2 nL 12 } and i ′′ ∈ { 1 , · · · , 2 nL 3 } , a ccording to Q n t =1 p ( v 1 t | u t ) . The indexing allows an alternati ve interpretation using binning. W e ran domly place the g enerated v 1 vectors into 2 nL 11 bins indexed by i ; for the co dewords in each bin, randomly place them into 2 nL 12 sub-bins indexed by i ′ ; thus i ′′ is the index for the c odeword in e ach sub-bin. • Similarly , for each codeword u , generate 2 n ( L 21 + L 22 + L 3 ) i.i.d. cod ew o rds v 2 ( j, j ′ , j ′′ ) acc ording to Q n t =1 p ( v 2 t | u t ) , where j ∈ { 1 , · · · , 2 nL 21 } , j ′ ∈ { 1 , · · · , 2 nL 22 } and j ′′ ∈ { 1 , · · · , 2 nL 3 } . Encoding: Enc oding in volves the mapping of mes sage indices to channel input, which is facilit ated by the auxiliary codewords generated above. October 27, 2018 DRAFT 17 T o s end messag e ( w 10 , w 20 , w 0 ) , we first calculate the correspon ding messa ge index k and ch oose the correspon ding codeword u ( k ) . Gi ven this u ( k ) , we have 2 n ( L 11 + L 12 + L 3 ) codewords of v 1 ( i, i ′ , i ′′ ) to choose from for message w 11 . Evenly map 2 nR 11 messag es w 11 to 2 nL 11 bins, then, gi ven (113), eac h bin correspon ds to at least one mess age w 11 . Thus, gi ven w 11 , the bin index i c an be decided. 1) If R 11 ≤ L 11 + L 12 , e ach bin corresponds to 2 R 11 − L 11 messag es w 11 . Evenly plac e the 2 nL 12 sub-bins into 2 R 11 − L 11 cells. Given w 11 , we can fin d the c orresponding cell, and rand omly choos e a su b-bin from tha t c ell, thus the sub-bin index i ′ can be decided . The co deword v 1 ( i, i ′ , i ′′ ) will be chosen from that sub-bin. 2) If L 11 + L 12 < R 11 ≤ L 11 + L 12 + L 3 , the n each sub-bin is mappe d to at least o ne message w 11 , so i ′ is dec ided giv e n w 11 . In each sub -bin, there a re 2 R 11 − L 11 − L 12 messag es. Evenly place those 2 nL 3 codewords v 1 into 2 R 11 − L 11 − L 12 cells. Given w 11 , we can find the corresponding ce ll. The codeword v 1 ( i, i ′ , i ′′ ) will be chosen from that cell. Gi ven w 22 , the selection of v j,j ′ ,j ′′ is c arried in exactly the same manner . From the given s ub-bins o r cells, the encod er cho oses the cod ew o rd pair ( v 1 ( i, i ′ , i ′′ ) , v 2 ( j, j ′ , j ′′ )) that satisfies ( v 1 ( i, i ′ , i ′′ ) , v 2 ( j, j ′ , j ′′ ) , u ( k )) ∈ A ( n ) ǫ ( V 1 , V 2 , U ) , (122) where A ( n ) ǫ ( · ) de notes the jointly typica l set. If there are mo re than one suc h pair , rando mly choos e one ; if there is no such pair , an error is declared . Gi ven v 1 and v 2 , we generate the chann el input x acco rding to i.i.d. p ( x | v 1 , v 2 ) , i . e., x ∼ Q n i =1 p ( x i | v 1 i , v 2 i ) where v 1 i and v 2 i are respectiv ely the i th ele ment of the vectors v 1 and v 2 . Decoding: Receiver Y 1 looks for u ( k ) suc h that ( u ( k ) , y 1 ) ∈ A ( n ) ǫ ( U, Y 1 ) . (123) If such u ( k ) exists and is unique, se t ˆ k = k ; otherwise , declare an error . Upon de coding k , rece i ver Y 1 looks for sequen ces v 1 ( i, i ′ , i ′′ ) such that ( v 1 ( i, i ′ , i ′′ ) , u ( k ) , y 1 ) ∈ A ( n ) ǫ ( V 1 , U, Y 1 ) . (124) If such v 1 ( i, i ′ , i ′′ ) exists a nd is unique, set ˆ i = i , ˆ i ′ = i ′ and ˆ i ′′ = i ′′ ; otherwise, declare an error . From the values of ˆ k , ˆ i , ˆ i ′ and ˆ i ′′ , the deco der can calculate the messa ge index ˆ w 0 , ˆ w 10 and ˆ w 11 . The d ecoding for receiv e r Y 2 is symmetric. Analysis of Err or P robability: W e only conside r P ( n ) e, 1 since P ( n ) e, 2 can be analyzed symmetrically . WLOG, we assume the transmitted codeword indices are k = i = i ′ = i ′′ = 1 . If an error is declared, one or more of the follo wing ev en ts occ ur . A 1 : T here is n o pair ( v 1 , v 2 ) such that (122) holds . A 2 : u (1 , 1) does not satisfy (123) . A 3 : u ( k , k ′ ) satisfies (123), where ( k , k ′ ) 6 = (1 , 1) . A 4 : v 1 (1 , 1 , 1) doe s no t satisfy (124) . A 5 : v 1 ( i, i ′ , i ′′ ) satisfies (124), where ( i, i ′ , i ′′ ) 6 = (1 , 1 , 1) . (125) October 27, 2018 DRAFT 18 The fact that P r { A 2 } ≤ ǫ and P r { A 4 } ≤ ǫ for sufficiently lar ge n follows directly from the asymp totic equipartition property . W e now examine e rror events A 1 , A 3 , A 5 . Let E ( v 1 , v 2 , u ) denote the ev en t (122). The n P r { E ( v 1 , v 2 , u ) } = X ( u , v 1 , v 2 ) ∈ A ( n ) ǫ p ( u ) p ( v 1 | u ) p ( v 2 | u ) (126) ≥ | A ( n ) ǫ | 2 − n ( H ( U ) + ǫ ) 2 − n ( H ( V 1 | U )+ ǫ ) 2 − n ( H ( V 2 | U )+ ǫ ) (127) ≥ 2 − n ( H ( U ) + H ( V 1 | U )+ H ( V 2 | U ) − H ( U V 1 V 2 )+4 ǫ ) (128) ≥ 2 − n ( I ( V 1 ; V 2 | U )+4 ǫ ) (129) So, P r { A 1 } ≤ Y ( v 1 , v 2 | k ) (1 − P r { E ( v 1 , v 2 , u ) } ) (130) ≤ Y ( v 1 , v 2 | k ) (1 − 2 − n ( I ( V 1 ; V 2 | U )+4 ǫ ) ) (131) From [28], [31], it is clear that if I ( V 1 ; Y 1 | U ) − ǫ 1 − R 11 + I ( V 2 ; Y 2 | U ) − ǫ 2 − R 22 ≥ I ( V 1 ; V 2 | U ) (132) P r { A 1 } ≤ ǫ . For A 3 , we have, from the decod ing rule, P r { A 3 } ≤ ǫ if R 0 + R 10 + R 20 ≤ I ( U ; Y 1 ) . (133) For A 5 , we first note that for ( i, i ′ , i ′′ ) 6 = (1 , 1 , 1) , P { v 1 ( i, i ′ , i ′′ ) , u ( k ) , y 1 ) ∈ A ( n ) ǫ ( V 1 , U, Y 1 ) } ≤ 2 − n ( I ( V 1 ; Y 1 | U ) − 4 ǫ ) (134) Gi ven that the total number of codewords for v 1 is L 11 + L 12 + L 3 = I ( V 1 ; Y 1 | U ) − ǫ 1 , it is e asy to show that if R 11 ≤ I ( V 1 ; Y 1 | U ) − ǫ 1 (135) then P { A 5 } < ǫ for n sufficiently large. Since P ( n ) e 1 ≤ P r ( 5 [ i =1 A i ) ≤ 5 X i =1 P r { A i } , (136) P ( n ) e 1 ≤ 5 ǫ whe n (53), (133) and (135) h old. Symmetrically , for P ( n ) e, 2 ≤ 5 ǫ as n is sufficiently large, we need (132), (133) and R 0 + R 10 + R 20 ≤ I ( U ; Y 2 ) (137) R 22 ≤ I ( V 2 ; Y 2 | U ) − ǫ 1 (138) October 27, 2018 DRAFT 19 Apply Fourier -Motzkin elimination on (132), (133), (135), (137) and (138) with the definition R 1 = R 11 + R 10 and R 2 = R 22 + R 20 , w e get (50)-(53). Equivocation: Now , we prove the bound on equiv oca tion rate (54). Eq. (55) follows b y symmetry . H ( W 1 | Y 2 ) ≥ H ( W 1 | Y 2 , V 2 , U ) (139) = H ( W 11 , W 10 | Y 2 , V 2 , U ) (140) ( a ) = H ( W 11 | Y 2 , V 2 , U ) (141) = H ( W 11 , Y 2 | V 2 , U ) − H ( Y 2 | V 2 , U ) (142) = H ( W 11 , V 1 , Y 2 | V 2 , U ) − H ( Y 2 | V 2 , U ) − H ( V 1 | Y 2 , V 2 , U , W 11 ) = H ( W 11 , V 1 | V 2 , U ) + H ( Y 2 | V 1 , V 2 , U , W 11 ) − H ( Y 2 | V 2 , U ) − H ( V 1 | Y 2 , V 2 , U , W 11 ) (143) ( b ) = H ( V 1 | V 2 , U ) − I ( V 1 ; Y 2 | V 2 , U ) − H ( V 1 | Y 2 , V 2 , U , W 11 ) = H ( V 1 | U ) − I ( V 1 ; V 2 | U ) − I ( V 1 ; Y 2 | V 2 , U ) − H ( V 1 | Y 2 , V 2 , U , W 11 ) (144) where (a) follo ws from the fact that given U , W 10 is unique ly de termined, a nd (b) follo ws from the fact that giv e n V 1 , W 11 is uniquely determined. Consider the first term in (144), the codeword generation ensu res that H ( V 1 | U ) = log 2 n ( L 11 + L 12 + L 3 ) = nI ( V 1 ; Y 1 | U ) − nǫ 1 . (145) For the se cond and third terms in (144), u sing the same approach as that in [18, Lemma 3], we obtain I ( V 1 ; V 2 | U ) ≤ nI ( V 1 ; V 2 | U ) + nǫ ′ 2 (146) I ( V 1 ; Y 2 | V 2 , U ) ≤ nI ( V 1 ; Y 2 | V 2 U ) + n ǫ ′ 3 (147) Now , we co nsider the last term of (144). W e first prove that, given V 2 , U and W 11 , the probab ility of error for Y 2 to decode V 1 satisfies P e ≤ ǫ for n s ufficiently lar ge . Y 2 looks for v 1 such that ( v 1 , v 2 , u , y 2 ) ∈ A ( n ) ǫ ( V 1 , V 2 , U, Y 2 ) . (148) Since R 11 ≥ L 11 , and giv e n the k nowledge of W 11 , the total number of possible codewords o f v 1 is N 1 ≤ 2 n ( L 12 + L 3 ) = 2 n ( I ( V 1 ; V 2 Y 2 | U ) − ǫ 1 ) . (149) Now define E ( v 1 , v 2 , u, y 2 ) the event in (148 ). W e have P r { E ( v 1 , v 2 , u, y 2 ) } = X ( u , v 1 , v 2 , y 2 ) ∈ A ( n ) ǫ p ( u ) p ( v 1 | u ) p ( v 2 , y 2 | u ) (15 0) ≤ | A ( n ) ǫ | 2 − n ( H ( U ) − ǫ ) 2 − n ( H ( V 1 | U ) − ǫ ) 2 − n ( H ( V 2 Y 2 | U ) − ǫ ) (151) ≤ 2 − n ( H ( U ) + H ( V 1 | U )+ H ( V 2 Y 2 | U ) − H ( U V 1 V 2 Y 2 ) − 4 ǫ ) (152) ≤ 2 − n ( I ( V 1 ; V 2 Y 2 | U ) − 4 ǫ ) (153) October 27, 2018 DRAFT 20 Now , the proba bility of error for Y 2 to decode V 1 is P e ≤ ǫ + N 1 · 2 − n ( I ( V 1 ; V 2 Y 2 | U ) − 4 ǫ ) (154) ≤ ǫ + 2 − n ( ǫ 1 − 4 ǫ ) (155) ≤ 2 ǫ (156) where the first ǫ acc ounts for the error tha t the true V 1 is not jointly typ ical with V 2 , U , Y 2 while the second term acco unts for the error when a diff e rent V 1 is jointly typical with V 2 , U , Y 2 . By Fano’ s inequality [32], we get H ( V 1 | Y 2 , V 2 , U , W 11 ) ≤ nǫ ′ n . (157) Combine (145), (146), (147) and (157), we have the bound (54). The above proof is only for the case when (11 1) and (11 2) are sa tisfied. By using the same con vexity argument as in Lemma 5 and Lemma 6 in [4], we can easily show that the region (48)-(55) is also achiev a ble. This completes the proof for Theorem 1. A P P E N D I X I I P RO O F O F T H E O U T E R B O U N D S I N T H E O R E M 2 W e on ly prove R O 2 and R O 3 are ou ter b ounds in this section. T he p roof o f Th eorem 2 is complete by the fact that R O 1 = R O 2 (cf. Proposition 10). W e first define the follo wing notations /quantities. All vec tors in volv e d are assumed to be leng th n . X i △ = ( X 1 , · · · , X i ); (158) ˜ X i △ = ( X i , · · · , X n ); (159) Σ 1 = n X i =1 I ( ˜ Y i +1 2 ; Y 1 i | Y i − 1 1 W 0 ); (160) Σ ∗ 1 = n X i =1 I ( Y i − 1 1 ; Y 2 i | ˜ Y i +1 2 W 0 ); (161) and (Σ 2 , Σ ∗ 2 ) , (Σ 3 , Σ ∗ 3 ) , (Σ 4 , Σ ∗ 4 ) are an alogously defined by replacing W 0 with W 0 W 1 , W 0 W 2 and W 0 W 1 W 2 in Eqs. (160) and (161), respectiv ely . In exactly the same fashion as in [4, Le mma 7], one can establish, for a = 1 , 2 , 3 , 4 , Σ a = Σ ∗ a . (162) W e begin by Fano’ s Le mma, H ( W 0 , W 1 | Y n 1 ) ≤ nǫ n , H ( W 0 , W 2 | Y n 2 ) ≤ nǫ n . October 27, 2018 DRAFT 21 where ǫ n → 0 a s n → ∞ . E qs. (57) and (58) follo w trivially from 0 ≤ H ( W 1 | Y n 2 ) ≤ H ( W 1 ) , (163) 0 ≤ H ( W 2 | Y n 1 ) ≤ H ( W 2 ) . (164) Next we c heck bound for R 0 . nR 0 = H ( W 0 ) = I ( W 0 ; Y n 1 ) + H ( W 0 | Y n 1 ) ≤ n X i =1 I ( W 0 ; Y 1 i | Y i − 1 1 ) + nǫ n (165) = n X i =1 ( I ( W 0 Y i − 1 1 ; Y 1 i ) − I ( Y i − 1 1 ; Y 1 i )) + nǫ n (166) ≤ n X i =1 ( I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) − I ( ˜ Y i +1 2 ; Y 1 i | Y i − 1 1 W 0 )) + nǫ n (167) = n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) − Σ 1 + nǫ n (168) ≤ n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) + nǫ n (169) (170) Similarly , nR 0 ≤ n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 2 i ) − Σ ∗ 1 + nǫ n (171) ≤ n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 2 i ) + nǫ n (172) Therefore nR 0 ≤ min " n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) , n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 2 i ) # + nǫ n . (173) Consider the sum rate bound for R 0 + R 1 . n ( R 0 + R 1 ) = H ( W 0 , W 1 ) = H ( W 0 ) + H ( W 1 | W 0 ) (174) = H ( W 0 ) + I ( W 1 ; Y n 1 | W 0 ) + H ( W 1 | Y n 1 W 0 ) (175) ≤ H ( W 0 ) + I ( W 1 ; Y n 1 | W 0 ) + nǫ n (176) October 27, 2018 DRAFT 22 where I ( W 1 ; Y n 1 | W 0 ) (177) = n X i =1 I ( W 1 ; Y 1 i | Y i − 1 1 W 0 ) (178) = n X i =1 ( I ( W 1 ˜ Y i +1 2 ; Y 1 i | Y i − 1 1 W 0 ) − I ( ˜ Y i +1 2 ; Y 1 i | Y i − 1 1 W 0 W 1 )) (179) = n X i =1 ( I ( W 1 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) + I ( ˜ Y i +1 2 ; Y 1 i | Y i − 1 1 W 0 ) − I ( ˜ Y i +1 2 ; Y 1 i | Y i − 1 1 W 0 W 1 )) (180 ) = n X i =1 I ( W 1 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) + Σ 1 − Σ 2 . (181) Combine (168), (176), and (181), we have n ( R 0 + R 1 ) ≤ n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) + n X i =1 I ( W 1 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) − Σ 2 + 2 nǫ n . (182) On the other hand, combining (171), (176), (181), and (162) yields n ( R 0 + R 1 ) ≤ n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 2 i ) + n X i =1 I ( W 1 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) − Σ 2 + 2 nǫ n . (183) Thus, n ( R 0 + R 1 ) ≤ min " n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) , n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 2 i ) # + n X i =1 I ( W 1 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) − Σ 2 + 2 nǫ n (184) ≤ min " n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) , n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 2 i ) # + n X i =1 I ( W 1 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) + 2 nǫ n (185) In an analogous fashion, we can ge t n ( R 0 + R 2 ) ≤ min " n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) , n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 2 i ) # + n X i =1 I ( W 1 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) − Σ 3 + 2 nǫ n (186) ≤ min " n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) , n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 2 i ) # + n X i =1 I ( W 1 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) + 2 nǫ n (187) October 27, 2018 DRAFT 23 Consider the sum rate bound for R 0 + R 1 + R 2 . n ( R 0 + R 1 + R 2 ) = H ( W 0 , W 1 ) + H ( W 2 | W 1 W 0 ) (188) = H ( W 0 , W 1 ) + I ( W 2 ; Y n 2 | W 1 , W 0 ) + H ( W 2 | Y n 2 W 0 W 1 ) (189) ≤ H ( W 0 , W 1 ) + I ( W 2 ; Y n 2 | W 1 , W 0 ) + nǫ n , (190) n ( R 0 + R 1 + R 2 ) = H ( W 0 , W 2 ) + H ( W 1 | W 2 W 0 ) (191) = H ( W 0 , W 2 ) + I ( W 1 ; Y n 1 | W 2 , W 0 ) + H ( W 1 | Y n 1 W 0 W 2 ) (192) ≤ H ( W 0 , W 2 ) + I ( W 1 ; Y n 1 | W 2 , W 0 ) + nǫ n . (193) Follo wing similar procedure as in (178)-(181), we can obtain I ( W 2 ; Y n 2 | W 1 , W 0 ) = n X i =1 I ( W 2 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 W 1 ) + Σ ∗ 2 − Σ ∗ 4 . (194) I ( W 1 ; Y n 1 | W 2 , W 0 ) = n X i =1 I ( W 1 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 W 2 ) + Σ 3 − Σ 4 , (195) Combine (184), (190), (194), and (162), we get n ( R 0 + R 1 + R 2 ) ≤ min " n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) , n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 2 i ) # + n X i =1 I ( W 1 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) + n X i =1 I ( W 2 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 W 1 ) + 3 nǫ n . (196) Alternati vely , c ombining (186), (193), (195), and (162) yields n ( R 0 + R 1 + R 2 ) ≤ min " n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 1 i ) , n X i =1 I ( W 0 Y i − 1 1 ˜ Y i +1 2 ; Y 2 i ) # + n X i =1 I ( W 2 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) + n X i =1 I ( W 1 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 W 2 ) + 3 nǫ n . (197) W e now conside r the equ i vocation rate bo und. R e 1 ≤ H ( W 1 | Y n 2 ) (198) = H ( W 1 | Y n 2 W 0 ) + I ( W 1 ; W 0 | Y n 2 ) (199) ≤ H ( W 1 | W 0 ) − I ( W 1 ; Y n 2 | W 0 ) + H ( W 0 | Y n 2 ) (200) = I ( W 1 ; Y n 1 | W 0 ) − I ( W 1 ; Y n 2 | W 0 ) + H ( W 1 | Y n 1 W 0 ) + H ( W 0 | Y n 2 ) (201) ≤ I ( W 1 ; Y n 1 | W 0 ) − I ( W 1 ; Y n 2 | W 0 ) + 2 nǫ n , (202) R e 1 ≤ H ( W 1 | Y n 2 ) (203) = H ( W 1 | Y n 2 W 0 W 2 ) + I ( W 1 ; W 0 W 2 | Y n 2 ) (204) ≤ H ( W 1 | W 0 W 2 ) − I ( W 1 ; Y n 2 | W 0 W 2 ) + H ( W 0 W 2 | Y n 2 ) (205) October 27, 2018 DRAFT 24 = I ( W 1 ; Y n 1 | W 0 W 2 ) − I ( W 1 ; Y n 2 | W 0 W 2 ) + H ( W 1 | Y n 1 W 0 W 2 ) + H ( W 0 W 2 | Y n 2 ) (206) ≤ I ( W 1 ; Y n 1 | W 0 W 2 ) − I ( W 1 ; Y n 2 | W 0 W 2 ) + 2 nǫ n .. (207) Of the terms in volved in (202) and (20 7), only I ( W 1 ; Y n 2 | W 0 ) and I ( W 1 ; Y n 2 | W 0 W 2 ) have yet to be determined. Similar to (178)-(181), we can get I ( W 1 ; Y n 2 | W 0 ) = n X i =1 I ( W 1 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) + Σ ∗ 1 − Σ ∗ 2 , (208) I ( W 1 ; Y n 2 | W 0 W 2 ) = n X i =1 I ( W 1 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 W 2 ) + Σ ∗ 3 − Σ ∗ 4 . (209) Therefore we get R e 1 ≤ n X i =1 I ( W 1 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) − n X i =1 I ( W 1 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) + 2 nǫ n , (210) R e 1 ≤ n X i =1 I ( W 1 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 W 2 ) − n X i =1 I ( W 1 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 W 2 ) + 2 nǫ n . (211) Bounds on R e 2 are analogous ly obtained: R e 2 ≤ n X i =1 I ( W 2 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) − n X i =1 I ( W 2 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 ) + 2 nǫ n , (212) R e 2 ≤ n X i =1 I ( W 2 ; Y 2 i | Y i − 1 1 ˜ Y i +1 2 W 0 W 1 ) − n X i =1 I ( W 2 ; Y 1 i | Y i − 1 1 ˜ Y i +1 2 W 0 W 1 ) + 2 nǫ n . (213) Let us introduce a random variable J , independe nt of W 0 W 1 W 2 X n Y n 1 Y n 2 , uniformly dis trib uted over { 1 , · · · , n } . Set U , W 0 Y J − 1 1 ˜ Y 2 J +1 J, V 1 , W 1 U, V 2 , W 2 U, X , X J , Y 1 , Y 1 J , Y 2 , Y 2 J . Substituting these definitions into Eqs. (173) , (185 ), (187), (196, (197), and (210) -(213), we obtain, through standard information the oretic ar gu ment, the de sired bounds as in Eqs. (57)-(65). The memoryles s property of the channe l guaran tees U → V 1 V 2 → X → Y 1 Y 2 . This complete s the proof. T o prove R O 3 is a lso an outer bo und, we follo w exac tly the sa me procedu re except that aux iliary random variables are defin ed d if ferently . Specifica lly , U , W 0 Y J − 1 1 ˜ Y J +1 2 J, V 1 , W 1 , V 2 , W 2 . A P P E N D I X I I I P RO O F O F P RO P O S I T I O N 1 2 By s imple alge bra, o ne can show R B C − O 3 ⊆ R N E . The fact that R B C − O 3 = R N E when R 0 = 0 can also be verified b y direct substitution. October 27, 2018 DRAFT 25 W e no w prove the e quiv alen ce unde r R 2 = 0 , and the case for R 1 = 0 can be estab lished b y index swapping. W ith R 2 = 0 , Eqs . (100)-( 104) of R B C − O 3 can be easily shown to be equiv a lent to R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] , (214) R 0 + R 1 ≤ I ( V 1 ; Y 1 | U ) + min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] , (215) W e note this is prec isely the capac ity region for DMBC with degraded mess age se t [4, Co rollary 5]. W ith R 2 = 0 , R N E in Proposition 7 reduces to R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] , (216) R 0 + R 1 ≤ I ( V 1 U ; Y 1 ) , (217) R 0 + R 1 ≤ I ( V 1 ; Y 1 | V 2 U ) + I ( U V 2 ; Y 2 ) . (218) Apparently R B C − O 3 ⊆ R N E , and it remains to check R N E ⊆ R B C − O 3 . Assume ( R 0 , R 1 ) ∈ R N E and ( U, V 1 , V 2 , X , Y 1 , Y 2 ) ∈ Q 3 are the variables suc h that Eqs. (216)-(218) a re s atisfied. Con sider three cases for analysis. 1) I ( U ; Y 1 ) ≤ I ( U ; Y 2 ) . The proof of ( R 0 , R 1 ) ∈ R B C − 03 is tri vial. 2) I ( U ; Y 1 ) ≥ I ( U ; Y 2 ) and I ( V 2 , U ; Y 1 ) ≥ I ( V 2 , U ; Y 2 ) . Define V ′ 1 = V 1 , U ′ = U V 2 . From (216), R 0 ≤ min[ I ( U ; Y 1 ) , I ( U ; Y 2 )] (219) ≤ min[ I ( U V 2 ; Y 1 ) , I ( U V 2 ; Y 2 )] (22 0) = min[ I ( U ′ ; Y 1 ) , I ( U ′ ; Y 2 )] (221) From (218), R 0 + R 1 ≤ I ( V 1 ; Y 1 | U V 2 ) + I ( U V 2 ; Y 2 ) (222) = I ( V ′ 1 ; Y 1 | U ′ ) + I ( U ′ ; Y 2 ) (223) Thus ( R 0 , R 2 ) also satisfies (214) and (215) for U ′ V ′ 1 → X → Y 1 Y 2 . 3) I ( U ; Y 1 ) ≥ I ( U ; Y 2 ) and I ( V 2 , U ; Y 1 ) ≤ I ( V 2 , U ; Y 2 ) . For this case, we c an always find a function g ( · ) such that I ( U g ( V 2 ); Y 1 ) = I ( U g ( V 2 ); Y 2 ) . (224) Define V ′ 1 = V 1 , U ′ = U g ( V 2 ) and we can verif y that ( R 0 , R 1 ) satisfies (214) and (215) f or U ′ V ′ 1 → X → Y 1 Y 2 . The above ar gument comp letes the proof of Propo sition 1 2. October 27, 2018 DRAFT 26 R E F E R E N C E S [1] C.E . S hannon, “Communication t heory of secrecy systems, ” Bell System T echnica l J ournal , vo l . 28, pp. 656–714, 1949. [2] R. L . Rivest, A. Shamir , and L. Adleman, “A method for obtaining digital signatures and public-key cryptosystems, ” Commun. ACM , vo l. 21, no. 2, pp. 120–126, 1978. [3] A. D. W yner , “The wire-tap channel, ” The Bell Systems T echnical J ournal , vo l . 54, pp. 1355–1387, October 1975. [4] I. Csisz ´ ar and J. K ¨ o rner, “Boradcast channels with confidential messages, ” IEEE Tr ans. Information Theory , vol. 24, pp. 339–34 8, May 1978. [5] Y . Oohama, “Coding for relay channels with confiden t ial messages, ” i n Proc. IE EE Information T heory W orkshop , Cairns, Australia, Sep. 2001 , pp. 87–89. [6] I. Csi sz ´ ar and P . Narayan, “Secrecy capacities for multiple terminal, ” IEEE Tr ans. Inform. Theory , vol. 50, no. 12, pp. 3047–3 061, Dec. 2004. [7] E. T ekin and A. Y ener, “The Gaussian multiple access wire-tap channel with collective secrecy constraints, ” in Pr oc. IEEE ISIT’06 , Seattle, USA, Jul. 2006. [8] Y . L iang and H.V . P oor , “Generalized multiple access channels wi th confidential messages, ” submitted t o IEEE Tr ans. Inform. Theory , A pril 2006. [9] R. Liu, I. Mari ´ c, P . Spasoje vi ´ c, and R.D. Y ates, “Discrete memoryless interfernce and broadcst channels with confidential messages, ” in Proc . Al lerton confer ence on Communication, Contr ol and Computing , S ep. 2006. [10] L. Lai and H. El Gamal, “The rel ay-eav esdropper channel: cooperation for secrecy, ” submitted to IEEE T rans. Inform. Theory , Dec. 2006 . [11] E. T ekin and A. Y ener , “The multiple access wire-tap channel: W i reless secrecy and cooperati ve j amming, ” in Pr oc. Information Theory and Application W orkshop , San Diego , USA, Jan. 2007. [12] E. T ekin and A. Y ener , “The general G aussian multi ple access and two-wa y wire-tap channels: achiev able rates and cooperati ve jamming, ” to appea r in the IE EE Tr ans. Inf. Theory , 2008. [13] Y . Li ang, H.V . Poor , and S. Shamai(Shitz), “Secure communication over fading channels, ” submitted to IEEE T rans. Inform. Theory , N ov . 2006. [14] P . Gopala, L. Lai, and H. El Gamal, “On the secrecy capacity of fading channels, ” in P r oc. IEEE Int. Symp. Information Theory (ISIT) , Nice, France, June 2007. [15] Z. Li, W . T r appe, and R.D. Y ates, “S ecret communication via multi-antenna transmission, ” in Pr oc. A nnual Confer ence on Information Sciences and Systems (CISS) , Baltimore, MD, Mar . 2007. [16] A. Khisti, G. W ornell, A. W i esel, and Y . Eldar , “On the Gaussian MIMO wiretap channel, ” in P r oc. IE EE Int. Symp. Information Theory (ISIT ) , Nice, France, Jun. 2007. [17] S. Shafiee, N. Liu, and S. Ulukus, “T ow ards the secrecy capcity of the Gaussian MIMO wi retap channel: the 2-2-1 channel, ” submitted to IEEE T rans. Inform. Theory , Sep. 2007. [18] R. Liu, I. Mari ´ c, P . Spasojev i ´ c, and R. D. Y ates, “Discrete memoryless interference and broadc ast channels w ith confidential messages: secrec y r ate r egions , ” to appear in the IEEE T rans. Information Theory , 2008. [19] K. Marton, “A coding theorem for the discrete memoryless broadcast channel, ” IEEE Tr ans. I nform. Theory , vol. 25, no. 1, pp. 306–311, May 1979. [20] S.I. Gel’fand and M.S. Pinsker , “Capacity of a broadcast channel with one deterministic compone nt, ” Pr obl. Inform. T ransm. , vol. 16, no. 1, pp. 17–25, Jan.-Mar . 1980. [21] C. Nair and A. El Gamal, “ An outer bound to the capacity region ofthe broadcast channel, ” EEE T rans. Inf. Theory , vol. 53, pp. 350–355, Jan. 2007. [22] Y . Liang and G. Kramer , “Capacity theorems f or cooperati ve relay broadcast channels, ” IEEE Tr ans. on Inform. Theory , vol. 53, no. 10, pp. 3517–3535, Oct. 2007. [23] Y . Liang, G. Kramer , and S. Shamai ( Shitz), “Capacity outer bounds for broadcast channels, ” in Pr oc. IEEE Information Theory W orkshop , Porto, Portugal, May 2008. [24] T .M. Cov er, “Broadcast channels, ” IEE E Tr ans. Inf. T heory , vol. 21, pp. 2–14, 1972. October 27, 2018 DRAFT 27 [25] T .M. Cov er, “Comments on broadcast channe ls, ” IEEE T rans. Inf. Theory , vo l . 44, pp. 2524–2530, Oct. 1998. [26] Y . L iang and G. Kramer , “Capacity t heorems for cooperati ve relay broadcast channels, ” in Proc. Annual Confer ence on Information Sciences and Systems , Princeton, NJ, Mar . 2006. [27] C. Nair and A. El Gamal, “An outer bound to the capacity region of the broadcast channe l, ” in Pr oc. Internationa l Symposium on Information T heory , Seattl e, W A, July 2006. [28] A. El Gamal and E.C. van der Meulen, “ A proof of marton’ s coding t heorem for the discrete memoryless broadcast channel, ” IE EE Tr ans. on Inf. Theory , vol. 27, pp. 120–122 , Jan. 1981. [29] Y . Chen and A.J. Han V inck, “W iretap channel with side information, ” in P r oc. International Symposium on Information Theory , Seattle, W A, July 2006. [30] J. K ¨ orner and K. Marton, “Comparison of two no isy channels, ” T opics in Information T heory , Keszthely (Hungary), 1975, Colloquia Mathematica Societati s Janos Bolyai, Amsterdam: North-Holland Publ., pp. 411-423, 1977. [31] G. Kramer , T opics in Multi-User Information T heory , NO W Publishers Inc., Hanov er, MA, 2008. [32] T .M. Cov er and J.A. Thomas, Elements of Information Theory , W i ley , New Y ork, 1991. October 27, 2018 DRAFT