New Bounds for the Capacity Region of the Finite-State Multiple Access Channel

Ne w Bounds for the Capac ity Re gion of the Finite-Stat e Mu ltiple Access Channel Haim Permuter ∗ , Tsachy W eissman ∗ † and Jun Chen ‡ , ∗ Departmen t of E lectrical Engineer ing, Stanford Un i versity , Stanfor d, CA, USA, { haim1, tsachy } @stanford.e du † Departmen t of E lectrical E ngineerin g, T echnion, Haifa, Israel ‡ Departmen t of E lectrical an d Com puter En gineerin g, McMaster Un iv ersity , Hamilton , Ontario, Canada, ju nchen@ece.m cmaster .ca Abstract — The capacity region of the Finite-S tate Multiple Access Channel (FS-MA C) with feedback that may be an arbitrary time-inv ariant function of the channel outpu t samples is considered. W e prov ided a sequence of inn er and outer b ounds fo r this region. These bounds are shown to coinci de, and hen ce yield the capacity region, of FS-MA Cs where th e state process is stationary and ergodic and not affected by the in puts, and for indecomposable FS-M A C when f eedback is not allowed. Though the capacity reg ion is ‘multi-l etter’ in general, our results yield explicit conclusions wh en appli ed to specific scenarios of int erest. I . I N T RO D U C T I O N The Multiple Access Chann el (MA C) has receiv ed much attention in the liter ature. T o put our con tributions in co ntext, we begin by briefly describing some o f the key r esults in the are a. The c apacity region f or the memory less MAC was derived by Ahlswede in [1]. Cover and Leun g derived an achiev able region for a mem oryless MA C with feedback in [2]. Ozarow der iv ed the cap acity of a me moryless Ga ussian MA C with feedb ack in [3] In [4] [5], Kr amer deri ved several capac ity r esults for discrete memory less networks with feedb ack. By using the idea of code -trees in stead of code- words, Kramer derived a ‘mulit-letter’ expre ssion for the cap acity of the discrete memory less MAC . One of the main results we de velop in the present p aper extend s Krame r’ s ca pacity result to the case o f a stationary and ergod ic Markov Finite-State MA C ( FS-MA C), to be fo rmally d efined below . In [6], Han u sed the inform ation-spectru m m ethod in order to deriv e the capacity of a g eneral MA C withou t fe edback. Han also conside red the additi ve mod - 2 MA C, which we shall use here to illustrate th e way in which our genera l results characterize special cases o f interest. In particu lar , ou r results will imp ly that feedback does not increase th e cap acity region of th e additive mo d- 2 MAC. In this work, we consider the capacity region of the Finite- State Mu ltiple Access Cha nnel (FS-MAC), with feedback th at may be a n arb itrary time-inv ariant fu nction of the channel output samples. W e character ize a sequen ce o f inn er an d outer bound s fo r this region and show that it yields the ca pacity region, fo r the importa nt subfamily of FS-MACs. Our d eriv ation of the c apacity region is ro oted in th e d eriv a - tion of the capac ity of finite-state channels in Gallager ’ s b ook [7, ch 4,5]. More recently , Lapido th and T elatar [8] have used it in order to derive the capacity of a compo und c hannel with out feedback , where the comp ound chann el consists of a family o f finite-state channe ls. In particular , they have introd uced into Gallager’ s proo f the ide a of concaten ating codewords, wh ich we extend here to co ncatenating code-trees. The pap er is organized as follows. W e co ncretely describe the co mmunicatio n model in Section I I. In Section III, we introdu ce the causal condition ing, d irected inform ation and an importan t idea of sup /sub-additivity of regions. W e state our main capac ity results in Section s IV and V, and we present a few applications o f the cap acity results in Sectio n VI . Because of space limitation we d o n ot provide the p roofs. The p roofs, with the exceptions of Lemmas 3 and 5, and Theorem 4, can be f ound in th e pr eprint [9 ]. I I . C H A N N E L M O D E L In this paper, we consider an FS-MA C (Finite State MA C) with a time in variant feed back as illustrated in Fig. 1. The P S f r a g r e p l a c e m e n t s Encoder 1 x 1 ,i ( m 1 , z i − 1 1 ) Encoder 2 x 2 ,i ( m 2 , z i − 1 2 ) m 1 { 1 , ..., 2 nR 1 } m 2 { 1 , ..., 2 nR 2 } Finite State MAC P ( y i , s i | x 1 ,i , x 2 ,i , s i − 1 ) z 2 , i = f 2 ( y i ) Time Time Inv ariant Inv ariant z 2 ,i ( y i ) z 1 ,i ( y i ) z 2 ,i − 1 z 1 ,i − 1 Decoder Unit Unit Delay Delay ˆ m 1 ( y n ) ˆ m 2 ( y n ) ˆ m 1 , ˆ m 2 F u n c t i o n z i − 1 ( y i − 1 ) y i y i ˆ m E s t i m a t e d M e s s a g e Fig. 1. Channel with feedback that is a time inv ariant deterministic function of the output. MA C setting co nsists of two senders an d o ne receiver . Each sender l ∈ { 1 , 2 } ch ooses an ind ex m l unifor mly fro m the set { 1 , ..., 2 nR l } and indep endently o f the other sender . The inpu t to the channel from encoder l is denote d by { X l 1 , X l 2 , X l 3 , ... } , and the output of th e chann el is deno ted by { Y 1 , Y 2 , Y 3 , ... } . T he state at time i , i.e., S i ∈ S , takes values in a finite set of possible states. The channel is stationary and is charac terized by a cond itional p robability P ( y i , s i | x 1 i , x 2 i , s i − 1 ) that satisfies P ( y i , s i | x i 1 , x i 2 , s i − 1 , y i − 1 ) = P ( y i , s i | x 1 i , x 2 i , s i − 1 ) , (1) where th e sup erscripts d enote sequ ences in the f ollowing way: x i l = ( x l 1 , x l 2 , ..., x li ) , l ∈ { 1 , 2 } . W e assume a commun ication with feedb ack z i l , where the element z li is a time-inv ariant f unction of the output y i . For examp le, z li could equal y i (perfect feedb ack), or a q uantized version of y i , or nu ll (n o fe edback) . The enco ders receive th e feedback samples with on e un it delay . A c ode with feed back con sists of two encod ing fun ctions g l : { 1 , ..., 2 nR 1 } × Z n − 1 l → X n l , l = 1 , 2 , where the k t h coordin ate of x n l ∈ X n l is giv en by the function x lk = g lk ( m l , z k − 1 l ) , k = 1 , 2 , . . . , n, l = 1 , 2 (2) and a d ecoding fu nction, g : Y n → { 1 , ..., 2 nR 1 } × { 1 , ..., 2 nR 2 } . (3) The averag e pr o bability of err or for ((2 nR 1 , 2 nR 2 , n ) code is defined as P ( n ) e = 1 2 n ( R 1 + R 2 ) X w 1 ,w 2 Pr { g ( Y n ) 6 = ( w 1 , w 2 ) | ( w 1 , w 2 ) sen t } . (4) A rate ( R 1 , R 2 ) is said to be achievable for the MA C if there exists a sequence of ((2 nR 1 , 2 nR 2 ) , n ) codes with P ( n ) e → 0 . Th e capacity re gion o f MAC is the closure of th e set of achiev able ( R 1 , R 2 ) r ates. I I I . P R E L I M I N A R I E S A. Causal condition ing and dir ected informatio n Throu ghout this paper we use th e causal co nditionin g notation ( ·||· ) . W e d enote the probability mass functio n (pm f) of Y N causally condition ed on X N − d , for some integer d ≥ 0 , as P ( y N || x N − d ) wh ich is d efined as P ( y N || x N − d ) , N Y i =1 P ( y i | y i − 1 , x i − d ) , (5) (if i − d ≤ 0 then x i − d is set to n ull). In p articular, we extensi vely use th e cases wher e d = 0 , 1 : P ( y N || x N ) , N Y i =1 P ( y i | y i − 1 , x i ) (6) Q ( x N || y N − 1 ) , N Y i =1 Q ( x i | x i − 1 , y i − 1 ) , (7) where the letters Q and P are b oth u sed f or d enoting pmfs. The Directed in formation was defined by Massey in [10] as I ( X N → Y N ) , N X i =1 I ( X i ; Y i | Y i − 1 ) , (8) and in [4], Kramer introd uced the notation I ( X N 1 → Y N || X N 2 ) , N X i =1 I ( X i 1 ; Y i | Y i − 1 , X i 2 ) . (9) Directed in ofrmation has b een widely u sed in the charac- terization of capacity of chan nels [ 4], [11]–[16 ], an d rate distortion f unction [17], [18]. Thr ougho ut th e pr oofs, we ar e using sev eral p roperties of causal conditionin g and dir ected informa tion. W e summarize them in the following lemm a. Lemma 1 T he following fou r p roperties ,(10)-(13), hold for any discrete rando m vectors ( X N 1 , X N 2 , Y N ) , P ( x N 1 , y N || x N 2 ) = P ( x N 1 || y N − 1 , x N 2 ) P ( y N || x N 1 , x N 2 ) . (10) | I ( X N 1 → Y N || X N 2 ) − I ( X N 1 → Y N || X N 2 , S ) | ≤ H ( S ) . (11) I ( Q ( x N 1 || y N − 1 ) , Q ( x N 2 || y N − 1 ) , P ( y N || x N 1 , x N 2 )) = I ( X N 1 → Y N || X N 2 ) , (1 2) where I ( Q ( x 1 ) , Q ( x 2 ) , P ( y | x 1 , x 2 )) d enotes the fun ctional I ( X 1 ; Y | X 2 ) , i.e., I ( Q ( x 1 ) , Q ( x 2 ) , P ( y | x 1 , x 2 )) , X y ,x 1 ,x 2 Q ( x 1 ) Q ( x 2 ) P ( y | x 1 , x 2 ) log P ( y | x 1 , x 2 ) P x ′ 1 Q ( x ′ 1 ) P ( y | x ′ 1 , x 2 ) . If there is no feedback , i.e., Q ( x N 1 , x N 2 || y N − 1 ) = Q ( x N 1 ) Q ( x N 2 ) , then I ( X N 1 ; Y N | X N 2 ) = I ( X N 1 → Y N || X N 2 ) . (13) B. Sup /Sub-a dditivity , a nd Con ver gence o f 2D r e gions In this subsection we define basic operation s ( summation and mu ltiplication by scalar) , conver gence, sup-ad ditivity and sub-add iti vity of 2D regions. Fur thermore we show that the limit o f a sup-additive sequence of r egions converges to the union o f all the region s, and the limit of a sub-ad ditive and conv ex sequ ence 2D r egions conver ges to the intersectio n o f all the regions. Let A , B be sets in R 2 , i.e., A and B are sets of 2D vectors. The sum of two r egions is denote d as A + B and defined as A + B = { a + b : a ∈ A , b ∈ B } , (14) and multiplication o f a set A with a scalar c is defined as c A = { c a : a ∈ A} . (15) A sequence {A n } , n = 1 , 2 , 3 , ..., of 2D r egions is said to conver ge to a region A , wr itten A = lim A n if lim sup A n = lim inf A n = A , (16) where lim inf A n = { a : a = lim a n , a n ∈ A n } , lim sup A n = { a : a = lim a k , a k ∈ A n k } , (17) and n k denotes an arbitrary increasing subsequ ence of the integers. Let us deno te A = cl  S n ≥ 1 A n  and A = cl  T n ≥ 1 A n  . W e say tha t a sequence {A n } n ≥ 1 is b ounde d if sup {|| a || : a ∈ A} < ∞ where || · || denotes a norm in R 2 . Lemma 2 L et A n , n = 1 , 2 , ... , be a bounded sequen ce of sets in R 2 that in cludes the origin, i.e., (0 , 0) . If n A n is sup - additive, i.e., for all n ≥ 1 and all N > n N A N ⊇ n A n + ( N − n ) A N − n (18) then lim n →∞ A n = A . (19) Lemma 3 L et A n , n = 1 , 2 , ... , be a sequence of conve x, closed and bou nded sets in R 2 . If n A n is sub-ad ditiv e, i.e., for all n ≥ 1 and all N > n N A N ⊆ n A n + ( N − n ) A N − n (20) then lim n →∞ A n = A . (21) I V . F S - M AC W I T H T I M E - I N V A R I A N T F E E D B A C K A. Inne r Boun d Let R n denote the following region in R 2 + (2D set of nonnegative real numbers): R n = [ Q      R 1 ≤ min s 0 1 n I ( X n 1 → Y n || X n 2 , W, s 0 ) − log |S | n , R 2 ≤ min s 0 1 n I ( X n 2 → Y n || X n 1 , W, s 0 ) − log |S | n , R 1 + R 2 ≤ min s 0 1 n I (( X 1 , X 2 ) n → Y n | W , s 0 ) − log |S | n , (22) where the union is over the set of all input distributions of the form Q ( w ) Q ( x n 1 || z n − 1 1 , w ) Q ( x n 2 || z n − 1 2 , w ) . Having th e auxiliary random variable W is equi valent to tak ing the con ve x hull of the region. Further more, the set of three-ineq ualities is equiv alent to an inter section of three regions, and min s 0 is equiv alent to ∩ s 0 . Hen ce, an eq uiv alent region is R n = conv [ Q \ s 0      R 1 ≤ 1 n I ( X n 1 → Y n || X n 2 , s 0 ) − log |S | n , R 2 ≤ 1 n I ( X n 2 → Y n || X n 1 , s 0 ) − log |S | n , R 1 + R 2 ≤ 1 n I (( X 1 , X 2 ) n → Y n | s 0 ) − log |S | n , (23) where conv de notes the conv ex hull, and th e input distribution is o f the f orm Q ( x n 1 || z n − 1 1 ) Q ( x n 2 || z n − 1 2 ) . In general, the right hand side (RHS) of each of the three inequalities that define R n can be n egati ve. In such a case, we assume that the RHS is ze ro. Theorem 1 ( Inner bound . ) F or any FS-MA C with ti me in v ari- ant f eedback as shown in Fig . 1, and fo r any integer n ≥ 1 , the region R n is ac hiev able. The proo f is similar to th e poin t-to-poin t FSC with tim e- in variant feedback , given in [14, Sec. V]. In the proo f we use Gallager’ s techniq ues to an alyze the erro r prob ability of a ML dec oder of a random ly-gene rated code. There are two main differences compared to the point-to -point FSC: 1) In the case of FSC, only on e message is sen t, and in the case o f FS-MAC, two ind epende nt messages are sent. This requir es that we analyze three different typ es of errors, an d th ey yield th ree in equalities in the achiev able region. 2) F or the FS-MAC case, we need to prove the achivebility for a set of in put distributions while for the point-to - point channel it was eno ugh to prove it on ly fo r the input distribution th at achieves the maximum. Because of this difference, we introd uce the idea of concaten ating code-tree s (see Fig. 2). T his difference influen ces the encodin g scheme an d th e analysis. The following lem ma establishes the sub-ad ditivity of {R n } . Lemma 4 ( sup-add itivity o f R n . ) For any FS-M A C, the sequence {R n } is sup- additive. Th erefore , lim n →∞ R n exists, it is an achievable region, and it equals to R . B. Outer Boun d The following outer b ound is proved using Fano’ s ineq ual- ity . Theorem 2 ( Outer boun d. ) Let ( R 1 , R 2 ) be an achiev able pair for a FS-MAC with time in variant f eedback, as sho wn in Fig. 1. Th en, for any n there exists a distribution Q ( x n 1 || z n − 1 1 ) Q ( x n 2 || z n − 1 2 ) such that the f ollowing in equalities hold: R 1 ≤ 1 n I ( X n 1 → Y n || X n 2 ) + ǫ n , R 2 ≤ 1 n I ( X n 2 → Y n || X n 1 ) + ǫ n , R 1 + R 2 ≤ 1 n I (( X 1 , X 2 ) n → Y n ) + ǫ n , (24) where ǫ n goes to zero as n goes to infinity . This theorem im plies that lim inf R n is an o uter b ound , where R n is defined as R n = [ Q      R 1 ≤ 1 n I ( X n 1 → Y n || X n 2 ) , R 1 ≤ 1 n I ( X n 2 → Y n || X n 1 ) , R 1 + R 2 ≤ 1 n I (( X 1 , X 2 ) n → Y n ) , (25) and th e union is over inpu t distributions o f the f orm Q ( x n 1 || z n − 1 1 ) Q ( x n 2 || z n − 1 2 ) . C. Capacity Based on the bou nds ab ove, we have the f ollowing capacity result. Theorem 3 For any FS-MA C of the form P ( y i , s i | x 1 ,i , x 2 ,i , s i − 1 ) = P ( s i | s i − 1 ) P ( y i | x 1 ,i , x 2 ,i , s i − 1 ) , (26) where the state process S i is stationar y and ergodic, the achiev able region is lim n →∞ R n , and th e limit exists. V . F S - M AC W I T H O U T F E E D BAC K The case where there is no feedback is a special case of deterministic time-inv ar iant feedback in w hich z i is null, and therefor e the theor ems in th e previous section hold for the case of no f eedback . Here we show additio nal r esults, which apply o nly for the case with out feedb ack. The re sults in clude a sequ ence of upp er bound s for all FS-MACs, and a capacity formu la for indecomp osable FS-MA Cs. P S f r a g r e p l a c e m e n t s x 1 = 0 x 2 = 1 i = 1 i = 1 i = 1 x 3 = 1 i = 2 i = 2 i = 2 x 4 = 0 i = 3 i = 3 i = 3 x 1 = 0 x 1 = 0 x 2 = 1 x 2 = 1 x 2 = 1 x 2 = 1 x 3 = 0 x 3 = 0 x 3 = 0 x 3 = 1 x 3 = 1 x 3 = 1 x 4 = 0 x 4 = 0 x 4 = 1 x 4 = 1 i = 4 z i − 1 = 0 z i − 1 = 1 (no feedback) codeword (case of no f eedback) code-tree (used in [14 ]) concatenated code-tree (u sed here an d in [9 ]) Fig. 2. Illust ration of codi ng scheme for setting without feedback, setting with feedback as used for point-to-point channel [14] and a code- tree that was create d by concatena ting sm aller code-tree s. In the case of no feedback each message is mapped to a codew ord, and in the case of feedback each message is mapped to a code-tree. The third scheme is a code-tree of depth 4 created by concate natin g two trees of depth 2. A. Outer boun d Let us denote, R n = conv [ Q      R 1 ≤ max P 1 n I ( X n 1 → Y n || X n 2 , S 0 ) + H ( S 0 ) n , R 2 ≤ max P 1 n I ( X n 2 → Y n || X n 1 , S 0 ) + H ( S 0 ) n , R 1 + R 2 ≤ max P 1 n I (( X 1 , X 2 ) n → Y n | S 0 ) + H ( S 0 ) n where the unio n is over all inpu t distributions of the form Q ( x n 1 ) Q ( x n 2 ) , a nd max P denote a ma ximization over distri- bution of the form P ( s 0 | x n 1 , x n 2 ) . The sup-additivity o f { R n } is th e key proper ty for estab lishing the outer bou nd. Lemma 5 ( sub-add itivity o f R n . ) For any FS-M A C, the sequence { R n } is sub -additive, i.e. , ( n + l ) R n + l ⊆ n R n + l R l . (27) Theorem 4 ( Outer bou nd ) For any FS-MAC and all n ≥ 1 , R n contains the cap acity regio n. B. Capacity Theorem 5 ( Capacity of FS-MAC without feedback. ) For a ny indecomp osable FS-MA C without feedbac k, lim n →∞ R n = lim n →∞ R n , (28) and the refore its capacity regio n is lim n →∞ R n , an d the limit exists. V I . A P P L I C A T I O N S In this section we use the c apacity results in order to derive the following conclusions: 1) F or a stationa ry ergodic Markovian ch annels, the capac- ity region is zero , if and only if the cap acity region with feedback is zer o. 2) F or the additive mod- |X | MAC, wh ere the noise m ay have memory: a) feedback does not enlarge the capacity; b) source-channe l coding separation holds for lossless reconstruc tion. A. Zer o ca pacity The first con clusion is given in T heorem 6 . Theorem 6 For the chan nel describ ed in (26), where th e state process S i is stationary and ergo dic, if the capac ity withou t feedback is zer o, then it is also ze ro in the case that the re is feedback . The pro of o f Theor em 6 is b ased on the fact that fo r any MA C max Q ( x n 1 || y n − 1 ) Q ( x n 2 || y n − 1 ) I ( X n 1 , X n 2 → Y n ) = 0 (29) if and o nly if max Q ( x n 1 ) Q ( x n 2 ) I ( X n 1 , X n 2 → Y n ) = 0 , (30) and on the fact that for the family of channels that is mentioned in the theo rem, the sequenc e R n is sup-add iti ve. For the case of additive Gau ssian MA C, one can deduce the result f rom the fact that feedb ack can at most dou ble its cap acity region [19 ]. Clearly , T heorem 6 also h olds for the c ase of a stationary an d ergodic FS-Mar kov point-to- point channel because a MA C is an e xtension of a point-to -point channel. Howev er , it does not ho ld for the case of a broad cast channel. B. Additive mod- |X | MAC In this section we consider th e a dditive mod- |X | M A C with and witho ut feedback . The chann el is described in Fig. 3. In the b inary case, the c hannel is simply Y = X 1 ,i ⊕ X 2 ,i ⊕ V i , where V i is the binar y no ise, p ossibly with memo ry , an d ⊕ denotes ad dition mod-2. The following theorem is an extension of o f Alajaj’ s result [20] to the additive MA C . Theorem 7 Feedback do es n ot enlarge the capacity region of a discrete additi ve (mo d- |X | ) no ise MA C. Mo reover , the capacity region is giv en by M X m =1 R m ≤ log q − H ( V ) , (31) where H ( V ) is the entropy rate of th e additiv e noise. delay delay P S f r a g r e p l a c e m e n t s γ W 1 W M X 1 , n ( W 1 ) X 2 , n ( W 2 ) Y i V i ( ˆ W 1 , ˆ W 2 ) X 1 ,i ( W 1 , Y n − 1 ) X 2 ,i ( W 2 , Y n − 1 ) Fig. 3. Additi ve noise MA C with feedback. The random v aria bles X 1 ,i , X 2 ,i , Y i , V i , i ∈ Z + , are from a common alphabet, and they denote the input from sende r 1,2, the output and the noise at time i , respecti vely . The output satisfies y i = x 1 ,i ⊕ x 2 ,i ⊕ v i where ⊕ denotes addition mod- |X | . The noise V i , possibly with memory , is independe nt of the messages W 1 , W 2 . The th eorem can b e shown to h old fo r a larger family o f MA Cs. The family includ es all th e MA Cs that can be rep- resented as m ultiplexer followed by a poin t-to-po int channel. The m ain idea of the p roof is that all three inequalities that defines th e region R n are m aximized b y un iform and i.i.d distribution, even if fee dback is allowed. C. Sou r ce-channel cod ing separation Cover , El- Gamal and Salehi [21] showed that, in general, the so urce ch annel coding separation does not ho ld for MA Cs ev en for a mem oryless ch annel without feedba ck. Howe ver , for the case where the MAC is a additive mod - |X | , a nd th e goal is to reconstruct the sources losslessly , then it d oes hold. Theorem 8 ( Source-channel codin g theor em for a add itive mod- |X | MAC. ) Let ( U 1 , U 2 ) n ≥ 1 be a finite alp habet, jointly stationary and ergodic pair of processes, and let the MAC channel b e an additive mo d- |X | MA C with station ary and ergodic noise. Define P ( n ) e , Pr(( ˆ U n 1 , ˆ U n 2 ) 6 = ( U n 1 , U n 2 )) , where ˆ U n 1 , ˆ U n 2 are th e reconstructed sou rces at the decoder . ( dir ect part.) Ther e exists a sequence of source-chann el codes with P ( n ) e → 0 , if H ( U 1 , U 2 ) < log |X | − H ( V ) , wh ere H ( U 1 , U 2 ) is the entro py rate of th e sources an d H ( V ) is the entropy rate of the noise. ( conver se part.) If H ( U 1 , U 2 ) > lo g |X | − H ( V ) , then the probab ility of er ror is boun ded away from zer o, indep endent of th e blockleng th. It is inter esting to n otice that, even though the source-chann el coding separatio n theorem hold s when the reconstruction of the sources h as to be lossless, the theorem d oes n ot hold when distortion is allowed. Such an example was shown by Nazar and Gastpar [2 2]. V I I . C O N C L U S I O N S A N D F U T U R E D I R E C T I O N S In this paper we have shown that directed infor mation and causal cond itioning emerge naturally in charac terizing th e ca- pacity region of FS-MA Cs in the presence of a time-inv ariant feedback . W e p rovided a sequence of inner a nd outer bound s, and f or some large families of chan nels we ch aracterize the capacity region in terms of a ‘multi-letter’ expression, which is a first step to ward d eriving usefu l c oncepts in co mmunicatio n. For in stance, we use this characterizatio n to show th at for a station ary and e rgodic Markovian channel, the capacity is zero if and only if the c apacity with feedback is z ero. Further, we identify FS-MACs for wh ich feedback d oes not e nlarge the cap acity region and for which sour ce-chann el separ ation holds. One futur e directio n is to use the c haracterization s devel- oped in th is paper to explicitly comp ute the capacity regions of classes of M A Cs with mem ory and feedback ( other than the additive mo d- |X | ch annel), and to find op timal coding schemes. R E F E R E N C E S [1] R. Ahlswede, “Multi-wa y communicat ion channe ls, ” in Pr ocee dings of 2nd International Symposium on Information Theory (Thakadsor , Arme- nian SSR, Sept. 1971) , Publishing House of the Hungarian Academy of Science , Budapest, 1973, pp. 23–52. [2] T . M. Cov er and C. S. K. Leung, “ An achie v abl e rate region for the multiple -acce ss channel with feedb ack, ” IE EE T rans. on Info. Theory , vol. 27, no. 3, pp. 292–298, 1981. [3] L. H. Ozarow , “The capacity of the white Gaussian multiple access channe l with feedback, ” IEEE T ransacti ons on Information Theory , vol. 30, no. 4, pp. 623–628, 1984. [4] G. Kramer , “Directe d information for channels with feedback, ” Ph.D. Dissertat ion, Swiss Federal Institute of T echnology Zurich, 1998. [5] ——, “Capacit y resul ts for the discrete m emoryless network , ” IEEE T rans. Inform. Theory , vol. 49, pp. 4–21, 2003. [6] T . S. Han, “ An informatio n-spectru m approa ch to capac ity theorems for the general m ultipl e-acc ess channel, ” IEE E Tr ans. Inform. Theory , vol. 44, no. 7, pp. 2773–2795, 1998. [7] R. G. Gallage r , Information theory and relia ble communicatio n . Ne w Y ork: W ile y , 1968. [8] A. L apidoth and I. T ela tar , “The compound channel capacity of a class of finite-stat e channels, ” IEEE T rans. Inform. Theory , vol. 44, pp. 973– 983, 1998. [9] H. H. Permuter and T . W eissman, “Capac ity region of the finite- state m ultipl e acc ess chann el wit h an d wit hout fe edback , ” Aug. 2007, submitted to IEEE Trans. Inform. Theory . A v ailbl e at arxi v .org/pd f/cs.IT/0608070 . [10] J. Massey , “Causali ty , feedback and direct ed information, ” Proc. Int. Symp. Informatio n Theory Application (ISIT A-90) , pp. 303–305, 1990. [11] S. T ati kond a, “Con trol under communicat ion constrai nts, ” Ph.D. diser- tation, MIT , Cambridge, MA , 2000. [12] J. Chen and T . Ber ger , “The capaci ty of finite-stat e Marko v channels with feed back, ” IEEE T rans. on Informati on theory , vol. 51, pp. 780– 789, 2005. [13] S. Y ang, A. Kavci c, and S. T atikond a, “Feedbac k capaci ty of finite-st ate machine channels, ” IEEE T ran s. Inform. Theory , pp. 799–810 , 2005. [14] H. H. Permute r , T . W eissman, and A. J. Goldsmith, “Finite stat e channels with time- in varia nt determini stic feedback, ” Sep 2006, submitte d to IEEE Trans. Inform. Theory . A v ailble at arxiv . org/p df/cs.IT/060807 0. [15] Y . Kim, “ A coding theorem for a class of stationary channels with feedbac k, ” Jan 2007, submitte d to IEEE T rans. Inform. Theory . A vailb le at arxiv .org/c s.IT/0701041. [16] B. Shrader and H. H. P ermuter , “On the compound finite s tate channel with feedback, ” in ISIT 2007 , Nice, France, 2007. [17] R. V enkatarama nan and S. S. Pradhan, “Source codi ng with feedforw ard: Rate-di stortio n function for genera l sources, ” in IEEE Information theory workshop (ITW) , 2004. [18] R. Zamir, Y . K ochman, and U. Erez, “ Achie ving the gaussian rate-di stortion function by predictio n, ” July 2006, submitted for publica tion in “IEEE Tra ns. Inform. T heory”. [Online]. A vai lable : http:/ /www .eng.tau.ac.il/ ∼ zamir/pa pers/dpc m.pdf [19] J. A. Thomas, “Feedback can at m ost double gaussian multiple access channe l capac ity , ” IEE E T rans. Inform. Theory , vol. 33, pp. 711–716, 1987. [20] F . Alajaji, “Feedback does not incr ease the capaci ty of discrete channel s with additi ve noise, ” IEEE T rans. Inform. Theory , vo l. 41, pp. 546–549, 1995. [21] T . M. Cov er , A. E. Gamal, and M. Salehi, “Multipl e access chann els with arbitrar ily correlated sources, ” IEE E T rans. Inform. Theory , vol. 26, pp. 648–657, 1980. [22] B. Nazer and M. Gastpar , “The case for structured random codes in net- work communication the orems, ” in 2007 nformat ion Theory W orkshop (ITW), Lake T ahoe, CA, , Sep. 2007.