Capacity of Block-Memoryless Channels with Causal Channel Side Information

1 Capacity of Block-Memoryle ss Channels with Causal Channel Side Information Hamid Farmanbar and Ami r K. Khandani Coding and Signal T ransmi ssion Laboratory Department of Electrical and Computer Engineering Uni v ersity of W aterloo W aterloo, Ontario, N2L 3G1 Email: { hamid,khandani } @cst.uwaterloo.ca Abstract The capacity of a time-v arying block-memo ryless channel in which the transmitter and the receiver have access to (possibly dif ferent) noisy causal cha nnel side information (CSI) is obtained. It is shown that the capacity fo rmula o btained in th is c orrespon dence reduces to the capacity fo rmula repo rted in [ 1] for the spe cial case wh ere the transmitter CSI is a deter ministic function of the recei ver CSI. Index T erms Channel cap acity , block-memor yless channel, time-varying chann el, causal side inf orma- tion. I . I N T R O D U C T I O N Motiv ated by the r esult reported in [1], this correspondence studies th e capacity of a stationary and ergodic time-varying block-m emoryless (BM) channel where the transmitter and the receiv er hav e acce ss to noi sy causal channel side information (CSI). 2 The CSI at the transmitter (CSIT) and the CSI at the receiv er (CSIR) can b e di f ferent. The ti me variations of the channel are modeled as a set of channel states where the channel is at some s tate at each tim e instant . A tim e-v arying (state-dependent) BM channel is memoryless b etween blocks, howev er , within each block t he stat e and t he channel conditioned on t he state can have memory . F or e xample, s uch a channel m odel applies to systems based on frequency h opping with slow mo bility . This results in a quasi-static (blo ck) fading channel model where the channel fading is s tatic withi n a block and changes independent ly between blocks as t he frequency hops to a differ ent carrier . A formal definition of a state-dependent BM channel is giv en in Section II. The capacity of time-var ying BM channel s with CSI at transm itter and recei ver has been studied in [1] and t he capacity is obt ained for the case that the CSIT is a deterministic function of the CSIR. As an example, the scenario w here the CSIT is a deterministic function of the CSIR o ccurs when the recei ver quantizes its ob serva tion of the channel state and transmits i t vi a a noi seless channel to the transmitter . Howe ver , when the feedback channel is noisy , the CSIT will no longer be a determinis tic functi on of the CSIR. In this correspondence, we obtain the capacity for such a general case. The key idea comes from the capacity results due to Shannon for stat e-dependent discrete memo ryless channels wi th causal side in formation at t he transmi tter [2]. In the model considered by Shannon, the state of the channel is perfectly known at the transmitter and unknown at the recei ver . Shannon’ s work wa s extended by Salehi [3] to the case that (possibly dif ferent) n oisy versions of the CSI are av ai lable at t he transmitter and at t he recei ver . It was later shown by Caire and Shamai [4] that the capacity with noisy CSI can be obt ained from Shannon’ s original work by con sidering a ne w state- dependent channel with CSIT alphabet as the new s tate alphabet. It is worth mentionin g that in our problem, since CSIT sy mbols are not av ailable up to the end of the current block, appl ying Shannon’ s results [2] to super symbo ls correspondi ng to bl ocks would not yield the capacity . W e will use the foll owing not ations throughout the correspondence. Random vari- 3 ables are deno ted by upper case letters ( X ) and their values are denoted by lower case letters ( x ). The sequence of random variables X m , . . . , X n is denoted by X n m ; and x n m denotes a particular realization of X n m . The sequences X n 1 and x n 1 are denoted by X n and x n , respective ly . Sets are denoted by calligraphic letters ( X ); |X | denotes the cardinality of X , and X n = X × · · · × X | {z } n is the n -th Cartesian powe r of X . I I . C H A N N E L M O D E L The channel model considered in t his correspondence i s the s ame as the one int ro- duced in [1] where a state-dependent block-m emoryless channel is defined by a finite channel input alphabet X , a finit e channel output alphabet Y , a finit e state alphabet S , and transition probabilit ies p ( y n 0 | x n 0 , s n 0 ) where n 0 is the channel block l ength. W e denote the CSIT and the CSIR by U ∈ U and V ∈ V , respecti vely . T he CSIT and the CSIR are dependent on th e state according to the jo int dis tribution p ( s n 0 , u n 0 , v n 0 ) . It is con venient to express the t ransition prob abilities of the channel i n term s of the CSIT and the CSIR as p ( y n 0 , v n 0 | x n 0 , u n 0 ) = X s n 0 p ( y n 0 , v n 0 | x n 0 , u n 0 , s n 0 ) p ( s n 0 | x n 0 , u n 0 ) = X s n 0 p ( y n 0 | x n 0 , u n 0 , s n 0 , v n 0 ) p ( v n 0 | x n 0 , u n 0 , s n 0 ) p ( s n 0 | x n 0 , u n 0 ) = X s n 0 p ( y n 0 | x n 0 , s n 0 ) p ( v n 0 | u n 0 , s n 0 ) p ( s n 0 | u n 0 ) = X s n 0 p ( y n 0 | x n 0 , s n 0 ) p ( s n 0 , u n 0 , v n 0 ) /p ( u n 0 ) , (1) where p ( u n 0 ) = P s n 0 ,v n 0 p ( s n 0 , u n 0 , v n 0 ) . For n = J n 0 uses of the channel, we have p ( y n , v n | x n , u n ) = J − 1 Y j =0 p  y ( j +1) n 0 j n 0 +1 , v ( j +1) n 0 j n 0 +1 | x ( j +1) n 0 j n 0 +1 , u ( j +1) n 0 j n 0 +1  , (2) and p ( s n , u n , v n ) = J − 1 Y j =0 p  s ( j +1) n 0 j n 0 +1 , u ( j +1) n 0 j n 0 +1 , v ( j +1) n 0 j n 0 +1  . (3) 4 W e define a (2 nR , n ) block code of length n for the state-dependent BM channel to be 2 nR sequences of n encoding function s f i : W × U n → X for i = 1 , . . . , n su ch that x i = f i ( w , u i 1 ) , where w ∈ W = { 1 , . . . , 2 nR } . Note that th e channel input at time i depends on the CSIT up to time i . In other w ords, we consider causal kno w ledge setting. At t he recei ver , a decoding function g : Y n × V n → W is used to d ecode the transmi tted message as ˆ w = g ( y n 1 , v n 1 ) . The rate of th e block cod e is R = 1 n log |W | , and P ( n ) e is defined as the probability that a mess age W , uniformly distributed ov er W , i s recei ved in error , i.e., P ( n ) e = Pr { ˆ W 6 = W } . (4) I I I . C A P A C I T Y O F B L O C K - M E M O RY L E S S C H A N N E L S W I T H C S I The capacity of a time-varying BM channel for the case that the CSIT , U n 0 , is a deterministic function of the CSIR, V n 0 , is given by [1] C = max p ( x n 0 | u n 0 ) 1 n 0 I ( X n 0 ; Y n 0 | V n 0 ) = X u n 0 p ( u n 0 ) max p ( x n 0 | u n 0 ) 1 n 0 I ( X n 0 ; Y n 0 | u n 0 , V n 0 ) (5) where the maximum is taken over all distributions sati sfying the causal side information constraint, i.e., p ( x n 0 | u n 0 ) = n 0 Y i =1 p ( x i | x i − 1 , u i ) . (6) The capacity is achie ved by a scheme that adapts itself to channel variations so that for e very realization of the CSIT , the encoder us es a code which is capacity-achieving for that specific realizati on. The final coding scheme will be si mply a m ultiplexed version of the coding schemes for al l po ssible CSIT realizatio ns. The scenario in which the CSIT is a function of the CSIR describes a situation where the CSIT is, for example, a quantized version of the CSIR due to rate restrictio ns on the capacity of the feedback li nk between th e receiv er and th e transmit ter . Howe ver , when th e feedback channel i ntroduces noise, the CSIT wi ll no longer be a deterministi c function 5 of the CSIR. In this case, the decoder will no longer kn ow the transm ission strategy and this complicates capacity analys is. In the following, we show that the Shannon’ s approach for state-dependent dis crete memoryless channels with causal side information at the transmitter , with some modifications , can b e used t o obtain th e capacity in this more general case. It sho uld be notes that apply ing Shannon’ s scheme to our channel with super symbols of size n 0 does not yield t he capacity since CSIT is ava ilable only up to t he current s ymbol, no t up t o the end of the current channel block (super symbol). W e will show that to achiev e the capacity , it is suf ficient to cons ider encoding schemes that use the CSIT u p to the current symbol and within the current super symbol. In other words, th ere is no loss in capacity by disregarding the past CSIT symbols th at are not within the current super symbol. Theor em 1: The capacity of a time-varying BM channel wi th the CSIT and the CSIR denoted b y U n 0 and V n 0 , respecti vely , i s equ al to C = max p ( t n 0 ) 1 n 0 I ( T n 0 ; Y n 0 | V n 0 ) , (7) where the equi valent channel from T n 0 to ( Y n 0 , V n 0 ) is defined by 1 p ( y n 0 , v n 0 | t n 0 ) = X u n 0 p ( u n 0 ) p  y n 0 , v n 0 | x i = t i ( u i ) | n 0 i =1 , u n 0  . (8) Pr oof : Achievability : Consider the fol lowing encoding scheme. A message w ∈ { 1 , . . . , 2 nR } is encoded to  t n 0 1 ( w ) , t 2 n 0 n 0 +1 ( w ) , . . . , t n ( J − 1) n 0 +1 ( w )  , where t j n 0 + i ∈ X |U | i is a functio n from U i to X 2 , j = 0 , 1 , . . . , J − 1 , i = 1 , 2 , . . . , n 0 . Then, for any CSI T sequence u n 1 , the channel input sequence x n 1 is giv en by x j n 0 + i = t j n 0 + i  u j n 0 + i j n 0 +1  , j = 0 , 1 , . . . , J − 1 , i = 1 , 2 , . . . , n 0 . The new chann el from T n 0 to ( Y n 0 , V n 0 ) defined by (8) is not state 1 Theorem 1 may equaiv alently be stated as follo ws. The capacity is gi ven by ( 7) in which the max imization is restricted to distrib utions satisfying p ( t n 0 , u n 0 , v n 0 , x n 0 , y n 0 ) = p ( t n 0 ) p ( u n 0 ) p ( x n 0 | t n 0 , u n 0 ) p ( y n 0 , v n 0 | x n 0 , u n 0 ) and x i = t i ( u i ) , i = 1 , . . . , n 0 . I.e., T n 0 is independent of U n 0 and p ( x n 0 | t n 0 , u n 0 ) takes values zero and one only . 2 There is a one-to-one correspo ndence between the elements of U i and the elements of ˘ 1 , 2 , . . . , |U | i ¯ . A function from U i to X can be represented by a |U | i -tuple composed of elements of X . E ach compone nt of t he |U | i -tuple represents the value of the function for a specific element of U i . 6 dependent and for which t he rate 1 n 0 I ( T n 0 ; Y n 0 , V n 0 ) is achie v able for a fixed p ( t n 0 ) . Howe ver , we hav e I ( T n 0 ; Y n 0 , V n 0 ) = I ( T n 0 ; V n 0 ) + I ( T n 0 ; Y n 0 | V n 0 ) = I ( T n 0 ; Y n 0 | V n 0 ) , (9) since T n 0 is independent of V n 0 . Hence, the rate C given in (7) is achie v able. Con verse : For any (2 nR , n ) code for the state-dependent BM channel with arbitrary small probability of error , we ha ve nR = H ( W ) (10) = I ( W ; Y n , V n ) + H ( W | Y n , V n ) (11) ≤ I ( W ; Y n , V n ) + nǫ n (12) = J − 1 X j =0 I  W ; Y ( j +1) n 0 j n 0 +1 , V ( j +1) n 0 j n 0 +1 | Y j n 0 1 , V j n 0 1  + nǫ n (13) ≤ J − 1 X j =0 I  W , Y j n 0 1 , V j n 0 1 ; Y ( j +1) n 0 j n 0 +1 , V ( j +1) n 0 j n 0 +1  + nǫ n (14) ≤ J − 1 X j =0 I  W , U j n 0 1 ; Y ( j +1) n 0 j n 0 +1 , V ( j +1) n 0 j n 0 +1  + nǫ n (15) = J − 1 X j =0 I  W , U j n 0 1 ; Y ( j +1) n 0 j n 0 +1 | V ( j +1) n 0 j n 0 +1  + nǫ n (16) = J − 1 X j =0 I  T j ; Y ( j +1) n 0 j n 0 +1 | V ( j +1) n 0 j n 0 +1  + nǫ n (17) ≤ nC + nǫ n , (18) where ǫ n = 1 n + P ( n ) e R → 0 for lar g e n ; (12) foll ows from Fano’ s inequal ity; (15) foll ows from the data p rocessing inequality for the Markov chain ( W , Y j n 0 1 , V j n 0 1 ) → ( W , U j n 0 1 ) → ( Y ( j +1) n 0 j n 0 +1 , V ( j +1) n 0 j n 0 +1 ) ; (16) fol lows si nce ( W, U j n 0 1 ) is i ndependent of V ( j +1) n 0 j n 0 +1 ; T j = ( W , U j n 0 1 ) ; and (18) follows by com paring I  T j ; Y ( j +1) n 0 j n 0 +1 | V ( j +1) n 0 j n 0 +1  with (7) and noting 7 that T j is independent of U ( j +1) n 0 j n 0 +1 and X j n 0 + i = f j n 0 + i  T j , U j n 0 + i j n 0 +1  , for j = 0 , . . . , J − 1 , i = 1 , . . . , n 0 . In the sequel, we show that t he capacity form ula (7), reduces to (5) when U n 0 is a deterministi c function of V n 0 , i .e., U n 0 = k ( V n 0 ) . Any distri bution p ( t n 0 ) induces a distribution p ( x n 0 | u n 0 ) according to p ( x n 0 | u n 0 ) = X t n 0 : t n 0 ( u n 0 )= x n 0 p ( t n 0 ) , ∀ x n 0 ∈ X n 0 , ∀ u n 0 ∈ U n 0 , (19) where t n 0 ( u n 0 ) = x n 0 implies x i = t i ( u i ) , i = 1 , . . . , n 0 . On the other h and, for any distribution p ( x n 0 | u n 0 ) , there is a correspondin g dist ribution p ( t n 0 ) which can be obtain ed by solving (19). Given a realization of the CSIT , u n 0 , we h a ve the Markov chain T n 0 → X n 0 | u n 0 → ( Y n 0 , V n 0 ) | u n 0 . Therefore, by av eraging over all realizations, we ha ve I ( T n 0 ; Y n 0 , V n 0 | U n 0 ) ≤ I ( X n 0 ; Y n 0 , V n 0 | U n 0 ) . (20) Howe ver , I ( T n 0 ; Y n 0 V n 0 | U n 0 ) = I ( T n 0 ; V n 0 | U n 0 ) + I ( T n 0 ; Y n 0 | V n 0 , U n 0 ) = I ( T n 0 ; Y n 0 | V n 0 ) , (21) since T n 0 is independent of ( U n 0 , V n 0 ) , and U n 0 = k ( V n 0 ) . Furtherm ore, I ( X n 0 ; Y n 0 , V n 0 | U n 0 ) = I ( X n 0 ; V n 0 | U n 0 ) + I ( X n 0 ; Y n 0 | V n 0 , U n 0 ) = I ( X n 0 ; Y n 0 | V n 0 ) , (22) Since V n 0 → U n 0 → X n 0 form a Markov chain. Hence, max p ( t n 0 ) I ( T n 0 ; Y n 0 | V n 0 ) ≤ max p ( x n 0 | u n 0 ) I ( X n 0 ; Y n 0 | V n 0 ) . (23) 8 On the other hand, I ( T n 0 ; Y n 0 | V n 0 ) = H ( Y n 0 | V n 0 ) − H ( Y n 0 | T n 0 , V n 0 ) (24) = H ( Y n 0 | V n 0 ) − H ( Y n 0 | T n 0 , V n 0 , U n 0 ) (25) = H ( Y n 0 | V n 0 ) − H ( Y n 0 | T n 0 , V n 0 , U n 0 , X n 0 ) (26) ≥ H ( Y n 0 | V n 0 ) − H ( Y n 0 | V n 0 , U n 0 , X n 0 ) (27) = H ( Y n 0 | V n 0 ) − H ( Y n 0 | X n 0 , V n 0 ) (28) = I ( X n 0 ; Y n 0 | V n 0 ) , (29) where (25 ) and (28) fol low since U n 0 = k ( V n 0 ) ; (26) follows since X n 0 is a function of T n 0 and U n 0 ; and (27) follows since conditioning reduces entropy . Hence, max p ( t n 0 ) I ( T n 0 ; Y n 0 | V n 0 ) ≥ max p ( x n 0 | u n 0 ) I ( X n 0 ; Y n 0 | V n 0 ) . (30) Comparing (23) and (30), we conclude the result. I V . C O N C L U S I O N In th is work, we obtained the capacity of time-varying block-memoryless channels where (possi bly dif ferent) noisy causal CSI is av ail able at the transm itter and at t he recei ver . W e showed that for the case that the CSIT is a d eterministic function of the CSIR, the obt ained resul t reduces to the capacity expression reported in [1]. R E F E R E N C E S [1] A. J. Goldsmith and M. Medard, “Capacity of time-v arying channels with causal channel side information, ” IEEE T rans. Inform. Theory , vol. 53, no. 3, pp. 881-899, Mar . 2007. [2] C. E. Shanno n, “Channels with side information at the transmitter , ” IBM Journ al of Resear ch and Dev elopment , vol. 2, pp. 289-2 93, Oct. 1958. [3] M. Salehi, “Capacity and coding for memories with real-t ime noisy defect information at the encoder and decoder , ” Pr oc. I nst. Elec. Eng.-Pt. I , vol. 139, no. 2, pp. 113-117, A pr . 1992. [4] G. Caire and S. Shamai,“On the capacity of some channels with channel state information, ” IE EE Tr ans. Inform. Theory , vol. 45, no. 6, pp . 2007-2019 , Sep. 1999.