On Capacity Computation for the Two-User Binary Multiple-Access Channel

On Capacity Computation f or the T wo-User Binary Multiple-Ac cess Channel J ¨ org B ¨ uhler Heinrich-Her tz-Chair for Mobile Communic ations T echnical Uni versity of Ber lin Einsteinufer 25, D-1 0587 Berlin, Germany Email: joerg.buehler@mk.tu-ber lin.de Gerhard W u nder Fraunho fer German-Sino Lab for Mobile Commu nications Einsteinufer 37, D-1 0587 Berlin, Germany Email: wu nder@hhi.f raunho fer .de Abstract — This paper deals with the pr oblem of computing the boundary of the capacity region f or the memoryless two-user binary-input binary-output multiple-access channel ( (2 , 2; 2) - MA C), or eq uiva lently , the compu tation of input probability distributions maximizing weighted sum-rate. Thi s is equivalent to solving a diffi cult noncon vex optimization problem. Fo r a restricted class of (2 , 2; 2) -MA Cs and weight vectors, it is sh own that, depend ing on an ordering property of the channel matrix, the optimal solution is located on the boundary , or the objectiv e function has at most one stationary point in the interior of the domain. Fo r th is, the problem is reduced to a pseudoconcav e one-dimensional optimization and t he single-user problem. I . I N T RO D U C T I O N For some mu ltiuser cha nnel mod els, the cap acity region can be cha racterized in terms of mu tual inform ation expressions. Howe ver , ev en for chan nels where such a single- letter repre- sentation is available, e valuation of the ca pacity region is often a difficult pro blem since c omputation of the cap acity region bound ary is generally a difficult and nonco n vex optimization problem . For the single-user discrete memoryless channel, computatio n of cap acity is a c on vex problem , an d several numerical metho ds th at allow to calculate the capacity within arbitrary precision h av e b een developed, e.g . th e Arimoto - Blahut algor ithm [1] [2]. For the d iscrete memor yless MA C, no alg orithms for the computation of the capacity region bound ary ar e known. A fund amental step in this directio n h as been taken in [3], wher e a nu merical method for calculating the sum-rate capa city (also called total capacity) of the two- user MA C with bina ry output has be en de veloped. This was achie ved by showing that the calculation of the sum capacity c an be redu ced to the calculation of th e sum cap acity for the two-user MAC with binary inp ut a nd binary output and by g iving necessary and su fficient condition s for sum- rate optimality by a partial mod ification of the Kuhn -T ucker condition s. Unfor tunately , furth er gen eralizations [4]- [6] of this appro ach to the most general ( n 1 , . . . , n m ; m ) -MAC (with m users, each with an a lphabet of size n k ) and con sequently the su bsequent work in [ 7] (whic h gener alizes the Arimoto- Blahut algo rithm for the sum cap acity compu tation of the ( n 1 , . . . , n m ; m ) -MAC based on the results in [4]) is p artially This research is supported by Deutsche Forschungsge meinschaft (DFG) under grant WU 598/1-1. incorrect. Th e work in [8] consider s the compu tation of not only the su m capacity , but of the whole capac ity region of the two-u ser discrete MAC. Here, the autho rs show that the only n on-conve xity in the problem stems from the require- ment o f the in put probability distrib utions to be indep endent, i.e. f rom the constraint f or the prob ability matrix specif ying the joint prob ability input distribution to be of rank one. They propose an approxima te solutio n to the prob lem b y removing this ind ependen ce con straint (i.e. r elaxation of the rank-o ne con straint), obtainin g a n o uter bou nd r egion to the actual cap acity region. By projecting the obtained p robability distribution to in depende nt d istributions by calcu lating the marginals, one obtains an inner boun d region. Even tho ugh the autho rs pr esent som e examples where th is approach gives the actual capacity region (i.e. the o uter bound region , the inner bound re gion and the capacity region coincide), the result is o ften subo ptimal, and it is no t clarified when the actual capacity region is obtain ed. Consequ ently , the solution of the capacity co mputation pro blem for the discrete memoryless MA C r emains an intere sting u nsolved pr oblem, ev en fo r th e case of two users and binary alph abets. Contributions. W e prove that fo r a c lass of (2 , 2; 2) -MAC s, the weighted sum ob jectiv e function has at most one stationary point in the interior o f the d omain. Beside the fact th at this is an in teresting structural property wh ich gi ves valuable insight into the gener al problem, it can also be employed for numerical solutio ns of the pr oblem. Since the maxim um of the objective fu nction on the boundary can be found by solving the single-user p roblem, it suffi ces to search for stationary poin ts in the inter ior of the dom ain: As there is at most o ne stationary point in the interior, method s such as grad ient d escent ca n return a subop timal solution only if the g lobal optim um is located o n th e b oundar y , which is then foun d by the bou ndary search. What is mo re, we p rove the statement by showing that the pro blem in the interior can be redu ced to a p seudoco ncave one-dim ensional p roblem, resulting in an efficient o ptimization proced ure for a spec ified to lerance of deviation fr om the optimal poin t fo r one of the inpu t paramete rs. W e rem ark tha t there is n umerical evidence fo r th e co njecture that also for the general (2 , 2; 2) -MAC, there is at most one stationar y p oint in the interior o f the d omain, which we un fortuna tely could not prove. Organization. The pa per is organiz ed as fo llows : Section II intro duces the problem f ormulation . In section III, we discuss some general properties of the (2 , 2; 2) -MAC. W e state the Karush-Kuhn-T ucker (KKT) cond itions, discuss some relations to previous work in the literature an d refo rmulate the optimization prob lem in te rms o f a on e-dimension al and the single-user problem . In section IV, we show that fo r a class of (2 , 2 ; 2) -MACs (the 3-parame ter (2 , 2; 2) -MAC ) and weigh t vectors with w 1 ≤ w 2 , this o ne-dime nsional prob lem can in turn be reduce d to a pseudo concave prob lem, also provin g that there is at most one stationar y p oint in the interior of th e domain. Imposing a further restriction on the channel tr ansition probab ilities, we find a closed- form expression for the solution of the on e-dimensio nal p roblem. Finally , section V co ncludes the paper . I I . P R O B L E M F O R M U L A T I O N The comm unication model un der study is the discrete and memo ryless two-user bina ry-inpu t binary-ou tput multiple- access channel, ter med as (2 , 2; 2) -MA C in this paper , which is specified by in put alph abets X 1 = X 2 = { 1 , 2 } , th e output alph abet Y = { 1 , 2 } an d cond itional channel tran sition probab ilities p ( y | x 1 , x 2 ) for y ∈ Y , x i ∈ X i . Let Q :=  q = ( q 1 , q 2 ) T ∈ R 2 + : q 1 + q 2 = 1  . It is well-known that the cap acity region C (2 , 2; 2) of the (2 , 2; 2) -MAC is given by [9]-[11] C (2 , 2; 2) = Co   [ q 1 , q 2 ∈ Q A ( q 1 , q 2 )   (1) where A ( q 1 , q 2 ) is th e set of all rate pa irs ( R 1 , R 2 ) T ∈ R 2 + that satisfy R 1 ≤ I ( X 1 ; Y | X 2 ) , R 2 ≤ I ( X 2 ; Y | X 1 ) , R 1 + R 2 ≤ I ( Y ; X 1 , X 2 ) . Here , q 1 , q 2 specify the input distri- bution by Pr [ X u = s ] = q us , whe re q us denotes the s -th compon ent o f q u , Co denotes th e c on vex closure ope ration and I is mutual information . The problem we con sider in this work is co mputing the bound ary of the capacity region, or equivalently , since the capacity region is convex, the maxim ization o f th e weig hted sum-rate in the capacity region f or a giv en weight vector w = ( w 1 , w 2 ) T > 0 : max r ∈C (2 , 2; 2) w T r . (2) Each poly hedron region A ( q 1 , q 2 ) is sp ecified by the cor- ner points C 1 ( q 1 , q 2 ) := ( I ( Y ; X 1 ) , I ( Y ; X 2 | X 1 )) T and C 2 ( q 1 , q 2 ) := ( I ( Y ; X 1 | X 2 ) , I ( Y ; X 2 ) T . It is easily verified that the weighted sum-rate o ptimization problem formulated above can be stated in terms of o ptimization over th e region defined b y the C 1 , C 2 points as follows: For w 1 ≤ w 2 , it h olds that max r ∈C (2 , 2; 2) w T r = max q 1 , q 2 ∈ Q w T C 1 ( q 1 , q 2 ) (3) and fo r w 1 > w 2 , the o ptimization can similarly b e perfo rmed by o ptimizing over the C 2 points. Notatio n and con ventions. For the transition probabilities of the ch annel, we write a := p (1 | 1 , 1 ) , b := p (1 | 1 , 2) , c := p (1 | 2 , 1) , d := p (1 | 2 , 2) and ∆ 1 := a − b, ∆ 2 := c − d . W e denote the natu ral logarithm by ln , an d express all entro py and mutual information quantities in nats. The binary entropy func- tion is den oted b y H . Finally , D ( p || q ) denotes the Kullback- Leibler d i vergence between two bin ary pro bability func tions defined by p, q ∈ [0 , 1 ] . Der i vati ves o n th e boun dary of closed intervals ar e to be understoo d as o ne-sided derivati ves. W e assume with out lo ss o f gen erality (w .l.o.g.) that 0 < w 1 ≤ w 2 : For the case w 1 > w 2 , we can use the fact that I ( Y ; X 2 ) and I ( Y ; X 2 | X 1 ) are obtain ed fro m I ( Y ; X 1 ) and I ( Y ; X 1 | X 2 ) by inter changin g the roles of q 1 and q 2 and the r oles of b an d c . For q 1 , q 2 ∈ Q, we d efine Ψ( q 1 , q 2 ) := w T C 1 ( q 1 , q 2 ) . (4) For cha nnels with a = b and c = d , it is I ( Y ; X 2 | X 1 ) = 0 for all q 1 , q 2 ∈ Q ; we exclude this d egenerate c ase from in vestigation. Similarly , fo r channels with a = c and b = d , I ( Y ; X 1 ) = 0 for all q 1 , q 2 ∈ Q , a nd we also omit this case. Optimization problem. I n the f ollowing, we are th us concern ed with the optimization p roblem max q 1 , q 2 ∈ Q Ψ( q 1 , q 2 ) . (5) Obviously , for w 1 = w 2 the problem (5) reduc es to the sum capacity pr oblem studied in [3]-[7]. I I I . T H E (2 , 2; 2) - M A C A. KKT condition s fo r the (2 , 2 ; 2) -MAC; r elation to prior work The work in [3]-[6] is prim arily con cerned with pr oving sufficiency of ( modified) Kar ush-Kuhn-Tucker (KKT) condi- tions. Assuming a, b, c, d / ∈ { 0 , 1 } to ensu re differentiability , the KKT conditions correspond ing to pr oblem (5) can be formu lated as ∂ Ψ( q 1 , q 2 ) ∂ q us = Ψ( q 1 , q 2 ) − w u , if q us > 0 , (6) ∂ Ψ( q 1 , q 2 ) ∂ q us ≤ Ψ( q 1 , q 2 ) − w u , if q us = 0 for u, s ∈ { 1 , 2 } . It can easily be checked that f or any po int satisfying (6), the linear indepen dence c onstraint qualification (LICQ) hold s, so that the KKT co nditions given ab ove are a necessary con dition for optimality . Note that these conditions are similar to the expression s giv en in [12] for the single- user pro blem and in [3]-[6] for the MAC sum -rate capacity . Unfortu nately , th e func tion Ψ is in g eneral not co ncave (and not e ven quasiconcave [13]), implying that the KKT conditions in (6) are not necessarily a sufficient cond ition for optimality and that solving the op timization pro blem (5) is d iffi cult. In [3], two classes o f (2 , 2; 2) -MACs ar e distinguished: case A and case B c hannels. For case B chann els, the KKT condition s as given above ( for w 1 = w 2 = 1 ) are proved to be sufficient for o ptimality . For case A channels, the con ditions have to b e slightly mo dified to be sufficient; essentially the modification consists in requiring the optimal point to be located on a certain boun dary of the doma in. W e also n ote ∂ I ( Y ; X 1 )( p 1 , p 2 ) ∂ p 1 = h 1 ( p 2 ) + h 2 ( p 2 ) ln  1 h 3 ( p 2 ) + p 1 h 2 ( p 2 ) − 1  (9) ∂ I ( Y ; X 2 | X 1 )( p 1 , p 2 ) ∂ p 1 = − p 2 H ( a ) + ( p 2 − 1) H ( b ) + p 2 H ( c ) − ( p 2 − 1) H ( d ) + H ( b + p 2 ( a − b )) − H ( d + p 2 ( c − d )) (10 ) 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 R 1 R 2 Fig. 1. Boundary of the noncon vex re gion G 1 for a = 2 / 3 , b = 1 / 4 , c = 10 − 3 , d = 5 / 8 and w 1 = w 2 = 1 . that case A channels are characterized by the condition ( a − c )( b − d ) < 0 , an d ca se B chan nels b y ( a − c )( b − d ) ≥ 0 , although this is n ot stated explicitly in [3]. In ou r case, th e situation is quite different: For c ase A ch annels, th e o ptimal input d istribution is n ot necessarily located on th e bou ndary . For example, th is is the c ase fo r the chann el with a = 1 / 5 , b = 2 / 5 , c = 1 / 2 , d = 3 / 10 an d w 1 = 1 / 5 , w 2 = 4 / 5 . The gen eralization to the ( n 1 , . . . , n m , m ) -MAC in [4] an d [6], where sufficiency of th e KKT con ditions for elementary MA Cs (i.e. MACs with n k ≤ m fo r all k ) is claimed , is not correct: For example, the (2 , 2; 2) -MAC with a = 2 / 3 , b = 1 / 4 , c = 10 − 3 , d = 5 / 8 an d w 1 = w 2 = 1 satisfies the KKT condition s in th ree points, among which actu ally only one is a g lobal o ptimum (located on the bou ndary) , one is only a local optimum ( also on the bound ary) and the third one is a saddlepoin t (in the interio r). Unlike stated in [6] (Pro position 3), not every KKT poin t is a local m aximum. Considering th e relation to the prob lem in th e r ate do main, it is true that every interior K KT po int corr esponds to a point on the b oundar y of G 1 = { C 1 ( q 1 , q 2 ) : q 1 , q 2 ∈ Q } (Propo sition 2 ). Howe ver , this region is gene rally not convex unlike implied b y the pr oof of Proposition 3. Figure 1 illustrates this nonco n vexity of G 1 for the example g i ven a bove. T he marked point on the bound ary of G 1 correspo nds to an inte rior KKT point an d has tangent slop e of -1, as indicated by the tang ent line drawn in the fig ure. Howe ver , the KKT point corr espondin g to it is not a local maximu m, but a saddle point. B. Red uction to a o ne-dimen sional pr oblem The form ulation of Ψ as a f unction o n Q × Q served mainly the purpose of relating to prior work. In the following, we consider Ψ as a fun ction on the dom ain [0 , 1] 2 instead, i.e. we are concer ned with the maxim ization o f Ψ( p 1 , p 2 ) where the input prob ability distribution is specified by p 1 = Pr [ X 1 = 1] an d p 2 = Pr [ X 2 = 1] . Ou r d eriv ation is based o n the following ob servations: First o f all, the b oundar y points of the capacity region on the two rate axis are ( e 1 , 0) T and (0 , e 2 ) T , where e 1 = m ax i ∈{ 0 , 1 } max p 1 ∈ [0 , 1] I ( X 1 , X 2 ; Y ) p 2 = i , (7) and e 2 is giv en similarly by fixing the value o f p 1 to 0 and 1 . Observe that e 1 and e 2 can b e foun d by solving the single-u ser capacity max imization pr oblem. Furthermo re, I ( X 2 ; Y | X 1 ) is linea r in p 1 , a nd for (all but at m ost o ne, na mely the on e th at satisfies h 2 ( p 2 ) = 0 , see below) fixed values of p 2 , I ( X 1 ; Y ) is strictly c oncave in p 1 . Hence, the first componen t in the stationarity equ ation ∇ Ψ( p 1 , p 2 ) ! = 0 , p 1 , p 2 ∈ (0 , 1 ) (8) has a u nique so lution in p 1 for fixed p 2 in the case o f strict concavity . What is more, we can find an explicit expression for this solution b y simp lifying the p artial d eriv ati ve of I ( X 1 ; Y ) with respect to p 1 such that p 1 occurs only o nce in the expression: Th e p artial deriv ati ves of mu tual informatio n at p 1 , p 2 ∈ (0 , 1) with r espect to p 1 are gi ven in (9) an d (1 0) at the top of this p age. Here, h 1 ( p 2 ) := H ( d + p 2 ( c − d )) − H ( b + p 2 ( a − b )) , (11 ) h 2 ( p 2 ) := − b + d + p 2 ( − a + b + c − d ) , (12) h 3 ( p 2 ) := 1 − d + p 2 ( d − c ) . (13 ) W e will also wr ite h 4 ( p 2 ) := ∂ I ( Y ; X 2 | X 1 )( p 1 ,p 2 ) ∂ p 1 . Let P 2 := { p ∈ (0 , 1) : h 2 ( p ) 6 = 0 } , P 2 := { p ∈ P 2 : f ( p ) ∈ (0 , 1) } . Note th at we h av e excluded the case a = c and b = d , so that there is at m ost one p ∈ (0 , 1) for which h 2 ( p ) = 0 . For fixed p ∈ P 2 , the explicit solution for p 1 in the first comp onent of (8) is given b y f ( p ) , where f : P 2 → R is d efined by f ( p ) := 1  e h ( p ) + 1  h 2 ( p ) − h 3 ( p ) h 2 ( p ) , (14) with h ( p ) := − w 2 w 1 h 4 ( p ) − h 1 ( p ) h 2 ( p ) . For p 2 ∈ (0 , 1) \ P 2 , it is easy to sho w that I ( X 1 ; Y ) = 0 for all p 1 . Define φ : P 2 → R by φ ( p ) := Ψ( f ( p ) , p ) . Collectively considering the above facts, we obtain Lemma 1 : If ∇ Ψ ( p 1 , p 2 ) = 0 for ( p 1 , p 2 ) ∈ (0 , 1 ) × P 2 , then p 2 ∈ P 2 , φ ′ ( p 2 ) = 0 and th ere is no ˜ p 1 ∈ (0 , 1) su ch that ˜ p 1 6 = p 1 and ∇ Ψ( ˜ p 1 , p 2 ) = 0 . Moreover, max p 1 ,p 2 ∈ [0 , 1] 2 Ψ( p 1 , p 2 ) = max  max p ∈ P 2 φ ( p ) , w 1 e 1 , w 2 e 2  . (15) I V . T H E 3 - P A R A M E T E R (2 , 2 ; 2) - M A C The ch annels th at we con sider n ow are (2 , 2; 2 ) -MA Cs with the restriction a = p (1 | 1 , 1) = p (1 | 1 , 2) = b on th e chann el transition prob abilities. W e call such a ch annel a 3 -parameter (2 , 2; 2) -MAC . The informa tion-theor etic interpretation of such channels is as follows: Condition ed o n the event th at user 1 transmits the sym bol 1 , the channel that user 2 sees is the single-user antisym metric binary channel, which has zero capacity . In other word s, whenever user 1 tr ansmits 1, the symbol o f u ser 2 canno t be disting uished at the receiv er . In fact, it is easily verified that I ( Y ; X 2 | X 1 = 1) = 0 . I f we giv e the sam e pr operty to u ser 1 for u ser 2 transmitting 1, i.e. a = b = c , then I ( Y ; X 1 | X 2 = 1) = 0 and all the p oints on th e bou ndary of the capacity r egion c an be achieved by ”time-sharing between the extrema l points on the r ate axis”. Moreover , we have e 1 = e 2 , so that the capacity region is an isosceles triang le. Furthermo re, fo r a = b , we can exchange the values c and d without cha nging the capacity region. In the following, we thus assume w .l.o.g. th at a 6 = c , a 6 = d and c > d . W e also restrict to weight vectors w = ( w 1 , w 2 ) T with 0 < w 1 ≤ w 2 . Unlike in the pr evious section, this actually is a restrictio n here: The case w 1 > w 2 cannot be treated by exchanging the roles of c an d b and p 1 and p 2 , since this would result in a (2 , 2; 2) -MAC that is not of 3-parameter typ e. Now consider the extension of φ to P 2 , i.e. ˆ φ : P 2 → R defined by ˆ φ ( p ) := Ψ( f ( p ) , p ) and which is e asily verified to be well-defin ed. W e will show that Ψ can have at mo st one stationary p oint in the interior . W e prove th is by showing that ˆ φ is pseudoco ncave on (0 , 1) . Recall that a twice differentiable function c : D ⊆ R → R is called pseu doconca ve if c ′ ( p ) = 0 ⇒ c ′′ ( p ) < 0 . In this case, ea ch lo cal maximu m of c is also a glob al maximum, and c has at most o ne stationary p oint. More precisely , we prove Pr o position 2: The fun ction ˆ φ has the fo llowing properties: • For a ∈ ( d, c ) : ˆ φ ′ ( p ) 6 = 0 for a ll p ∈ P 2 . • For a / ∈ ( d, c ) : ˆ φ is pseud oconcave o n P 2 = (0 , 1) . Pr o of: W e first pr ove the following proper ties o f h : • For a ∈ ( d, c ) : h ′ ( p ) 6 = 0 for all p ∈ P 2 . • For a ∈ [0 , d ) : h ′ ( p ) = 0 ⇒ h ′′ ( p ) < 0 . • For a ∈ ( c, 1 ] : h ′ ( p ) = 0 ⇒ h ′′ ( p ) > 0 . The fir st der iv ati ve of h can be written as h ′ ( p ) = ∆ 2 w 1 − w 2 w 1 D ( a || d + p ∆ 2 ) + w 2 w 1 δ ( a, c, d ) h 2 ( p ) 2 , (16) where δ ( a, c, d ) := ( c − d )( H ( d ) − H ( a )) − ( H ( c ) − H ( d ))( d − a ) . F or a ∈ (0 , 1) , ∂ 2 ∂ a 2 δ ( a, c, d ) = ( c − d )  1 1 − a + 1 a  > 0 , (17) implying that δ is strictly con vex in a . No w δ ( c, c, d ) = δ ( d, c, d ) = 0 , so that δ ( a, c, d ) > 0 fo r a / ∈ ( d, c ) and δ ( a, c, d ) < 0 for a ∈ ( d, c ) . A similar argumen t shows δ (0 , c, d ) > 0 and δ (1 , c, d ) > 0 . Moreover , D ( a || d + p ∆ 2 ) ≥ 0 by the n onnegativity of Kullback- Leibler d i vergence. This implies that h ′ ( p ) < 0 f or a ∈ ( d, c ) . Now consider the situation a / ∈ ( d, c ) . For a ∈ [0 , d ) , we h av e h 2 ( p ) > 0 for all p ∈ (0 , 1) a nd for a ∈ ( c, 1] , h 2 ( p ) < 0 for all p ∈ (0 , 1) , so that P 2 = (0 , 1) fo r a / ∈ ( d, c ) . Since h ′′ ( p ) = w 2 − w 1 w 1 h ′′ 1 ( p ) − 2∆ 2 h ′ ( p ) h 2 ( p ) , (18) the claimed p roperty of h follows fo r w 1 < w 2 from the stric t concavity o f h 1 . If w 1 = w 2 , then h ′ ( p ) > 0 f rom (1 6), so that the statement also h olds in this case. W e n ow show that f ( p ) < 1 fo r all p ∈ (0 , 1) . For this, we first prove that f ( p ) 6 = 1 f or all p ∈ (0 , 1) . T his follows easily for a ∈ { 0 , 1 } , so th at we let a ∈ (0 , 1) . W e also assum e c, d ∈ (0 , 1 ) ; the situations c ∈ { 0 , 1 } o r d ∈ { 0 , 1 } can be treated similarly . It suffices to pr ove tha t v ( p ) := ∂ I ( Y ; X 1 )( p 1 , p ) ∂ p 1     p 1 =1 < 0 (1 9) and h 4 ( p ) < 0 fo r all p ∈ (0 , 1) . T o see this, we first no te that it ca n be shown that v ′ ( p ) = ln  1 1 − a − 1  h ′ 2 ( p ) + h ′ 1 ( p ) 6 = 0 (2 0) for all p ∈ (0 , 1) . Mo reover , we get v (0) = − D ( d || a ) < 0 and v (1) = − D ( c || a ) < 0 , which to gether with (20) imply (19). For the seco nd statement, observe that h ′′ 4 ( p ) = ∆ 2 2 ( p ∆ 2 + d )(1 − ( p ∆ 2 + d )) > 0 , (21) so that h 4 ( p ) is strictly conve x in p . h 4 ( p ) < 0 then follows from the fact that h 4 (0) = h 4 (1) = 0 . Secondly , also u sing non-n egati vity of Kullback- Leibler divergence, one c an prove lim p → 0+ f ( p ) < 1 , which together with f ( p ) 6 = 1 and the continuity of f implies that f ( p ) < 1 for all p ∈ (0 , 1) . T o co nclude the proo f of th e proposition , on e can find the following factorized rep resentation for ˆ φ ′ : ˆ φ ′ ( p ) = w 1 (1 − f ( p )) h 2 ( p ) h ′ ( p ) . (22) W ith the shown p roperties of h and f , the statement follows directly from ( 22) for th e case a ∈ ( d, c ) . For th e o ther c ase, we ha ve that ˆ φ ′′ ( p ) = w 1 h ′ ( p ) ( − f ′ ( p ) h 2 ( p ) + ∆ 2 (1 − f ( p ))) (23) + w 1 (1 − f ( p )) h 2 ( p ) h ′′ ( p ) , implying that with the pro perties o f h and f follows that ˆ φ ′ ( p ∗ ) = 0 ⇒ h ′ ( p ∗ ) = 0 ⇒ ˆ φ ′′ ( p ∗ ) < 0 . (24) It can be verified that fo r h 2 ( p 2 ) = 0 , it is Ψ ( p 1 , p 2 ) 6 = 0 for all p 1 ∈ (0 , 1) . As a con sequence, Propo sition 2 and Lemma 1 imp ly that in th e case a ∈ ( d, c ) (i. e. for 3-p arameter channels o f case A), the f unction Ψ h as no stationary point in the interior ; the optimum input pro bability distribution is located on the boun dary and it suffices to solve the single-user problem s. Sp eaking in terms o f the KKT con ditions (6), this means that ea ch p oint satisfying these equation s is located on the b oundar y , as in the case of the sum- rate prob lem [ 3]. For the case a / ∈ ( d, c ) (ca se B), p seudocon cavity of ˆ φ implies that there is at most on e stationary poin t (o r , equiv alently , at most one KKT p oint) in the interior of the domain. W e summarize this in the following theorem: Theor em 3: For th e 3-pa rameter (2 , 2; 2) - MA C with a 6 = c , a 6 = d , c > d and w 1 ≤ w 2 , the fo llowing h olds: • For a ∈ ( d, c ) , th e optim al input distribution is located on the bound ary of [0 , 1] 2 , a nd there is no station ary p oint of Ψ in the in terior of [0 , 1] 2 . • For a / ∈ ( d, c ) , there is at most one stationary po int of Ψ in the interior of [0 , 1] 2 . By Lemma 1, the pro blem of findin g th e maximizing in put distribution can b e redu ced to th e single-user pr oblem and the optimization of φ (for wh ich it suffices to optimize ˆ φ , as describ ed in the following). Since ˆ φ is pseu doconcave, it can efficiently b e op timized using a simple standard b isection algorithm : For a g iv en tolerance ε , we start with the in terval [ ǫ, 1 − ǫ ] and determine if on e of the intervals [ ǫ, 1 / 2 ] , [1 / 2 , 1 − ǫ ] con tains th e optim al point by checkin g the sign of h ′ (i.e. the sign of ˆ φ ′ by (22)) a t the interval bou ndaries. If this is not the case, we assume th e optimal point to be on the boun dary . Otherwise, we con tinue bisecting the inter val that co ntains the stationary poin t until the interval length is smaller th an ε , an d find a solution p ǫ within ǫ de viation tolerance from the optimal using only O (log (1 /ǫ )) ev aluations of h ′ , which is much more efficient than a bru te-force search. By the defin ition of φ , ( p ∗ 1 , p ∗ 2 ) = ( f ( p ǫ ) , p ǫ ) is used as o ptimization outpu t if f ( p ǫ ) ∈ (0 , 1) . If f ( p ǫ ) / ∈ (0 , 1 ) , we assum e that the stationar y point of ˆ φ is outside of P 2 , and we also assume the optimal point fo r Ψ to be o n the b oundar y . Note that in principle, we could fin d the op timal p 1 for this c hoice of p 2 = p ε by solving the single-user problem f or fixed p 2 = p ε . However , it is clear that for sufficiently sma ll ǫ , we will hav e f ( p ǫ ) ∈ (0 , 1 ) if Ψ has a stationary p oint in th e inter ior . Note that we can only giv e a deviation toleran ce fo r p ∗ 2 . However , p ∗ 1 = f ( p ǫ ) is still a reason able solution since it is optimal ” condition ed” on the choice of p ∗ 2 = p ǫ . W e remark that a gradient d escent algorithm employed for Ψ also typically shows fast c on vergence to the optimal in put d istribution; we also refer to th e in troductor y discussion. W e finally consider the 3-pa rameter (2 , 2 ; 2) -MAC wit h a = b = 0 , 0 < d < c and w 1 < w 2 . For this chann el, it is possible to give an explicit solu tion for the optimization o f φ . It can be shown that P 2 = (0 , 1) , so that ˆ φ ≡ φ . Here, we can fin d the zero of (22) b y solvin g h ′ ( p ) = 0 for p , resulting in p = p ∗ ( c, d, w ) := 1 − d − e − w 2 δ (0 ,c,d ) ∆ 2 ( w 2 − w 1 ) ∆ 2 , (25) where δ (0 , c , d ) = ( c − d ) H ( d ) − d ( H ( c ) − H ( d )) . The maximum weighted sum-ra te is gi ven by max r ∈C (2 , 2; 2) w T r =  φ ( p ∗ ( c, d, w )) , p ∗ ( c, d, w ) ∈ (0 , 1 ) max { w 1 e 1 , w 2 e 2 } , otherwise. (26) V . C O N C L U S I O N S In this w ork, we studied the p roblem of maximizing weighted sum -rate (or c omputing th e bou ndary of th e capac ity region) for the memoryle ss two-user binary-in put bin ary- output multiple-access channel, called (2 , 2; 2) -MAC. T he KKT co nditions were f ormulated as a necessary optim ality condition . However , the objecti ve function is not co ncave, and th e KKT con ditions are n ot su fficient for op timality . W e demon strated that this is the case even for the sum - rate pr oblem, unlike stated in prior work. In this paper, we proved some stru ctural p roperties o f the p roblem. For th is, it was first redu ced to a on e-dimensio nal prob lem an d the single user p roblem and then studied fo r the 3-p arameter (2 , 2; 2) -MAC, for wh ich p (1 | 1 , 1) = p (1 | 1 , 2) . For weights satisfying w 1 ≤ w 2 , we sho wed that, depending on an ordering proper ty of the chan nel tr ansition probab ility m atrix, either the maximum is attained on the boun dary , o r there is at most o ne stationar y po int in the in terior o f the optimiza tion domain. The pro of was ob tained by showing that in the latter case, the reduction to the one-d imensional problem leads to a pseudoc oncave form ulation, which can also be used n umer- ically for the capacity op timization. For a further restricted class o f (2 , 2 ; 2) -MAC chan nels, an explicit solution for the one-dim ensional p roblem c ould be given. Fu ture work will consider the unre stricted (2 , 2; 2) -MA C a nd the most gene ral situation o f the ( n 1 , . . . , n m ; m ) -MAC. R E F E R E N C E S [1] S. Arimoto, “ An algori thm for computing the capacity of arbitrary discrete m emoryless channels, ” IEEE T rans. Inform. Theory , vol. IT -18, no. 1, pp. 14–20, Jan. 1972. [2] R. Blahut, “Computati on of channel capacity and rate-distort ion func- tions, ” IEEE T rans. Inform. Theory , vol. IT -18, no. 4, pp. 460–473, Jul. 1972. [3] Y . W atana be, “The total capaci ty of two-user multiple -access channel with binary output, ” IE EE T rans. Inform. Theory , vol. IT -42, no. 5, pp. 1453–1465, Sep. 1996. [4] Y . W ata nabe and K. Kamoi, “The total capacity of multiple-ac cess channe l, ” in Proc . IEEE Int. Symp. Information Theory , Lausanne, Switzerl and, June/July 2002, p. 308. [5] ——, “Channel configuratio n of multiple- access channel, ” in Proc. IEEE Int. Symp. Information Theory , Chicago, USA , June/July 2004, p. 105. [6] ——, “ A formulation of the channel capacity of multipl e-acce ss chan - nel, ” IEEE T ran s. Inform. Theory , vol. 55, no. 5, pp. 2083–2096, May 2009. [7] M. Rezae ian and A. Grant, “Computation of total capacity for discrete memoryless multiple- access channels, ” IEEE T rans. Inform. Theory , vol. IT -50, no. 11, pp. 2779–2784, Nov . 2004. [8] E. Calvo , D. P . Palomar , J. R. Fonollo sa, and J . V ida l, “The computation of the capacit y regi on of the discrete mac is a rank-one non-con vex optimiza tion problem, ” in Pr oc. IEEE Int. Symp. Informat ion Theory , Nice, France, June 2007, pp. 2396–2400. [9] R. Ahlswede, “Multi -way communication channels, ” in P r oc. 2nd. Int. Symp. Information Theory , Tsakhadsor , Armenia n SSR, 1971, pp. 23– 52. [10] H. Liao, “Multip le access channels, ” Ph.D. dissertation , Department of Electric al Engineering, Univ ersity of Hawaii , Honolulu, 1972. [11] T . Co ver and J. Thomas, Elements of Information Theory . Ne w Y ork, NY : W ile y , 1991. [12] R. G. Gallager , Information Theory and Reli able Communication . New Y ork, NY : Wile y , 1968. [13] S. Boyd and L. V andenbe rghe , Conv ex Optimization . Cambri dge, U.K.: Cambridge Uni versit y Press, 2004.