Evaluate the Word Error Rate of Binary Block Codes with Square Radius Probability Density Function

1 Ev aluate the W ord Error Rate of Binary Block Codes with Squ are Radius Probability Density Function Xiaogang Chen, Student Member , I EEE, Hongwe n Y ang, Member , IEEE, Jian Gu an d Ho ngkui Y ang Abstract The word error rate (WER) o f soft-decision-decode d binary block codes rarely has closed-form. Bounding techniques are widely used to ev aluate the performance of maximum-likelihood decoding algorithm. But the existing bounds are not tight enough especially for l o w si gnal-to-noise rat ios and become looser when a suboptimum decoding algorithm is u sed. This paper proposes a ne w concept named sq uar e radius proba bility density function (SR-PDF) of d ecision region to e valuate the WER. Based on the SR-PDF , T he WER of binary block codes can be calculated precisely for ML and suboptimum decoders. Furthermore, for a long binary block code, SR-PDF can be approximated by Gamma distribution with only tw o parameters that can be measured easily . Using this prope rty , two closed-form approximati ve e xpressions a re proposed which are very close to the simulation results of the WER of interesting. Index T erms Binary block codes, bounds, decision re gion, sq uare radius probability density fun ction, word error rate. I . I N T R O D U C T I O N The performan ce evaluation of binary b lock codes with soft-dec ision-deco ding in additive wh ite Gaussian no ise (A WGN) and fading cha nnels has lo ng been a problem in coding th eory and practice. A clo sed-form expression of word er ror rate ( WER) for po pularly used long co des hasn’t been derived as yet. Thu s, bo unding tech niques are widely used for performan ce evaluation of maximum-likelihood ( ML) decoding. The most po pular upper boun d is the u nion bou nd . When the weight en umerating function of a co de is k nown, union boun d presents a tight up per bound at signal-to- noise ratios (SNRs) ab ove the cutoff rate limit but becomes u seless at SNRs below the cu toff rate limit [1]. Based on Gallager’ s first bo unding techniq ue [2] , some tighte r up per bo unds ar e pr esented [3 ] [4 ] [5]. These bounds are t ighter relati ve to union bound. B u t e ven Poltyrev’ s tangential sph er e boun d [6], which has been believed as the tightest bo und f or binar y block co des, still has a gap to the r eal value of WER [7] [8 ] [ 9]. The authors are with the School of T elecommunicati on Engineeri ng, Bei jing Univ ersity of Posts and T elecommunicati ons, Beijin g, 100876, China. (e-mail: debug3000@gmai l.com; yanghongwen @263.net.cn) Novem ber 24, 2021 DRAFT 2 Additionally , these bounds are all based on ML decoding, which is too complex to implement in practice. When a suboptimu m deco der is used , the WER will cha nge b ut these bounds still keep their original v a lue. In this paper, a new concep t named squa r e radius prob ability density function (SR-PDF) of d ecision region is propo sed to calculate th e WER precisely at SNRs of inter esting and any decodin g a lgorithm. The basic premise is that in A WGN ch annel, when the enco der and decodin g alg orithm are given, th e decision region is fixed, an d WER is c ompletely determined by the decision region. Th e SR-PDF prop osed in this pap er i s unique for e very encoder- channel- decoder mode ls, thu s any ch anges of cha nnel an d d ecoding algorith m c an b e reflected by it. Furthe rmore, when the co dew ord leng th N → ∞ , the asymptotic WER is exactly the co mplementar y cu mulative distribution probab ility of the no rmalized squ are r adius an d can b e rough ly charac terized by the maxim um p oint o f th e SR-PDF . Moreover , for po pular long c odes such as T ur bo co des, Low-density parity check (L DPC) cod es an d Con volution al codes, e tc, their SR-PDFs are close to Gamma distribution, which implies that th e exh austing measur ement of SR-PDF is no t need ed, and only the me an an d variance of th e square radius are eno ugh to get an ap proxim ated SR-PDF . Based on these prop erties, two closed-form app roximative expre ssions of WER are pro posed and their approx imation erro rs are aro und the ord er o f 0.1 dB and 0 .3dB respectively for WER ab ove 1 0 − 3 . The rest of this paper is o rganized as follows: Section II introdu ces the system model and the conce pt of SR- PDF of decision region, as well as th e meth od of mea suring the pdf. In section III, the WER in A WGN c hannel is derived using SR-PDF an d several example s are followed to p rove the validity o f th e prop osed meth od. Sectio n IV illustra tes so me im portant asymptotic p roper ties o f th e r elation of WER to the norm alized SR-PDF . In section V , two closed-fo rm appro ximative exp ressions a re d erived for WER calcu lation. Sectio n VI extending th e SR-PDF method to flat fading chan nel. Th e conclu sion is d rawn in section VII , includin g some d iscussion and remark s on the SR-PDF metho d an d othe r applicatio ns. I I . P R E L I M I N A R I E S A. System Mod el The following system m odel is used in the paper . Binary inf ormation b its are first en coded to binar y blo ck cod e. The cod e rate is R = k / N , where N is the c odeword length an d k is the info rmation length . Th en enco ded bits are BPSK modu lated (the resu lting signal set is S ), an d transm itted in A WGN channel. Th e received signal is y = s + n (1) where s = [ s 1 , s 2 , ..., s N ] T ∈ S , s i ∈ {± √ E s = ± √ RE b } , E s and E b are, respec ti vely , the energy per code bit and the energy pe r in formation b it. n = [ n 1 , n 2 , ...n N ] T is the additive wh ite Gaussian noise with zero- mean and variance N 0 2 . In this pap er , SNR (signal to noise power ratio) is define d as β = 2 E s N 0 . It is also assum ed th at the signal s is de tected coh erently an d deco ded with some algorith m at the r eceiver , and th e chann el side info rmation is known if it is requ ired. DRAFT November 24, 2021 3 B. Decision Region an d Squ ar e Rad ius Pr oba bility D ensity Fu nction The decision region V s of signal s is a set in N dimensiona l Euclidean space R N . Con sider th e dec oder as a function that m aps the received vector y to a transmitted sign al s ∈ S , f decoder : R N → S , then the decision region can be defin ed as V s =  y | y ∈ R N , f decoder ( y ) = s  , i.e. the doma in o f s . Whenever th e d ecoder is specified , th e decision region is fixed. Wh en the re ceiv e d vector y lies inside V s , it will b e correctly de coded to s , oth erwise a decodin g erro r occur s. For linea r block codes investigated in this pape r , all the cod ew o rds have the same decisio n region. Fig. 1 is an examp le of two d imensional decision region, where n is the additive noise, θ is the directio n of n . Point p is on the bo undary of decision r egion a nd the radial origin ated fro m s along θ . Conne cting s a nd p , the length o f vector r ( θ ) is th e radiu s of the decision region along the d irection o f n . Define l ( θ ) = | r ( θ ) | 2 as the squar e radius in the direction of θ . Becau se n is a random variable, so l ( θ ) is a ra ndom variable. The pdf of l ( θ ) is d enoted as p l ( l ) and is abbreviated as SR-PDF . This concept can be extende d to N dimension al space, where θ is determ ined by N − 1 ang les (e.g. az imuth and e lev ation in three dimension al space) . The SR-PDF is too co mplex to work o ut in analytical way fo r lon g c odes, but it can be measured w ith simulatio n: For th e system mo del ab ove, g enerate a white Gaussian noise vector n = ( n 1 , n 2 , ...n N ) , normalize n to n ′ = n | n | = (1 , θ ) , where θ is the direc tion of n , scale n ′ by λ and send the vector y = s + λ n ′ to the dec oder . Th ere exists a ˆ λ > 0 such that f decoder ( s + λ n ′ ) = s , ∀ λ ≤ ˆ λ an d f decoder ( s + λ n ′ ) 6 = s , ∀ λ > ˆ λ . In this paper, we will a ssume that the d ecision r egion is simply co nnected 1 . Consequen tly , | r ( θ ) | = ˆ λ is the radiu s in d irection of n , and l ( θ ) = ˆ λ 2 is the squ are radius. No te that r ( θ ) is related to the co de bit en ergy and co dew o rd leng th. This effect can b e rem oved by nor malizing the square radius by N E s , i.e. l n = l N E s . The r esulting SR-PDF is ref erred to as normalized SR- PDF an d is de noted as p l n ( l n ) h ereafter . With sufficient tests, the appro ximated normalized SR-PDF can be o btained. Fig. 2 is the normalized SR-PDF of T urb o co des with different cod ew or d lengths, code rate s and decodin g algorithm s in A WGN ch annel. Turbo cod es used in this p aper is defin ed in [ 13]. Obviously , all the chan ges in the encoder and d ecoder will be r eflected by the SR-PDF . Larger decision region implies th at the cod e can tolerate larger noise, th erefor e, the SR-PDF which oc curs in the righter side will ha ve a be tter WER perform ance. Sub sequent sections will mainly d iscuss the relation between SR-PDF and WER. I I I . W O R D E R R O R R A T E I N AWG N C H A N N E L In A WGN ch annel, the d ecision region and the correspo nding SR-PDF is co mpletely determ ined whe never th e decoder is specified, no matter whether it is a maxim um likeliho od (ML) deco der like the V iterbi decodin g for Con volution al codes or a subop timal decod er such a s the iterati ve decoders for T urbo and LDPC co des. For a binary linear block code, all the codewords have the same er ror rate. Th erefore , withou t loss of gener ality , assume 1 Note that when iterati ve decoding algorithm is used for Turbo and LDPC decoding, as the iterat ion increases, it is possible, particu larly on very noisy channel, for the decoder to con ver ge to the correct decision and then div erge again [10]. i.e. it is possible that s + λ 1 n ′ can be decode d in error e ven if f decoder ( s + λ 2 n ′ ) = s , where λ 1 < λ 2 . T his implies that the decisio n region of iterat ive decoder may not be a simply connect ed re gion , but the probability of this except ion is m uch smaller than the WE R. Novem ber 24, 2021 DRAFT 4 a codeword s ∈ S is transmitted. Th e word error ra te is the prob ability that the received vector y / ∈ V s condition ed on s , that is P e = 1 − I V s p ( y | s ) d y = 1 − I V s − s p ( y − s ) d y = 1 − I V ′ s p ( n ) d n (2) where V ′ s = V s − s is th e shift of the decisio n region fro m s to origin . Den oting n in polar co ordinates, ( 2) becom es P e = 1 − I V ′ s p ρ, θ ( ρ, θ ) dρd θ = 1 − I V ′ s p ρ | θ ( ρ ) p θ ( θ ) dρd θ = 1 − Z θ " Z ρ ≤| r ( θ ) | p ρ | θ ( ρ ) dρ # p θ ( θ ) d θ = Z θ " Z ρ> | r ( θ ) | p ρ | θ ( ρ ) dρ # p θ ( θ ) d θ = E θ " Z ρ> | r ( θ ) | p ρ | θ ( ρ ) dρ # (3) Note that the integral in the bracket is the d ecoding error proba bility conditioned on the directio n of a noise realization, so P e | θ = Z ρ> | r ( θ ) | p ρ | θ ( ρ ) dρ = P [ ρ ≥ | r ( θ ) | ] = P  ρ 2 ≥ l ( θ )  (4) Expectation over θ is equivalent to expec tation over l , th us the av erage error p robability is P e = E θ  P e | θ  = E l  P  ρ 2 ≥ l  = Z ∞ 0 P  ρ 2 ≥ x  p l ( x ) dx = Z ∞ 0 P  ρ 2 ≥ N E s x  p l n ( x ) dx (5) DRAFT November 24, 2021 5 where ρ 2 = | n | 2 = N P i =1 n 2 i . Define x = ρ 2 , then x is chi-square distributed [11] with N degrees of freedo m. T he pdf o f x is p ( x ) = 1 N N/ 2 0 Γ ( N /2) x N /2 − 1 e − x / N 0 , x ≥ 0 (6) where Γ( x ) is th e g amma function [1 2]. When x is an integer and x > 0 , Γ( x + 1) = x ! (the factorial of x ). So P e | θ = P  ρ 2 ≥ l  = 1 − Z N E s l n 0 1 N N/ 2 0 Γ ( N /2) x N/ 2 − 1 e − x/ N 0 dx = 1 − 1 Γ ( N /2) Z N β l n / 2 0 e − x x N 2 − 1 dx = 1 − 1 Γ ( N /2) γ  N 2 , N β l n 2  (7) where γ ( α, x ) is the incomplete ga mma functio n defined as [12 ] γ ( α, x ) = Z x 0 e − t t α − 1 dt, α > 0 (8) when α is an integer [12], γ ( α, x ) = ( α − 1)! 1 − e − x α − 1 X m =0 x m m ! ! (9) substitute (7) into (5 ), P e = E θ  P e | θ  = E l  P  ρ 2 > l  = Z ∞ 0  1 − 1 Γ ( N /2) γ  N 2 , N β x 2  p l n ( x ) dx (10) (10) show that, WER of linear bin ary cod es und er A WGN channel is completely determined b y the normalized SR-PDF thro ugh a one dim ensional integral. Simulations are used to verify (10). Three error c ontrol co des comm only used in wireless com munication s are considered , includ ing Con volution al codes [12], Turbo cod es [1 2] an d L DPC co des [ 14]. Fig. 3 is the co mparison o f WER between simu lation and tha t ev aluated from ( 10). For th e simu lation, each poin t on the curve is obtained by 10 6 tests; For the results of (1 0), 10 5 radius are mea sured to get p l n ( l n ) , which is th en substituted into ( 10) to get the WER. Fig. 3( a) shows th e WER of T u rbo co de for different max imum iteratio ns. The p arameter is N = 57 6 , R = 1 / 3 an d the d ecoding algorith ms are Log-M AP and Max-lo g-MAP . Fig. 3(b ) are the WERs of T urb o, LDPC and Conv olutio nal c odes with d ifferent code le ngths and code rates. The dec oding algo rithms are log -MAP with 8 maximum iterations for turb o code, soft V iterb i algo rithm fo r Conv olutional cod e, sum-pr oduct algor ithm (SP A) and min-sum algorithm ( MSA) with 25 maximu m iterations and la yered deco ding for LDPC cod es. It can b e seen fr om these figu res th at th e WER ev alu ated from (10) m atches very well with the simulation results except for large SNRs. The mismatch in large SNR region may be caused by the inaccur ateness o f th e simulation , or the in accuratene ss o f the “left tail” (this will be f urther explained in section IV) of p l n ( l n ) , bo th o f which a re difficult to be mea sured Novem ber 24, 2021 DRAFT 6 precisely owning to infinitesimal p robability . Moreover , th e SR-PDF meth od can trace the change of dec oder, e.g . the num ber o f iteration s, any m odifications of the algo rithm an d etc. w hile the b ounds such as u nion b ound an d so on cann ot do this. I V . A S Y M P T O T I C P RO P E RT I E S O F W E R Eq.(10) shows that the WER o f a binary linear c ode is completely d etermined by th e n ormalized SR-PDF , p l n ( x ) . When the codeword le ngth, N → ∞ , th ere are some asym ptotic prop erties of WER which will b e d iscussed in th is section. Pr operty 1: Defin e τ = 1 /β and P a e as the WER wh en cod ew o rd len gth N → ∞ , i.e. P a e ( τ ) = lim N →∞ P e ( N , β ) where P e ( N , β ) is defined by (1 0). Then, P a e ( τ ) is exactly the cum ulativ e d istribution function (CDF) of the normalized squ are radius: P a e ( τ ) = lim N →∞ P e ( N , β ) = Z 1 /β 0 p l n ( x ) dx = Z τ 0 p l n ( x ) dx = F l n ( τ ) , (11) where F l n ( x ) is the CDF o f nor malized squ are-rad ius l n . Pr oof: Define f ( α, t ) , 1 Γ( α ) γ ( α, αt ) = R αt 0 e − x x α − 1 dx Γ( α ) , t ≥ 0 , α > 0 (12) It is obvious that f ( α, 0) = 0 and f ( α, ∞ ) = 1 . T aking d eriv ative with respe ct to t , ∂ f ∂ t = αe − αt ( αt ) α − 1 Γ( α ) , t ≥ 0 (13) For a large α , Γ( α ) can be app roximated with Stirling’ s Series [12] Γ( α ) = e − α α α − 1 2 √ 2 π  1 + 1 12 α + 1 288 α 2 − · · ·  ≈ e − α α α − 1 2 √ 2 π (14) Substitute into (13) ∂ f ∂ t = αe − αt ( αt ) α − 1 e − α α α − 1 2 √ 2 π = r α 2 π t 2  te 1 − t  α , t ≥ 0 (15) The term te 1 − t is a lw ays less than 1 wh en t 6 = 1 . Th us, lim α →∞ ∂ f ∂ t =      0 t 6 = 1 ∞ t = 1 (16) f ( α, t ) is a continu ous f unction of t . Theref ore, as α ap proach es infinity , f ( α, t ) approa ches to a un it step fu nction: lim α →∞ 1 Γ( α ) γ ( α, αt ) =      f ( α, ∞ ) = 1 t > 1 f ( α, 0) = 0 0 ≤ t < 1 (17) Applying (1 7) to (10) will obtain ( 11). DRAFT November 24, 2021 7 This prop erty can also be explained with the law of la r ge numbers : Dividing ρ 2 and l ( θ ) in (7) by codew o rd length N P e | θ = P  ρ 2 N > l ( θ ) N  (18) because ρ 2 = N P i =1 n 2 i , whe re n 2 i , i = 1 , 2 , ...N are ind ependen t and identically distributed (i.i.d) variables and E ( n 2 i ) = N 0 2 , ba sed on th e law of large numb ers lim N →∞ P                N P i =1 n 2 i N − N 0 2         =     ρ 2 N − E s β     < ε        = 1 (19) where ε is a positive n umber arbitrarily small. (19) implies that ρ 2 N approa ches a co nstant, E s β , as N appr oaches infinity . Based on (18) and ( 19) lim N →∞ P e | θ = P  l ( θ ) N < E s β  = P  l n ( θ ) < 1 β  =      1 1 β > l n ( θ ) 0 else (20) Substituting (20) into ( 10) will get ( 11). The con ditional WER (7), i.e. th e term in the sq uare bracket of (1 0) is a decr easing fu nction arou nd l n = 1 /β with r espect to l n . T his is illustrated in Fig.5 together with th e SR-PDF p l n ( l n ) . Th e WER o f (10) is the integral of the produ ct P e | θ ( β l n ) p l n ( l n ) . I t is clear that when SNR is h igh, WER is d ominated b y the lef t tail of the n ormalized SR-PDF . Pr operty 2: Th e inflectio n p oint τ 0 of P a e ( τ ) , is the maximum point o f the n ormalized SR-PDF . Pr oof: Based on pr op erty 1 , taking the second deriv ativ e of (1 1) with respect to τ d 2 P a e ( τ ) d τ 2 = d p l n ( τ ) d τ = 0 (21) The result τ 0 that make (2 1) equal to zer o is the inflection poin t of P a e and mu st be the max imum poin t of p l n ( τ ) . The inflection poin t τ 0 tells th e position wher e the WER curve falls rap idly . Defin e critical SNR as the in verse of the inflection p oint: β c = 1 / τ 0 (22) then β c can b e viewed as a sing le p arameter wh ich ca n be u sed to ch aracterize th e WER perfor mance of lon g c odes. This has been shown in Fig . 6, wh ere we have d rawn the simulated WER with lin er c oordin ates. For the po pular codes, th e normalized SR-PDF tends to be symmetr ic abo ut th e m aximum p oint a nd th e n ormalized SR-PDF p l n ( x ) arrives to its maximum rou ghly at µ l n = E [ l n ] . T hus, the critical SNR of a cod e can be app roximated as β c ≈ 1 µ l n . The critical SNRs of the codes in Fig. 3(b ) a re listed in T ab le I. Novem ber 24, 2021 DRAFT 8 Pr operty 3: For the c apacity achiev able codes, decision region is a mu lti-dimension al sphere with constant radius. The inverse of the no rmalized square radius, i.e. th e cr itical SNR β c , is the Sh annon limits. By ” Shannno n lim it”, it means such a thr eshold SNR β s that for a given family of codes with fixed co de r ate and codeword leng th N → ∞ , if the chann el SNR is greater th an β s , the co de will be dec oded successfully , oth erwise, if SNR is less th an β s , the decodin g pro cess will fail. Pr oof: Based on 10 a nd 35 P a e ( τ ) = Z 1 /β 0 p l n ( x ) dx =      1 β ≤ β c 0 β > β c (23) The β s that satisfy 23 is u nique for the giv en family o f codes, thus with the definition of β s , it is clearly that β s = β c . 2 3 im plies tha t p l n ( x ) = δ ( x − 1 β s ) , whe re δ ( x ) is the Dirac im pulse function . A rando m variable with δ pdf is in fact a constant which is also the mean . T hus the decision region mu st be a sp here with constant radius. It is well known that threshold SNR of iterati ve soft-dec ision d ecoding can be obtained by means of Density Evolution [16] [17] or the e quiv alen t (Ex trinsic In formatio n Transfer) EXIT chart [18 ] [1 9] [ 20]. So p roperty 3 implies that measur ing th e average squ are rad ius of the dec ision region is an other way to determin e th reshold SNR. For the T u rbo code in T able I, which is exactly the same as the o ne used in [17], the thresho ld SNR o btained with Density E volution can be found in [17 ] as 0 .70dB f or 1/2 cod e rate and 0. 02dB for 1/3 co de rate, the difference with β c is with in 0.07 dB. For the 1/2 cod e rate L DPC code in T able I, its EXI T chart is shown in Fig.4, wher e the two cur ves intersect un til I E v = 1 imp lying that the thr eshold SNR is about 0.98d B, while th e critical SNR obtained fr om µ l n is 0 .92dB, the d ifference is 0.06 dB. Note that tho ugh the thresho ld SNR is generally viewed as a kind o f analytical value , this value can only be obtain ed via nu merical simulatio ns with possible a pprox imations (such as Gaussian app roximatio n [ 16] [1 7]). Therefo re, it is hard to say which o ne amon g β c and β s is mor e accurate. Property 3 has provided us a simple method to estimate the a symptotic p erform ance (Shann on limit) fo r a given family of cod e. W e on ly n eed to me asure th e me an o f th e square-r adius with a code of adeq uate length because the mean is independe nt o f codeword length if the co de family is given. Note tha t m easuring the m ean is much simpler than measurin g th e p df, with o ur experience, 10 00 radius is enou gh to get a relatively accurate result. The d ifference between good co des (th e codes that can achie ve Shann on limits when N → ∞ ) an d p ractical codes is that, The WER of go od codes falls steep ly at the critical SNR while the WER o f p ractical cod es falls in a rolling off fashion aroun d the cr itical SNR. Define ∆ ǫ as the rang e co rrespon ding to P a e ( τ ) falls fro m 1 − ǫ to ǫ , wh ere 0 < ǫ ≪ 1 , i.e. ∆ ǫ , τ 1 − ǫ − τ ǫ = F − 1 l n (1 − ǫ ) − F − 1 l n ( ǫ ) (24) Then ∆ ǫ can be viewed as a measure of per fectness of pra ctical co des comp ared with the go od cod es. I f the code is c apacity achiev able, the WER is a step fun ction th us ∆ ǫ = 0 . Otherwise, ∆ ǫ will be a positive nu mber . Th e ∆ ǫ for co des in Fig. 3b are listed in table I. DRAFT November 24, 2021 9 Assume th at p l n ( τ ) is sy mmetrical abou t the mean µ l n when co dew o rd len gth N → ∞ , ∆ ǫ is bou nded by ∆ ǫ ≤ q 2 σ 2 l n /ǫ (25) where σ 2 l n is th e variance o f l n . This is becau se that, with the Chebysh ev Inequ ality : 2 ǫ = P  | τ − τ 0 | ≥ ∆ τ 2  ≤ 4 σ 2 l n ∆ 2 τ (26) In pra ctice, the WER falling r ange in decibel d omain may be mo re interesting , i.e . ∆ ǫ (dB) , 10 log 10 τ 2 τ 1 = 10 log 10 F − 1 l n (1 − ǫ ) F − 1 l n ( ǫ ) (27) The Cheby shev boun d now ch anges to ∆ ǫ (dB) ≤ 10 log 10 τ 0 + ∆ τ 2 τ 0 − ∆ τ 2 ! = 10 log 10 µ l n + σ l n p 1 / 2 ǫ µ l n − σ l n p 1 / 2 ǫ ! dB . (28) The bo unds in dB for codes in Fig. 3 b are a lso listed in table I. It is obvious th at the b ound is qu ite loose, r ough ly 3 ∼ 5 times larger . V . A P P RO X I M AT I O N O F W E R Eq.(10) inv olves an in tegration of the pro duct of inc omplete gamm a fu nction and SR-PDF . Mo reover , accur ate measuremen t of SR-PDF requir es a large num ber of d ecoding tests. Thus, it is inconvenient for p ractical use, and approx imation form ulas will be welco me. The appr oximation of (10) is to ap proxim ate the nor malized SR-PDF p l n ( x ) . Alth ough other a pprox imations such as Gaussian are also possible, it is fo und that Gamma a pprox imation is mor e accurate. Th e pd f of Gamma distribution is g iv e n b y p ( x ) = 1 b a Γ( a ) x a − 1 e − x b (29) where a and b a re the parameters of Gamma distribution that h ave the following re lationship with µ l n and σ 2 l n [15]:      a = µ 2 l n /σ 2 l n b = σ 2 l n /µ l n (30) Substitute (29) into (10), P e ≈ Z ∞ 0 1 b a Γ( a ) x a − 1 e − x b  1 − 1 Γ ( N /2) γ  N 2 , N β x 2  dx (31) Novem ber 24, 2021 DRAFT 10 For lo ng codes, in creasing code length from N to N + 1 gene rally brin gs no n otable perfo rmance difference. So assuming th at N is even and recalling (9), the WER c an be simplified as P e ≈ N 2 − 1 X m =0  N β 2  m b a Γ( a ) m ! Z ∞ 0 x m + a − 1 e − x ( 1 b + N β x 2 ) dx = N 2 − 1 X m =0  N β 2  m  2 b 2+ N β b  m + a Γ( m + a ) b a Γ( a )Γ( m + 1) Z ∞ 0 1  2 b 2+ N β b  m + a Γ( m + a ) x m + a − 1 e − x (2+ N β b ) 2 b dx = N 2 − 1 X m =0 (1 − u ) m u a mB ( a, m ) (32) where B ( a, m ) = Γ( a )Γ( m ) Γ( a + m ) is th e Beta Fun ction [12] an d u = 2 N β b +2 < 1 . The ad vantage of (32) over (10) is tha t only µ l n and σ 2 l n need to be measur ed. In the v iewpoint of statistics, th e nu mber o f r adiuses requ ired to estimate µ l n and σ 2 l n is much smaller than to estimate the SR-PDF . T able I I lists the mean and variance o f l n and a, b for some cod es in A WGN chann el. Fig. 7 is the co mparison of WER between simu lation and ap prox imation using (32). I t can be seen that the approx imation for all the codes only d eviates fro m the simu lation within abou t 0.1d B for WER ab ove 10 − 3 . If th e error floor of app roximated Turbo code WER do es no t o ccur at high E b / N 0 , this is a n ice appr oximation for WER of intere st with a significan tly lower c omputatio nal comp lexity . For a large codew o rd len gth, the WER expre ssion can be further simplified using pr op erty 1 in section III. Substitute (29) into (11), the WER c an be appr oximated as P e ≈ Z 1 /β 0 1 b a Γ( a ) x a − 1 e − x b dx = 1 Γ( a ) γ  a, 1 β b  (33) In (33), roun ding a to its lower integer an d recallin g ( 9), the WER c an be appr oximated as P e ≈ 1 − e − 1 β b ⌊ a ⌋− 1 X m =0  1 β b  m m ! (34) (34) and ( 32) u se the same par ameters to app roxima te WER. (3 4) is simple r to evaluate but less acc urate than ( 32). Fig. 8 is th e com parison b etween simulation WER and appro ximation of ( 34). Similar to th e result o f Fig. 7, If the er ror floor o f T ur bo cod e d oes n ot occur within the range th at WER is ab ove 10 − 3 , the approx imation r esult deviate fr om the simulation within abo ut 0 .3dB. When (32) an d (34) are app lied to evaluate the WER, an im portant pro blem is how accurate shou ld the p arameters a and b be me asured. Gen erally , fo r a lon g co de a possesses a large value. It c an b e verified that the term u a B ( a,m ) in (32) is in sensitiv e to the error of a , and the last f ew terms in the sum mation of (34) are very small ( on the or der of two magnitud es lower than th e sum ). Thus an error o f a within ± 1 is safely accep table for the WER evaluation. DRAFT November 24, 2021 11 The parameter b on ly o ccurs in (34) in th e fo rm of prod uct β b . T hus, an error of b by ∆ dB is eq uiv alent to that b is exact but β (SNR) is biased by − ∆ d B. From Fig. 7 an d Fig. 8, it can be ob served that the accur acies of (3 2) and (34) ar e on the order o f 0.1d B and 0.3dB re spectiv ely , therefo re, the me asurement erro r o f b should be less than 0. 1dB. Sev er al hun dreds of rad iuses are g enerally enou gh for this req uiremen t. V I . W O R D E R RO R R AT E I N F L A T F A D I N G C H A N N E L In flat fading ch annel, (1) changes to y = diag ( h ) · s + n (35) where h = [ h 1 , h 2 , ..., h N ] T is the chan nel gain , whic h scales every elemen t of the tr ansmitted vector . Thus, th e decision region is depen dent on a spec ific channel vector . Given the ch annel vector, th e co nditional WER can be calculated by (10) with p l n ( l n ) be replaced b y p l n | h ( l n ) which is the nor malized SR-PDF c ondition ed on a c hannel vector . Th e average WER for flat fading chann el is then obtain ed by takin g expe ctation over all possible h : P e = E h  Z ∞ 0  1 − 1 Γ ( N /2) γ  N 2 , N β x 2  p l n | h ( x ) dx  = Z ∞ 0  1 − 1 Γ ( N /2) γ  N 2 , N β x 2  E h  p l n | h ( x )  dx = Z ∞ 0  1 − 1 Γ ( N /2) γ  N 2 , N β x 2  ¯ p l n ( x ) dx (36) where ¯ p l n ( x ) , E h [ p l n | h ( x )] is the normalize d SR-PDF averaged over the ensemb le of h . Fig. 9 is the no rmalized average SR-PDF fo r several cod es inves tigated in th is p aper . In th ese examples, fully interleaved Rayleigh flat fading chan nel is con sidered. T he elements of h a re i.i.d. Rayleig h rand om variables with pdf p h ( h i ) = 2 h i e − h 2 i , h i > 0 , i = 1 , 2 , · · · , N (37) Fig. 9 indic ates th at the average SR-PDF in fully interleaved Rayleigh flat fading chann el still keeps th e same shape as in A WGN chan nel. Th erefore, the appro ximation s p resented in Sectio n IV , i.e. ( 32) a nd (34) can also be used to ev aluate the average WER in flat fading channels, on ly with a and b replaced by a and b respec ti vely , which are av e raged over all chan nel gain realizations. T able II lists the mean and variance of the average l n and a, b used in (32) and (34) in Rayleigh flat fadin g chann el. Fig. 1 0 is th e co mparison betwe en simu lation WER and (36), ( 32), (34). It is o bvious th at the SR-PDF method and its appro ximations ar e still applicable in flat fading cha nnel. V I I . C O N C L U S I O N SR-PDF of dec ision region introduces a new meth od to ev aluate the performan ce of binary block co des. T he WER can be calculated u sing th is p df precisely , and ev en th e closed-fo rm appro ximations are mo re p recise th an existing tightest boun ds for practically used long block cod es at SNRs o f interesting. De spite that the SR-PDF Novem ber 24, 2021 DRAFT 12 method is demonstrated with bin ary c odes in A WGN and flat fading chan nel in this paper, it is straigh tforward to generalize this metho d to any situations wh ere th e error r ate is ch aracterized by th e decision region , such as memory less modu lation, MIMO detection, co ded-m odulation , equalizatio n, etc. In th ese situations, the decision region may no t have the same shape for different transmitted sign als. Nevertheless, the av erage error ra te can still be ev a luated by the average SR-PDF in a similar way as in fading chan nel. R E F E R E N C E S [1] D. Di vsalar , S. Dolinar , F . Pollara, and R. J. McEliece, “Tra nsfer function bounds on the performance of turbo codes Pasadena, ” CA: Jet Propulsion L ab ., TDA Progr . Rep. 42-122, pp. 44-55, Aug. 15, 1995. [2] D. Divsala r, “ A simple tight bound on error probability of block codes with applicati on to turbo codes, ” NASA, JPL , Pasadena , CA, TMO Progr . Rep. 42-139, 1999. [3] E. R. Berlek amp, “The technology of error-corre cting codes, ” Proc IEEE , vol. 68, pp. 564-593, May 1980. [4] H. Herzber g and G. Poltyre v , “T echniqu es of bounding the probability of decoding error for block-code d modulati on structures, ” IEE E T rans.Inform. Theory , vol. 40, pp. 903-911, May 1994. [5] T . M. Duman and M. Salehi, ”Ne w performance bounds for T urbo codes, ” IEEE T rans. Commun. , vol. 46, pp. 717-723, June 1998. [6] G. Poltyre v , “Bounds on the decoding error probabili ty of binary linear codes via their spectra, ” IEEE T rans. Inform. Theory , vol.40, pp.1284-1292, July 1994. [7] I. Sason and S. Shamai (Shitz), “Improv ed upper bounds on the decodin g error probabilit y of parallel and serial concatena ted turbo codes via their ensemble distanc e P DF , ” IEEE T rans. Inform. T heory , vol. 46, pp. 1-23, Jan. 2000. [8] I. Sason and S. Shamai (Shitz), “V ariati ons on the Gallager bounds, connections and applica tions, ” IEEE T rans. Inform. Theory , vol. 48, pp.3029-3051, Dec. 2002. [9] I. Sason and S. Shamai (Shitz), “Improv ed upper bounds on the ensemble performance of ML decoded low density parity check codes, ” IEEE Commun. Lett ., vol. 4, pp. 89-91, Mar . 2000. [10] Shu Lin, Daniel J.Costello,Jr Er r or Contr ol Coding. Decond Edition , Pearson Prentice Hall, 2004, pp.839. [11] John G . Prokis, Digital Communications, F ourth Edition , McGraw-Hil l, 2001 pp. 45. [12] I.S. Gradsbte yn, I.M. Ryzbik, T able of Inte grals, Series, and Pr oducts. Sixth Edition . Academic Press, 2000, pp. 883-892. [13] 3GPP2 C. S0024-B cdma2000 High Rate Packe t Data Air Interfac e Specificat ion. May 2006 [14] IEEE P802.16e/D12, Oct. 2005 [15] Eric W . W eisstein, CRC Concise Encyclopedi a of Mathemat ics , CRC P ress, 1999. [16] Sae-Y oung Chung, Thomas J. Richardson, and Rdiger L. Urbanke, “ Analysis of Sum-Product Decoding of Low-Densit y Parity-Ch eck Codes Using a Gaussian Approximation”,IEEE Transacti on On Information Theory , V ol. 47, NO. 2, Feb . 2001. [17] Hesham El Gamal, and A. Roger Hammons, Jr ., “ Analyzing the Turbo Decoder Using the Gaussian Approximation”, IE EE Transact ion On Information T heory , vol. 47, NO. 2, Feb . 2001. [18] Stepha n ten Brink,“Con vergenc e Behavior of Iterati vely Decoded Paralle l Concatenate d Codes”, IEEE Transacti on On Communications, V ol. 49, NO. 10, Oct. 2001. [19] Eran Sharon, Alexe i Ashikhmin, and Simon Litsyn, “ Analysis of Low-Densi ty Parity-Che ck Codes Based on EXIT Functions”, IEE E Tra nsactio n On Communications, vol. 54, NO. 8, Aug. 2006. [20] K ollu, S. R.; Jafarkhani, H., “On the EXIT chart analysi s of low-densit y parity-check codes”, GL OBECOM ’05. IEE E. V olume 3, 2005. DRAFT November 24, 2021 13 T ABLE I T H E C R I T I C A L S N R A N D ∆ ǫ ( ǫ = 0 . 0 1 ) F O R S O M E C O D E S . Codes Critic al SNR ( E b / N 0 dB ) ∆ ǫ (dB) upper bound of ∆ ǫ (dB) T urbo 1152 1/3 (Log-MAP) 0.09 1.65 5.80 T urbo 576 1/3 (Log-MAP) 0.03 2.12 9.88 T urbo 1152 1/2 (Log-MAP) 0.76 1.64 4.54 LDPC 1152 1/2 (SP A) 0.92 1.15 4.46 LDPC 1152 3/4 (SP A) 2.22 1.21 3.73 * The parameters µ l n and σ 2 l n used to calculate the criti cal SNR and the upper bound are listed in table II. T ABLE II P A R A M E T E R S O F N O R M A L I Z E D S R - P D F F O R R E P R E S E N TA T I V E E R RO R C O N T R O L C O D E S I N AW G N C H A N N E L codes decodin g algorit hm mean va riance a b T urbo 1152 1/3 Log-MAP 1.47 1.47e-2 147.45 1.00e-3 T urbo 1152 1/2 Log-MAP 0.84 3.25e-3 219.55 3.85e-3 T urbo 1152 1/2 Max-Log-MAP 0.79 3.06e-3 202.18 3.89e-3 T urbo 576 1/3 Log-MAP 1.49 2.94e-2 75.39 1.98e-2 T urbo 576 1/3 Max-Log-MAP 1.39 2.81e-2 68.21 2.03e-2 T urbo 576 1/2 Log-MAP 0.85 6.30e-3 115.23 7.40e-3 T urbo 1152 2/3 Log-MAP 0.51 1.13e-3 233.37 2.20e-3 T urbo 1152 3/4 Log-MAP 0.40 7.84e-4 204.04 1.96e-3 LDPC 1152 1/2 Sum-Product 0.81 2.93e-3 221.19 3.64e-3 LDPC 1152 1/2 Min-Sum 0.72 2.13e-3 245.43 2.95e-3 LDPC 1152 3/4 Sum-Product 0.40 5.25e-4 298.66 1.33e-3 LDPC 1152 3/4 Mean-Sum 0.37 4.76e-4 286.14 1.29e-3 LDPC 1152 2/3 Sum-Product 0.50 8.89e-3 280.85 1.78e-3 Con voluti on 576 1/4 V iterbi-Soft 2.13 1.68e-1 27.08 7.88e-2 Con voluti on 576 1/3 V iterbi-Soft 1.45 6.01e-2 34.85 4.15e-2 Con voluti on 576 1/2 V iterbi-Soft 0.82 1.45e-2 46.68 1.76e-2 Novem ber 24, 2021 DRAFT 14 Fig. 1. Decision regi on of s and its radius for two dimensional codes. 0.5 1 1.5 2 2.5 0 0.01 0.02 0.03 0.04 0.05 0.06 l n p ln (x) 0.4 0.6 0.8 1 1.2 0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 l n p ln (x) 0.5 1 1.5 2 2.5 0 0.002 0.004 0.006 0.008 0.01 l n p ln (x) 1 1.5 2 2.5 0 1 2 3 4 5 x 10 −3 l n p ln (x) N iter =2 N iter =4 N iter =8 Max−LogMAP LogMAP Turbo 576 1/3 Turbo 576 1/2 Turbo 1152 1/3 Turbo 576 1/3 Turbo 4095 1/3 (a) (b) (c) (d) Turbo 576 1/3 Turbo 1152 1/2 Fig. 2. Normalize d SR-PDF of Turbo codes for diff erent (a) decoding algorit hm; (b) iteratio ns; (c) code rate; (d) codew ord length. “T urbo 1152 1/3” stands for a turbo code with code rate 1/3 and code word length 1152. DRAFT November 24, 2021 15 0.5 1 1.5 2 2.5 3 10 −4 10 −3 10 −2 10 −1 10 0 EbNo(dB) WER Dotted:Approximation Solid:Simulation +:LogMAP o:Max−LogMAP 8 iterations 4 iterations 2 iterations (a) −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 10 −4 10 −3 10 −2 10 −1 10 0 Eb/No[dB] WER LDPC 1152 3/4 MSA Convolutional 576 1/4 LDPC 1152 1/2 SPA Solid:simulation Dashed:square radius method LDPC 1152 3/4 SPA Turbo 1152 1/3 Turbo 1152 1/2 Turbo 576 1/3 (b) Fig. 3. Co mparison of the simulated WE R and the WER ev aluate d with (10) in A WGN channel. (a)The same Turbo codes with differe nt maximum iterat ions. N = 576 , R = 1 / 3 , decoding algorithm is L og-MAP and Max-Log-MAP; (b)Differe nt codes, includi ng Con volutiona l codes, Turbo codes and LDPC codes. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 I Av /I Ec I Ac /I Ev Variable node Check node E b /N 0 =0.98dB Fig. 4. EXIT chart for the LDPC code in T able I. The code word length is 11520 and degree distributi on is the same as the 1/2 rate LDPC code of T able I. I A v and I E c are the a priori and extrinsi c mutual information betwee n the transmitted signal and the soft information input to v ariabl e nodes and output from check nodes respecti vely . I E v and I A c are the ext rinsic and a priori m utual information between the transmitted signal and the soft information output from varia ble nodes and input to check nodes respect i vely . Novem ber 24, 2021 DRAFT 16 T ABLE III P A R A M E T E R S O F A V E R A G E N O R M A L I Z E D S R - P D F F O R R E P R E S E N TA T I V E E R RO R C O N T RO L C O D E S I N R A Y L E I G H FL AT FA D I N G C H A N N E L codes decodin g algorit hm mean varianc e a b T urbo 1152 1/3 Log-MAP 0.935 1.12e-2 77.70 1.20e-2 T urbo 1152 1/2 Log-MAP 0.442 2.89e-3 67.59 6.54e-3 T urbo 576 1/3 Log-MAP 0.958 2.30e-2 39.70 2.41e-2 LDPC 1152 1/2 Sum-Product 0.419 2.27e-3 77.18 5.43e-3 Fig. 5. Illustrat ion of P e | θ ( β l n ) and p l n ( l n ) .( β 1 > β 2 ). Fig. 6. Critica l S NR and ∆ ǫ . WER is the s imulati on result. Critical SNR is estimated by 1 /µ l n . ∆ ǫ is calculate d from (24). DRAFT November 24, 2021 17 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 10 −4 10 −3 10 −2 10 −1 10 0 Eb/No[dB] WER LDPC 1152 3/4 MSA Convolutional 576 1/4 LDPC 1152 1/2 SPA Solid:simulation Dashed:approximation LDPC 1152 3/4 SPA Turbo 1152 1/2 Turbo 1152 1/3 Fig. 7. Compari son of WER between simulation and approximati on using (32) for Conv olutional codes, Turbo codes and LD PC codes in A WGN channel. The decodin g algorithms are soft decision V iterbi decoding for Con volutio nal codes, Sum-Product algorit hm (SP A) and Min-Sum algorithm (MSA) respecti vely with 25 maximum iterations for LDPC codes and Log-MAP with 8 maximum iterat ions for Turbo codes. −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 10 −4 10 −3 10 −2 10 −1 10 0 Eb/No[dB] WER Convolutional 576 1/4 LDPC 1152 1/2 SPA LDPC 1152 3/4 SPA LDPC 1152 3/4 MSA Solid:simulation Dashed:approximation Turbo 1152 1/3 Turbo 1152 1/2 Fig. 8. Compari son of WER between simulation and approximati on using (34) for Conv olutional codes, Turbo codes and LD PC codes in A WGN channel. The decodin g algorithms are soft decision V iterbi decoding for Con volutio nal codes, Sum-Product algorit hm (SP A) and Min-Sum algorithm (MSA) respecti vely with 25 m aximum iterati ons for LDPC codes and Log-MAP with 8 m aximum iterations for T urbo code. Novem ber 24, 2021 DRAFT 18 0.2 0.3 0.4 0.5 0.6 0 0.01 0.02 0.03 0.04 0.05 0.06 l n p ln (x) LDPC 1152 1/2 0.2 0.4 0.6 0.8 0 0.005 0.01 0.015 0.02 0.025 Turbo 1152 1/2 l n p ln (x) 0 0.5 1 0 0.01 0.02 0.03 0.04 0.05 l n p ln (x) Turbo 576 1/2 0 0.5 1 1.5 2 0 0.01 0.02 0.03 0.04 l n p ln (x) Turbo 576 1/3 Fig. 9. Normalized av erage S R-PDF ¯ p l n ( x ) for se vera l representat ive error control codes in Rayleigh flat fadi ng channel. 1 2 3 4 5 6 7 10 −4 10 −3 10 −2 10 −1 10 0 Eb/No[dB] WER Turbo 576 1/3 Turbo 1152 1/2 LDPC 1152 1/2 SPA Turbo 1152 1/3 Fig. 10. Compa rison of WER between simulatio n and approximations of Turbo codes, L DPC codes in Raylei gh flat fadi ng channel. The decodin g algorit hm s are Log-MAP with 8 maximum iterati ons for Turbo codes and SP A with 25 maximum iterations for LDPC codes. Solid: simulatio n; Dashed: approximation using (36); Dash dot: approximation using (34); Dotted: approxima tion using (32 ) DRAFT November 24, 2021