Information-theoretic resolution of perceptual WSS watermarking of non i.i.d. Gaussian signals

Information-the oretic resolut ion of p erceptu al WSS w atermarking of non i.i.d. Gauss ian signals Ga ¨ etan L e Guelvouit, St ´ ephane Pateux and Christine Guil lemot IRISA/INRIA, Campus de Beaulieu, 35042 Rennes Cedex, FRANCE T el: +33 2 99 84 73 60; fax: +33 2 99 84 25 31 e-mail: Gaetan.Le Guelvouit@i risa.fr ABSTRA CT The theoretical foundations of data hiding hav e b een re- veal ed by form ulating the problem as message comm unica- tion ov er a noisy channel. W e revisit the problem in ligh t of a more general c haracterization of the watermark chan- nel and of w eighted distortion measures. Considering spread sp ectru m based information hiding, we release the usual as- sumption of an i.i.d. cov er signal. The game-theoretic reso- lution of the problem reveal s a generalized chara cterization of optimum attacks. The pap er then d erives closed-form ex- pressions for the different parameters exhib iting a practical em b eddin g and extraction technique. 1 INTR ODUCTION Information hiding refers to nearly in visible embedding of a message within a host signa l. This pa per fo cuses on the data hiding problem, assuming a blind and sym- metric system. In the spirit of a communication problem one seeks the max im um rate of reliable tr ansmission ov er any hiding and attack strategies. This ra te is called the hiding capa city and depe nds o n admissible distortion levels and on the watermark channel characterization. W ater ma rking is often regar ded as a form o f spread sp ectrum communication with v arious fo rms of chan- nel c haracteriz a tions. The perceptual sensitivity of the host signal is often taken into a ccount fo r cho os- ing e mbedding sites and strength [1, 7]. The a ttacks are often mo delled as the addition o f White Ga us sian noise (A W GN) [8, 4], or as linear filtering plus additive noise [9, 3]. The authors in [10] show that the optim um attack is obtained by Wiener filtering and that to b e maximally robust, the watermark should ha ve a p ow er sp ectrum matching the one of the or iginal s ignal. The problem of robust embedding and e xtraction based on spread sp ectrum is r evisited here in light o f a mor e general mo del of the cov er signal a nd of the wa- termark channel. Most of the a pproaches introduced so far consider that the cov er signa l can b e mo delled as an erg o dic wide sense sta tionary Gaussia n r a ndom pro cess. This a ssumption is ra rely s atisfied for rea l sig - nals. W e ass ume instead tha t it can be mo delled as the realization of a set of indepe ndent non iden tically dis- tributed Gaussian ra ndom v aria bles (referred to as non i.i.d. sig nals). The attack c hannel is consider e d to b e of the t yp e amplitude scaling and additive white Gaussia n noise (SA WGN) [2]. The g ame-theoretic r e s olution of the problem with weighted distor tion measur es leads to a characterizatio n of optimum attack do mains. By max- imizing the w atermar king channel sig nal to nois e ratio, we then derive a closed- form expr e ssion of the water- mark sp ectra l density co r resp onding to the b est defense. The perfor mance limits of the approa ch in terms of hid- ing capacity are then analyz e d. The appro ach can b e seen as a generaliza tion o f previo us w or k to the case of non i.i.d. Gaussian sourc e s , considering weigh ted dis- tortion measur es and a more general SA WGN attack channel, with the exhibition of closed-form expressio ns for a practical embedding and ex tr action scheme. 2 PR OBLEM ST A TEMENT Let b = { b 1 , b 2 , . . . , b n } with b i ∈ {− 1 , +1 } ∀ i ∈ { 1 , 2 , . . . , n } be the message to b e embedded in a host sig na l x . Many a pproaches intro duced so far as- sume that the signal x can be mo delled as an erg o dic zero-mea n wide sense statio na ry Ga ussian r andom pro- cess [4, 9]. This as s umption is r arely satisfied for real signals or for conten t adaptive watermarks. W e as- sume instead that the host s ignal x can be mo delled as the realiza tion of a set of no n stationar y Gaus- sian random v ariables X = { X 1 , X 2 , . . . , X m } where X i ∼ N (0 , σ X i ). The information is then used a s a key for indexing pseudo- random noise sequences whic h are additively combined with the signal. Let G be a n × m matrix comp osed o f n pseudo-r andom generated vectors G j ∈ {− 1 , +1 } m . The watermarked sig nal is obtained by y i = x i + w i = x i + α i n X j =1 G i,j b j , (1) where x i represents the i th site of the host signal a nd y i the corr esp onding w ater marked site. In or der to e x tract each embedded bit b i , a co rrelation pro duct b etw een the vector G j and y is computed. The term α i is a weigh t- ing fac tor a llowing to adjust the amplitude (or energy) of the mark. In the following we derive a c lo sed-form expression of this pa r ameter in the case o f SA W GN a t- 1 tacks, with weigh ted distortion mea sures and non i.i.d Gaussian cover sig na ls. The attack channel is often as - sumed to b e A W GN [6]. This model assumes that the distortion induced b y the attack is indep endent of the watermarked signa l, hence can hardly a pply to attacks such as filtering and co mpression. Mor e accura te mod- els a ssuming that the distortion depends o n the water- marked sig nal and based on linear filtering plus additive noise hav e b een co ns idered in [9, 3]. Here, we consider that the attack ed signal y ′ can b e expre s sed as y ′ i = γ i y i + δ i = γ i x i + γ i α i n X j =1 G i,j b j + δ i , (2) where γ i is an atten uation factor o n each watermarked site. This amo un ts to cons ider the attack channel a s a SA WGN channel (amplitude scaling by the factor γ i , and additive white Gaussia n nois e o f δ i ∼ N (0 , σ 2 δ i )). The distortion mea sure is defined as a weighted sum of the MSE on each sample of the host sig nal, in order to reflect the p erceptua l quality . The embedding distortion is therefore given by D xy = E " m X i =1 ϕ 2 i ( y i − x i ) 2 # = m X i =1 ϕ 2 i nα 2 i , (3) where ϕ i is a p erceptual factor . Similar ly , the exp ected attack distortion is given b y D xy ′ = m X i =1 ϕ 2 i  σ 2 X i (1 − γ i ) 2 + nγ 2 i α 2 i + σ 2 δ i  . (4) 3 MAP W A TERMARK ESTIMA TION The maximun a p osteriori (MAP) estimation of the bit b j is defined as b b j = ar g max b j  P ( B j = b j | Y ′ m = y ′ )  . (5) The a p os teriori pro bability P = P ( B j = b j | Y ′ m = y ′ ) can be rewritten (using Bay es law) as P = P ( Y ′ m = y ′ | B j = b j ) × P ( B j = b j ) P ( Y ′ m = y ′ ) . (6) Since the received vector y ′ is fixed, and tha t no a pri- ori knowledge o n the messa ge b is a ssumed, we have the a p osteriori pro bability P ∝ P ( Y ′ m = y ′ | B j = b j ), where ∝ deno tes an obvious renormalization. Assum- ing that the w ater ma rked sites are indepe nden t, it c a n be shown [5] that the quantit y can be expressed as a pro duct o f Gaussia n distributions of the for m P i ∼ N (0 , γ 2 i ( σ 2 X i + α 2 i ( n − 1)) + σ 2 δ i ), i.e. as P ∝ m Y i =1 1 √ 2 π q γ i σ X 2 i + α 2 i ( n − 1 )) + σ 2 δ i exp " − ( y ′ i − γ i α i b j G i,j ) 2 2  γ 2 i ( σ 2 X i + α 2 i ( n − 1)) + σ 2 δ i  # , (7) ∝ C 2 exp − Λ 2 , (8) where C is a constant and Λ = n X j =1  b j − b b j  2 σ 2 b j (9) with b b j = P m i =1 γ i α i y ′ i G i,j γ 2 i ( σ 2 X i + α 2 i ( n − 1))+ σ 2 δ i P m i =1 γ 2 i α 2 i γ 2 i ( σ 2 X i + α 2 i ( n − 1))+ σ 2 δ i , (10) σ 2 b j = m X i =1 γ 2 i α 2 i γ 2 i ( σ 2 X i + α 2 i ( n − 1)) + σ 2 δ i ! − 1 . (11) The term b b j represents the optimal es tima tor. F rom Eqn.(8) and (9), w atermar king c hannel can b e s een a s a gaussian channel. Es tima to r’s p erformance can b e mea- sured in terms o f the signal to no ise ratio E b / N 0 of the watermarking channel. This quantit y is defined as the ratio b etw een the ener g y of the embedded bit a nd the ov erall noise introduced by the cov er signal ( σ 2 X i ), b y the other em b edded bits ( α 2 i ( n − 1 )) and by the a ttack (i.e. σ 2 δ i ). It is then ex pr essed a s E b N 0 = E ( b 2 j ) σ 2 b j = 1 σ 2 b j = m X i =1 ρ i , (12) where ρ i = α 2 i γ 2 i γ 2 i ( σ 2 X i + α 2 i ( n − 1)) + σ 2 δ i . (13) 4 GAME-THEORETIC RE SOLUTION The optimization of the embedding and attack pa ram- eters can be for m ulated a s a g ame betw een an attack er and a hider. The a ttack searches the t wo vectors γ and σ δ minimizing the extractor per formance (i.e. E b / N 0 ) while maintaining the distortion b elow an acceptable level ( D xy ′ < D max xy ′ ). This problem can be solved by a Lagra ng ian optimization:  γ ⋆ , σ δ ⋆  = arg min γ ,σ δ  J λ = E b N 0 + λ  D xy ′ − D max xy ′   , where λ > 0 is a Lagrangian m ultiplier. F r o m Eqn.(4) and (12) it app ear s that J λ is an additive functional. The optimization can then b e made separa tely on ea ch J λ,i given by J λ,i ( γ i , σ δ i ) = ρ i + λϕ 2 i  σ 2 X i (1 − γ i ) 2 + nγ 2 i α 2 i + σ 2 δ i  by setting its deriv atives with respect to the attack pa- rameters γ i and σ δ i ∂ J λ,i ∂ γ i = 2 γ i α 2 i σ 2 δ i  γ 2 i  σ 2 X i + α 2 i ( n − 1)  + σ 2 δ i  2 +2 λϕ 2 i  nγ i α 2 i − σ 2 X i (1 − γ i )  , (14) and ∂ J λ,i ∂ σ δ i = − 2 γ 2 i α 2 i σ δ i  γ 2 i  σ 2 X i + α 2 i ( n − 1 )  + σ 2 δ i  2 + 2 λϕ 2 i σ δ i (15) to zero on the v a lidit y do main. The resolution o f the resulting set of eq uations leads to the following tw o ex- pressions for σ 2 δ i : 2 σ 2 δ i = − γ i  nγ i α 2 i − σ 2 X i (1 − γ i )  (16) and σ 2 δ i = γ i  α i √ λϕ i − γ i  σ 2 X i + α 2 i ( n − 1)   . (17) Equating Eqn.(16) and (17) leads to the optimum v alues γ ⋆ i = √ λϕ i σ 2 X i − α i √ λϕ i α 2 i (18) and σ 2 δ ⋆ i = γ ⋆ i ( γ w i − γ ⋆ i )  σ 2 X i + nα 2 i  , (19) where γ w i = σ 2 X i σ 2 X i + σ 2 w i , and σ 2 w i = nα 2 i . The term γ w i represents the resp onse of a Wiener filter. Since the attack parameter s σ 2 δ ⋆ i and γ ⋆ i m ust verify σ 2 δ ⋆ i ≥ 0 and γ ⋆ i ≥ 0, the solutions of Eqn.(18) and (19) ar e v alid only for µ − α i ≥ 0 (20) ( α i − µ )  σ 2 X i + nα 2 i  + µα 2 i ≥ 0, (21) where µ = √ λϕ i σ 2 X i . This set o f inequatio ns defines three domains D 1 , D 2 and D 3 shown in Fig. 1. The optimum attacks can then b e characterized in terms of the do mains of v alidity of the attack param- eters γ i and σ δ i . Let us fir st consider their limits of v alidity ( γ i = 0 and σ δ i = 0). If γ ⋆ i = 0 (the marked v alue y i is er ased), a minim um is obta ined for σ δ ⋆ i = 0. A grea ter v alue fo r the additive noise will increa se D xy ′ but will not decrease E b / N 0 . This attack is referred to as the Erase attack. I f σ δ i = 0, another minimum is given b y γ ⋆ i = γ w i . This a ttack is a Wiener filtering. The last attac k (defined for γ ⋆ i > 0 a nd for σ δ ⋆ i > 0) is a combination of filtering and a dditiv e Gaussian noise. This is ca lle d her e the In termedi ate a ttack. T able 1 gives the corres p onding expr e s sions of the cost function J λ,i ( γ i , σ δ i ) denoted J E , J W and J I for resp ectively the erase, Wiener and intermediate attacks. T o find the op- tim um attac k, o ne has to find on each doma in (defined in terms o f α i and σ X i ), the a ttack that will minimize J λ,i ( γ i , σ δ i ). F r om table 1 and co nstraints (20)-(2 1) the minim um v alues of J λ,i ( γ i , σ δ i ), in the domains of v a lid- it y of γ i and σ δ i , a re g iven b y J E and J W on D 1 and D 3 resp ectively (see [5] for details ). Simila rly , on D 2 , J I ≤ J E and J I ≤ J W . Thus, if the v alidity constraint γ ⋆ i ≥ 0 of the Intermediate attac k domain is satisfied, the optimum attack is g iven by the Intermediate attack (with para meter s given b y Eqn.(1 8) and (19)). Other- wise, the a ttack er sho uld use instead the Erase or the Wiener solution. Figur e 1: Domains define d by the validity c onstr aints γ ⋆ i ≥ 0 and σ δ ⋆ i ≥ 0 ( λ = 0 . 2 , χ = 0 . 00 2 , n = 1 bit). V alue of J λ,i ( γ i , σ δ i ) J E λϕ 2 i σ 2 X i J W α 2 i σ 2 X i + α 2 i ( n − 1) + λ nϕ 2 i α 2 i σ 2 X i σ 2 X i + nα 2 i J I 2 √ λϕ i σ 2 X i α i − 1 + λϕ 2 i σ 2 X i  1 − σ 2 X i α 2 i  T able 1: Cost function J λ,i ( γ i , σ δ i ) for the differ ent typ es of att ack (Er ase, Wiener, In t erme diate). Given the o ptim um attack, we then s earch the pa- rameters α i (strength o f the watermark) that maximize E b / N 0 , under constra int s of a max im um watermarked signal disto rtion ( D max xy ). This lea ds to a Lagrang ian approach: α ⋆ = ar g max α  J χ = J λ − χ  D xy − D max xy  , (22) where χ > 0 is a La grangia n m ultiplier. The cost function J χ being the additive functional J χ = P m i J λ,i ( γ i , σ δ i ) − χ  D xy − D max xy  , the o ptimiza tion can b e carrie d out separately o n each J χ,i ( α i ) = J λ,i ( γ i , σ δ i ) − χnϕ 2 i α 2 i . Let us c o nsider the three at- tack strateg ies. In the case of the Er ase a ttack, i.e. ( α i , σ X i ) ∈ D 1 , J χ,i ( α i ) = λϕ 2 i σ 2 X i − χnϕ 2 i α 2 i . The func- tion J χ,i ( α i ) is a decrea sing function a nd the minimum v alid v a lue of α i is g iven by α ⋆ i = √ λϕ i σ 2 X i . In D 2 , setting the deriv a tive ∂ J χ,i ∂ α i = − 2 √ λϕ i σ 2 X i α 2 i + 2 λϕ 2 i σ 4 X i α 3 i − 2 χnϕ 2 i α i (23) to zero leads to µ 2 − µα i − χnϕ 2 i α 4 i = 0 w her e µ = √ λϕ i σ 2 X i . The deriv a tive is nega tive for α i = µ and po sitive for α i = 0. The po lynomial being monotonous on the interv al [0 ; µ ], one ca n conclude that Eqn.(23) has a v alid solution on [0; µ ]. If the deriv ative is negative on D 2 , the s o lution a dopted is α i such that µα 2 i = ( µ − α i )( σ 2 X i + nα 2 i ). Le t n α 2 i = σ 2 W i , nλ = λ ′ and nχ = χ ′ . Let us as s ume that σ 2 W i is very close to α 2 i ( n − 1). In the Wiener case (( γ i , σ δ i ) ∈ D 3 ), the cost function J χ,i ( α i ) expressed in terms of σ 2 W i = nα 2 i is given by J χ,i ( σ 2 W i ) =  1 + λ ′ ϕ 2 i σ 2 X i  σ 2 W i σ 2 W i + σ 2 X i − χ ′ ϕ 2 i σ 2 W i . (24) Setting the deriv ative of J χ,i ( σ 2 W i ) with resp ect to σ 2 W i ∂ J χ,i ∂ σ 2 W i =  1 + λ ′ ϕ 2 i σ 2 X i  σ 2 X i  σ 2 W i + σ 2 X i  2 − χ ′ ϕ 2 i (25) to zero, given that σ 2 W i = nα 2 i , leads to α ⋆ i = v u u t σ X i q 1 + λ ′ ϕ 2 i σ 2 X i − √ χ ′ ϕ i σ 2 X i √ χ ′ nϕ i . (26) This provides a closed-for m of the optim um embedding parameter α i in terms of the host signal power s p ectr um ( σ 2 X i ) and for an SA W GN attack. The Wiener filtering can restor e the signal, hence may lea d to D xy ′ < D xy . This can b e avoided b y filtering the watermarked sig- nal (after e m b edding). The distor tion meas ur e is then 3 Figur e 2: Optimum values of α i for λ = 0 . 0 02 , χ = 0 . 0028 and n = 100 bits. Figur e 3: Performanc es of the pr op ose d scheme in pr es- enc e of a c ompr ession attack JPEG. D xy = P m i =1 ϕ 2 i σ 2 X i σ 2 W i σ 2 X i + σ 2 W i , where σ 2 W i = nα 2 i . The r eso- lution of the problem leads to new parameters (see [5] for details): α ⋆ i ≃ √ λϕ i σ 2 X i . Fig . 2 illustrates the v ar i- ations of the parameter α i in terms of σ 2 X i for b o th ap- proaches i.e., witho ut (bold curve) and with a Wiener po st-filtering (light curve) o f the watermarked signa l (with ϕ i = (1 + σ X i ) − 1 / 2 ). Unlik e [1], it can be ob- served that for high v alues of σ 2 X i , no w aterma rk can be robustly embedded on the cor resp onding sites. 5 RESUL TS The appr o ach has b een tested on ima g es a g ainst tech- niques using α i = constan t , and α i = c | x i | [6], consider- ing embedding in the wa velet transform do main. Fig . 3 depicts the respec tive E b / N 0 per formances in terms of the attack distor tion. A message of 156 bits is embed- ded (i.e. n = 156) in the 51 2 × 5 12 gr ay scale L ena image (i.e. m = 2 62 14 4). The La grag ia n multipliers λ and χ are se t so that D xy = D max xy and D xy ′ = D max xy ′ . The embedding parameters hav e b een tuned in order to get the same per ceptual distortion D xy /m = 1 (with ϕ i = (1 + σ X i ) − 1 / 2 ) with the different techniques. The watermarked image has b een attack ed with a lossy com- pression JPEG, from 95% to 5 % quality . The tests, us- ing the Stirmark b enchmark have shown that the tech- nique is robust to all the no n-geometric attacks. 6 CONCLUSION This pap er provides an information-theo retic a nalysis of informatio n hiding in non i.i.d sig nals with per cep- tual distortion metrics. Note that previous work, when considering p er ceptual watermaking was often led by int uition. Her e we hav e der ived closed-form expre s sions of the different extraction and embedding par a meters, revealing an efficient and pra ctical informa tio n hiding system. References [1] I. J. Cox, J. Kilian, T. Leightom, and T. Sh amoon. Secure spread sp ectrum w atermarking for multimedia. IEEE T r ans. I mage Pr o c. , 6(12):1673–1687, Dec. 1997. [2] J. J. Eggers, R. B¨ auml, and B. Girod. Digital water- marking facing attacks by amplitude scaling and addi- tive white noise. In 4th Int. ITG Conf . on Sour c e and Channel Co ding , Jan. 2002. [3] P . Moulin and A. Iv anovic. The waterma rk selection game. In Pr o c. Conf. on Info. Scienc es and Systems , Mar. 2001. [4] P . Moulin and J. A . O ’Sulliv an. I nformation-theoretic analysis of watermarking. In Pr o c. Int. Conf. on A c- c oustic, Sp e e ch and Signal Pr o c essing. , Istanbul, T urkey , Jun. 2000. [5] S. P ateux, G. Le Guelvouit, and C. Guillemot. Information-theoretic analysis of WSS watermarking of non i.i.d. Gaussian signals. submitte d to IEEE T r ans. Signal Pr o c. , Dec. 2001. [6] A. Piv a, M. Barni, F. Bartolini, and V. Capp ellini. Threshold selection for correlation-based watermark de- tection. In Pr o c. COST 254 Worksho p on Intel l igent Communic ations , pages 66–72, L’Aquila, Italy , Jun. 1998. [7] C. I. P od ilc huk and W. Zeng. Image-adaptive wa ter- marking using visual mo dels. IEEE Journal on Sp e cial Ar e as in Communi c ations , 16(4):525–539, Ma y 1998. [8] S. Serv etto, C. I. Podilch uk, and K. Ramchandran. Ca- pacit y issues in digital image watermarking. In Pr o c. Int. Conf. on Image Pr o c essing , volume 1, pages 445– 449, Chicago, I L, Oct. 1998. [9] J. K. Su, J. J. Eggers, and B. Girod . A nalysis of digital w atermarks sub jected to optimum linear filtering and additive n oise. IEEE T r ans. Signal Pr o c.: Sp e ci al Issue on Information The or etic I ssues in Digital Watermark- ing , 81(6), Jun . 2001. [10] J. K. Su and B. Giro d. Po wer-spectru m condition for energy-efficient w atermarking. submitte d to IEEE T r ans. M ultime dia , Jul. 1999. 4