Blind Fingerprinting

1 Blind Fingerprinting Y ing W ang and Pierre Mou lin Abstract W e study blin d fin gerprin ting, whe re th e ho st sequ ence into which fing erprints are embedde d is partially or completely unknown to the de coder . T his prob lem r elates to a multiuser version of the Gel’fand-Pinsker prob lem. The nu mber of colluders and the collusion channel are unknown, and the colluders and the fing erprint embedder are subject to distortio n constraints. W e pro pose a condition ally constant-co mposition rando m binning scheme and a universal decodin g rule and derive the corresp onding false-po siti ve and false-negative error exponents. The enc oder is a stacked binning scheme and makes use of an auxiliary random sequen ce. The decoder is a ma ximum doubly- penalized mutual information d ecoder , where the significance of ea ch candidate coalition is assessed relati ve to a threshold th at trades off false-positiv e and false-negati ve er ror exponents. The penalty is p ropor tional to coalition size and is a function of the con ditional typ e of h ost sequence. Positi ve exponents are o btained at all r ates below a certain value, which is ther efore a lower boun d on public fingerprinting capacity . W e conjectur e that this v alue is the public fingerprintin g capacity . A simpler threshold decoder is also given, wh ich has similar un iv ersality proper ties but a lso lower achievable rates. An u pper bound on pu blic fin gerprin ting capacity is also derived. Index T erms. Fing erprinting, traitor tracing, watermarking, data hiding, randomized codes, uni versal codes, method o f types, ma ximum mutual information dec oder , minimum equiv ocation de coder , cha nnel coding with side information, random b inning, c apacity , e rror expon ents, multiple acces s chan nels, mode l order se lection. Y . W ang is with Qualcomm, Bedminster , NJ. P . Moulin is with the ECE Department, the Coordinated Science Lab- oratory , and the Beckman Institute at the Univ ersity of Illi nois at Urbana-Champaign , U rbana, IL 61801, USA. Email: moulin@ifp.ui uc.edu . This work was suppo rted by NSF under grants CCR 03-25924, C CF 06-35137 and CCF 07-29061. Part of this work was presented at ISIT’06 in Seattle, W A. Nov ember 5, 2018 DRAFT 2 I . I N T RO D U C T I O N Content fingerprinting finds applications to do cument protec tion for multimedia distribution, broad- casting, and traitor tracing [1]–[4]. A coverte xt—image, video , audio, o r text—is to be distributed to many users. A fingerprint, a mark u nique to each us er , is embed ded into eac h copy of the covertext. In a c ollusion attack, several u sers may combine their copies in an attempt to “remove” their fingerprints and to forge a pirated copy . The distortion betwee n the pirated copy and the colluding c opies is bou nded by a certain tolerance level. T o trace the forgery bac k to the co alition memb ers, we need fingerprinting codes that can reliably iden tify the fingerprints of those members. Es sentially , from a c ommunication viewpoint, the fingerprinting prob lem is a multiuser version of the watermarking prob lem [5]–[10]. For watermarking, the a ttack is b y o ne user a nd is based on one single copy , where as for finge rprinting, the attack is mode led a s a multiple-access chan nel (MA C). The covertext plays the role of side information to the encode r and pos sibly to the d ecode r . Depending on the av a ilability of the original covertext to the de coder , there are two basic versions of the problem: priv a te and pu blic. In the priva te finger printing setup, the covertext is av a ilable to both the encod er and d ecoder . In the public fingerpr inting setup, the covertext is available to the encoder but not to the decoder , and thus decoding p erformance is g enerally worse. Howev er public fingerprinting presents a n impo rtant a dvantage over priv ate fin gerprinting, in that it doe s no t require the vast s torage and co mputational resources that are need ed for media registration in a large databas e. For example, a D VD player could detec t fingerprints from a movie d isc and refuse to play it if fingerprints other tha n the owner’ s are prese nt. Or W e b crawling programs can b e use d to automa tically s earch for u nauthorized content on the Internet o r other public networks [3]. The sc enario c onsidered in this pap er is one where a d egraded version S d of ea ch h ost symbo l S is av a ilable to the decode r . Pri vate and public fi ngerprinting are ob tained as spe cial c ases with S d = S and S d = ∅ , res pectiv ely . W e refer to this s cenario as e ither blind or s emiprivate fi ngerprinting . Th e moti vation is a nalogous to s emipri vate watermarking [11], where some information about the host signal is provided to the rec eiv er in orde r to improve decoding pe rformance. This may be necess ary to gu arantee an acc eptable performance level when the number of c olluders is lar g e. The ca pacity an d reliability limits of private finger printing have b een studied in [7]–[10]. The decod er of [10] is a variation of Liu an d Hughes’ minimum equ i vocation decoder [12], ac counting for the p resence of side information and for the fact that the n umber of chann el inputs is un known. T wo basic types of decode rs are of interest: detect-all and detect-one. The detect-all decod er aims to catch all members of Nov ember 5, 2018 DRAFT 3 the coalition and an error o ccurs if some colluder esc apes detection. The de tect-one decoder is content with c atching at least o ne o f the culprits a nd an error oc curs o nly whe n no ne of the co lluders is identified. A third type of e rror (arguably the most damaging one) is a false positive , by which the decode r accus es an innoce nt user . In the same way a s finge rprinting is related to the MA C problem, blind fingerprinting is related to a multiuser extension of the Ge l’fand-Pinsker problem. The capacity region for the latter problem is unknown. An inner region, a chiev able using random binning, was g i ven in [13]. This paper deri ves rand om-coding expo nents an d an uppe r b ound on de tect-all c apacity for semipriv a te fingerprinting. Neither the enco der nor the decod er know the number o f collude rs. The collusion c hannel has arbitrary memo ry but is su bject to a distortion con straint between the pirated co py an d the colluding copies. Our fingerprinting s cheme uses r andom binning because, unlik e in the pri vate setup, t he a vail ability of side i nformation to the encoder and decoder is as ymmetric. T o optimize the error exponents, we propo se an extension of the stacked-binning s cheme that was developed for single-use r c hanne l cod ing with side information [11]. Here the c odeboo k cons ists o f a s tack of variable-size codeword-arrays indexed by the conditional type of the covertext sequenc e. The decod er is a minimum doubly-pe nalized e quivocation (M2PE) dec oder o r equiv alently , a maximum doub ly-penalized mu tual information (M2PMI) decod er . The propose d fingerprinting system is universal in that it c an cope with unknown collusion channe ls and un known n umber of collude rs, as in the priv ate fing erprinting setup of [10 ]. A tunab le pa rameter ∆ trades o f f false-pos iti ve an d false-negativ e error exponents. The deriv ation o f the se expone nts c ombines techniques from [10] and [11]. A preliminary version of our work, as suming a fixed numb er of c olluders, was giv en in [14], [15]. A. Or ganization of This P aper A mathematical statement of our g eneric finge rprinting p roblem is gi ven in Se c. II, together with the basic definitions of error p robabilities, capacity , error exponents, an d fair coalitions. Sec. III presen ts our random coding s cheme. Sec. IV presen ts a s imple but s uboptimal decode r that comp ares empirical mutual information scores be tween rece i ved data an d individual fingerprints, a nd o utputs a guilty d ecision whenever the sco re excee ds a certain tun able threshold. S ec. V p resents a joint decod er that ass igns a penalized empirical mu tual information score to ca ndidate co alitions an d s elects the coalition with the highest score. Sec. V I e stablishes an upp er bo und on blind finge rprinting c apacity un der the de tect-all criterion. Finally , c onclusions are giv en in Sec. VII. The proofs of the theo rems a re giv en in a ppendice s. Nov ember 5, 2018 DRAFT 4 B. Notation W e use upp ercase letters for rand om variables, lowercase letters for their individual values, ca lligraphic letters for finite alphabets, and boldface letters for seq uences . W e denote by M ⋆ the set of seque nces of a rbitrary length (including 0) who se elements a re in M . Th e prob ability ma ss fun ction (p.m.f.) of a random v ariable X ∈ X is d enoted b y p X = { p X ( x ) , x ∈ X } . The entropy of a random variable X is denoted by H ( X ) , and the mutual information between two random variables X and Y is de noted by I ( X ; Y ) = H ( X ) − H ( X | Y ) . Shou ld the de penden cy on the unde rlying p.m.f.s be exp licit, we write the p.m.f.s as subsc ripts, e.g., H p X ( X ) a nd I p X ,p Y | X ( X ; Y ) . Th e Kullback-Leibler d i ver gence between two p.m.f.s p and q is de noted by D ( p || q ) ; the conditional Kullback-Leibler div er gence of p Y | X and q Y | X giv en p X is de noted by D ( p Y | X || q Y | X | p X ) = D ( p Y | X p X || q Y | X p X ) . All logarithms a re in bas e 2 un less specified othe rwise. Denote by p x the type, or empirical p.m.f. induced by a seque nce x ∈ X N . The type class T x is the set of a ll sequen ces of type p x . Like wise, we de note by p xy the joint type of a p air o f seque nces ( x , y ) ∈ X N × Y N and by T xy the type clas s a ssociated with p xy . The conditional typ e p y | x of a p air of se quence s ( x , y ) is define d by p xy ( x, y ) /p x ( x ) for all x ∈ X such that p x ( x ) > 0 . The conditional type class T y | x giv en x , is the set of all s equenc es ˜ y such that ( x , ˜ y ) ∈ T xy . W e denote by H ( x ) the empirical entropy of the p.m.f. p x , by H ( y | x ) the e mpirical conditional entropy , and by I ( x ; y ) the empirical mutual information for the joint p.m.f. p xy . W e use the calligraphic fonts P X and P [ N ] X to repres ent the set o f all p.m.f.s and all emp irical p.m. f. ’ s, respectively , on the alphabe t X . Likewise, P Y | X and P [ N ] Y | X denote the s et of a ll c onditional p.m.f.s and all empirical conditional p.m.f. ’ s on the alphab et Y . A s pecial symbol W K will be used to deno te the feasible set of collusion cha nnels p Y | X 1 , ··· ,X K that can be selected by a size- K coalition. Mathematical expectation is de noted by the symbol E . The shorthands a N . = b N and a N  ≤ b N denote asymptotic relations in the exponential s cale, res pectively lim N →∞ 1 N log a N b N = 0 and lim su p N →∞ 1 N log a N b N ≤ 0 . W e defi ne | t | + , max( t, 0) an d exp 2 ( t ) , 2 t . The indicator function of a set A is denoted by 1 { x ∈A} . Finally , we ad opt the conv ention that the minimum of a function over a n e mpty set is + ∞ a nd the maximum of a function over a n empty set is 0. I I . S TA T E M E N T O F T H E P RO B L E M A. Overview Our model for blind finge rprinting is diagrammed in Fig. 1. Let S , X , an d Y be three finite alpha bets. The covertext s equenc e S = ( S 1 , · · · , S N ) ∈ S N consists of N independ ent an d identically distributed Nov ember 5, 2018 DRAFT 5 class W K D Host sequence Secret key S V X 2 NR 2 1 Fingerprints fingerprints Decoded Pirated copy Coalition copies Y g Decoder Collusion channel 1 mK m1 Y|X , ..., X P Fingerprinted . . K 2 NR 2 N N mK m2 m1 1 X K X X Choose . . X X Encoder f . . h(.) S d Fig. 1. Model for semipri v ate (blind) fingerprinting game, where S d is a degrad ed version of the cov ertext S . P riv ate and public fingerprinting ari se as special cases wit h S d = S and S d = ∅ , r especti vely . (i.i.d.) sa mples drawn from a p.m.f. p S ( s ) , s ∈ S . A ran dom variable V taking values in an a lphabet V N is shared b etween e ncoder a nd d ecoder , and not pu blicly revealed. The rand om variable V is indep endent o f S and p lays the role of a cryptograph ic key . The re are 2 N R users, e ach o f w hich receiv es a fingerprinted copy: X m = f N ( S , V , m ) , 1 ≤ m ≤ 2 N R , (2.1) where f N : S N × V N × { 1 , · · · , 2 N R } → X N is the enc oding func tion, and m is the index of the user . The e ncoder binds each fing erprinted copy x m to the covertext s v ia a distortion co nstraint. Let d : S × X → R + be the distortion meas ure and d N ( s , x ) = 1 N P N i =1 d ( s i , x i ) the extension o f this measure to len gth- N sequenc es. The co de f N is sub ject to the d istortion cons traint d N ( s , x m ) ≤ D 1 1 ≤ m ≤ 2 N R . (2.2) Let K , { m 1 , m 2 · · · , m K } be a coalition o f K users, called colluders. No constraints are imposed on the formation of c oalitions. The colluders combine their copies X K , { X m , m ∈ K } to produc e a p irated co py Y ∈ Y N . W ithout los s of generality , we assume that Y is generated stocha stically as the output of a c ollusion chann el p Y | X K . Fidelity cons traints are impo sed on p Y | X K to ensure that Y is “close” to the fingerprinted copies X m , m ∈ K . These constraints can take the form of distortion constraints, a nalogous ly to (2.2). They are formulated below and result in the defin ition of a fea sible class W K of attacks . The dec oder knows neither K n or p Y | X K selected by the K colluders a nd has acce ss to the pirated co py Y , the se cret key V , as well as to S d , a degraded version of the host S . T o simplify the exposition, the Nov ember 5, 2018 DRAFT 6 degradation arises via a deterministic symbo lwise mapping h : S → S d . The s equen ce s d = h ( s ) co uld represent a coa rse version of s , o r s ome othe r features of s . T wo special c ases are private fingerprinting where S d = S , and public fi ngerprinting whe re S d = ∅ . The deco der p roduces an e stimate ˆ K = g N ( Y , S d , V ) (2.3) of the coalition. A possible decision is the emp ty se t, ˆ K = ∅ , which is the reaso nable choice whe n an accus ation would be unreliable. T o summarize, we have Definition 2 .1: A randomize d rate- R length- N fing erprinting code ( f N , g N ) with e mbedding distortion D 1 is a pair of encod er mapp ing f N : S N × V N × { 1 , 2 , · · · , 2 N R } → X N and decoder mapping g N : Y N × ( S d ) N × V N → { 1 , 2 , · · · , 2 N R } ⋆ . The randomization is v ia the secre t key V and can take the form of permutations of the symbol positions { 1 , 2 , · · · , N } , p ermutations of the 2 N R fingerprint a ssignments, and an a uxiliary time-sharing sequen ce, as in [6]— [10], [16]. W e now state the attack mode ls an d define the error probab ilities, capac ities, an d error expone nts. B. Collusion Ch annels The c onditional type p y | x K is a random variable whos e conditional dis trib u tion g i ven x K depend s o n the co llusion c hannel p Y | X K . Our fidelity cons traint o n the c oalition is of the gen eral form P r [ p y | x K ∈ W K ] = 1 , (2.4) where W K is a conv ex subset of P Y | X K . Tha t is, the empirical conditional p.m.f. o f the pirated copy giv en the marked c opies is restricted. Exa mples o f W K are giv en in [10], including ha rd distortion constraints on the coalition: W K = ( p Y | X K : X x K ,y p X K ( x K ) p Y | X K ( y | x K ) E φ d 2 ( φ ( x K ) , y ) ≤ D 2 ) (2.5) where φ : X K → S is a (poss ible randomize d) permutation-in variant estimator ˆ S = φ ( X K ) of each host signal sample bas ed on the corresp onding marked samples; d 2 : S → Y is the coalition’ s distortion function; p X K is a reference p.m.f.; a nd D 2 is the maximum allowed distortion. Ano ther possible ch oice for W K is obtaine d u sing the Boneh-Shaw con straint [1], [10]. Fair Coalitions. Denote b y π a p ermutation of the elements of K . The set of fair , feasible c ollusion channe ls is the subs et of W K consisting of pe rmutation-in variant channe ls: W f air K =  p Y | X K ∈ W K : p Y | X π ( K ) = p Y | X K , ∀ π  . (2.6) Nov ember 5, 2018 DRAFT 7 The co llusion chan nel p Y | X K is sa id to b e fair if P r [ p y | x K ∈ W f air K ] = 1 . For any fair collusion ch annel, the co nditional type p y | x K is in variant to permutations of the colluders. Strongly exchange able c ollusion channels [7]. Now de note by π a permutation o f the samples of a length- N sequenc e. For s trongly exchang eable channe ls, p Y | X K ( π y | π x K ) is inde penden t of π , for every ( x K , y ) . The channel is defined by a probability assignmen t P r [ T y | x K ] on the conditional type classe s. The distribution of Y conditioned on Y ∈ T y | x K is uniform: p Y | X K ( ˜ y | x K ) = P r [ T y | x K ] | T y | x K | , ∀ ˜ y ∈ T y | x K . (2.7) C. Err o r P r oba bilities Let K be the actual coalition and ˆ K = g N ( Y , S d , V ) the decoder’ s ou tput. The three error probabilities of interest in this pa per are the probability of false p ositi ves (one or more innocent users are accused ), P F P ( f N , g N , p Y | X K ) = P r [ ˆ K \ K 6 = ∅ ] , the prob ability o f failing to catch a sing le colluder , P one e ( f N , g N , p Y | X K ) = P r [ ˆ K ∩ K = ∅ ] , and the prob ability of failing to catch the full coa lition: P all e ( f N , g N , p Y | X K ) = P r [ K 6⊆ ˆ K ] . These three probab ilities are obtained by averaging over S , V , and the output of the collusion channe l p Y | X K . In each case the worst-case p robability is d enoted by P e ( f N , g N , W K ) = max p Y | X K P e ( f N , g N , p Y | X K ) (2.8) where P e denotes either P F P , P one e or P all e , an d the max imum is over all fea sible c ollusion c hanne ls, i.e., such that (2.4) h olds. D. Capacity and Random-Coding Expo nents Definition 2 .2: A rate R is achiev able f or embedding distortion D 1 , collusion class W K , and detect-one criterion if there exists a s equenc e o f ( N , ⌈ 2 N R ⌉ ) randomized codes ( f N , g N ) with maximum embedd ing distortion D 1 , suc h tha t both P one e,N ( f N , g N , W K ) and P F P, N ( f N , g N , W K ) vanish a s N → ∞ . Definition 2 .3: A rate R is achievable f or embedding d istortion D 1 , collusion c lass W K , an d de tect-all criterion if there exists a s equenc e o f ( N , ⌈ 2 N R ⌉ ) randomized codes ( f N , g N ) with maximum embedd ing distortion D 1 , suc h tha t both P all e,N ( f N , g N , W K ) a nd P F P, N ( f N , g N , W K ) vanish a s N → ∞ . Nov ember 5, 2018 DRAFT 8 Definition 2 .4: Fingerprinting ca pacities C one ( D 1 , W K ) and C all ( D 1 , W K ) are the suprema of all achiev able rates with respe ct to the detec t-one a nd detec t-all criteria, respec ti vely . For random co des the error exponents c orresponding to (2.8) are defined as E { one,all, F P } ( R, D 1 , W K ) = lim in f N →∞  − 1 N log P { one,all, F P } e ( f N , g N , W K )  . (2.9) W e have C all ( D 1 , W K ) ≤ C one ( D 1 , W K ) and E all ( R, D 1 , W K ) ≤ E one ( R, D 1 , W K ) because an error ev ent for the de tect-one problem is also an e rror event for the detec t-all problem. I I I . O V E R V I E W O F R A N D O M - C O D I N G S C H E M E A brief overvie w of our sch eme is given in this se ction. T he decode rs will be sp ecified la ter . The scheme is designed to achieve a false-positiv e error exponent equa l to ∆ a nd a ssumes a nominal value K nom for coalition size. T wo arbitrarily large integers L w and L u are selected, defining alphabets W = { 1 , 2 , · · · , L w } and U = { 1 , 2 , · · · , L u } , respectively . The parameters ∆ , K nom , L w , L u are used to identify a certain o ptimal type class T ∗ w and conditional type c lasses T ∗ U | S d W ( s d , w ) , T ∗ U | S W ( s , w ) and T ∗ X | U S W ( u , s , w ) for every p ossible ( u , s , w ) . Optimality is defined relativ e to either the thresh olding decode r of Sec. IV o r the joint decod er of Sec. V. The sec ret key V consists of a random seque nce W ∈ T w ∗ and the collection (3.1) o f random c odeboo ks indexed by s d , w , λ . A. Codeboo k A rando m constant-comp osition c ode C ( s d , w , λ ) = { u ( l, m , λ ) , 1 ≤ l ≤ 2 N ρ ( λ ) , 1 ≤ m ≤ 2 N R } (3.1) is genera ted for each pair o f seque nces ( s d , w ) ∈ ( S d ) N × T ∗ w and conditional type λ ∈ P [ N ] S | S d W by drawing 2 N [ R + ρ ( λ )] random seque nces indepe ndently a nd uniformly from a n optimized co nditional type class T ∗ U | S d W ( s d , w ) , an d a rranging the m into an array with 2 N R columns and 2 N ρ ( λ ) rows. Similarly to [11] (see F ig. 2 there in), we refer to ρ ( λ ) as the de pth parameter of the array . B. Encoding Scheme Prior to e ncoding, a seque nce W ∈ W N is drawn independe ntly of S and uniformly from T ∗ w , and shared with the recei ver . Given ( S , W ) , the encode r determines the conditional type λ = p s | s d w and performs the following two steps for each use r 1 ≤ m ≤ 2 N R . Nov ember 5, 2018 DRAFT 9 1) Find l su ch that u ( l , m, λ ) ∈ C ( s d , w , λ ) T T ∗ U | S W ( s , w ) . If more tha n one s uch l exists, p ick one of them randomly (with u niform distribution). Let u = u ( l , m, λ ) . If no such l can be found, generate u uniformly from the conditional type class T ∗ U | S W ( s , w ) . 2) Generate X m uniformly distrib uted over the conditional type class T ∗ X | U S W ( u , s , w ) , a nd assign this marked se quence to us er m . C. W orst Collusion Chann el The fingerprinting co des used in this paper are rand omly-modulated (RM) codes [10, Def. 2.2]. For such c odes we have the follo wing proposition, which is a straightforward variation of [10 , Prop. 2.1] with S d in place of S a t the deco der . Pr op osition 3. 1: For any RM c ode ( f N , g N ) , the maximum of the error probability criteria (2.8) over all feasible p Y | X K is ac hiev ed by a strongly excha ngeab le co llusion c hannel, as d efined in (2.7 ). T o de ri ve e rror exponents for suc h chann els, it suffices to u se the following upper bo und: p Y | X K ( ˜ y | x K ) = P r [ T y | x K ] | T y | x K | ≤ 1 | T y | x K | 1 { p y | x K ∈ W K } , ∀ ˜ y ∈ T y | x K (3.2) which holds uniformly over all fea sible probab ility assignmen ts to co nditional type clas ses T y | x K . D. Encoding and De coding Error s The array depth parame ter ρ ( λ ) takes the form ρ ( λ ) = I ( u ; s | s d , w ) + ǫ where u is any element of T ∗ U | S W ( s , w ) , and ǫ > 0 is an arbitraril y s mall number . The ana lysis sh ows that given any ( s , w ) , the probability of encoding errors v anishes doubly expo nentially . The analysis also shows that the decod ing error proba bility is d ominated by a single joint type c lass T yusw . Den ote by ( y , u , s , w ) an arbitrary rep resentative of tha t class . The no rmalized log arithm of the size of the array is gi ven by R + ρ ( λ ) = I ( u ; y | s d , w ) − ∆ , and the prob ability of false positi ves vanishes as 2 − N ∆ . Nov ember 5, 2018 DRAFT 10 I V . T H R E S H O L D D E C O D E R A. Decoding The de coder has acce ss to ( y , s d , w ) but does not k now the c onditional type λ = p s | s d w realized at the encode r . T he dec oder ev a luates the use rs one at a time a nd makes an innocen t/guilty decision on each user independ ently of the other users . Sp ecifically , the receiv er o utputs an estimated co alition ˆ K if and only if ˆ K satisfies the following condition: ∀ m ∈ ˆ K : max λ ∈ P [ N ] S | S d W max 1 ≤ l ≤ 2 N ρ ( λ ) I ( u ( l , m, λ ); y | s d w ) − ρ ( λ ) > R + ∆ . (4.1) If no s uch ˆ K is foun d, the receiver o utputs ˆ K = ∅ . This dec oder outputs a ll us er indices whose empirical mutual information score, penalized by ρ ( λ ) , exceed s the thresh old R + ∆ . Observe that the ma ximizing λ in (4.1) may depend on m . W ith high probab ility , this event implies a decoding error . Improvements can only be obtained us ing a mo re complex joint dec oder , as in Sec. V. B. Err o r E xponen ts Define the follo wing set o f cond itional p .m.f. ’ s for ( X U ) K , ( X K , U K ) g i ven ( S, W ) : M ( p X U | S W ) = { p ( X U ) K | S W : p X m U m | S W = p X U | S W , m ∈ K} , i.e., the co nditional marginal p.m.f. p X U | S W is the same for e ach ( X m , U m ) , ∀ m ∈ K . Also defi ne the sets P X U | S W ( p S W , L w , L u , D 1 ) =  p X U | S W : E [ d ( S, X )] ≤ D 1  , P ( X U ) K W | S ( p S , L w , L u , D 1 ) = ( p ( X U ) K W | S = p W Y k ∈K p X k U k | S W : p X 1 U 1 | S W = · · · = p X K U K | S W , and E [ d ( S, X 1 )] ≤ D 1  (4.2) where in (4. 2) the random variables ( X k , U k ) , k ∈ K , are cond itionally i.i.d. giv en ( S, W ) . Define for each m ∈ K the set of con ditional p.m.f. ’ s P Y ( X U ) K | S W ( p W , ˜ p S | W , p X U | S W , W K , R, L w , L u , m ) ,  ˜ p Y ( X U ) K | S W : ˜ p ( X U ) K | S W ∈ M ( p X U | S W ) , ˜ p Y | X K ∈ W K , I p W ˜ p S | W ˜ p Y ( X U ) K | S W ( U m ; Y | S d W ) − I p W ˜ p S | W p X U | S W ( U ; S | S d W ) ≤ R o (4.3) Nov ember 5, 2018 DRAFT 11 and the ps eudo sphe r e packing exponent ˜ E psp,m ( R, p W , ˜ p S | W , p X U | S W , W K ) = min ˜ p Y ( X U ) K | S W ∈ P Y ( X U ) K | S W ( p W , ˜ p S | W ,p X U | S W , W K ,R,L w ,L u ,m ) D ( ˜ p Y ( X U ) K | S W ˜ p S | W k ˜ p Y | X K p K X U | S W p S | p W ) . (4.4) T aking the maximum and minimum of ˜ E psp,m above over m ∈ K , we respec ti vely d efine ˜ E psp ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W K ) = max m ∈K ˜ E psp,m ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W K ) , (4.5) ˜ E psp ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W K ) = min m ∈K ˜ E psp,m ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W K ) . (4.6) For a fair c oalition ( W K = W f air K ), ˜ E psp,m is indepen dent of m ∈ K , a nd the two expressions above coincide. Define E psp ( R, L w , L u , D 1 , W K ) = max p W ∈ P W min ˜ p S | W ∈ P S | W max p X U | S W ∈ P X U | S W ( p W ˜ p S | W ,L w ,L u ,D 1 ) ˜ E psp, 1 ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W f air K ) . (4.7) Denote by p ∗ W and p ∗ X U | S W the maximizers in (4.7), the latter to be viewed as a function of ˜ p S | W . Both p ∗ W and p ∗ X U | S W implicitly depe nd on R an d W f air K . Finally , de fine E psp ( R, L w , L u , D 1 , W K ) = min ˜ p S | W ∈ P S | W ˜ E psp ( R, L w , L u , p ∗ W , ˜ p S | W , p ∗ X U | S W , W K ) (4.8) E psp ( R, L w , L u , D 1 , W K ) = min ˜ p S | W ∈ P S | W ˜ E psp ( R, L w , L u , p ∗ W , ˜ p S | W , p ∗ X U | S W , W K ) . (4.9) The terminology pse udo sphere-pac king exponent is used bec ause d espite its sup erficial similarity to a real sphere-pa cking exponent, (4.4) d oes not p rovide a fund amental a symptotic lo wer bound on error probability . Theorem 4.1: The d ecision rule (4.1) yields the follo wing error expone nts. (i) The false-positi ve error exponent is E F P ( R, D 1 , W K , ∆) = ∆ . (4.10) (ii) The detec t-all error exponen t is E all ( R, L w , L u , D 1 , W K , ∆) = E psp ( R + ∆ , L w , L u , D 1 , W K ) . (4.11) (iii) The detec t-one e rror expon ent is E one ( R, L w , L u , D 1 , W K , ∆) = E psp ( R + ∆ , L w , L u , D 1 , W K ) . (4.12) (i v) A fair co llusion strategy is optimal und er the de tect-one error criterion: E one ( R, L w , L u , D 1 , W K , ∆) = E one ( R, L w , L u , D 1 , W f air K , ∆) . Nov ember 5, 2018 DRAFT 12 (v) The detect-one and detect-all error expon ents are the sa me when the colluders emply a fair strategy: E one ( R, L w , L u , D 1 , W f air K , ∆) = E all ( R, L w , L u , D 1 , W f air K , ∆) . (vi) For K = K nom , the sup remum of all rates for which the detect-one error expon ent of (4. 12) is positiv e is C thr ( D 1 , W K ) = C thr ( D 1 , W f air K ) = lim L w ,L u →∞ max p W ∈ P W max p X U | S W ∈ P X U | S W ( p W p S ,L w ,L u ,D 1 ) min p Y | X K ∈ W f air K [ I ( U ; Y | S d , W ) − I ( U ; S | S d , W )] . (4.13) V . J O I N T F I N G E R P R I N T D E C O D E R The funda mental improvement over the simple thresholding s trategy for decoding in Se c. IV resides in the use of a joint decoding rule. Specifica lly , the decode r maximizes a penalized empirical mutual information sco re over all possible coalitions of any size. Th e penalty de pends on the conditional hos t sequen ce type p s | s d w , as in Sec. IV, a nd is p roportional to the s ize of the c oalition, as in [10, Sec . V]. W e call this blind finge rprint deco der the maximum doubly-pen alized mutual infor mation (M2PMI) deco der . Mutual Inf ormation of k Ra ndom V ariables. The mutual information o f k rando m variables X 1 , · · · , X k is define d as the sum of their individual en tropies minus their joint entropy [21, p. 57] or equiv alently , the diver g ence b etween their joint distrib ution an d the p roduct of the ir marginals: ◦ I ( X 1 ; · · · ; X k ) = H ( X 1 ) + · · · + H ( X k ) − H ( X 1 , · · · , X k ) (5.1) = D ( p X 1 ··· X k k p X 1 · · · p X k ) . The symbol ◦ I is use d to distinguish it from ordinary mutual information I between two ran dom vari- ables. Similarly one can define a con ditional mutual information ◦ I ( X 1 ; · · · ; X k | Z ) = P i H ( X i | Z ) − H ( X 1 , · · · , X k | Z ) conditioned on Z , and an empirical mutua l information ◦ I ( x 1 ; · · · ; x k | z ) be tween k sequen ces x 1 , · · · , x k , con ditioned on z , as the conditional mutua l information with respec t to the joint type of x 1 , · · · , x k , z . Some properties o f ◦ I are gi ven in [10, S ec. V .A]. Recall that x A denotes { x m , m ∈ A} and that the cod ew ords in (3.1) take the form u ( l , m, λ ) . In the follo wing, we shall use the compact notation ( xu ) A , ( x A , u A ) , a nd u ( l A , m A , λ ) , { u ( l m 1 , m 1 , λ ) , · · · , u ( l m |A| , m |A| , λ ) } for A = { m 1 , · · · , m |A| } . Nov ember 5, 2018 DRAFT 13 A. M2PMI Criterion Gi ven y , s d , w , the decod er seek s the coa lition siz e k , the conditional ho st se quenc e type λ ∈ P [ N ] S | S d W , and the cod ew ords u ( l, m, λ ) in C ( s d , w , λ ) that maximize the M2PMI criterion below . The c olumn indices m ∈ K , corresp onding to the decode d words form the dec oded coa lition ˆ K . If the maximizing k in (5.2) is zero, the receiv er outputs ˆ K = ∅ . The Maximum Do ubly-P e nalized Mutual Information criterion is de fined as max k ≥ 0 M 2 P M I ( k ) (5.2) where M 2 P M I ( k ) =        0 : if k = 0 max λ ∈ P [ N ] S | S d W max u K ∈C k ( s d , w , λ )  ◦ I ( u K ; y | s d w ) − k ( ρ ( λ ) + R + ∆)  : if k = 1 , 2 , · · · (5.3) B. Pr op erties The follo wing lemma shows that 1) eac h subs et of the e stimated coalition is s ignificant, a nd 2 ) any further extens ion of the coalition would fail a s ignificanc e test. The proof parallels that of Lemma 5.1 in [10] a nd is therefore omitted. Lemma 5.1: Let ˆ K , λ , l ˆ K achieve the maximum in (5.3) (5.2), i.e., u ˆ K = u ( l ˆ K , m ˆ K , λ ) . Then for each subset of the estimated co alition ˆ K , we have ∀A ⊆ ˆ K : ◦ I ( u ( l A , m A , λ ); yu ( l ˆ K \A , m ˆ K\A , λ ) | s d w ) > |A| ( ρ ( λ ) + R + ∆) . (5.4) Moreover , for every A disjoint with ˆ K , ◦ I ( u ( l A , m A , λ ); yu ( l ˆ K , m ˆ K , λ ) | s d w ) ≤ |A| ( ρ ( λ ) + R + ∆) . (5.5) C. Err o r E xponen ts Define for each A ⊆ K the se t of c onditional p.m.f. ’ s P Y ( X U ) K | S W ( p W , ˜ p S | W , p X U | S W , W K , R, L w , L u , A ) ,  ˜ p Y ( X U ) K | S W : ˜ p ( X U ) K | S W ∈ M ( p X U | S W ) , 1 |A| ◦ I p W ˜ p S | W ˜ p Y ( X U ) K | S W ( U A ; Y U K\A | S d , W ) ≤ I p W ˜ p S | W p X U | S W ( U ; S | S d , W ) + R  (5.6) Nov ember 5, 2018 DRAFT 14 and the ps eudo sphe r e packing exponent ˜ E psp, A ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W K ) = min ˜ p Y ( X U ) K | S W ∈ P Y ( X U ) K | S W ( p W , ˜ p S | W ,p X U | S W , W K ,R,L w ,L u , A ) D ( ˜ p Y ( X U ) K | S W ˜ p S | W k ˜ p Y | X K ˜ p ( X U ) K | S W p S | p W ) . (5.7) T aking the maximum 1 and the minimum of ˜ E psp, A above over all subsets A of K , we define ˜ E psp ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W K ) = ˜ E psp, K ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W K ) , (5.8) ˜ E psp ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W K ) = min A⊆K ˜ E psp, A ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W K ) . (5.9) Now defin e E psp ( R, L w , L u , D 1 , W K ) = max p W ∈ P W min ˜ p S | W ∈ P S | W max p X U | S W ∈ P X U | S W ( p W , ˜ p S | W ,L w ,L u ,D 1 ) ˜ E psp, K ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W f air K nom ) . (5.10) Denote by p ∗ W and p ∗ X U | S W the maximizers in (5.10), where the latter is to be viewed as a fun ction of ˜ p S | W . Both p ∗ W and p ∗ X U | S W implicitly d epend on R and W f air K . Finally , de fine E psp ( R, L w , L u , D 1 , W K ) = min ˜ p S | W ∈ P S | W ˜ E psp ( R, L w , L u , p ∗ W , ˜ p S | W , p ∗ X U | S W , W K ) , (5.11) E psp ( R, L w , L u , D 1 , W K ) = min ˜ p S | W ∈ P S | W ˜ E psp ( R, L w , L u , p ∗ W , ˜ p S | W , p ∗ X U | S W , W K ) . (5.12) Theorem 5.2: The d ecision rule (5.2) yields the follo wing error expone nts. (i) The false-positi ve error exponent is E F P ( R, D 1 , W K , ∆) = ∆ . (5.13) (ii) The detec t-all error exponen t is E all ( R, L w , L u , D 1 , W K , ∆) = E psp ( R + ∆ , L w , L u , D 1 , W K ) . (5.14) (iii) The detec t-one e rror expon ent is E one ( R, L w , L u , D 1 , W K , ∆) = E psp ( R + ∆ , L w , L u , D 1 , W K ) . (5.15) (i v) E one ( R, L w , L u , D 1 , W K , ∆) = E one ( R, L w , L u , D 1 , W f air K , ∆) . (v) E all ( R, L w , L u , D 1 , W f air K , ∆) = E one ( R, L w , L u , D 1 , W f air K , ∆) . 1 The property that K achiev es max A⊆K ˜ E psp, A is established i n the proof of Theorem 5.2, Part (iv). Nov ember 5, 2018 DRAFT 15 (vi) If K = K nom , the s upremum of all rates for whic h the error exponen t of (5.15) a nd (5.14) are positiv e is C one ( D 1 , W K ) = C one ( D 1 , W f air K ) = lim L w ,L u →∞ max p W ∈ P W max p ( X U ) K | S W ∈ P ( X U ) K | S W ( p W ,p S ,L w ,L u ,D 1 ) min p Y | X K ∈ W f air K  1 K I ( U K ; Y | S d , W ) − I ( U ; S | S d , W )  (5.16) under the “detect-one” criterion, an d by C all ( D 1 , W K ) = lim L w ,L u →∞ max p W ∈ P W max p ( X U ) K | S W ∈ P ( X U ) K | S W ( p W ,p S ,L w ,L u ,D 1 ) min p Y | X K ∈ W K  min A⊆K 1 |A| I ( U A ; Y | S d , W, U K\A ) − I ( U ; S | S d , W )  (5.17) under the “detect-all” criterion. If the colluders selec t a fair collus ion channe l, as is their collectiv e interes t, the minimization is res tricted to W f air K in (5.17), and then C all ( D 1 , W K ) = C one ( D 1 , W K ) . For the special case of pri vate fing erprinting ( S d = S ), the term I ( U ; S | S d , W ) in (5. 16) is zero. Since I ( U K ; Y | S, W ) ≤ I (( X U ) K ; Y | S, W ) , it suffices to choos e L u = |X | a nd U = X to a chieve the maximum in (5.16). The resu lting expre ssion co incides with the capac ity formula in [10 , Theorem 3.2]. Similarly to the single-us er ca se [11], when U = X the binning sch eme is degen erate. D. Bounded Coalition Size Assume now that K is known not exceed some maximum value K max . The sa me random cod ing scheme ca n be u sed. In the evaluation of the M2PMI criterion of (5.2), the max imization is now limited to 0 ≤ k ≤ K max . In L emma 5.1, property (5.4) holds, and prop erty (5.5) now holds for every A disjoint with ˆ K , and of size |A| ≤ K max − | ˆ K| . Following the deriv ation of the error exponents in the appe ndix, we see tha t these exponents rema in the sa me a s those g i ven by Theorem 5.2. Blind watermar king . The case K max = 1 repres ents blind watermark deco ding with a gu arantee that the false-positi ve exponent is at least equa l to ∆ . In this scenario, there is no need for a time-sharing sequen ce w , and the decoder’ s inp ut y is e ither an unwatermarked s equenc e ( K = 0 ) o r a watermarked sequen ce ( K = 1 ). The M2PMI criterion o f (5.3) red uces to M 2 P M I ( k ) = max λ max u ∈C ( s d ) I ( u ; y | s d ) − ( ρ ( λ ) + R + ∆) for k = 1 . The resulting false-pos iti ve a nd false-negati ve e xponen ts are gi ven by ∆ a nd E psp ( R + ∆ , 0 , L u , D 1 , W K ) , respectively . Nov ember 5, 2018 DRAFT 16 V I . U P P E R B O U N D S O N P U B L I C F I N G E R P R I N T I N G C A PAC I T Y Deri ving pub lic finge rprinting capacity is a challenge beca use the ca pacity region for the Gel’f and- Pinsker version of the MA C is still unknown, in fact an outer b ound for this region ha s yet to b e established . Even in the case of a MA C with side information cau sally av ailable at the transmitter but not at the rece i ver , the expres sions for the inn er and ou ter c apacity regions do n ot coinc ide [23]. L ike wise, the expres sion deriv ed be lo w is an upp er bo und on public fing erprinting cap acity unde r the d etect-all criterion. Recall the definition of the se t P ( X U ) K W | S ( p S , L w , L u , D 1 ) in (4.2), whe re W a nd U are random variables defined over alph abets W = { 1 , 2 , · · · , L w } and U = { 1 , 2 , · · · , L u } , respecti vely . Here we define the lar ger se t P outer ( X U ) K W | S ( p S , L w , L u , D 1 ) = ( p ( X U ) K W | S = p W Y k ∈K p X k | S W ! p U K | X K S W : p X 1 | S W = · · · = p X K | S W , and E [ d ( S, X 1 )] ≤ D 1  (6.1) where X k , k ∈ K , a re still c onditionally i.i.d. given ( S, W ) b ut the ran dom variables U k , k ∈ K , are generally con ditionally depend ent. Define C all L w ,L u ( D 1 , W K ) = max p ( X U ) K W | S ∈ P outer ( X U ) K W | S ( p S ,L w ,L u ,D 1 ) min p Y | X K ∈ W K min A⊆K 1 |A| h I ( U A ; Y , S d | U K\A ) − I ( U A ; S | U K\A ) i . (6.2) Using the same deriv ation a s in Lemma 2.1 of [11], it ca n be shown that C all L w ,L u ( D 1 , W K ) is a nonde- creasing function of L w and L u and con ver ges to a finite limit. Moreover , the gap to the limit ma y be bounde d by a polynomial function of L w and L u , se e [11, Sec. 3.5] for a s imilar deriv ation. Theorem 6.1: Public fing erprinting capac ity is upper-bounded b y C all ( D 1 , W K ) = lim L w ,L u →∞ C all L w ,L u ( D 1 , W K ) (6.3) under the “ detect-all” criterion. Pr oo f : s ee appe ndix. W e co njecture that the uppe r bound on capacity giv en by Theorem 6.1 is gene rally not tight. The insight here is that the upper bound remains valid if the class of e ncoding functions is enlarged to include feedba ck from the receiver: X k i = ˜ f i ( S , M k , Y i − 1 ) for 1 ≤ i ≤ N . It can indeed be verified that all the inequa lities in the proof a nd the Markov chain properties hold. The que stion is now whether Nov ember 5, 2018 DRAFT 17 feedback c an increase public finge rprinting capa city . W e con jecture the an swer is yes, beca use fee dback is known to increase MA C ca pacity [24]. W e also make the stronger conje cture that the maximum over p ( X U ) K | S W is achieved by a p.m.f. that decoup les the compone nts ( X k , U k ) , k ∈ K , con ditioned on ( S, W ) . If this is tr ue, the set P outer ( X U ) K W | S ( p S , L w , L u , D 1 ) in the formula (6.2) c an be replace d with the sma ller set P ( X U ) K W | S ( p S , L w , L u , D 1 ) of (4.2), and the random co ding sche me o f Sec. V is capa city-achieving. V I I . C O N C L U S I O N W e have proposed a communication model and a random-co ding scheme for blind fingerprinting. While a standa rd binning sche me for commun ication with asymme tric side information at the transmitter and the receiver may se em like a reaso nable candidate, such a sc heme would be u nable to trade false-pos iti ve error expo nents against false-negati ve err or expon ents. Our propo sed binning s cheme combine s two ide as. The first is the u se of a stacked binn ing s cheme as in [11], which demo nstrated the advantages (in terms of dec oding error exponents ) of s electing c odewords from a n a rray w hose s ize d epends on the co nditional type of the host se quence . The s econd is the us e of an auxiliary time-sharing rando m variable as in [10]. The blind fingerprint decode rs of Secs. IV and V combine the ad vantages o f b oth methods and provide positiv e error exponents for a range of c ode rates. The tradeof f betwee n the two fundame ntal types o f error probabilities is determined b y the value of the parameter ∆ . Nov ember 5, 2018 DRAFT 18 A P P E N D I X I P R O O F O F T H E O R E M 4 . 1 W e d eri ve the error expone nts for the thresh olding rule (4.1). W e hav e W = { 1 , 2 , · · · , L w } and U = { 1 , 2 , · · · , L u } . Fix s ome arbitrarily s mall ǫ > 0 . Define for all m ∈ K P [ N ] Y ( X U ) K | S W ( p w , p s | w , p xu | sw , W K , R, L w , L u , m ) =  p y ( xu ) K | sw : p ( xu ) K | sw ∈ M ( p xu | sw ) , p y | x K ∈ W K , I ( u m ; y | s d w ) ≤ ρ ( p s | s d w ) + R o ˘ E psp,m,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) = min p y ( xu ) K | sw ∈ P [ N ] Y ( X U ) K | S W ( p w ,p s | w ,p xu | sw , W K ,R,L w ,L u ,m ) D ( p y ( xu ) K | sw k p y | x K p K xu | sw | p sw ) , (A.1) ˆ E psp,m,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) = D ( p s | w k p S | p w ) + ˘ E psp,m,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) = min p y ( xu ) K | sw ∈ P [ N ] Y ( X U ) K | S W ( p w ,p s | w ,p xu | sw , W K ,R,L w ,L u ,m ) D ( p y ( xu ) K | sw p s | w k p y | x K p K xu | sw p S | p w ) , (A.2 ) ˆ E psp,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) = max m ∈K ˆ E psp,m,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) (A.3) ˆ E psp,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) = min m ∈K ˆ E psp,m,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) (A.4) where (A.2) is obtained b y application of the chain rule for diver g ence. Also define E psp,N ( R, L w , L u , D 1 , W K ) = ma x p w ∈ P [ N ] W min p s | w ∈ P [ N ] S | W max p xu | sw ∈ P [ N ] X U | S W ( p w p s | w ,L w ,L u ,D 1 ) ˆ E psp, 1 ,N ( R, L w , L u , p w , p s | w , p xu | sw , W f air K nom ) . (A.5) Denote by p ∗ w and p ∗ xu | sw the ma ximizers a bove, the latter viewed as a func tion of p s | w . Both maximizers depend implicitly on R and W f air K nom . Let E psp,N ( R, L w , L u , D 1 , W K ) = min p s | w ∈ P [ N ] S | W ˆ E psp,N ( R, L w , L u , p ∗ w , p s | w , p ∗ xu | sw ) , (A.6) E psp,N ( R, L w , L u , D 1 , W K ) = min p s | w ∈ P [ N ] S | W ˆ E psp,N ( R, L w , L u , p ∗ w , p s | w , p ∗ xu | sw ) . (A.7) Nov ember 5, 2018 DRAFT 19 The expon ents (A.2)—(A.7) d if fe r from (4.4)—(4.9) in tha t the op timizations are pe rformed over co ndi- tional types instead of gen eral conditional p.m.f. ’ s. W e hav e lim N →∞ E psp,N ( R, L w , L u , D 1 , W K ) = E psp ( R, L w , L u , D 1 , W K ) (A.8) lim N →∞ E psp,N ( R, L w , L u , D 1 , W K ) = E psp ( R, L w , L u , D 1 , W K ) (A.9) by continuity of the diver ge nce and mutual-information functionals. Consider the max imization over the co nditional type p xu | sw in (A.5). As a result of this maximization, we may associate the following: • t o any ( s , w ) , a conditional type class T ∗ U | S W ( s , w ) , T ∗ u | sw ; • t o any ( s d , w ) , a conditional type c lass T ∗ U | S d W ( s d , w ) , T ∗ u | s d w ; • t o any ( s , w ) a nd u ∈ T ∗ U | S W ( sw ) , a cond itional typ e class T ∗ X | U S W ( u , s , w ) , T ∗ x | usw ; • t o a ny type p sw , a conditional mutual information I ∗ U S | S d W ( p sw ) , I ( u ; s | s d , w ) where u , s , w a re any three sequ ences with joint type p ∗ u | sw p sw . Codebook . Defi ne the func tion ρ ( p s | s d w ) = I ∗ U S | S d W ( p sw ) + ǫ, ∀ p sw ∈ P [ N ] S W . A rando m constant-comp osition code C ( s d , w , p s | s d w ) = { u ( l, m, p s | s d w ) , 1 ≤ l ≤ exp 2 { N ρ ( p s | s d w ) , 1 ≤ m ≤ 2 N R } is ge nerated for each s d ∈ ( S d ) N , w ∈ T ∗ w , an d p s | s d w ∈ P [ N ] S | S d W by d rawing exp 2 { N ( R + ρ ( p s | s d w )) } random se quenc es ind epende ntly and uniformly from the cond itional type class T ∗ U | S d W ( s d , w ) , and arranging them into an a rray with 2 N R columns and exp 2 { N ρ ( p s | s d w ) } rows. Encoder . Prior to e ncoding, a s equen ce W ∈ W N is drawn indepen dently of S and uniformly from T ∗ w , an d s hared with the rece i ver . Gi ven ( S , W ) , the encod er de termines the cond itional type p s | s d w and performs the following two steps for each use r 1 ≤ m ≤ 2 N R . 1) Find l s uch that u ( l , m, p s | s d w ) ∈ C ( S d , W , p s | s d w ) T T ∗ U | S W ( s , w ) . If more than one suc h l exists, pick one of them randomly (with u niform distrib ution). Let u = u ( l , m, p s | s d w ) . If no such l can be found, generate u uniformly from the conditional type clas s T ∗ U | S W ( s , w ) . 2) Generate X m uniformly d istrib u ted over the conditional type class T ∗ X | U S W ( u , s , w ) . Collusion cha nnel. By Prop. 3.1, it is sufficient to res trict our a ttention to strongly exchange able collusion cha nnels in the error probability analys is. Decoder . Giv en ( y , s d , w ) , the decode r outputs ˆ K if and on ly if (4.1) is satisfie d. Nov ember 5, 2018 DRAFT 20 Encoding errors. Ana logously to [11], the p robability of encoding errors vanishes doubly e xponen tially with N because ρ ( p s | s d w ) > I ( u ; s | s d w ) . Indeed an enc oding error for user m arises under the follo wing ev ent: E m = { ( C , s , w ) : ( u ( l, m, p s | s d w ) ∈ C and u ( l, m , p s | s d w ) / ∈ T ∗ U | S W ( s , w )) for 1 ≤ l ≤ 2 N ρ ( p s | s d w ) } . (A.10) The probability that a se quence U u niformly distributed ov er T ∗ U | S d W ( s d , w ) also belon gs to T ∗ U | S W ( s , w ) is equal to exp 2 {− N I ∗ U S | S d W ( p sw ) } on the expo nential scale . The refore the e ncoding error probab ility , conditioned on type class T sw , sa tisfies P r [ E m | ( S , W ) ∈ T sw ] = 1 − | T ∗ U | S W ( S , W ) | | T ∗ U | S d W ( S d , W ) | ! 2 N ρ ( p s | s d w ) . = (1 − 2 − N I ∗ U S | S d W ( p sw ) ) 2 N ρ ( p s | s d w ) ≤ exp {− exp 2 ( N [ ρ ( p s | s d w ) − I ∗ U S | S d W ( p sw )]) } = exp {− 2 N ǫ } (A.11) where the inequality follows from 1 − a ≤ e − a . The de ri vati on o f the dec oding error exponents is based on the follo wing tw o asymp totic e qualities which are special cases of (C.2) and (C.5 ) establishe d in Lemma 3.1. 1) Fix y , s d , w and d raw u uniformly from some fi xed type class , inde pende ntly of ( y , s d , w ) . The n P r [ I ( u ; y | s d w ) ≥ ν ] . = 2 − N ν . ( A.12) 2) Given s , w , draw ( x k , u k ) , k ∈ K , i.i.d. uniformly from a conditional type class T xu | sw , an d then draw y u niformly over a single conditional type class T y | x K . For any ν > 0 , we have P r [ I ( u m ; y | s d w ) ≤ ρ ( p s | s d w ) + ν ] . = exp 2 {− N ˘ E psp,m,N ( ν, L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K ) } . (A.13 ) (i). False Positiv es. From (4.1 ), the occurrence of a false p ositi ve implies that ∃ λ ∈ P [ N ] S | S d W , l , m / ∈ K : I ( u ( l , m, λ ); y | s d w ) > ρ ( λ ) + R + ∆ . (A.14) By co nstruction of the code book, u ( l , m, λ ) is indepen dent of y for m / ∈ K . For any given λ , there are at mos t 2 N ρ ( λ ) possible values for l and 2 N R − K po ssible values for m in (A.14). Hence the probability Nov ember 5, 2018 DRAFT 21 of false po siti ves, cond itioned on the joint type clas s T y ( xu ) K sw , is P F P ( T y ( xu ) K sw , W K ) ≤ X λ (2 N R − K ) 2 N ρ ( λ ) P r [ I ( u ( l, m, λ ); y | s d w ) > ρ ( λ ) + R + ∆] ( a ) . = X λ 2 N ( R + ρ ( λ )) 2 − N ( R +∆+ ρ ( λ )) ( b ) ≤ ( N + 1) |S | L w 2 − N ∆ . = 2 − N ∆ (A.15) where (a) is obtained b y ap plication of (A.12) with ν = ρ ( λ ) + R + ∆ , and (b) because the number of conditional types λ is a t most ( N + 1) |S | L w . A veraging over a ll type classes T y ( xu ) K sw , we o btain P F P  ≤ 2 − N ∆ , from wh ich (4.10) follows. (ii). Detect-One Error Criterion (Miss All Co lluders). W e first derive the e rror expon ent for the e vent that the deco der miss es a spe cific colluder m ∈ K . Any coalition ˆ K that con tains m fails the test (4.1), i.e., for any suc h ˆ K , ∀ λ ∈ P [ N ] S | S d W : max l I ( u ( l , m, λ ); y | s d w ) ≤ ρ ( λ ) + R + ∆ . (A.16) This implies tha t I ( u ( l , m, p s | s d w ); y | s d w ) ≤ ρ ( p s | s d w ) + R + ∆ (A.17) where l is the r ow index ac tually selected by the encoder , and p s | s d w is the actual host sequence c onditional type. T he probab ility of the miss- m event, gi ven the joint type p ∗ w p s | w p ∗ xu | sw , is therefore up per- bounde d by the probability of the ev ent (A.17): p miss − m ( p ∗ w , p s | w , p ∗ xu | sw , W K ) ≤ P r h I ( u ( l , m, p s | s d w ); y | s d w ) ≤ ρ ( p s | s d w ) + R + ∆ i ( a )  ≤ exp 2 n − N ˘ E psp,m,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K ) o where (a) follows from (A.13) w ith ν = R + ∆ . Nov ember 5, 2018 DRAFT 22 The miss-all event is the interse ction of the miss- m events over m ∈ K . Its conditional probability is p miss − all ( p ∗ w , p s | w , p ∗ xu | sw , W K ) = P r " \ m ∈K n miss m | p ∗ w , p s | w , p ∗ xu | sw o # ≤ min m ∈K p miss − m ( p ∗ w , p s | w , p ∗ xu | sw , W K ) . = exp 2  − N max m ∈K ˘ E psp,m,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K )  . (A.18) A veraging over S , we obtain p miss − all ( W K ) ≤ X p s | w P r [ T s | w ] p miss − all ( p ∗ w , p s | w , p ∗ xu | sw , W K ) . = max p s | w P r [ T s | w ] p miss − all ( p ∗ w , p s | w , p ∗ xu | sw , W K ) ( a ) . = max p s | w exp 2  − N  D ( p s | w k p S | p ∗ w ) + max m ∈K ˘ E psp,m,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K )  ( b ) . = exp 2  − N E psp,N ( R + ∆ , L w , L u , D 1 , W K )  ( c ) . = exp 2  − N E psp ( R + ∆ , L w , L u , D 1 , W K )  which establishes (4.1 2). He re (a) follows from (C.3) a nd (A.18), (b) from (A.3) and (A.6), and (c) from (A.8). (iii). Detect-All Error Criterion (Miss Some Colluders). The mi ss-some e vent is the union of the miss- m e vents over m ∈ K . Given the joint type p ∗ w p s | w p ∗ xu | sw , the prob ability o f this event is p miss − some ( p ∗ w , p s | w , p ∗ xu | sw , W K ) (A.19) = P r " [ m ∈K n miss m | p ∗ w , p s | w , p ∗ xu | sw o # ≤ X m ∈K p miss − m ( p ∗ w , p s | w , p ∗ xu | sw , W K ) . = max m ∈K exp 2 n − N ˘ E psp,m,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K ) o = exp 2  − N min m ∈K ˘ E psp,m,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K )  . (A.20) Nov ember 5, 2018 DRAFT 23 A veraging over S , we obtain p miss − some ( W K ) ≤ X p s | w P r [ T s | w ] p miss − some ( p ∗ w , p s | w , p ∗ xu | sw , W K ) ( a ) . = max p s | w exp 2  − N  D ( p s | w k p S | p ∗ w ) + m in m ∈K ˘ E psp,m,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K )  ( b ) ≤ exp 2  − N E psp,N ( R + ∆ , D 1 , W K )  ( c ) . = exp 2  − N E psp ( R + ∆ , L w , L u , D 1 , W K )  which establishes (4.1 1). He re (a) follows from (C.3) a nd (A.20), (b) from (A.7) and (A.4), and (c) from (A.9). (iv). Fair Collusion Channe ls. Th e proof parallels that of [10, Theorem 4.1(iv)], u sing the conditional div er gence D ( ˜ p Y ( X U ) K | S W ˜ p S | W k ˜ p Y | X K p K X U | S W p S | p W ) in p lace of D ( ˜ p Y X K | W k ˜ p Y | X K p K X | W | p W ) . (v). Immedia te, becau se E psp = E psp in this cas e. (vi). Positi ve Error Expone nts. From Part (v) above, we ma y restrict our attention to W K = W f air K . Consider any W = { 1 , · · · , L w } and p W that is positi ve over its support set (if it i s not, reduce the v alue of L w accordingly .) For any m ∈ K , the miniman d in the expression (4.4) for ˜ E psp,m ( R, L w , L u , p W , p X U | S W , W f air K ) is z ero if a nd only if ˜ p Y ( X U ) K | S W ˜ p S | W = ˜ p Y | X K p K X U | S W p S , with ˜ p Y | X K ∈ W f air K . Such ( ˜ p Y ( X U ) K | S W , ˜ p S | W ) is feasible for (4.3) if and only i f ( p X U | S W , ˜ p Y | X K ) is such that I ( U m ; Y | S d , W ) ≤ I ( U m ; S | S d , W ) + R . It is not feasible, and thus a positiv e exponent E one is guarantee d, if R < I ( U 1 ; Y | S d , W ) − I ( U 1 ; S | S d , W ) . T he supremum of all such R is gi ven by (4.13) and is ach iev ed by letting ǫ → 0 , ∆ → 0 , and L w , L u → ∞ . ✷ Nov ember 5, 2018 DRAFT 24 A P P E N D I X I I P R O O F O F T H E O R E M 5 . 2 W e deri ve the error exponen ts for the M2PMI decision rule (5.2). Define for all A ⊆ K P [ N ] Y ( X U ) K | S W ( p w , p s | w , p xu | sw , W K , R, L w , L u , A ) =  p y ( xu ) K | sw : p ( xu ) K | sw ∈ M ( p xu | sw ) , p y | x K ∈ W K , ◦ I ( u A ; yu K\A | s d w ) ≤ |A| ( ρ ( p s | s d w ) + R )  (B.1) ˘ E psp, A ,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) = min p y ( xu ) K | sw ∈ P [ N ] Y ( X U ) K | S W ( p w ,p s | w ,p xu | sw , W K ,R,L w ,L u , A ) D ( p y ( xu ) K | sw k p y | x K p K xu | sw | p sw ) , (B.2) ˆ E psp, A ,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) = D ( p s | w k p S | p w ) + ˘ E psp, A ,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) = min p y ( xu ) K | sw ∈ P [ N ] Y ( X U ) K | S W ( p w ,p s | w ,p xu | sw , W K ,R,L w ,L u , A ) D ( p y ( xu ) K | sw p s | w k p y | x K p K xu | sw p S | p w ) , (B.3) ˆ E psp,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) = ˆ E psp, K ,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) , (B.4) ˆ E psp,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) = min A⊆K ˆ E psp, A ,N ( R, L w , L u , p w , p s | w , p xu | sw , W K ) , (B.5) E psp,N ( R, L w , L u , D 1 , W K ) = max p w ∈ P [ N ] W min p s | w ∈ P [ N ] S | W max p xu | sw ∈ P [ N ] X U | S W ( p w p s | w ,L w ,L u ,D 1 ) ˆ E psp, K ,N ( R, L w , L u , p w , p s | w , p xu | sw , W f air K nom ) . (B.6) Denote by p ∗ w and p ∗ xu | sw the maximizers in (B.6), the latter viewed a s a func tion of p s | w . Both maximizers dep end implicitly on R , D 1 , and W f air K nom . Let E psp,N ( R, L w , L u , D 1 , W K ) = min p s | w ˆ E psp,N ( R, L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K ) (B.7) E psp,N ( R, L w , L u , D 1 , W K ) = min p s | w ˆ E psp,N ( R, L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K ) . (B.8) The exponents (B.3)—(B.8 ) dif fer from (5.7)—(5.12) in that the optimizations are performed ov er Nov ember 5, 2018 DRAFT 25 conditional types instead of gen eral conditional p.m.f. ’ s. W e hav e lim N →∞ E psp,N ( R, L w , L u , D 1 , W K ) = E psp ( R, L w , L u , D 1 , W K ) (B.9) lim N →∞ E psp,N ( R, L w , L u , D 1 , W K ) = E psp ( R, L w , L u , D 1 , W K ) (B.10) by continuity of the diver ge nce and mutual-information functionals. The co debook and enc oding proced ure a re exactly as in the proo f of The orem IV, the difference being that p ∗ w and p ∗ xu | sw are so lutions to the optimization problem (B.6) instead of (A.5). The de coding rule is the M2PMI rule o f (5.2). T o a nalyze the error p robability for this ran dom-coding scheme, it is a gain sufficient to res trict our attention to strongly-excha ngeab le channe ls and us e the bound (3.2 ) on the c onditional proba bility of the collusion cha nnel output. W e also us e Le mma 3.1. (i). False Positiv es. By application of (5.4), a false p ositi ve oc curs if ˆ K \ K 6 = ∅ a nd ∃ λ ∈ P [ N ] S | S d W : ∀A ⊆ ˆ K : ∃ l ˆ K : ◦ I ( u ( l A , m A , λ ); yu ( l ˆ K \A , m ˆ K\A , λ ) | s d w ) > |A| ( ρ ( λ ) + R + ∆ ) . (B.11) The proba bility of this ev ent is upper-bounded by the probab ility of the larger ev ent ∀A ⊆ ˆ K : ∃ λ, l ˆ K : ◦ I ( u ( l A , m A , λ ); yu ( l ˆ K \A , m ˆ K\A , λ ) | s d w ) > |A| ( ρ ( λ ) + R + ∆ ) . (B.12) Denote by p ∗ s | s d w the conditional type of the host se quenc e and by l ∗ K the ro w indice s s elected b y the encode r . T o e ach triple ( ˆ K , λ, l ˆ K ) , we associa te a u nique subse t B of K ∩ ˆ K d efined as follows: • I f λ 6 = p ∗ s | s d w then B = ∅ • I f λ = p ∗ s | s d w then B is the (pos sibly empty) set of all indice s k ∈ K ∩ ˆ K s uch that l k = l ∗ k . Thus B is the set of colluder indices k ∈ K for which the decod er co rrectly ide ntifies the c onditional host sequen ce type p ∗ s | s d w and the codewords u ( l ∗ k , k , p ∗ s | s d w ) that were ass igned b y the encode r . Denoting b y Ω( B ) the set of pairs ( λ, l ˆ K ) a ssociated with B , we rewrit e (B.12 ) as ∀A ⊆ ˆ K : ∃B ⊆ K ∩ ˆ K , ∃ ( λ, l ˆ K ) ∈ Ω( B ) : ◦ I ( u ( l A , m A , λ ); yu ( l ˆ K \A , m ˆ K\A , λ ) | s d w ) > |A| ( ρ ( λ ) + R + ∆) . (B.13) Define the complement set A = ˆ K \ B whic h is co mprised of all inc orrectly accus ed use rs as well as any colluder k such that λ 6 = p ∗ s | s d w or l k 6 = l ∗ k . Since B ⊆ ˆ K and there is at leas t o ne innocen t u ser in ˆ K , the cardinality of A is at least equal to 1 . By construction of the codeb ook and definition o f A Nov ember 5, 2018 DRAFT 26 and B , u ( l A , m A , λ ) is inde penden t of y and u ( l ∗ B , m B , p ∗ s | s d w ) . The probability of the event (B.13) is upper-bounded by the probab ility of the larger event ∃B ⊆ K , ∃ λ, l A , m A : ◦ I ( u ( l A , m A , λ ); yu ( l ∗ B , m B , p ∗ s | s d w ) | s d w ) > |A| ( ρ ( λ ) + R + ∆) . (B.14) Hence the probability of false po siti ves, co nditioned on T y ( xu ) K sw , sa tisfies P F P ( T y ( xu ) K sw , W K ) = P r   [ B⊆K [ |A|≥ 1  ∃ λ, l A , m A : ◦ I ( u ( l A , m A , λ ); yu ( l ∗ B , m B , p ∗ s | s d w ) | s d w ) > |A| ( ρ ( λ ) + R + ∆ ) } ] ≤ X B⊆K X |A|≥ 1 P B , |A| ( T y ( xu ) K sw , W K ) (B.15) where P B , |A| ( T y ( xu ) K sw , W K ) = P r [ ∃ λ, l A , m A : ◦ I ( u ( l A , m A , λ ); yu ( l ∗ B , m B , p ∗ s | s d w ) | s d w ) > |A| ( ρ ( λ ) + R + ∆)] . (B.16) By d efinition of B , the re are at mos t P λ 6 = p s | s d w 2 N |A| ρ ( λ ) possible values for l A and 2 N |A| R possible values for m A in (B.16). Hence P B , |A| ( T y ( xu ) K sw , W K ) ≤ X λ 2 N |A| ( R + ρ ( λ )) P r [ ◦ I ( u ( l A , m A , λ ); yu ( l ∗ B , m B , p ∗ s | s d w ) | s d w ) > |A| ( ρ ( λ ) + R + ∆)] ( a ) . = X λ 2 N |A| ( R + ρ ( λ )) 2 − N |A| ( R +∆+ ρ ( λ )) ≤ ( N + 1) |S | 2 − N |A| ∆ . = 2 − N |A| ∆ (B.17) where (a) is obtained by application of (C.2 ) with yu ( l ∗ B , m B , p ∗ s | s d w ) in place of z . Combining (B.15) and (B.17) we obtain P F P ( T y ( xu ) K sw , W K ) ≤ X B⊆K X |A|≥ 1 2 − N |A| ∆ . = 2 − N ∆ . A veraging over a ll joint typ e classe s T y ( xu ) K sw , we o btain P F P  ≤ 2 − N ∆ , from wh ich (5.13) follows. Nov ember 5, 2018 DRAFT 27 (ii). Detect-All Cr iterion. (Miss So me Colluders.) Under the miss-some e rror event, any c oalition ˆ K that con tains K fails the test. By (5. 4), this implies ∀ λ ∈ P [ N ] S | S d W : ∃ A ⊆ ˆ K : max l ˆ K ◦ I ( u ( l A , m A , λ ); yu ( l ˆ K \A , m ˆ K\A , λ ) | s d w ) ≤ |A| ( ρ ( λ ) + R + ∆) . (B.18) In particular , for ˆ K = K we have ∃A ⊆ K : ◦ I ( u ( l A , m A , p s | s d w ); yu ( l K\A , m K\A , p s | s d w ) | s d w ) ≤ |A| ( ρ ( p s | s d w ) + R + ∆) . (B.19) where l K are the ro w indices actua lly selected by the encoder , an d p s | s d w is the actual host sequ ence conditional type. The p robability of the miss-some event, c onditioned on ( s , w ) , is the refore uppe r bounde d by the probability of the event (B.19): p miss − some ( p ∗ w , p s | w , p ∗ xu | sw , W K ) ≤ P r   [ A⊆K  ◦ I ( u ( l A , m A , p s | s d w ); yu ( l K\A , m K\A , p s | s d w ) | s d w ) ≤ |A| ( ρ ( p s | s d w ) + R + ∆)    ≤ X A⊆K P r  ◦ I ( u ( l A , m A , p s | s d w ); yu ( l K\A , m K\A , p s | s d w ) | s d w ) ≤ |A| ( ρ ( p s | s d w ) + R + ∆)  ( a )  ≤ X A⊆K exp 2 n − N ˘ E psp, A ,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K ) o . = max A⊆K exp 2 n − N ˘ E psp, A ,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K ) o = exp 2  − N min A⊆K ˘ E psp, A ,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K )  (B.20) where (a) follows from (C.5) with ν = R + ∆ . A veraging over S , we obtain p miss − some ( W K ) = X p s | w P r [ T s | w ] p miss − some ( p ∗ w , p s | w , p ∗ xu | sw , W K ) ( a ) . = max p s | w exp 2  − N [ D ( p s | w k p S | p w ) + min A⊆K ˘ E psp,N ( R + ∆ , L, p ∗ w , p s | w , p ∗ xu | sw , W K )]  ( b ) = max p s | w exp 2 n − N ˆ E psp,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K ) o ( c ) = exp 2  − N E psp,N ( R + ∆ , L w , L u , D 1 , W K )  ( d ) . = exp 2  − N E psp ( R + ∆ , L w , L u , D 1 , W K )  Nov ember 5, 2018 DRAFT 28 which prov es (5.14). Here (a) follo ws from (C.3) a nd (B.20), (b) from the definitions (B.5) and (B.3), (c) from (B.8), and (d) from the limit property (B.10). (iii). Detect-One Criterion (Miss All Colluders. ) E ither the es timated co alition ˆ K is empty , or it is a set I of innoc ent u sers (disjoint with K ). Hence P one e ≤ P r [ ˆ K = ∅ ] + P r [ ˆ K = I ] . The first probability , conditioned on ( s d , w ) , is bounded as P r [ ˆ K = ∅ ] = P r [ ∀K ′ : M 2 P M I ( K ′ ) ≤ 0] ≤ P r [ M 2 P M I ( K ) ≤ 0] = P r [ ◦ I ( u K ; y | s d w ) ≤ K ( ρ ( p s | s d w ) + R + ∆)] (B.21) ( a ) . = exp 2 n − N ˘ E psp, K ,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K ) o . where (a) follo ws from (C.5) with ν = R + ∆ . T o bound P r [ ˆ K = I ] , we use property (5.5) with ˆ K = I and A = K , whic h yields ◦ I ( u K ; yu I | s d w ) ≤ K ( ρ ( p s | s d w ) + R + ∆) . Since ◦ I ( u K ; yu I | s d w ) = ◦ I ( u K ; y | s d w ) + I ( u K ; u I | ys d w ) ≥ ◦ I ( u K ; y | s d w ) combining the two ine qualities above y ields ◦ I ( u K ; y | s d w ) ≤ K ( ρ ( p s | s d w ) + R + ∆) . The proba bility of this ev ent is again giv en b y (B.21); we conclud e tha t p miss − all ( p ∗ w p s | w , p ∗ xu | sw , W K ) . = exp 2 n − N ˘ E psp, K ,N ( R + ∆ , L w , L u , p ∗ w , p s | w , p ∗ xu | sw , W K ) o . A veraging over S and proce eding as in Part (ii) above, we ob tain p miss − all ( W K ) ≤ X p s | w P r [ T s | w ] p miss − all ( p ∗ w p s | w , p ∗ xu | sw , W K ) . = exp 2  − N E psp ( R + ∆ , L w , L u , D 1 , W K )  which estab lishes (5.15). (iv). Optimal Collusion Channels are F air . The proof p arallels that of [10, Theorem 4 .1(i v)] and is omitted. (v). Detect-All Exponent f or Fair Collusion Channels. The proof pa rallels that of [10, Theo - rem 4.1(v)] and is o mitted. Nov ember 5, 2018 DRAFT 29 (vi). Ac hievable Rates. Co nsider a ny W = { 1 , · · · , L w } a nd p W that is po siti ve over its s upport se t (if it is not, reduce the value of L w accordingly .) For any A ⊆ K , the diver ge nce to be minimized in the expression (5.7 ) for ˜ E psp, A ( R, L w , L u , p W , ˜ p S | W , p X U | S W , W K ) is z ero if a nd only if ˜ p Y ( X U ) K | S W = ˜ p Y | X K p K X U | S W and ˜ p S | W = p S . These p.m.f. ’ s are fea sible for (5.6) if and only if the ine quality below h olds: 1 |A| I ( U A ; Y U K\A | S d , W ) > I ( U ; S | S d , W ) + R. They a re infeasible, and thu s po siti ve error exponents are guaranteed, if R < min A⊆K 1 |A| I ( U A ; Y U K\A | S d , W ) − I ( U ; S | S d , W ) . From Part (i v ) above, we may restrict our a ttention to W K = W f air K under the dete ct-one c riterion. Since the p.m.f. of ( S, W, ( X U ) K , Y ) is permutation-in variant, by application o f [10, Eqn. (3.3)] with ( U K , S d ) in p lace of ( X K , S ) , we have min A⊆K 1 |A| I ( U A ; Y U K\A | S d W ) = 1 K I ( U K ; Y | S d W ) . (B.22) Hence the supremum of all R for error expone nts are positi ve is gi ven by C one ( D 1 , W K ) in (5.16) and is obtaine d b y letting ǫ → 0 , ∆ → 0 a nd L w , L u → ∞ . For any W K , under the detect-all criterion, the supremum of all R for wh ich error exponents are positi ve is giv en b y C all ( D 1 , W K ) in (5.17) and is o btained by letting ǫ → 0 , ∆ → 0 and L w , L u → ∞ . Since the optimal con ditional p.m.f. is n ot necessa rily permutation-in variant, (B.22) does not hold in gene ral. Howe ver , if W K = W f air K , (B.22) ho lds, and the same ac hiev a ble rate is obtained for the detect-one and detect-all problems . ✷ A P P E N D I X I I I Lemma 3.1: 1) Fix ( s d , w ) and z ∈ Z N , an d d raw u K = { u m , m ∈ K} i.i.d. uniformly over a common type class T u | s d w , indep endently o f z . W e have the asymptotic equ ality P r [ T u K | zs d w ] = | T u K | zs d w | | T u | s d w | K . = 2 − N [ K H ( u | s d w ) − H ( u K | zs d w )] = 2 − N ◦ I ( u K ; z | s d w ) (C.1) P r [ ◦ I ( u K ; z | s d w ) ≥ ν ] . = 2 − N ν . (C.2) 2) Given w , draw s i.i.d. p S . W e h av e [21 ] P r [ T s | w ] . = 2 − N D ( p s | w k p S | p w ) . (C.3) Nov ember 5, 2018 DRAFT 30 3) Giv en ( s , w ) , draw ( x k , u k ) , k ∈ K , i.i.d. uniformly from a cond itional typ e class T xu | sw , and then draw Y uniformly from a single con ditional type class T y | x K . W e have P r [ T y ( xu ) K | sw ] = | T y | ( xu ) K sw | | T y | x K | | T ( xu ) K | sw | | T xu | sw | K . = exp 2 n − N D ( p yx K | sw k p y | x K p K xu | sw | p sw ) o . (C.4) For any fea sible, strongly excha ngeab le co llusion chann el, for any A ⊆ K and ν > 0 , we have P r [ ◦ I ( u A ; yu K\A | s d w ) ≤ |A| ( ν + ρ ( p s | s d w ))] . = exp 2 n − N ˘ E psp, A ,N ( ν, L, p ∗ w , p s | w , p ∗ xu | sw , W K ) o . (C.5) Pr oo f: Th e deriv a tion of (C.4), (C.3 ), and (C.5 ) parallels that of (D.12), (D.1 5) and (D.16) in [10]. A P P E N D I X I V P R O O F O F T H E O R E M 6 . 1 Let K be size of the coalition a nd ( f N , g N ) a seq uence of length- N , rate- R randomize d codes. W e show that for any seque nce of such codes, reliab le decoding of a ll K fin gerprints is possible o nly if R ≤ C all ( D 1 , W K ) . Recall that the encoder generates marked copies x m = f N ( s , v , m ) for 1 ≤ m ≤ 2 N R and that the deco der outpu ts an estimated c oalition g N ( y , s d , v ) ∈ { 1 , · · · , 2 N R } ⋆ . W e use the no tation M K , { M 1 , · · · , M K } a nd X K , { X 1 , · · · , X K } . T o prove that C all ( D 1 , W K ) is a n up per boun d on c apacity , it s uffices to identify a family o f collusion channe ls for which reliable decod ing is impossible at rates ab ove C all ( D 1 , W K ) . As s hown in [10], it is sufficient to deri ve su ch a boun d for the compo und family W K of memor yless cha nnels . Our de ri vati on is an extension of the single-us er compound Gel’fand-Pinsker p roblem [11] to the multiple-access ca se. A lower bound on e rror prob ability is ob tained when an oracle informs the de coder that the coalition size is at most K . There are   2 N R K   ≤ 2 K N R possible c oalitions of size ≤ K . W e repres ent such a co alition as M K , { M 1 , · · · , M K } , wh ere M k , 1 ≤ k ≤ K , are drawn i.i.d. uniformly from { 1 , · · · , 2 N R } . Gi ven a memoryless channe l p Y | X K ∈ W K , the joint p.m.f. of ( M K , V , S , X K , Y ) is given by p M K V SX K Y = p N S p V Y 1 ≤ k ≤ K  p M k 1 { X k = f N ( S ,V ,M k ) }  p N Y | X K . (D.1) Our deriv ations make repeated u se of the identity I ( U A ; Y | Z, U K\A ) − I ( U A ; S | Z, U K\A ) = I ( U A ; Y , Z | U K\A ) − I ( U A ; S, Z | U K\A ) Nov ember 5, 2018 DRAFT 31 which follows from the c hain rule for conditional mutual information and h olds for any ( U K , S, Y , Z ) . The total error probability (includ ing false pos iti ves and false negatives) for the detec t-all decode r is P e ( p Y | X K ) = P r [ ˆ K 6 = K ] (D.2) when collus ion chann el p Y | X K ∈ W K is in ef fect. Step 1 . Follo w ing the deri vation of [10, Eqn. (B.20)] with ( Y , S d , V ) in place of ( Y , S , V ) at the receiv er , for the error prob ability P e ( p Y | X K ) to vanish for e ach p Y | X K ∈ W K , we need R ≤ lim in f N →∞ min p Y | X K ∈ W K min A⊆K 1 N |A | I ( M A ; Y | S d , V ) . (D.3) Step 2 . Define the i.i.d. random variables W i = { V , S j , j 6 = i } ∈ V N × S N − 1 , 1 ≤ i ≤ N . (D.4) Also define the random variables V k i = ( M k , V , S N i +1 ) , U k i = ( V k i , ( Y S d ) i − 1 ) = ( M k , V , S N i +1 , ( Y S d ) i − 1 ) , 1 ≤ k ≤ K, 1 ≤ i ≤ N (D.5) where S N i +1 , ( S i +1 , · · · , S N ) and ( Y S d ) i − 1 , ( Y 1 , S d 1 , · · · , Y i − 1 , S d i − 1 ) . Henc e V K i − 1 = ( V K i , S i ) , V K 1 = U K 1 , V K N = ( M K , V ) . (D.6) The following properties hold for each 1 ≤ i ≤ N : • By (D.1 ) and (D.5), ( S i , W i , U K i ) = ( M K , V , S , Y i − 1 ) → X K i → Y i forms a Markov cha in. • The rand om variables X k i , 1 ≤ k ≤ K , are conditionally i.i.d. gi ven ( S , V ) = ( S i , W i ) . • Due to the term Y i − 1 in (D.5), the random variables U k i , 1 ≤ k ≤ K , a re conditionally depende nt giv en ( S , V ) = ( S i , W i ) . The joint p. m.f. of ( S i , W i , X K i , U K i , Y i ) may thus be written as p S i p W i   Y 1 ≤ k ≤ K p X ki | S i W i   p U K i | X K i S i W i p Y | X K , 1 ≤ i ≤ N . (D.7) Step 3 . Consider a time-sha ring r andom v ariable T that is uniformly distrib uted over { 1 , · · · , N } and in- depend ent of the other ran dom variables, and define the tuple of rando m variables ( S, S d , W, U K , X K , Y ) as ( S T , S d T , W T , U K T , X K T , Y T ) . Also let W = ( W T , T ) and U k = ( U k ,T , T ) , 1 ≤ k ≤ K , which are defined over alpha bets of resp ectiv e ca rdinalities L w ( N ) = N |V N | |S | N − 1 Nov ember 5, 2018 DRAFT 32 and L u ( N ) = N |V N | 2 N [ R +l og max( |S | , |Y | |S d | )] . Since ( S i , W i , U K i ) → X K i → Y i forms a Ma rkov cha in, so does ( S, W, U K ) → X K → Y . From (D.7), the joint p.m.f. of ( S, W , U K , X K , Y ) takes the form p S p W   Y 1 ≤ k ≤ K p X k | S W   p U K | X K S W p Y | X K . (D.8) In (6.1) we have defin ed the set P outer X K U K W | S ( p S , L w , L u , D 1 ) = ( p X K U K W | S = p W K Y k =1 p X k | S W ! p U K | X K S W : p X 1 | S W = · · · = p X K | S W , and E d ( S, X 1 ) ≤ D 1  (D.9) where |W | = L w and |U | = L u . Observe that p X K U K W | S defined in (D.8) belongs t o P X K U K W | S ( p S , L w , L u , D 1 ) . Define the collection of K indices K = { 1 , 2 , · · · , K } and the follo wing functionals indexed b y A ⊆ K : J L w ,L u , A ( p S , p X K U K W | S , p Y | X K ) = 1 |A| [ I ( U A ; Y S d | U K\A ) − I ( U A ; S | U K\A )] . (D.10) Step 4 . W e have I ( M K ; Y | S d , V ) ( a ) = I ( M K ; Y | S d , V ) − I ( M K , V ; S | S d ) = I ( M K , V ; Y | S d ) − I ( V ; Y | S d ) − I ( M K , V ; S | S d ) ≤ I ( M K , V ; Y | S d ) − I ( M K , V ; S | S d ) ( b ) = I ( M K , V ; YS d ) − I ( M K , V ; S ) ( c ) ≤ N X i =1 [ I ( U K ,i ; Y i S d i ) − I ( U K ,i ; S i )] = I ( U K ,T ; Y S d | T ) − I ( U K ,T ; S | T ) = I ( U K ,T , T ; Y S d ) − I ( T ; Y S d ) − I ( U K ,T , T ; S ) + I ( T ; S ) ( d ) ≤ I ( U K ,T , T ; Y S d ) − I ( U K ,T , T ; S ) ( e ) = I ( U K ; Y S d ) − I ( U K ; S ) = K J L w ( N ) ,L u ( N ) , K ( p S , p X K U K W | S , p Y | X K ) , (D.11) where (a) holds beca use M K , V , S are mutually independen t, and (b) follows from the chain rule for mutual information, (c) from [20, Le mma 4], using V K i and U K i in place of V i and U i , resp ectiv ely , (d) holds bec ause I ( T ; S ) = 0 , and (e) by definition of U K . Nov ember 5, 2018 DRAFT 33 For all A ⊂ K , we have I ( M A ; Y | S d , V ) = I ( M A , V ; Y | S d , V ) ( a ) = I ( M A , V ; Y | S d , V ) − I ( M A , V ; S | S d , M K\A , V ) ( b ) = I ( M A , V ; Y | S d , M K\A , V ) − I ( M A , V ; S | S d , M K\A ) = I ( M A , V ; YS d | M K\A , V ) − I ( M A , V ; S | S d , M K\A , V ) ( c ) = N X i =1 [ I ( U A ,i ; Y i S d i | U K\A ,i ) − I ( U A ,i ; S i | U K\A ,i )] (D.12) = N [ I ( U A ,T ; Y S d | U K\A ,T , T ) − I ( U A ,T ; S | U K\A ,T , T )] = N [ I ( U A ,T , T ; Y S d | U K\A ,T , T ) − I ( U A ,T , T ; S | U K\A ,T , T )] ( d ) = N [ I ( U A ; Y S d | U K\A ) − I ( U A ; S | U K\A )] = N |A | J L w ( N ) ,L u ( N ) , A ( p S , p X K U K W | S , p Y | X K ) . (D.13) where (a) and (b) hold beca use M K , S , and V are mutually independen t, the equality (c) is proved a t the en d o f this s ection, and (d) follo ws from the defin ition o f U K . Combining (D.3), (D.1 1), and (D.13), we ob tain R ≤ lim inf N →∞ min p Y | X K ∈ W K min A⊆K J L w ( N ) ,L u ( N ) , A ( p S , p X K U K W | S , p Y | X K ) ( a ) ≤ sup L w ,L u min p Y | X K ∈ W K min A⊆K J L w ,L u , A ( p S , p X K U K W | S , p Y | X K ) ≤ sup L w ,L u max p X K U K W | S ∈ P X K U K W | S ( p S ,L w ,L u ,D 1 ) min p Y | X K ∈ W K min A⊆K J L w ,L u , A ( p S , p X K U K W | S , p Y | X K ) ( b ) = sup L w ,L u C all L w ,L u ( D 1 , W K ) = lim L w ,L u →∞ C all L w ,L u ( D 1 , W K ) ( c ) = C all ( D 1 , W K ) , (D.14) where (a) holds be cause the functiona ls J L w ,L u , A ( · ) are nondec reasing in L w , L u , (b) uses the definition of C all L w ,L u in (6.2), an d (c) the fact that the sequen ce { C all L w ,L u } is nondec reasing. Nov ember 5, 2018 DRAFT 34 Proof of (D.12) . Reca ll the definitions of V K ,i = ( M K , V , S N i +1 ) and U K ,i = ( V K ,i , ( Y S d ) i − 1 ) in (D.5) and the rec ursion (D.6) for V K ,i . W e p rove the following inequality: I ( U A ,i ; Y i S d i | U K\A ,i ) − I ( U A ,i ; S i | U K\A ,i ) = [ I ( V A ,i ; ( Y S d ) i | V K\A ,i ) − I ( V A ,i ; S i | V K\A ,i )] − [ I ( V A ,i − 1 ; ( Y S d ) i − 1 | V K\A ,i − 1 ) − I ( V A ,i − 1 ; S i − 1 | V K\A ,i − 1 )] . (D.15) Then summing b oth sides of this equ ality from i = 2 to N , c ancelling terms, and using the p roperties V k , 1 = U k , 1 and V k ,N = ( M k , V ) yields (D.1 2). The first o f the six terms in (D.15) may be expande d a s follows: I ( U A ,i ; Y i S d i | U K\A ,i ) = I ( V A ,i , ( Y S d ) i − 1 ; Y i S d i | V K\A ,i , ( Y S d ) i − 1 ) = I ( V A ,i ; Y i S d i | V K\A ,i , ( Y S d ) i − 1 ) = I ( V A ,i , ( Y S d ) i − 1 ; Y i S d i | V K\A ,i ) − I (( Y S d ) i − 1 ; Y i S d i | V K\A ,i ) = I ( U A ,i ; Y i S d i | V K\A ,i ) − I (( Y S d ) i − 1 ; Y i S d i | V K\A ,i ) . (D.16) Similarly for the seco nd term, replacing ( Y S d ) with S in the a bove deriv ation, we obtain I ( U A ,i ; S i | U K\A ,i ) = I ( U A ,i ; S i | V K\A ,i ) − I (( Y S d ) i − 1 ; S i | V K\A ,i ) . (D.17) The s ix terms in (D.15) ca n be expande d using the ch ain rule for mutual information, in the sa me way as in [20 , Lemma 4.2]: I ( V A ,i ; ( Y S d ) i | V K\A ,i ) = I ( V A ,i ; ( Y S d ) i − 1 | V K\A ,i ) + I ( V A ,i ; ( Y S d ) i | V K\A ,i ) (D.18) I ( V A ,i ; S i | V K\A ,i ) = I ( V A ,i ; S i − 1 | V K\A ,i ) + I ( V A ,i ; S i | V K\A ,i ) (D.19) I ( V A ,i − 1 ; S i − 1 | V K\A ,i − 1 ) = I ( V A ,i ; S i − 1 | S i , V K\A ,i − 1 ) (D.20) I ( V A ,i − 1 ; ( Y S d ) i − 1 | V K\A ,i − 1 ) = I ( V A ,i ; ( Y S d ) i − 1 | S i , V K\A ,i − 1 ) (D.21) I ( U A ,i ; S i | V K\A ,i ) = I (( Y S d ) i − 1 ; S i | V K\A ,i ) + I ( V A ,i ; S i | ( Y S d ) i − 1 , V K\A ,i ) (D.22) I ( U A ,i ; ( Y S d ) i | V K\A ,i ) = I (( Y S d ) i − 1 ; ( Y S d ) i | V K\A ,i ) + I ( V A ,i ; ( Y S d ) i | ( Y S d ) i − 1 , V K\A ,i ) . (D.23) Moreover , expanding the con ditional mutual information I ( V A ,i ; S i , ( Y S d ) i − 1 | V K\A ,i ) in two diff erent ways, we obtain I ( V A ,i ; ( Y S d ) i − 1 | V K\A ,i ) + I ( V A ,i ; S i | ( Y S d ) i − 1 , V K\A ,i ) = I ( V A ,i ; S i − 1 | V K\A ,i ) + I ( V A ,i ; ( Y S d ) i − 1 | S i , V K\A ,i ) . (D.24) Nov ember 5, 2018 DRAFT 35 Substracting the sum of (D.17), (D.18), (D.20), (D.22), (D.24) from the sum of (D.16), (D.19), (D.21 ), (D.23), an d cance lling terms, we obtain (D.15), from which the claim follows. ✷ R E F E R E N C E S [1] D. Boneh and J. Shaw , “Collusion–Secure Fingerprinting for Digital Data, ” in Advances in Cryptology: Pro c. CRYPTO’95 , Springer–V erlag, New Y ork, 1995. [2] I. J. Cox, J. Ki llian, F . T . Leighton and T . Shamoon, “Secure S pread Spectrum W atermarking for Multimedia, ” IEEE Tr ans. Imag e Proc. , V ol. 6, N o. 12, pp. 1673—16 87, Dec. 1997. [3] M. W u, W . T rappe, Z. J. W ang and K. J. R. Liu, “Collusion-Resistant Fi ngerprinting for Multimedia, ” IEEE Signal Pr ocessing Magazine , V ol. 21, No. 2, pp. 15—27, March 2004. [4] K. J. R. Li u, W . Trappe , Z . J. W ang, M. W u and H. Z hao, Multimedia Fin gerprinting F or ensics for Tr aitor Tr acing , EURASIP Book Series on Signal Processing, 2006. [5] P . Moulin and A. Briassouli, “The Gaussian Fingerprinting Game, ” Pro c. Conf . Information Scien ces and Systems , Princeton, NJ, March 2002. [6] P . Moulin and J. A. O’Sulliv an, “Information-theoretic analysis of information hiding, ” IEEE T rans. on Information Theory , V ol. 49, No. 3, pp. 563—593 , March 2003. [7] A. Somekh-Baruch and N. Merhav , “On the capacity game of priv ate fingerprinting systems under collusion attacks, ” IEEE T rans. Information Theory , vol. 51, no. 3, pp. 884—899, Mar . 2005. [8] A. Somekh-Baruch and N. Merhav , “ Achie v able error expon ents for the priv at e fingerprinting game, ” IEEE T rans. Information Theory , V ol. 53, No. 5, pp. 1827—1838, May 2007. [9] N. P . Anthapadm anabhan, A. Barg and I. Dumer , “On the F ingerprinting Capacity Under the Marking Assumption, ” submitted to IE EE T rans. Information Theory , arXiv:cs/0612 073v2, July 2007. [10] P . Moulin, “Univ ersal Fingerprinting: Capacity and R andom-Coding E xponents, ” submitted to IEEE T rans. Information Theory . A v ailable fr om arXiv:0801.383 7 v1 [cs.IT] 24 Jan 2008. [11] P . Moulin and Y . W ang, “Capacity and Random-Coding E xponen ts for Channel C oding with Side Information, ” IEEE T rans. on Information Theory , V ol. 53, No. 4, pp. 1326—1347, Apr . 2007. [12] Y .-S. Li u and B. L. Hughes, “ A ne w uni versal random coding bou nd for the multiple-access channel, ” IE EE T rans. Information Theory , vo l. 42, no. 2, pp. 376–386, Mar . 1996. [13] A. Somekh-Baruch and N. Merhav , “On t he R andom C oding Error E xponents of the Single-User and t he Multiple-Access Gel’fand-Pinsk er Channels, ” Pr oc. IEEE Int. Symp. Info. Theory , p. 448, Chicago, IL, June-July 2004. [14] Y . W ang and P . Moulin, “Capacity and Random-Coding E rror Exponent for P ublic Fingerprinting Game, ” P r oc. Int. Symp. on Information T heory , Seattle, W A, July 2006. [15] Y . W ang, Detection- and Information-Theor etic A nalysis of Ste gano grap hy and F ingerprinting, Ph. D. T hesis, ECE Department, Univ ersity of Illinois at Urbana-Champaign, Dec. 2006 . [16] G. T ardos, “Optimal Probabilistic Fingerprinting Codes, ” STOC , 2003. [17] R. Ahlswede, “Multiway Communication Channels, ” Pr oc. ISIT , pp. 23—52, Tsahkadsor , Armenia, 1971. [18] H. Liao, “Multiple Access Channels, ” Ph. D. dissertation , EE Department, U. of Hawaii, 1972. [19] A. Lapidoth and P . Narayan, “Reliable Communication Under Channel Uncertainty , ” IEE E T rans. Information T heory , V ol. 44, No. 6, pp. 2148—21 77, Oct. 1998. Nov ember 5, 2018 DRAFT 36 [20] S. I. Gel’fand and M. S. P insker , “Coding for Channel with Random Parameters, ” Prob lems of C ontr ol and I nformation Theory , V ol. 9, No. 1, pp. 19—31, 1980. [21] I. Csisz ´ ar and J. K ¨ o rner , Information Theory: Coding Theory f or Discr ete Memoryless Systems , Academic Press, NY , 1981. [22] A. Das and P . Narayan, “Capacities of T ime-V arying Multiple-Access Channels With Si de Information, ” IEEE T rans. Information Theory , V ol. 48, No. 1, pp. 4—25, Jan. 2002. [23] S. Sigurj ´ onsson and Y .-H. Kim, “On Multiple User Channels with State Information at the Transmitters, ” Pro c. ISIT 2005. [24] N. T . Gaarder and J. K. W olf, “The Capacity Region of a Multiple-Access Discrete Memoryless Channel Can Increase with Feedback, ” IE EE T rans. Information Theory , V ol. 21, No. 1, pp. 100—102, Jan. 1975. Nov ember 5, 2018 DRAFT