An approximation trichotomy for Boolean #CSP

An appro ximatio n tric hotom y for Bo olean #CSP ∗ Martin Dy er Sc ho ol of Computing Univ ersit y of Leeds Leeds LS2 9JT, UK Leslie Ann Goldb erg Departmen t of Computer Science, Univ ersit y of Liv erp o ol, Liv erp o o l L69 3BX, UK Mark Jerrum Sc ho ol of Mathematical Sciences, Queen Mary , Univ ersit y of London Mile End Road, London E1 4NS, UK Octob er 31, 2018 Abstract W e giv e a trichotom y theorem for the complexit y of a pproximately counting the num b er of sa tisfying assig nment s o f a Bo ole a n CSP instance . Such pr oblems are parameterised b y a constraint language sp ecifying the relations that may be used in constra ints. If every re lation in the con- straint language is a ffine then the num b e r of satisfying ass ig nments can be exactly counted in po lynomial time. Otherwise, if every rela tio n in the constra int la ng uage is in the co -clone I M 2 from Post’s lattice, then the problem of counting satisfying assig nment s is co mplete with r esp ect to a pproximation-preserving r e ductio ns for the complexity class #RHΠ 1 . This means that the pr oblem of approximately counting satisfying as- signments of such a CSP insta nce is e q uiv alen t in complexity to several other known counting pr o blems, including the pro blem of approximately counting the num b er o f independent sets in a bipa rtite gr aph. F or every other fixed constraint langua ge, the pr oblem is complete for #P with re- sp ect to approximation-preserv ing reductions, meaning tha t there is no ful ly p olynomial r andomise d appr oximation scheme for counting satisfying assignments unless NP= RP . 1 In tro duc tion This pap er giv es a tric hotom y th eorem for the complexit y of appro ximately coun ting the num b er of satisfying assignm ents of a Bo olean CSP instance. Su c h problems are parameterised by a constrain t language Γ w hic h sp ecifies r elations that may b e used in constraint s. In the Boolean case, th e relations are on a domain whic h has t wo elemen ts. Then #CSP (Γ) will denote the problem of ∗ This work was partially supp orted by the EPSRC grant “The Complexit y of Counting in Constrain t Satisfaction Problems” 1 determining the n umber of (distinct) satisfying assignmen ts of a CS P in stance with constrain t language Γ. F urther d etails are giv en in Section 1.1 b elo w . Creignou an d Hermann [6] ha ve giv en a dic hotom y theorem for the exact coun ting problem. They hav e shown th at if ev ery relation in Γ is affin e, then #CSP (Γ) is in FP . Otherwise, it is # P -complete. The complexit y classes F P and #P are the analog ues of P an d NP for coun ting problems. F P is the class of functions computable in deterministic p olynomial time. #P is the class of in teger functions that can b e expressed as the n umb er of acc epting computations of a p olynomial-time n on-deterministic T uring mac hine. In this pap er we build on previous w ork on the complexit y of appro ximate coun ting to iden tify a trichoto my in the complexit y of appro ximate coun ting for Bo olean #CSP . T ogether wit h Greenhill [9], we ha v e previously studied app ro ximation- preserving red u ctions (AP-reductions) b et w een coun ting p roblems. W e will giv e details of AP-reductions in Section 1.2. F or no w it suffices to note that if an AP-redu ction exists fr om a counti ng prob lem f to a coun ting problem g and g has a F ul ly Polynomial R andomise d Appr oximation Scheme (FPRAS) th en f also has an FPRAS. If an AP-reduction from f to g exists we write f ≤ AP g , and sa y that f is A P -r e ducible to g . If f ≤ AP g and g ≤ AP f then we say that f and g ar e AP-interr e ducible , and write f = AP g . W e pr eviously identified [9 ] three natural classes of counting problems that are in terreducible und er AP-reductions. These are (i) those problems that ha v e an FPRAS, (ii) those pr oblems that are complete for #P with resp ect to AP-reducibilit y , and a t hir d c lass of intermediate complexit y . Tw o counting problems pla yed a sp ecial role in [9 ]. Name. #SA T . Instanc e. A Bo olean form ula ϕ in conjunctive norm al form. Output. T he n umb er of satisfying assignments of ϕ . Name. #BIS . Instanc e. A bipartite graph B . Output. T he n umb er of indep enden t sets in B . All pr oblems in #P are AP-red u cible to # SA T (see [9, Section 3]). Thus #SA T is complete for #P w ith resp ect to AP-r ed ucibilit y . This means that #SA T cannot ha ve an FPRAS unless NP = RP. The same is true of any problem in #P to wh ic h #SA T is AP-reducible. W e show ed in [9, Sections 4, 5] that #BIS is AP-int erredu cible with m any other n atural coun ting problems s uc h as counting do wnsets in a partial order. Moreo ver, #BIS is complete for #RHΠ 1 , a logically-defined sub class of #P , with resp ect to AP-reductions. The m ain theorem of our current p ap er (Theorem 3) sho ws that eve ry prob- lem #CSP (Γ) falls neatly in to one of the three classes from [9]: If ev ery relation 2 in Γ is affine, then trivially #CSP (Γ) has an FPRAS since it is in FP . Other- wise, if ev ery relation in Γ is in a certain set I M 2 , then #CS P (Γ) = AP #BIS . Otherwise #CSP (Γ) = AP #SA T . A formal definition of I M 2 app ears in Sec- tion 1.4 — it is the set of relations which can b e expressed as conjunctions in vo lving only binary implication and unary r elations. It is worth p ointing out that, w hile ev ery problem #CS P (Γ) falls in to on e of the three approximat ion classes fr om [9], the three classes m a y well not pro vide a partition of all appro ximate counting problems in #P . F or example, the problem of appro ximately counting 3-colourings of a b ipartite graph is a problem that ma y well lie b et w een #BIS and #SA T in approxima bility (see [9 ]). 1.1 Constrain t satisfaction Constr aint Satisfaction , which originated in Artificial Inte lligence, pro vides a general f ramew ork for mo delling decision problems, and has many practical applications. (See, for example [18].) Deci sions are mo delled b y variables , whic h are sub ject to c onstr aints , mo delling logical and resour ce restrictions. The paradigm is sufficien tly broad that man y inte resting problems can b e mo delled, from satisfiability problems to sc hed u ling problems and graph-theory problems. Understanding the complexit y of constraint satisfaction p roblems h as b ecome a ma jor and activ e area within computational complexit y [7, 11]. A Constr ain t Satisfaction Pr ob lem (CSP) typical ly has a finite domain , whic h we d enote b y { 0 , . . . , q − 1 } for a p ositiv e inte ger q . In this pap er w e are intereste d in the Bo ole an case q = 2. A c onstr aint language Γ with d omain { 0 , . . . , q − 1 } is a set of relations on { 0 , . . . , q − 1 } . F or example, tak e q = 2. The relation R = { (0 , 0 , 1), (0 , 1 , 0), (1 , 0 , 0), (1 , 1 , 1) } is a 3-ary relation on the domain { 0 , 1 } , with fou r tuples. Once we ha v e fixed a constraint language Γ , an instanc e of the CSP is a set of variables V = { v 1 , . . . , v n } and a set of c onstr aints . Eac h constrain t has a sc op e, whic h is a tuple of v ariables (for example, ( v 4 , v 5 , v 1 )) an d a relation fr om Γ of the same arit y , which constrains the v ariables in th e scop e. An assignment σ is a function from V to { 0 , . . . , q − 1 } . Th e assignmen t σ is satisfying if the scop e of ev ery constrain t is m app ed to a tuple that is in the corresp onding relation. In our example ab o v e, an assignment σ satisfies the constraint with scop e ( v 4 , v 5 , v 1 ) and r elation R , written R ( v 4 , v 5 , v 1 ), if and only if it maps an o dd num b er of the v ariables in { v 1 , v 4 , v 5 } to the v alue 1. Giv en an instance I of a CSP with constraint language Γ, the de cision pr oblem CS P (Γ) asks us to determine w hether an y assignment satisfies I . Th e c ounting pr oblem # CSP (Γ) asks us to determine the numb e r of (distinct) satisfying assignmen ts of I , whic h w e will denote by #csp ( I ). V arying the constrain t language Γ defines the classes CSP and #CSP of decision and counti ng problems. These con tain problems of different com- putational complexities. F or example, consider the binary relations defined b y OR = { (0 , 1) , (1 , 0) , (1 , 1) } , Im pli es = { (0 , 0) , (0 , 1) , (1 , 1) } , and NAND = { (0 , 0) , (0 , 1) , (1 , 0) } . If Γ = { OR , Implies , NAND } th en CSP (Γ) is the classical 2-Satisfiabilit y pr ob lem, which is in P . O n the other hand , there is a similar con- strain t language Γ ′ with four relations of arit y 3 suc h that 3-Satisfiabilit y (which 3 is NP -complete) can b e represen ted in CS P (Γ ′ ). It ma y happ en, as here, th at the coun ting pr ob lem is h ard er than the decision problem: #CSP (Γ) con tains the problem of counting in dep end en t sets in graph, and is thus # P -complete. An y decision problem CSP (Γ) is in NP , but not every pr ob lem in NP can b e represent ed as a CS P . F or example, the question “Is G Hamiltonian?” cann ot b e exp ressed as a CS P , b ecause the prop ert y of b eing Hamiltonian cannot b e captured by r elations of b ounded size. Th is limitation of the class CS P has an imp ortant adv an tage. If P 6 = NP , then there are problems w hic h are neither in P nor NP -complete [15]. But, f or w ell-b ehav ed smaller classes of decision problems, the situation can b e simp ler. W e ma y hav e a dichotomy the or em , partitioning all problems in th e class in to those whic h are in P and those whic h are NP -complete. There are no “lefto v er” problems of intermediate complex- it y . It has b een conjectured th at there is a d ic hotom y theorem for CS P . Th e conjecture is that CSP (Γ) is in P for some constrain t languages Γ, and CSP (Γ) is NP -complete for all other constraint languages Γ. Th is conjecture app eared in a seminal pap er of F eder and V ardi [13], but has n ot yet b een prov ed. A similar dic hotom y , b et w een FP and #P -complete, is conjectured for # CSP [4]. Recen tly , Bulato v [3 ] has announced a p ositiv e resolution of this conjecture. There h av e b een many imp ortan t results for sub classes of CSP and #CSP . W e mentio n the most relev ant to our pap er here. T he first decision dic hotomy w as that of Schaefer [19], for the Bo olean domain { 0 , 1 } . S c haefer’s result is as follo w s. Theorem 1 (Sc haefer [19]) . L et Γ b e a c onstr aint language with domain { 0 , 1 } . The pr oblem CS P (Γ) is in P if Γ satisfies one of the c onditions b elow. Other- wise, CSP (Γ) i s NP -c omplete. (i) Γ is 0 -valid or 1 -valid. (ii) Γ is we akly p ositive or we akly ne gative. (iii) Γ i s affine. (iv) Γ is bijunctive . W e will not giv e detailed definitions of the conditions in Theorem 1, bu t the int erested reader is referred to the pap er [19] or to Theorem 6.2 of the textb o ok [7]. An in teresting feature is that the conditions in [7, Theorem 6.2] are all c hec k able. T h at is, there is an algo rithm to determine whether CSP (Γ) is in P or NP -complete, given a constrain t language Γ w ith domain { 0 , 1 } . W e sa y in this case that the dic hotom y is effe ctive . A Bo olean relation R is said to b e affine if the set of tup les x ∈ R is the set of solutions to a system of linear equations o ve r GF(2). Creignou and Hermann [6 ] adapted S c haefer’s decision dic hotom y to obtain a counting dichot omy for the Bo olean domain. Their result is as follo ws. Theorem 2 (Cr eignou and Hermann [6]) . L et Γ b e a c onstr aint language with domain { 0 , 1 } . The pr oblem # CSP (Γ) i s in FP if eve ry r elation in Γ is affine. Otherwise, #CSP (Γ) is # P -c omplete. Creignou and Herman n ’s resu lt is an imp ortant starting p oin t for our work, and w e will discuss it f urther b elo w. Note that th ere is an algorithm for deter- mining whether a r elation is affine, so the dichot omy is effectiv e. 4 W e ha v e recen tly [10] extended C reignou and Hermann’s dic hotom y to the domain of weighte d Bo olean #CSP giving an effectiv e dic h otom y b etw een FP and FP #P for the problem of computing the p artition function of a we ighte d Bo olean CSP instance. 1.2 The complexit y of appro ximate coun ting W e no w recall th e necessary bac kground from [9 ]. A r andomise d appr oxima- tion scheme is an algorithm for appr o ximately computing the v alue of a func- tion f : Σ ∗ → N . Th e appro ximation sc heme h as a parameter ε > 0 whic h sp ecifies the error tolerance. A r ando mise d appr oximatio n scheme for f is a randomised algorithm that take s as input an instance x ∈ Σ ∗ (e.g., an enco din g of a CSP instance) and an error tolerance ε > 0, and outp uts an in teger z (a random v ariable on the “coin tosses” made by the algorithm) such that, for ev ery instance x , Pr  e − ε f ( x ) ≤ z ≤ e ε f ( x )  ≥ 3 4 . (1) The ran d omised appro ximation scheme is said to b e a ful ly p olynomial r an- domise d appr oximatio n scheme , or FPRAS , if it run s in time b ounded by a p olynomial in | x | and ε − 1 . (See Mitzenmac her and Up f al [16, Definition 10.2].) Note that the q u an tit y 3 / 4 in Equ ation (1) could b e changed to any v alue in the op en in terv al ( 1 2 , 1) with ou t changing the set of problems th at h av e r andomised appro ximation schemes [14, Lemma 6.1]. Supp ose that f and g are fun ctions fr om Σ ∗ to N . An “approximati on- preserving r eduction” (AP-reduction) from f to g giv es a wa y to turn an FPRAS for g into an FPRAS for f . An AP-reduction from f to g is a randomised algorithm A for co mp u ting f using an oracle f or g 1 . The algo rithm A tak es as input a pair ( x, ε ) ∈ Σ ∗ × (0 , 1), and satisfies the follo wing three conditions: (i) ev ery oracle call made by A is of the f orm ( w , δ ), where w ∈ Σ ∗ is an instance of g , and 0 < δ < 1 is an err or b ound s atisfying δ − 1 ≤ p oly( | x | , ε − 1 ); (ii) the algorithm A meets the sp ecification for b eing a randomised appro ximation sc heme for f (as describ ed ab o ve ) w henev er the oracle meets the sp ecificati on for b eing a rand omised approximat ion scheme for g ; and (iii) the r un-time of A is p olynomial in | x | and ε − 1 . In form ulating a d efinition of app ro ximation- presering redu ction, a num b er of choice s must b e faced. Th e k ey r equ iremen t is that the class of f u nctions computable by an FPRAS should b e closed under AP- reducibilit y . Informally , we hav e gone for the most lib er al notion of red uction meeting this requir emen t. 1.3 Notation for relations Define the un ary relations δ 0 = { (0) } and δ 1 = { (1) } . Recall the bin ary relation Implies = { (0 , 0) , (0 , 1) , (1 , 1) } . F or con venience , according to con text, w e view a k -ary r elation R either as a set of k -tuples or as a k -ary predicate. Thus the notations R ( x 1 , . . . , x k ) = 1 (or 1 The reader who is not familiar with oracle T uring machines can just think of th is as an imaginary (u nwritten) subroutine for compu ting g . 5 just R ( x 1 , . . . , x k )) and ( x 1 , . . . , x k ) ∈ R are equiv alen t. F or example, δ 0 ( x ) = x , δ 1 ( x ) = x and Impli es ( x, y ) = x ∨ y . 1.4 The set of relations I M 2 An n -ary relation R is in I M 2 if and only if R ( x 1 , . . . , x n ) is logically equiv alent to a conjunction of p redicates of the form δ 0 ( x i ), δ 1 ( x i ) and Implies ( x i , x j ). As we will d iscuss b elo w, C reignou, K olaitis, and Zanuttini [8] h a v e sho wn that I M 2 is a co-clone in Post’s lattice (see [2]). 1.5 Our result W e can n ow state our main theorem. Theorem 3. L et Γ b e a c onstr aint language with domain { 0 , 1 } . If every r ela- tion in Γ is affine then #CSP (Γ) is in FP . Otherwise i f every r elation i n Γ is in I M 2 then #CSP (Γ) = AP #BIS . Otherwise #CSP (Γ) = AP #SA T . The main ingredients in the p ro of are: (1) the AP-reduction tec hnology of [9], which allo w s us to effecti vel y “pin” certain CSP v ariables in hardness pro ofs (see Section 2.3); (2) the “implementat ions” of Cr eignou, Khanna and Sudan [7], which sho w h o w to construct the k ey r elations OR , Implies , and NAND from a non-affine relation and δ 0 or δ 1 (see Section 2.5); (3) the complexit y class #RHΠ 1 from [9], consisting of th ose problems wh ich are AP-in terredu cible with #BIS ; and (4) the co-clone I M 2 in P ost’s lattice (see Section 2.8), since th e complexit y of #CSP (Γ) for Γ ⊆ I M 2 turns out to b e closely connected to th e complexit y of #B IS . 2 The pieces of the pro of 2.1 T yp es of relations A r elation R is 0-valid if the all-zero tup le is in R . Similarly , R is 1-valid if the all-ones tuple is in R . F ollo wing [7 ], w e sa y that a k -ary r elation R is c omplement-close d (C-closed in [7]) if ( x 1 , . . . , x k ) ∈ R ⇔ ( x 1 ⊕ 1 , . . . , x k ⊕ 1) ∈ R, where ⊕ is the exclusive or op erator. W e say that Γ is 0-v alid if ev ery R ∈ Γ is 0-v alid and w e defi ne what it means for Γ to b e 1-v alid or complemen t-closed similarly . 2.2 Some preliminary complexity results W e start by observing that ev ery problem #CSP (Γ) is AP-redu cible to #S A T Observ ation 4. L et Γ b e a c onstr aint language with domain { 0 , 1 } . Then #CSP (Γ) ≤ AP #SA T . 6 Observ ation 4 follo ws from the fact that all pr oblems in # P are AP- reducible to #S A T [9]. Another, v ery simple, but useful, observ ation is th e follo w ing. Observ ation 5. L et Γ b e a c onstr aint language with do main { 0 , 1 } . Supp ose Γ ′ ⊆ Γ . Then #CSP (Γ ′ ) ≤ AP #CSP (Γ) . Observ ation 5 is true for the simple reason th at ev ery instance of #CSP (Γ ′ ) is an in s tance of #CSP (Γ). Recall the r elations OR = { (0 , 1) , (1 , 0 ) , (1 , 1) } and NAND = { (0 , 0) , (0 , 1) , (1 , 0) } . T h ese r elations are particularly fundament al for us, and we start with complexit y results ab out these. Lemma 6. #SA T ≤ AP #CSP ( { NAND } ) . Pr o of. It was shown in [9 ] that the follo w ing p r oblem is AP-int erredu cible w ith #SA T . Name. #IS . Instanc e. A graph G . Output. T he n umb er of indep enden t sets in G . W e s ho w that #IS ≤ AP #CSP ( { NAND } ). Let G = ( V , E ) b e an instance of #IS . Cons tr uct an in stance I of #CSP ( { NAND } ) w ith v ariable set V . F or ev ery edge ( u, v ) ∈ E , add constrain t NAND ( u, v ). T h ere is no w a bijectio n b et wee n indep endent sets of G and satisfying assig nm en ts σ of I : v ariables v with σ ( v ) = 1 corresp ond to v ertices in the ind ep endent set. Lemma 7. #SA T ≤ AP #CSP ( { OR } ) . Pr o of. The p ro of that #IS ≤ AP #CSP ( { OR } ) is similar (ju s t asso ciate v ariables v with σ ( v ) = 1 w ith v ertices th at are out of the indep endent set). Finally , w e will need a couple of complexit y results inv olving #BIS . Lemma 8. #BIS ≤ AP #CSP ( { Implies } ) . Pr o of. Let G b e an instance of #BIS with verte x s ets U and V and edge set E . Construct an instance I of #CSP ( { Implies } ) with v ariable set U ∪ V . F or every edge ( u, v ) ∈ E with u ∈ U add constrain t Implies ( u, v ). There is now a b ijection b et wee n indep endent sets of G and satisfying assignmen ts σ of I : a v ariable u ∈ U with σ ( u ) = 1 is in th e ind ep endent set and a v ariable v ∈ V with σ ( v ) = 0 is in th e indep end en t set. Lemma 9. Supp ose Γ ⊆ I M 2 . Then #CSP (Γ) ≤ AP #BIS . 7 Pr o of. It is straigh tforward to sh o w that #CSP (Γ) is in th e complexit y class #RHΠ 1 whic h has #B IS as a complete pr oblem [9]. Ho w ev er, to av oid giving a defin ition of #RHΠ 1 , whic h requires some nota- tion, we will instead sho w #CS P (Γ) ≤ AP #Do wns ets , wh ere #Downsets is the follo wing counting problem w h ic h w as sho wn in [9] to b e AP-in terredu cible with #BIS . Name. #Do wns ets . Instanc e. A partially order ed set ( X,  ). Output. T he n umb er of do wnsets 2 in ( X,  ). Consider an instance I of #CSP (Γ) with v ariables v 1 , . . . , v n . The set of constrain ts can b e view ed as an equiv alen t set of constraint s of the form δ 0 ( v i ), δ 1 ( v i ) or Implies ( v i , v j ). Denote b y Implies ∗ the transitiv e closure of the Implies relation on { v 1 , . . . , v n } : thus Implies ∗ ( v i , v j ) if there is a sequence of v ariables, starting with v i and ending with v j , suc h th at ev ery adjacen t pair in the se- quence is constrained b y Implies . Let N 0 ( I ) b e the s et of v ariables v i for whic h either (i) a constrain t δ 0 ( v i ) o ccurs in I , or (ii) there exists a v ariable v j suc h that Implies ∗ ( v i , v j ) and a constrain t δ 0 ( v j ) o ccur s in I . These are the v ariables that are forced to b e 0 in any satisfying assignment of I . Define N 1 ( I ) analogously to b e the set of v ariables that are forced to b e 1 in an y s atisfying assignmen t. W e can assume without loss of generalit y that N 0 ( I ) and N 1 ( I ) are disjoint. Otherwise th e instance I h as no satisfying assignments, and we can determine this without ev en us in g the do wn s ets oracle. No w remo ve all the v ariables in N 0 ( I ) and N 1 ( I ) from the instance I : this do es not affect the num b er of s atisfying assignmen ts, since these v ariables d o not constrain an y of the others. Also iden tify all p airs of v ariables v i , v j suc h that Implies ∗ ( v i , v j ) and Implies ∗ ( v j , v i ): again, this do es not affect the n umb er of satisfying assignments. The r emaining v ariables and relations define a partial order ( X,  ) since our construction forces an tisymmetry . The satisfying assignmen ts of I corresp ond 1–1 with the downsets of ( X ,  ). 2.3 A useful to ol: pinning Pinning is the abilit y to tie certain C SP v ariables to sp ecific v alues in hard- ness pr o ofs. Th is idea w as used b y C reignou and Hermann in th eir dic hotom y theorem [6]. S imilar ideas hav e b een used in man y other hardn ess p ro ofs and dic hotom y theorems [4, 5, 10, 12]. A s we s ho w in th is section, AP-reductions facilitat e a particularly useful form of pinnin g. Lemma 10. L et Γ b e a c onstr aint language with domain { 0 , 1 } . Supp ose ther e is a r elation R ∈ Γ for which, for some p osition j , R has mor e tuples t with t j = 0 th an with t j = 1 . Then #CSP (Γ ∪ { δ 0 } ) ≤ AP #CSP (Γ) . Similarly, i f 2 A downset in ( X ,  ) is a subset D ⊆ X that is closed und er  ; i.e., x  y and y ∈ D implies x ∈ D . 8 ther e i s a r elation R ∈ Γ for which, for som e p osition j , R has mor e tuples t with t j = 1 than with t j = 0 then #CSP (Γ ∪ { δ 1 } ) ≤ AP #CSP (Γ) . Pr o of. Consider an instance I of #CSP (Γ ∪ { δ 0 } ) w ith n v ariables. Sup p ose there is an arit y- k relation R ∈ Γ for wh ic h, for p osition j , R h as w tu ples t with t j = 0 and w ′ < w tuples t with t j = 1. As in the pro of of Lemma 9, let N 0 ( I ) b e the set of v ariables x to which one or more constrain ts δ 0 ( x ) o ccurs in I and let N 1 ( I ) b e the set of v ariables y to whic h one or more constrain ts δ 1 ( y ) o ccurs. Let n 0 = | N 0 ( I ) | . Let m = ⌈ ( n + 2) / lg( w /w ′ ) ⌉ . Constru ct an instance I ′ of #CSP (Γ). Include all constrain ts in I other than th ose in vo lving δ 0 . F or eac h v ariable x ∈ N 0 ( I ), and ev ery a ∈ { 1 , . . . , m } , in tro du ce k − 1 new v ariables x ′ a,b for b ∈ { 1 , . . . , k } − { j } . In tro d uce a new constrain t in I ′ with relation R and v ariable x in the j th p osition, and x ′ a,b in the b th p osition, for all b . No w a satisfying assignment for I can b e extended in w mn 0 w a ys to satisfying assignmen ts of I ′ . An assignmen t for I that violates one of the δ 0 ( x ) constraints can b e extended in at most w m ( n 0 − 1) w ′ m w a ys to satisfying assignments of I ′ . Th us , #csp ( I ) w mn 0 ≤ #csp ( I ′ ) ≤ #csp ( I ) w mn 0 + 2 n w m ( n 0 − 1) w ′ m , i.e., #csp ( I ) ≤ #csp ( I ′ ) w mn 0 ≤ #csp ( I ) + 2 n ( w ′ /w ) m . So, by defin ition of m , #csp ( I ) ≤ #csp ( I ′ ) w mn 0 ≤ #csp ( I ) + 1 4 . Th us we hav e constructed a red u ction from # CSP (Γ ∪ { δ 0 } ) to #CSP (Γ): Giv en an instance I of #CS P (Γ ∪ { δ 0 } ), u se an oracle for #CSP (Γ) to app ro xi- mate #csp ( I ′ ), d ivide by w mn 0 , and round to the nearest int eger (alw a ys down). Note that th e reduction mak es only one oracle call (and uses no rand omisation). T o sho w that the reduction is indeed an AP-reduction, we add some tec hni- cal details concerning th e c hoice of the accuracy parameter δ in the oracle call (see the definition of AP-redu ction in S ection 1.2). These details are here to mak e the pro of complete, but they are not essen tial for un derstanding th e rest of the pap er. If w e h ad #csp ( I ) = #csp ( I ′ ) w mn 0 , w e could simply set δ = ε , since division by a constan t pr eserves relativ e err or. Instead w e h a v e #csp ( I ) =  #csp ( I ′ ) w mn 0  . The discontin u ous flo or function could sp oil th e approxima tion when its argu- men t is small. The situation h ere is that the tru e answe r N = #csp ( I ) is obtained by rounding the fraction Q = #csp ( I ′ ) w mn 0 where we hav e | Q − N | ≤ 1 / 4. 9 Supp ose that the oracle pro vides an appro ximation b Q to Q satisfying Qe − δ ≤ b Q ≤ Qe δ (as it is required to do w ith probabilit y at least 3 / 4). Set δ = ε/ 21, where ε is th e accuracy parameter go verning the final result. There are t w o cases. If N ≤ 2 /ε , th en a sh ort calculation yields | b Q − Q | < 1 / 4 implying that the result returned by the algorithm is exact. If N > 2 /ε , then the result returned is in th e range [( N − 1 / 4) e − δ − 1 / 2 , ( N + 1 / 4) e δ + 1 / 2] wh ich, for the c hosen δ , is con tained in [ N e − ε , N e ε ]. Th us , we ha ve an AP-redu ction f rom #CSP (Γ ∪ { δ 0 } ) to #CSP (Γ). The reduction sho wing #CSP (Γ ∪ { δ 1 } ) ≤ AP #CSP (Γ) is similar. 2.4 Affine relations W e use the follo win g w ell-kno wn facts ab out affin e relations. Lemma 11. (i) A k -ary Bo ole an r elation R is affine if and only if a, b, c ∈ R implies d = a ⊕ b ⊕ c ∈ R , wher e the ⊕ op er ator is applie d c omp onentwise. (ii) If R is not affin e, then for any fixe d a ∈ R ther e ar e b, c ∈ R such that a ⊕ b ⊕ c 6∈ R . (iii) If R i s not affine, then ther e ar e a, b in R such that a ⊕ b 6∈ R . Pr o of. F or P art (i) see, for example, Lemma 4.10 of [7]). Pa rt (ii) is pro v ed in the same place, bu t since it is a little less w ell-kno w n, we provide the pro of: Supp ose the con trary that R is not affine, bu t for all b, c ∈ R , a ⊕ b ⊕ c ∈ R . Cho ose s 0 , s 1 , s 2 ∈ R suc h th at s 0 ⊕ s 1 ⊕ s 2 6∈ R . F r om b = s 0 , c = s 1 , d = a ⊕ s 0 ⊕ s 1 w e ha ve d ∈ R . F rom b = s 2 , c = d we hav e a ⊕ s 2 ⊕ d = s 0 ⊕ s 1 ⊕ s 2 ∈ R , a contradicti on. T o see Part (iii), note that the cond ition “ ∀ a, b : a, b ∈ R implies a ⊕ b ∈ R ” implies that R is affin e, so, if R is n ot affine then the condition is f alse. 2.5 Implemen t ation Let Γ b e a constraint language with domain { 0 , 1 } . Γ is said to implement 3 a k -ary r elation R if, for some k ′ ≥ k there is a C SP instance I with v ariables x 1 , . . . , x k ′ and constraint s in Γ such that, for ev ery tuple ( s 1 , . . . , s k ) ∈ R , there is exactly one s atisfying assignmen t σ of I w ith σ ( x 1 ) = s 1 , . . . , σ ( x k ) = s k and for ev ery tu p le ( s 1 , . . . , s k ) 6∈ R , there are no satisfying assignmen ts σ of I w ith σ ( x 1 ) = s 1 , . . . , σ ( x k ) = s k . Note the follo wing str aightforw ard observ ation, whic h is essen tially a parsimon ious redu ction [17, p.441]. Observ ation 12. If Γ implements R then #CSP (Γ ∪ { R } ) ≤ AP #CSP (Γ) . W e will use sev eral imp lemen tations of Cr eignou, Kh anna and Sudan. Proofs are provided in the app endix in order to mak e the pap er self-con tained. Lemma 13. (Cr eignou, Khanna and Sudan, [7, L emmas 5.24 and 5.25]) L et Γ b e a c onstr aint language with domain { 0 , 1 } . 3 There are many va riants of “implement” defined in the literature. See [7, Chapter 5], where the kind of implementation we defin e here is called “faithful” and “perfect”. 10 (i) If Γ c ontains a r elation R that is 0-valid, 1-valid and not c omplement- close d then Γ implements the r elation R ′ = { (0 , 0) , (1 , 1) , (1 , 0) } . (ii) If Γ c ontains a r elation R that is not 0-valid, not 1-valid and not c omple- ment-close d then Γ implements δ 0 and δ 1 . (iii) If Γ c ontains a r elation R th at is 0- v alid and not 1-valid then Γ i mple- ments δ 0 . (iv) If Γ c ontains a r elation R that is 1-v alid and not 0-valid then Γ i mple- ments δ 1 . Lemma 14. (Cr eignou, Khanna and Sudan, [7, Claim 5.31]) L et R b e a ternary r elation c ontaining (0 , 0 , 0) , (0 , 1 , 1) and (1 , 0 , 1) but not (1 , 1 , 0) . Then { R, δ 0 } implements one of Implies and NAND . Lemma 15. (Cr eignou, Khanna and Sudan, [7, L emma 5.30]) If R is a r ela- tion over { 0 , 1 } that is not affine then { R, δ 0 } implements one of OR , Implies , and NAND and so do es { R, δ 1 } . 2.6 Pinning revisited Com bining th e u seful pinn ing that we get fr om AP-reductions (Lemma 10) with the implementat ions of OR , Implies and NAND in Section 2.5, w e obtain a u seful lemma whic h says that w e can always do some pin ning. Lemma 16. L et Γ b e a c onstr aint language with domain { 0 , 1 } . Then either #CSP (Γ ∪ { δ 0 } ) ≤ AP #CSP (Γ) or #CSP (Γ ∪ { δ 1 } ) ≤ AP #CSP (Γ) (or b oth). Pr o of. First, su pp ose that Γ is not complement-c losed. If Γ con tains a r elation R that is not 0-v alid, not 1-v alid and not complement- closed then we finish by Observ ation 12 an d P art (ii) of Lemma 13. If Γ conta ins a relation R that is 0-v alid, 1-v alid an d not complement- closed then it imp lemen ts the relation R ′ from P art (i) of Lemma 13 so by Observ ation 12, #CSP (Γ ∪ { R ′ } ) ≤ AP #CSP (Γ). But Lemma 10 sh o ws b oth #CSP (Γ ∪ { R ′ , δ 0 } ) ≤ AP #CSP (Γ ∪ { R ′ } ) and #CSP (Γ ∪ { R ′ , δ 1 } ) ≤ AP #CSP (Γ ∪ { R ′ } ). Otherw ise Γ con tains a relation R that is 0-v alid and not 1-v alid (or vice-v ers a) and w e finish by Part (iii) (or P art (iv)) of Lemm a 13, and Observ ation 12. Second (and fi n ally), sup p ose that Γ is complemen t-closed. Here is a sim- ple AP-reduction from #CSP (Γ ∪ { δ 0 } ) to #CSP (Γ). Let I b e an in stance of #CSP (Γ ∪ { δ 0 } ). C onstruct an instance I ′ of #CSP (Γ) by adding a new v ari- able z 0 . F or all x ∈ N 0 ( I ) (all v ariables x to w hic h one or more constraint s δ 0 ( x ) in I apply), replace all o ccurrences of v ariable x with z 0 in I ′ . Now note that 2 #csp ( I ) = #csp ( I ′ ) since there is a one-to-t wo map from satisfying assign- men ts of I and satisfying assignments of I ′ . In particular, if s is an assignment to all v ariables of I other than those in N 0 ( I ) and s is satisfying, provided the rest of the v ariables are assigned v alue 0, then s is mapp ed to s ; z 0 = 0 and s ; z 0 = 1, wh er e s is the tuple obtained from s b y complemen ting the assign- men t of every v ariable. Both satisfy I ′ since Γ is complement-c losed. It is clear that all satisfying assignments of I ′ arise in this wa y . 11 2.7 Notation for Bo olean functions The follo wing definitions are from [1, 2]. An m -ary Bo olean f unction f is monotonic if and only if ( a 1 , . . . , a m ) ≤ ( b 1 , . . . , b m ) comp onen t wise implies f ( a 1 , . . . , a m ) ≤ f ( b 1 , . . . , b m ). Let M 2 b e the set of all monotone Bo olean functions f satisfying f (0 , . . . , 0) = 0 and f (1 , . . . , 1) = 1. Given a set B of Bo olean f unctions, the closure [ B ] consists of all functions that can b e defined b y pr op ositional form ulas with connectiv es from B (see [1]). An m -ary Boolean fu nction f is said to b e a p olymorphism of an n -ary relation R ( x 1 , . . . , x n ) if applying f comp onent w ise to m tuples in R results in a tuple that is also in R . 2.8 P olymorphisms and I M 2 In the termin ology of u niv ersal algebra, Creignou, Kolaitis, and Zan uttini [8] ha v e shown that I M 2 is precisely th e co-clone corresp ond ing to M 2 , wh ic h is a clone in P ost’s lattic e (see [2]). The direction of this r esult that w e will use is the follo wing. Lemma 17. (Cr eignou, Kolaitis, Zanuttini, [8]) If the r elation R is not in I M 2 then ther e is an f ∈ M 2 that is not a p olymorphism of R . Corollary 18. If the n -ary r elation R is not in I M 2 then ther e ar e Bo ole an tuples ( a 1 , . . . , a n ) ∈ R and ( b 1 , . . . , b n ) ∈ R such that either ( a 1 ∧ b 1 , . . . , a n ∧ b n ) 6∈ R or ( a 1 ∨ b 1 , . . . , a n ∨ b n ) 6∈ R (or b oth). Pr o of. W e w ill use the fact (see [1 ]) that M 2 = [ {∨ , ∧} ] where x ∨ y is th e OR of the Bo olean v alues x and y and x ∧ y is the AND of x and y . Thus, ev ery function f ∈ M 2 can b e defin ed by a pr op ositional formula usin g the 2-ary connectiv es ∨ and ∧ . The pro of is b y ind uction on the num b er of conn ectiv es used in the prop o- sitional formula used to r epresent the fun ction f from Lemma 17. The case f ( x ) = x (in wh ic h f has no connectiv es) cann ot arise since the iden tit y fu nction is a p olymorphism of ev ery r elation. The cases f ( x, y ) = x ∨ y and f ( x, y ) = x ∧ y (in whic h f has one conn ectiv e) im m ediately giv e the corollary . F or the ind uctiv e step, w e assume either f ( x 1 , . . . , x m ) = f ′ ( x 1 , . . . , x m ) ∨ f ′′ ( x 1 , . . . , x m ) or f ( x 1 , . . . , x m ) = f ′ ( x 1 , . . . , x m ) ∧ f ′′ ( x 1 , . . . , x m ) w here f ′ and f ′′ ha v e fewer connectiv es th an f . N ote that f ′ and f ′′ ma y not actually u se all of the v ariables in x 1 , . . . , x m . These t wo cases are similar, s o supp ose we are in the fir st of them. That is, supp ose f ( x 1 , . . . , x m ) = f ′ ( x 1 , . . . , x m ) ∨ f ′′ ( x 1 , . . . , x m ) . Supp ose also that f ′ and f ′′ are p olymorphisms of R (otherwise w e w ill apply the indu ctiv e hyp othesis to one of these fun ctions w hic h has f ew er connectiv es). Let t 1 , . . . , t m b e m n -tuples in R , su c h that th e tuple obtained b y applying f comp onen t wise to t 1 , . . . , t m is n ot in R . Let t ′ b e the n -tuple obtained by applying f ′ comp onent wise to t 1 , . . . , t m and let t ′′ b e th e n -tuple obtained by 12 applying f ′′ comp onent wise to t 1 , . . . , t m . S ince f ′ and f ′′ are p olymorp hisms of R , w e kno w that t ′ and t ′′ are in R . Ho we ve r, since f is not a p olymorph ism of R , the tup le t ′ ∨ t ′′ is not in R , pro ving the corollary . 3 Putting it all together: the pro of of Theorem 3 W e start with a lemma establishing a reduction from #S A T . Lemma 19. L et R 1 and R 2 b e r elations on { 0 , 1 } . If R 1 is not affine and R 2 is not in I M 2 then #SA T ≤ AP #CSP ( { R 1 , R 2 } ) . Pr o of. Apply Lemma 16 with Γ = { R 1 , R 2 } . T hen either #CSP ( { R 1 , R 2 , δ 0 } ) ≤ AP #CSP ( { R 1 , R 2 } ) or #CS P ( { R 1 , R 2 , δ 1 } ) ≤ AP #CSP ( { R 1 , R 2 } ). Assume the former (the latter case is s y m metric). No w use Lemma 15 together with Ob serv ation 12. Since R 1 is n ot affine this sho ws one of the follo win g. • #CSP ( { R 1 , R 2 , δ 0 , OR } ) ≤ AP #CSP ( { R 1 , R 2 , δ 0 } ), or • #CSP ( { R 1 , R 2 , δ 0 , NAND } ) ≤ AP #CSP ( { R 1 , R 2 , δ 0 } ), or • #CSP ( { R 1 , R 2 , δ 0 , Implies } ) ≤ AP #CSP ( { R 1 , R 2 , δ 0 } ). In the fir st t w o of th ese cases, w e are fi nished by Observ ation 5 and Lem- mas 6 and 7, so assu me the final case. Using Lemma 10 with the second p osition of Implies , we get #CSP ( { R 1 , R 2 , δ 0 , Implies , δ 1 } ) ≤ AP #CSP ( { R 1 , R 2 , δ 0 } ). Simplifying the chain of reductions and using Observ ation 5 to dr op R 1 from the left-hand side, w e get #CSP ( { Implies , R 2 , δ 0 , δ 1 } ) ≤ AP #CSP ( { R 1 , R 2 } ). W e will n o w fin ish b y showing #SA T ≤ AP #CSP ( { Implies , R 2 , δ 0 , δ 1 } ). Case 1. Using C orollary 18, supp ose that t and t ′ are tup les in R 2 but the tup le t ∧ t ′ (in wh ic h the op erator ∧ is applied component w ise) is n ot in R 2 . W e will sho w that { Implies , R 2 , δ 0 , δ 1 } implements one of OR and X OR = { (0 , 1) , (1 , 0) } . Let k b e the arit y of R 2 . As in th e implementa tions of Creignou et al. [7], define r i to b e u if t i = t ′ i = 0 or x if t i = 0 , t ′ i = 1 or y if t i = 1 , t ′ i = 0, or v if t i = t ′ i = 1. Let R ′ b e th e r elation implement ed by R ′ ( x, y ) = R 2 ( r 1 , . . . , r k ) ∧ δ 0 ( u ) ∧ δ 1 ( v ). Note that b oth x and y app ear as arguments of R ′ since t 6 = t ∧ t ′ and t ′ 6 = t ∧ t ′ . If t ∨ t ′ is in R 2 then R ′ ( x, y ) implemen ts OR ( x, y ), so w e are finish ed. Other w ise R ′ = XOR (which we n o w assume). Using Observ ation 12 and 5, we hav e #CSP ( { Implies , XOR } ) ≤ AP #CSP ( { R 1 , R 2 } ) . W e will finish b y sho wing that { Implies , X OR } imp lements NAND . (Th e result then follo ws b y Lemma 6 and Ob serv ation 12.) The implement ation is given by NAND ( x, z ) = Implies ( x, y ) ∧ X OR ( y , z ). Case 2. Otherwise, b y Corollary 18, there are t and t ′ in R 2 suc h th at t ∨ t ′ is not in R 2 . This case is dual to Case 1. 13 W e can n ow pro ve the main th eorem. Theorem 3. Let Γ b e a constrain t language with d omain { 0 , 1 } . If every relation in Γ is affine then #CSP (Γ) is in FP . Otherwise if every relation in Γ is in I M 2 then #CSP (Γ) = AP #BIS . Otherwise #CS P (Γ) = AP #SA T . Pr o of. First, su p p ose that every relation in Γ is affine. In th is case, the num b er of satisfying assignmen ts of an instance I of #CS P (Γ) is the num b er of solutions to a system of linear equations ov er GF(2). This can b e computed exactly , b y Gauss ian elimination, in p olynomial time, as Creignou and Hermann hav e noted [6]. Next, supp ose that Γ con tains a r elation R that is not affine, bu t ev ery relation in Γ is in I M 2 . By Lemma 9, #CSP (Γ) ≤ AP #BIS T o see that #BIS ≤ AP #CSP (Γ), apply Lemma 16. Then w e know that either #CSP (Γ ∪ { δ 0 } ) ≤ AP #CSP (Γ) or # CSP (Γ ∪ { δ 1 } ) ≤ AP #CSP (Γ) (or b oth). W e will sh o w #BIS ≤ AP #CSP (Γ ∪ { δ 0 } ) (2) and #BIS ≤ AP #CSP (Γ ∪ { δ 1 } ) (3) and then we will b e able to conclude #BIS ≤ AP #CSP (Γ). The p ro ofs of Equations (2) and (3) are similar, so we just pro ve (2). By Lemm a 15, Γ ∪ { δ 0 } implemen ts one of OR , Implies , and NAND . So b y Observ ation 12 we hav e (at least) one of the f ollo wing. (i) #CSP (Γ ∪ { δ 0 , OR } ) ≤ AP #CSP (Γ ∪ { δ 0 } ) (ii) #CSP (Γ ∪ { δ 0 , Implies } ) ≤ AP #CSP (Γ ∪ { δ 0 } ) (iii) #CSP (Γ ∪ { δ 0 , NAND } ) ≤ AP #CSP (Γ ∪ { δ 0 } ) Equation (2) follo ws f r om the combinatio n of Lemma 8 and (ii) u sing Obser- v ation 5. Also, since # BIS ≤ AP #SA T (see [9]), Equation (2) f ollo ws from the com bination of Lemma 7 and (i) using Observ ation 5. Similarly , it f ollo w s f r om the com bination of Lemma 6 and (iii) using Observ ation 5. Finally , supp ose that Γ con tains a relatio n R 1 that is not affine and a r ela- tion R 2 that is not in I M 2 . ( R 1 and R 2 migh t p ossibly b e th e same relatio n.) The fact that #CSP (Γ) ≤ AP #SA T f ollo w s from Observ ation 4 and the fact that #SA T ≤ AP #CSP (Γ) follo ws from Lemma 19 and Observ ation 5. References [1] E. B¨ ohler, N. C reignou, S. Reith and H. V ollmer, Pla ying with Bo olean blo c ks, Part I: P ost’s lattice w ith applications to complexit y theory , ACM SIGACT Newsletter 34 (200 3), 38–52 . [2] E. B¨ ohler, S. Reith, H. S c hno or and H. V ollmer, Bases for Boolean co-clones, Information Pr o c essing L etters 96 (2005 ), 59–66. [3] A. Bulato v, The complexit y of the count ing constrain t satisfaction p roblem, Pr o c. 35 th International Col lo quium for Auto mata, L anguages and Pr o gr am- ming , Lecture Notes in Computer Science 5125 , S pringer-V er lag, 2008, 646– 661. 14 [4] A. Bulato v and V. Dalmau, T o w ards a dichot omy theorem for th e coun ting constrain t satisfaction problem, in Pr o c . 44 th Annual IE EE Symp osium on F oundations of Computer Scienc e , 2003, 562–573. [5] A. Bulato v and M. Grohe, T he complexit y of partition fun ctions, The or etic al Computer Scienc e 348 (200 5), 148–18 6. [6] N. Creignou and M. Hermann, Complexit y of generalized satisfiabilit y coun ting problems, Information and Computation 125 (1996), 1–12. [7] N. Creignou, S. Khann a and M. Sudan, Complexity classific ations of Bo ole an c onstr aint satisfaction pr oblems , S IAM Press, 2001. [8] N. Cr eignou, P . Kolaitis and B. Zan uttini, Preferred represen tations of Bo olean relations, E le ctr onic Col lo quium on Computational Complexity , Re- p ort No. 119, 2005. [9] M. Dy er, L.A. Goldb erg, C. Greenh ill and M. J er r um, The relativ e complex- it y of app r o ximate counting p roblems, Algorith mic a 38 (2004), 471 –500. [10] M. Dyer, L.A. Goldb erg and M. Jerrum, The Complexit y of W eigh ted Bo olean #CSP , SIAM Journal on Computing 38 (200 9), 1970– 1986. [11] P . Hell and J. Ne ˇ set ˇ ril, Gr aphs and homomorp hisms , Oxford Univ ersit y Press, 2004. [12] M. Dy er and C. Greenhill, The complexit y of coun ting graph homomor- phisms, R andom Structur es and Algorith ms 17 (2000), 260–289. [13] T. F eder and M. V ardi, The compu tational s tr ucture of m onotone monadic SNP and constrain t satisfaction: a study through Datalog and group theory , SIAM Journal on Computing 28 (199 9), 57–10 4. [14] M. Jerru m , L. V alian t and V. V azirani, Rand om generation of combinato- rial structures from a uniform d istribution, The or etic al Computer Scienc e 43 (1986), 169–188 . [15] R. Ladn er, On the str ucture of p olynomial time red u cibilit y , Journal of the Asso ciation for Computing Machinery 22 (1975), 155–171 . [16] M. Mitzenmac her and E . Up fal, Pr ob ability and Computing , Cam bridge Univ ersit y Press, 2005. [17] C.H. P apadimitriou, Computational Complexity , Addison-W esley , 1994. [18] F. Rossi, P . v an Be ek and T. W alsh (Ed s.), Handb o ok of c onstr aint pr o- gr amming , Elsevier, 2006. [19] T. S c haefer, The complexit y of satisfiabilit y problems, in Pr o c. 10 th Annual ACM Symp osium on The ory of Computing , A CM Press, 1978, 216– 226. 15 App endix: The implemen tations of Creignou, Khanna and S u dan In order to make ou r pap er self-con tained, w e giv e the details of the implemen- tations of Creignou, Kh anna and S udan th at we use. In particular, w e provide the pro ofs for Lemmas 13, 14 and 15. (These pr o ofs can b e found in [7 ].) W e start with the construction for Lemma 13. Su pp ose R ∈ Γ is not complemen t-closed. Cho ose ( s 1 , . . . , s k ) in R su c h that ( s 1 ⊕ 1 , . . . , s k ⊕ 1) is not in R . No w consider the relation R ′ implemen ted by R ′ ( x, y ) = R ( r 1 , . . . , r k ) where r i = x if s i = 1 and r i = y otherwise. In the firs t case, R ′ is the relation { (0 , 0) , (1 , 1) , (1 , 0) } . In the s econd case, R ′ = { (1 , 0) } so R ′ giv es an implemen- tation of b oth δ 1 and δ 0 . The construction for th e third and fourth cases are the trivial implementa tions δ 0 ( x ) = R ( x, . . . , x ) and δ 1 ( x ) = R ( x, . . . , x ). W e now giv e the construction for Lemma 14. If R exclud es exactly one of (0 , 1 , 0) and (1 , 1 , 1) th en R ( x, y , x ) implements Implies ( y , x ) or NAND ( x, y ) (dep endin g on w h ic h is excluded). Similarly , if R exclud es exactly one of (1 , 0 , 0) and (1 , 1 , 1) then R ( x, y , y ) implements Implies ( x, y ) or NAND ( x, y ). If b oth (0 , 1 , 0) and (1 , 0 , 0) are in R then f R ( x, y , z ) ∧ δ 0 ( z ) imp lemen ts f NAND ( x, y ). If (0 , 1 , 0), (1 , 1 , 1) and (1 , 0 , 0) are excluded f r om R and so is (0 , 0 , 1) then R ( x, y , z ) implements NAND ( x, y ). Finally , if (0 , 1 , 0), (1 , 1 , 1) and (1 , 0 , 0) are excluded but (0 , 0 , 1) is in R th en R ( x, y , z ) ∧ δ 0 ( x ) implement s Implies ( y , z ). Finally , w e giv e the construction for Lemma 15. W e will sh o w that { R, δ 0 } implemen ts one of the n amed relations. A similar argument shows that { R, δ 1 } do es. Let k b e the arit y of R . First, supp ose that R is 0-v alid. Using part (iii) of Lemma 11, let s and s ′ b e tuples in R s uc h that s ⊕ s ′ is n ot in R . Let r i = w if s i = s ′ i = 0. Let r i = x if s i = 0 , s ′ i = 1. Let r i = y if s i = 1 , s ′ i = 0. Let r i = z if s i = s ′ i = 1. No w we know th at at least one of x and y o ccurs as an r i , since s 6 = s ′ . Let R ′ b e the relation imp lemented by R ( r 1 , . . . , r k ) ∧ δ 0 ( w ). There are a few cases to consider. If x o ccurs as an argum en t to R bu t y do es n ot then z o ccurs since s 6 = 0. Thus, th e relation R ′ ( x, z ) is Implies . (T ec h nically , this is a ternary relation in v ariables x , y and z , but it can b e view ed as a binary relation sin ce y do es not ap p ear.) T he situ ation is similar if y o ccurs as an argument to R but x do es not. If b oth x and y o ccur as argum en ts b ut z d o es n ot th en the relation R ′ ( x, y ) is NAND . Otherwise, x , y and z all o ccur as arguments. F urth er m ore, since R is 0-v alid, lemma 14 applies to the relation giv en b y R ′ ( x, y , z ). Second (and finally), supp ose that R is n ot 0-v alid. Note th at { R, δ 0 } can implemen t δ 1 . T o see this, let s b e a tuple in R . Let r i = x if s i = 1 and let r i = y otherwise. Then δ 1 ( x ) is implemen ted by R ( r 1 , . . . , r k ) ∧ δ 0 ( y ). No w consider t wo s ub-cases. F or the first su b-case, supp ose that for an y t wo tuples, t and t ′ , in R , the tuple t ∧ t ′ ,where ∧ is app lied comp onent wise, is also in R . Let s b e th e in tersection of all tuples in R . T hen s ∈ R . By Pa rt (ii) of Lemma 11, there are tw o tup les s ′ and s ′′ in R such that s ⊕ s ′ ⊕ s ′′ is n ot in R . Let r i = u if s i = s ′ i = s ′′ i = 0. Let r i = x if s i = 0 , s ′ i = 0 , s ′′ i = 1. Let r i = y if s i = 0 , s ′ i = 1 , s ′′ i = 0. L et r i = z if s i = 0 , s ′ i = 1 , s ′′ i = 1. Let r i = v if s i = s ′ i = s ′′ i = 1. 16 Let R ′ b e the relation implemente d by R ( r 1 , . . . , r k ) ∧ δ 0 ( u ) ∧ δ 1 ( v ). If y do es not o ccur as an argument of R ′ then R ′ ( x, z ) imp lemen ts Impli es . Similarly , if x do es n ot o ccur as an argument of R ′ then R ′ ( y , z ) im p lemen ts Implies . If z do es not o ccur as an argument of R ′ then R ′ ( x, y ) implement s NAND . So we assume that x , y and z o ccur as argments. Then app ly Lemma 14 to R ′ ( x, y , z ). F or the final sub case, su pp ose that th ere are tup les t and t ′ in R suc h that t ∧ t ′ is n ot in R . Defin e r i to b e u if t i = t ′ i = 0 or x if t i = 0 , t ′ i = 1 or y if t i = 1 , t ′ i = 0, or v if t i = t ′ i = 1. Let R ′ b e the relation implement ed by R ′ ( x, y ) = R ( r 1 , . . . , r k ) ∧ δ 0 ( u ) ∧ δ 1 ( v ). If t ∨ t ′ is in R then R ′ ( x, y ) implemen ts OR ( x, y ), so we are fin ished. Otherwise R ′ = { (0 , 1) , (1 , 0) } (which we n o w assume). No w using P art (i) of Lemma 11, let s , s ′ and s ′′ b e tuples in R so that s ⊕ s ′ ⊕ s ′′ is not in R . Define r i as follo ws . s i s ′ i s ′′ i r i 0 0 0 u 0 0 1 x 0 1 0 y 0 1 1 z 1 0 0 z ′ 1 0 1 y ′ 1 1 0 x ′ 1 1 1 u ′ Let R ′′ b e the r elation implemen ted by R ( r 1 , . . . , r k ) ∧ δ 0 ( u ) ∧ R ′ ( u, u ′ ) ∧ R ′ ( x, x ′ ) ∧ R ′ ( y , y ′ ) ∧ R ′ ( z , z ′ ) . By writing x ′ = x , y ′ = y and z ′ = z , we can think of R ′′ as a function of x , y and z . If x d o es not o ccur as an argumen t then R ′′ ( y , z ) implement s Implies ( y , z ). Similarly , we can assume that y and z o ccur as arguments. No w consider the relation R ′′ ( x, y , z ). W e kno w that (0 , 0 , 0) , (0 , 1 , 1) , (1 , 0 , 1) ∈ R ′′ , since s , s ′ , s ′′ ∈ R . Also (1 , 1 , 0) / ∈ R ′′ since s ⊕ s ′ ⊕ s ′′ / ∈ R . Then ap p ly Lemma 14 to R ′′ . 17