Product theorems via semidefinite programming

Product theorems via semidefinite programming T roy Lee Department of Computer Science Rutgers Uni versity ∗ Rajat Mittal Department of Computer Science Rutgers Uni versity † Abstract The tendency of semidefinite programs to compose perfectly under product has been e x- ploited many times in comple xity theory: for example, by Lov ´ asz to determine the Shannon capacity of the pentagon; to show a direct sum theorem for non-deterministic communication complexity and direct product theorems for discrepancy; and in interacti ve proof systems to sho w parallel repetition theorems for restricted classes of games. Despite all these examples of product theorems—some going back nearly thirty years—it was only recently that Mittal and Szegedy began to dev elop a general theory to explain when and why semidefinite programs behave perfectly under product. This theory captured many examples in the literature, but there were also some notable exceptions which it could not explain—namely , an early parallel repetition result of Feige and Lov ´ asz, and a direct product theorem for the discrepancy method of communication complexity by Lee, Shraibman, and ˇ Spalek. W e extend the theory of Mittal and Szegedy to explain these cases as well. Indeed, to the best of our knowledge, our theory captures all examples of semidefinite product theorems in the literature. 1 Intr oduction A pre valent theme in complexity theory is what we might roughly call product theorems. These results look at how the resources to accomplish sev eral independent tasks scale with the resources needed to accomplish the tasks indi vidually . Let us look at a fe w examples of such questions: Shannon Capacity If a graph G has an independent set of size α , how large an independent set can the product graph G × G ha ve? Ho w does α compare with amortized independent set size lim k →∞ α ( G k ) 1 /k ? This last quantity , known as the Shannon capacity , gives the effecti ve alphabet size of a graph where v ertices are labeled by letters and edges represent letters which can be confused if adjacent. ∗ Supported by a NSF Mathematical Sciences Postdoctoral Fellowship. Email: troyjlee@gmail.com † Supported by NSF Grant 0523866. Email: ramittal@cs.rutgers.edu 1 Hardness Amplification Product theorems naturally arise in the context of hardness amplifica- tion. If it is hard to ev aluate a function f ( x ) , then an ob vious approach to create a harder function is to ev aluate two independent copies f 0 ( x, y ) = ( f ( x ) , f ( y )) of f . There are different ways that f 0 can be harder than f —a direct sum theorem aims to sho w that e valuation of f 0 requires twice as man y resources as needed to ev aluate f ; direct product theorems aim to sho w that the error probability to compute f 0 is larger than that of f , gi ven the same amount of resources. Soundness Amplification V ery related to hardness amplification is what we might call sound- ness amplification. This arises in the context of interactiv e proofs where one wants to reduce the error probability of a protocol, by running se veral checks in parallel. The celebrated parallel repeti- tion theorem sho ws that the soundness of multiple pro ver interactiv e proof systems can be boosted in this manner [Raz98]. These examples illustrate that many important problems in complexity theory deal with product theorems. One successful approach to these types of questions has been through semidefinite programming. In this approach, if one wants to know how some quantity σ ( G ) beha ves under product, one first looks at a semidefinite approximation ¯ σ ( G ) of σ ( G ) . One then hopes to show that ¯ σ ( G ) provides a good approximation to σ ( G ) , and that ¯ σ ( G × G ) = ¯ σ ( G ) ¯ σ ( G ) . In this way one obtains that the original quantity must approximately product as well. Let us see ho w this approach has been used on some of the above questions. Shannon Capacity Perhaps the first application of this technique was to the Shannon capacity of a graph. Lo v ´ asz de veloped a semidefinite quantity , the Lov ´ asz theta function ϑ ( G ) , showed that it was a bound on the independence number of a graph, and that ϑ ( G × G ) = ϑ ( G ) 2 . In this way he determined the Shannon capacity of the pentagon, resolving a long standing open problem [Lov79]. Hardness Amplification Karchmer , Kushile vitz, and Nisan [KKN95] notice that another pro- gram introduced by Lov ´ asz [Lo v75], the fractional cover number , can be used to characterize non-deterministic communication complexity , up to small factors. As this program also perfectly products, they obtain a direct sum theorem for non-deterministic communication comple xity . As another example, Linial and Shraibman [LS06] sho w that a semidefinite programming quan- tity γ ∞ 2 characterizes the discrepancy method of communication complexity , up to constant factors. Lee, Shraibman and ˇ Spalek [LS ˇ S08] then use this result, together with the fact that γ ∞ 2 perfectly products, to show a direct product theorem for discrepancy , resolving an open problem of Shaltiel [Sha03]. Soundness Amplification Although the parallel repetition theorem was ev entually proven by other means [Raz98, Hol07], one of the first positiv e results did use semidefinite programming. Feige and Lov ´ asz [FL92] show that the acceptance probability ω ( G ) of a two-prov er interacti ve proof on input x can be represented as an integer program. They then study a semidefinite relax- ation of this program, and use this to sho w that if ω ( G ) < 1 then sup k →∞ ω ( G k ) 1 /k < 1 , for a certain class of games G . More recently , Clev e et al. [CSUU07] look at two-prover games where 2 the provers share entanglement, and sho w that the value of a special kind of such a game known as an XOR game can be exactly represented by a semidefinite program. As this program perfectly products, they obtain a perfect parallel repetition theorem for this game. W e hope this selection of examples sho ws the usefulness of the semidefinite programming ap- proach to product theorems. Until recently , howe ver , this approach remained an ad hoc collection of examples without a theory to explain when and why semidefinite programs perfectly product. Mittal and Szegedy [MS07] began to address this lacuna by gi ving a general suf ficient condition for a semidefinite program to obey a product rule. This condition captures many examples in the literature, notably the Lov ´ asz theta function [Lov79], and the parallel repetition for XOR games with entangled prov ers [CSUU07]. Other examples cited above, ho wev er , do not fit into the Mittal and Szegedy framew ork: namely , the product theorem of Feige and Lov ´ asz [FL92] and that for discrepancy [LS ˇ S08]. W e extend the condition of Mittal and Szegedy to capture these cases as well. Indeed, in our (admit- tedly imperfect) search of the literature, we have not found a semidefinite product theorem which does not fit into our frame work. 2 Pr eliminaries W e begin with some notational con ventions and basic definitions which will be useful. In general, lo wer case letters like v will denote column v ectors, and upper case letters like A will denote matrices. V ectors and matrices will be over the real numbers. The notation v T or A T will denote the transpose of a vector or matrix. W e will say A  0 if A is positive semidefinite, i.e. if A is symmetric and v T Av ≥ 0 for all v ectors v . W e will use se veral kinds of matrix products. W e write AB for the normal matrix product. For two matrices A, B of the same dimensions, A ◦ B denotes the matrix formed by their entrywise product. That is, ( A ◦ B )[ x, y ] = A [ x, y ] B [ x, y ] . W e will use A • B for the entrywise sum of A ◦ B . Equi valently , A • B = T r( AB T ) . W e will use the notation v ≥ w to indicate that the vector v is entrywise greater than or equal to the vector w . In applications we often face the situation where we would like to use the framew ork of semidefinite programming, which requires symmetric matrices, b ut the problem at hand is rep- resented by matrices which are not symmetric, or possibly not e ven square. Fortunately , this can often be handled by a simple trick. This trick is so useful that we will give it its o wn notation. For an arbitrary real matrix A , we define b A =  0 A A T 0  W e will refer to this as the bipartite version of A , as such a matrix corresponds to the adjacency matrix of a (weighted) bipartite graph. In many respects b A behaves similarly to A , b ut has the adv antages of being symmetric and square. More generally , we will refer to a matrix M which can be written as M =  0 A B 0  3 as block anti-dia gonal and a matrix M which can be written M =  D 1 0 0 D 2  as block dia gonal . One subtlety that arises in w orking with the bipartite version b A instead of A itself is in defining the product of instances. Mathematically , it is most con venient to work with the normal tensor product b A ⊗ b A =     0 0 0 A ⊗ A 0 0 A ⊗ A T 0 0 A T ⊗ A 0 0 A T ⊗ A T 0 0 0     Whereas what naturally arises in the product of problems is instead the “bipartite tensor” product of A : \ A ⊗ A =  0 A ⊗ A A T ⊗ A T 0  K empe, Rege v , and T oner [KR T07] observe, ho wev er , that a product theorem for the tensor product implies a product theorem for the bipartite tensor product. This essentially follows because \ A ⊗ A is a submatrix of b A ⊗ b A , and so positi ve semidefiniteness of the latter implies positive semidefiniteness of the former . See [KR T07] for full details. 3 Pr oduct rule with non-negativity constraints In this section we prove our main theorem extending the product theorem of Mittal and Szegedy [MS07] to handle non-negati vity constraints. As our work builds on the framew ork dev eloped by Mittal and Szegedy , let us first explain their results. Mittal and Szegedy consider a general affine semidefinite program π = ( J, A , b ) . Here A = ( A 1 , . . . , A m ) is a v ector of matrices, and we extend the notation • such that A • X = ( A 1 • X, A 2 • X , . . . , A m • X ) . The value of π is gi ven as α ( π ) = max X J • X such that A • X = b X  0 . W e take this as the primal formulation of π . Part of what makes semidefinite programming so use- ful for proving product theorems is that w e can also consider the dual formulation of π . Dualizing in the straightforward way gi ves: α ∗ ( π ) = min y y T b y T A − J  0 4 A necessary pre-condition for the semidefinite programming approach to proving product theorems is that so-called strong duality holds. That is, that α ( π ) = α ∗ ( π ) , the optimal primal and dual v alues agree. W e will assume this throughout our discussion. For more information about strong duality and suf ficient conditions for it to hold, see [BV06]. W e define the product of programs as follows: for π 1 = ( J 1 , A 1 , b 1 ) and π 2 = ( J 2 , A 2 , b 2 ) we define π 1 × π 2 = ( J 1 ⊗ J 2 , A 1 ⊗ A 2 , b 1 ⊗ b 2 ) . If A 1 is a tuple of m 1 matrices and A 2 is a tuple of m 2 matrices, then the tensor product A 1 ⊗ A 2 is a tuple of m 1 m 2 matrices consisting of all the tensor products A 1 [ i ] ⊗ A 2 [ j ] . It is straightforward to see that α ( π 1 × π 2 ) ≥ α ( π 1 ) α ( π 2 ) . Namely , if X 1 realizes α ( π 1 ) and X 2 realizes α ( π 2 ) , then X 1 ⊗ X 2 will be a feasible solution to π 1 × π 2 with v alue α ( π 1 ) α ( π 2 ) . This is because X 1 ⊗ X 2 is positiv e semidefinite, ( A 1 ⊗ A 2 ) • ( X 1 ⊗ X 2 ) = ( A 1 • X 1 ) ⊗ ( A 2 • X 2 ) = b 1 ⊗ b 2 , and ( J 1 ⊗ J 2 ) • ( X 1 ⊗ X 2 ) = ( J 1 • X 1 ) ⊗ ( J 2 • X 2 ) = α ( π 1 ) α ( π 2 ) . Mittal and Szegedy show the follo wing theorem giving sufficient conditions for the re verse inequality α ( π 1 × π 2 ) ≤ α ( π 1 ) α ( π 2 ) . Theorem 1 (Mittal and Szegedy [MS07]) Let π 1 = ( J 1 , A 1 , b 1 ) , π 2 = ( J 2 , A 2 , b 2 ) be two affine semidefinite pr ograms for which str ong duality holds. Then α ( π 1 × π 2 ) ≤ α ( π 1 ) α ( π 2 ) if either of the following two conditions hold: 1. J 1 , J 2  0 . 2. (Bipartiteness) Ther e is a partition of r ows and columns into two sets such that with r espect to this partition, J i is block anti-diagonal, and all matrices in A i ar e block diagonal, for i ∈ { 1 , 2 } . W e extend item (2) of this theorem to also handle non-negati vity constraints. This is a class of constraints which seems to arise often in practice, and allo ws us to capture cases in the literature that the original work of Mittal and Szegedy does not. More precisely , we consider programs of the follo wing form: α ( π ) = max X J • X such that A • X = b B • X ≥ 0 X  0 Here both A and B are vectors of matrices, and 0 denotes the all 0 vector . W e should point out a subtlety here. A program of this form can be equiv alently written as an af fine program by suitably extending X and modifying A accordingly to enforce the B • X ≥ 0 constraints through the X  0 condition. The catch is that two equiv alent programs do not nec- essarily lead to equiv alent product instances. W e e xplicitly separate out the non-ne gativity con- straints here so that we can define the product as follo ws: for two programs, π 1 = ( J 1 , A 1 , b 1 , B 1 ) and π 2 = ( J 2 , A 2 , b 2 , B 2 ) we say π 1 × π 2 = ( J 1 ⊗ J 2 , A 1 ⊗ A 2 , b 1 ⊗ b 2 , B 1 ⊗ B 2 ) . 5 Notice that the equality constraints and non-negati vity constraints do not interact in the product, which is usually the intended meaning of the product of instances. It is again straightforward to see that α ( π 1 × π 2 ) ≥ α ( π 1 ) α ( π 2 ) , thus we focus on the reverse inequality . W e extend Condition (2) of Theorem 1 to the case of programs with non-negati vity constraints. As we will see in Section 4, this theorem captures the product theorems of Feige- Lov ´ asz [FL92] and discrepancy [LS ˇ S08]. Theorem 2 Let π 1 = ( J 1 , A 1 , b 1 , B 1 ) and π 2 = ( J 2 , A 2 , b 2 , B 2 ) be two semidefinite pr ograms for which str ong duality holds. Suppose the following two conditions hold: 1. (Bipartiteness) Ther e is a partition of r ows and columns into two sets such that, with r espect to this partition, J i and all the matrices of B i ar e block anti-diagonal, and all the matrices of A i ar e block dia gonal, for i ∈ { 1 , 2 } . 2. Ther e ar e non-ne gative vectors u 1 , u 2 such that J 1 = u T 1 B 1 and J 2 = u T 2 B 2 . Then α ( π 1 × π 2 ) ≤ α ( π 1 ) α ( π 2 ) . Proof: T o pro ve the theorem it will be useful to consider the dual formulations of π 1 and π 2 . Dualizing in the standard fashion, we find α ( π 1 ) = min y 1 y T 1 b 1 such that y T 1 A 1 − ( z T 1 B 1 + J 1 )  0 z 1 ≥ 0 and similarly for π 2 . Fix y 1 , z 1 to be vectors which realizes this optimum for π 1 and similarly y 2 , z 2 for π 2 . The key observ ation of the proof is that if we can also sho w that y T 1 A 1 + ( z T 1 B 1 + J 1 )  0 and y T 2 A 2 + ( z T 2 B 2 + J 2 )  0 (1) then we will be done. Let us for the moment assume Equation (1) and see why this is the case. If Equation (1) holds, then we also hav e  y T 1 A 1 − ( z T 1 B 1 + J 1 )  ⊗  y T 2 A 2 + ( z T 2 B 2 + J 2 )   0  y T 1 A 1 + ( z T 1 B 1 + J 1 )  ⊗  y T 2 A 2 − ( z T 2 B 2 + J 2 )   0 A veraging these equations, we find ( y 1 ⊗ y 2 ) T ( A 1 ⊗ A 2 ) −  ( z T 1 B 1 + J 1 ) ⊗ ( z T 2 B 2 + J 2 )   0 . Let us work on the second term. W e hav e ( z T 1 B 1 + J 1 ) ⊗ ( z T 2 B 2 + J 2 ) = ( z 1 ⊗ z 2 ) T ( B 1 ⊗ B 2 ) + z T 1 B 1 ⊗ J 2 + J 1 ⊗ z T 2 B 2 + J 1 ⊗ J 2 = ( z 1 ⊗ z 2 ) T ( B 1 ⊗ B 2 ) + ( z 1 ⊗ u 2 ) T B 1 ⊗ B 2 + ( u 1 ⊗ z 2 ) T B 1 ⊗ B 2 + J 1 ⊗ J 2 . 6 Thus if we let v = z 1 ⊗ z 2 + z 1 ⊗ u 2 + u 1 ⊗ z 2 we see that v ≥ 0 as all of z 1 , z 2 , u 1 , u 2 are, and also ( y 1 ⊗ y 2 ) T ⊗ ( A 1 ⊗ A 2 ) − ( v T ( B 1 ⊗ B 2 ) + J 1 ⊗ J 2 )  0 . Hence ( y 1 ⊗ y 2 , v ) form a feasible solution to the dual formulation of π 1 × π 2 with value ( y 1 ⊗ y 2 )( b 1 ⊗ b 2 ) = α ( π 1 ) α ( π 2 ) . It no w remains to show that Equation (1) follows from the condition of the theorem. Giv en y A − ( z T B + J )  0 and the bipartiteness condition of the theorem, we will sho w that y A + ( z T B + J )  0 . This argument can then be applied to both π 1 and π 2 . W e hav e that y T A is block diagonal and z T B + J is block anti-diagonal with respect to the same partition. Hence for any v ector x T =  x 1 x 2  , we hav e  x 1 x 2   y T A − ( z T B + J )   x 1 x 2  =  x 1 − x 2   y T A + ( z T B + J )   x 1 − x 2  Thus the positi ve semidefiniteness of y A + ( z T B + J ) follows from that of y A − ( z T B + J ) . 2 One may find the condition that J lies in the positiv e span of B in the statement of Theorem 2 some what unnatural. If we remov e this condition, howe ver , a simple counterexample shows that the theorem no longer holds. Consider the program α ( π ) = max X  0 − 1 − 1 0  • X such that I • X = 1 ,  0 1 0 0  • X ≥ 0 ,  0 0 1 0  • X ≥ 0 , X  0 . Here I stands for the 2 -by- 2 identity matrix. This program satisfies the bipartiteness condition of Theorem 2, but J does not lie in the positi ve span of the matrices of B . It is easy to see that the v alue of this program is zero. The program π × π , howe ver , has positiv e v alue as J ⊗ J does not hav e any ne gativ e entries but is the matrix with ones on the main anti-diagonal. 4 A pplications T wo notable examples of semidefinite programming based product theorems in the literature are not captured by Theorem 1. Namely , a recent direct product theorem for the discrepancy method of communication complexity , and an early semidefinite programming based parallel repetition result of Feige and Lov ´ asz. As we no w describe in detail, these product theorems can be explained by Theorem 2. 4.1 Discrepancy Communication complexity is an ideal model to study direct sum and direct product theorems as it is simple enough that one can often hope to attain tight results, yet powerful enough that such 7 theorems are non-tri vial and hav e applications to reasonably po werful models of computation. See [KN97] for more details on communication complexity and its applications. Shaltiel [Sha03] began a systematic study of when we can expect direct product theorems to hold, and in particular looked at this question in the model of communication complexity for exactly these reasons. He showed a general counterexample where a direct product theorem does not hold, yet also prov ed a direct product for communication complexity lo wer bounds shown by a particular method—the discrepancy method under the uniform distrib ution. Shaltiel does not explicitly use semidefinite programming techniques, b ut proceeds by relating discrepancy under the uniform distribution to the spectral norm, which can be cast as a semidefinite program. This result was recently generalized and strengthened by Lee, Shraibman, and ˇ Spalek [LS ˇ S08] who show an essentially optimal direct product theorem for discrepancy under arbitrary distribu- tions. This result follo ws the general plan for sho wing product theorems via semidefinite program- ming: they use a result of Linial and Shraibman [LS06] that a semidefinite programming quantity γ ∞ 2 ( M ) characterizes the discrepanc y of the communication matrix M up to a constant factor , and then show that γ ∞ 2 ( M ) perfectly products. The semidefinite programming formulation of γ ∞ 2 ( M ) is not affine but in volv es non-negati vity constraints, and so does not fall into the original frame work of Mittal and Szegedy . Let us no w look at the semidefinite program describing γ ∞ 2 : γ ∞ 2 ( M ) = max X c M • X such that X • I = 1 X • E ij = 0 for all i 6 = j ≤ m, i 6 = j ≥ m X • ( c M ◦ E ij ) ≥ 0 for all i ≤ m, j ≥ m, and i ≥ m, j ≤ m X  0 . Here E i,j is the 0/1 matrix with exactly one entry equal to 1 in coordinate ( i, j ) . In this case, A is formed from the matrices I and E ij for i 6 = j ≤ m and i 6 = j ≥ m . These matrices are all block diagonal with respect to the natural partition of c M . Further , the objecti ve matrix c M and matrices of B are all block anti-diagonal with respect to this partition. Finally , we can express c M = u T B by simply taking u to be the all 1 vector . 4.2 F eige-Lov ´ asz In a seminal paper , Babai, Fortno w , and Lund [BFL91] sho w that all of non-deterministic expo- nential time can be captured by interacti ve proof systems with tw o-provers and polynomially many rounds. The attempt to characterize the power of two-prover systems with just one round sparked interest in a parallel repetition theorem—the question of whether the soundness of a two-pro ver system can be amplified by running sev eral checks in parallel. Feige and Lov ´ asz [FL92] ended up showing that two-prov er one-round systems capture NEXP by other means, and a proof of a parallel repetition theorem turned out to be the more dif ficult question [Raz98]. In the same paper , ho wev er , Feige and Lov ´ asz also take up the study of parallel repetition theorems and show an early positi ve result in this direction. 8 In a two-pro ver one-round game, the V erifier is trying to check if some input x is in the language L . The V erifier chooses questions s ∈ S, t ∈ T with some probability P ( s, t ) and then sends question s to prov er Alice, and question t to pro ver Bob . Alice sends back an answer u ∈ U and Bob replies w ∈ W , and then the V erifier answers according to some Boolean predicate V ( s, t, u, w ) . W e call this a game G ( V , P ) , and write the acceptance probability of the V erifier as ω ( G ) . In much the same spirit as the result of Lov ´ asz on the Shannon capacity of a graph, Feige and Lov ´ asz show that if the value of a game ω ( G ) < 1 then also sup k ω ( G k ) 1 /k < 1 , for a certain class of games kno wn as unique games. The proof of this result proceeds in the usual way: Feige and Lov ´ asz first sho w that ω ( G ) can be represented as a quadratic program. They then relax this quadratic program in the natural way to obtain a semidefinite program with v alue σ ( G ) ≥ ω ( G ) . Here the proof faces an e xtra complication as σ ( G ) does not perfectly product either . Thus another round of relaxation is done, thro wing out some constraints to obtain a program with v alue ¯ σ ( G ) ≥ σ ( G ) which does perfectly product. P art of our moti vation for proving Theorem 2 was to uncov er the “magic” of this second round of relaxation, and e xplain why Feige and Lov ´ asz remove the constraints they do in order to obtain something which perfectly products. Although the parallel repetition theorem w as eventually pro ven by different means [Raz98, Hol07], the semidefinite programming approach has recently seen rene wed interest for sho wing tighter parallel repetition theorems for restricted classes of games and where the prov ers share entanglement [CSUU07, KR T07]. 4.2.1 The relaxed pr ogram As mentioned above, Feige and Lov ´ asz first write ω ( G ) as an integer program, and then relax this to a semidefinite program with v alue σ ( G ) ≥ ω ( G ) . W e no w describe this program. The objecti ve matrix C is a | S | × | U | -by- | T | × | W | matrix where the ro ws are labeled by pairs ( s, u ) of possible question and answer pairs with Alice and similarly the columns are labeled by ( t, w ) possible dialogue with Bob . The objecti ve matrix for a game G = ( V , P ) is gi ven by C [( s, u ) , ( t, w )] = P ( s, t ) V ( s, t, u, w ) . W e also define an auxiliary matrices B st of dimensions the same as b C , where B st [( s 0 , u ) , ( t 0 , w )] = 1 if s = s 0 and t = t 0 and is zero otherwise. W ith these notations in place, we can define the program: σ ( G ) = max X 1 2 b C • X such that (2) X • B st = 1 for all s, t ∈ S ∪ T (3) X ≥ 0 (4) X  0 (5) W e see that we cannot apply Theorem 2 here as we ha ve global non-ne gativity constraints (not confined to the of f-diagonal blocks) and global equality constraints (not confined to the diagonal blocks). Indeed, Feige and Lov ´ asz remark that this program does not perfectly product. Feige and Lov ´ asz then consider a further relaxation with value ¯ σ ( G ) whose program does fit into our frame work. They throw out all the constraints of Equation (3) which are of f-diagonal, 9 and remove the non-negati vity constraints for the on-diagonal blocks of X . More precisely , they consider the follo wing program: ¯ σ ( G ) = max X 1 2 b C • X such that (6) X u,w ∈ U | X [( s, u ) , ( s 0 , w )] | ≤ 1 for all s, s 0 ∈ S (7) X u,w ∈ W | X [( t, u ) , ( t 0 , w )] | ≤ 1 for all t, t 0 ∈ T (8) X • E ( s,u ) , ( t,w ) ≥ 0 for all s ∈ S , t ∈ T , u ∈ U, w ∈ W (9) X  0 (10) Let us see that this program fits into the framew ork of Theorem 2. The vector of matrices B is composed of the matrices E ( s,u ) , ( t,w ) for s ∈ S, u ∈ U and t ∈ T , w ∈ W . Each of these matrices is block diagonal with respect to the natural partition of b C . Moreov er , as b C is non-negati ve and bipartite, we can write b C = u T B for a non-negati ve u , namely where u is gi ven by concatenation of the entries of C and C T written as a long vector . The on-diagonal constraints gi ven by Equations (7), (8) are not immediately seen to be of the form needed for Theorem 2 for two reasons: first, they are inequalities rather than equalities, and second, they hav e of absolute value signs. Fortunately , both of these problems can be easily dealt with. It is not hard to check that Theorem 2 also works for inequality constraints A • X ≤ b . The only change needed is that in the dual formulation we have the additional constraint y ≥ 0 . This condition is preserv ed in the product solution constructed in the proof of Theorem 2 as y ⊗ y ≥ 0 . The difficulty in allo wing constraints of the form A • X ≤ b is in fact that the opposite direction α ( π 1 × π 2 ) ≥ α ( π 1 ) α ( π 2 ) does not hold in general. Essentially , what can go wrong here is that a 1 , a 2 ≤ b does not imply a 1 a 2 ≤ b 2 . In our case, ho we ver , this does not occur as all the terms in volv ed are positiv e and so one can show ¯ σ ( G 1 × G 2 ) ≥ ¯ σ ( G 1 ) ¯ σ ( G 2 ) . T o handle the absolute v alue signs we consider an equi valent formulation of ¯ σ ( G ) . W e replace the condition that the sum of absolute values is at most one by constraints saying that the sum of e very possible ± combination of v alues is at most one: ¯ σ 0 ( G ) = max X 1 2 b C • X such that X u,w ∈ U ( − 1) x uw X [( s, u ) , ( s 0 , w )] ≤ 1 for all s, s 0 ∈ S and x ∈ { 0 , 1 } | U | 2 X u,w ∈ W ( − 1) x uw X [( t, u ) , ( t 0 , w )] ≤ 1 for all t, t 0 ∈ T and x ∈ { 0 , 1 } | W | 2 X • E ( s,u ) , ( t,w ) ≥ 0 for all s ∈ S , t ∈ T , u ∈ U, w ∈ W X  0 This program now satisfies the conditions of Theorem 2. It is clear that ¯ σ ( G ) = ¯ σ 0 ( G ) , and 10 also that this equiv alence is preserved under product. Thus the product theorem for ¯ σ ( G ) follows from Theorem 2 as well. 5 Conclusion W e have no w de veloped a theory which co vers all e xamples of semidefinite programming product theorems we are a ware of in the literature. Having such a theory which can be applied in black-box fashion should simplify the pursuit of product theorems via semidefinite programming methods, and we hope will find future applications. That being said, we still think there is more work to be done to arriv e at a complete understanding of semidefinite product theorems. In particular , we do not know the extension of item (1) of Theorem 1 to the case of non-negati ve constraints, and it would nice to understand to what e xtent item (2) of Theorem 2 can be relaxed. So far we hav e only considered tensor products of programs. One could also try for more general composition theorems: in this setting, if one has a lo wer bound on the complexity of f : { 0 , 1 } n → { 0 , 1 } and g : { 0 , 1 } k → { 0 , 1 } , one would like to obtain a lo wer bound on ( f ◦ g )( ~ x ) = f ( g ( x 1 ) , . . . , g ( x n )) . What we ha ve studied so f ar in looking at tensor products corresponds to the special cases where f is the P ARITY or AND function, depending on if the objecti ve matrix is a sign matrix or a 0 / 1 v alued matrix. One e xample of such a general composition theorem is known for the adversary method, a semidefinite programming quantity which lower bounds quantum query comple xity . There it holds that AD V ( f ◦ g ) ≥ ADV( f )AD V ( g ) [Amb03, HL ˇ S07]. It would be interesting to de velop a theory to capture these cases as well. Acknowledgements W e would like to thank Mario Szegedy for many insightful con versations. W e would also like to thank the anonymous referees of ICALP 2008 for their helpful comments. Refer ences [Amb03] A. Ambainis. Polynomial de gree vs. quantum query complexity . In Pr oceedings of the 44th IEEE Symposium on F oundations of Computer Science , pages 230–239. IEEE, 2003. [BFL91] L. Babai, L. Fortno w , and C. Lund. Non-deterministic exponential time has two-pro ver interacti ve protocols. Computational Complexity , 1:3–40, 1991. [BV06] S. Boyd and L. V andenberghe. Con ve x optimization . Cambridge Uni versity Press, 2006. [CSUU07] R. Clev e, W . Slofstra, F . Unger , and S. Upadhyay . Perfect parallel repetition theorem for quantum XOR proof systems. In Pr oceedings of the 22nd IEEE Conference on Computational Complexity . IEEE, 2007. 11 [FL92] U. Feige and L. Lov ´ asz. T wo-pro ver one-round proof systems: their power and their problems. In Pr oceedings of the 24th ACM Symposium on the Theory of Computing , pages 733–744. A CM, 1992. [HL ˇ S07] P . Høyer , T . Lee, and R. ˇ Spalek. Negati ve weights mak e adv ersaries stronger . In Pr oceedings of the 39th A CM Symposium on the Theory of Computing . A CM, 2007. [Hol07] T . Holenstein. P arallel repetition theorem: simplifications and the no-signaling case. In Pr oceedings of the 39th ACM Symposium on the Theory of Computing , pages 411– 419, 2007. [KKN95] M. Karchmer , E. Kushile vitz, and N. Nisan. Fractional cov ers and communication complexity . SIAM Journal on Discr ete Mathematics , 8(1):76–92, 1995. [KN97] E. Kushile vitz and N. Nisan. Communication Complexity . Cambridge Uni versity Press, 1997. [KR T07] J. Kempe, O. Rege v , and B. T oner . The unique game conjecture with entangled provers is false. T echnical Report 0712.4279, arXiv , 2007. [Lov75] L. Lov ´ asz. On the ratio of optimal integral and fractional cov ers. Discr ete Mathemat- ics , 13:383–390, 1975. [Lov79] L. Lov ´ asz. On the Shannon capacity of a graph. IEEE T ransactions on Information Theory , IT -25:1–7, 1979. [LS06] N. Linial and A. Shraibman. Learning comple xity versus communication complexity . In Pr oceedings of the 23r d IEEE Confer ence on Computational Comple xity . IEEE, 2008. [LS ˇ S08] T . Lee, A. Shraibman, and R. ˇ Spalek. A direct product theorem for discrepancy . In Pr oceedings of the 23r d IEEE Confer ence on Computational Complexity . IEEE, 2008. [MS07] R. Mittal and M. Szegedy . Product rules in semidefinite programming. In 16th Inter- national Symposium on Fundamentals of Computation Theory , 2007. [Raz98] R. Raz. A parallel repetition theorem. SIAM J ournal on Computing , 27(3):763–803, 1998. [Sha03] R. Shaltiel. T ow ards pro ving strong direct product theorems. Computational Com- plexity , 12(1–2):1–22, 2003. 12