Limits of Preprocessing

Limits of Preprocessing ∗ Stefan Szeider V ienna Univ ersity of T echnology , V ienna, Austria stefan@szeider .net Abstract W e present a first theoretical analysis of the power of polynomial-time preprocessing for important combinatorial problems from various areas in AI. W e consider problems from Constraint Satisfaction, Global Constraints, Satisfiability , Nonmonotonic and Bayesian Reasoning. W e show that, subject to a complexity theoretic assumption, none of the considered problems can be reduced by polynomial-time preprocessing to a problem kernel whose size is polynomial in a structural problem parameter of the input, such as induced width or backdoor size. Our results provide a firm theoretical boundary for the performance of polynomial-time preprocessing algorithms for the considered problems. K eywor ds: Fixed-P arameter T ractability , Constraint Satisfaction, Global Constraints, Satisfiability , Bayesian Networks, Normal Logic Programs, Computational Complexity . 1 Intr oduction Many important computational problems that arise in various areas of AI are intractable. Ne vertheless, AI research was very successful in de veloping and implementing heuristic solv ers that work well on real- world instances. An important component of virtually ev ery solver is a po werful polynomial-time prepro- cessing procedure that reduces the problem input. For instance, preprocessing techniques for the proposi- tional satisfiability problem are based on Boolean Constraint Propagation (see, e.g., E ´ en and Biere, 2005), CSP solvers make use of various local consistency algorithms that filter the domains of variables (see, e.g., Bessi ` ere, 2006); similar preprocessing methods are used by solvers for Nonmonotonic and Bayesian reasoning problems (see, e.g., Gebser et al., 2008, Bolt and van der Gaag, 2006, respecti vely). Until recently , no pro v able performance guarantees for polynomial-time preprocessing methods hav e been obtained, and so preprocessing was only subject of empirical studies. A possible reason for the lack of theoretical results is a certain inadequacy of the P vs NP framework for such an analysis: if we could reduce in polynomial time an instance of an NP-hard problem just by one bit, then we can solv e the entire problem in polynomial time by repeating the reduction step a polynomial number of times, and P = NP follows. W ith the advent of parameterized comple xity (Downey , Fello ws, and Stege, 1999), a new theoretical framew ork became av ailable that provides suitable tools to analyze the power of preprocessing. Param- eterized complexity considers a problem in a two-dimensional setting, where in addition to the input size n , a pr oblem parameter k is taken into consideration. This parameter can encode a structural aspect of the problem instance. A problem is called fixed-parameter tractable (FPT) if it can be solv ed in time f ( k ) p ( n ) where f is a function of the parameter k and p is a polynomial of the input size n . Thus, for FPT problems, the combinatorial explosion can be confined to the parameter and is independent of the input size. It is known that a problem is fixed-parameter tractable if and only if ev ery problem input can be reduced by polynomial-time preprocessing to an equiv alent input whose size is bounded by a function of the parameter (Downey , Fellows, and Stege, 1999). The reduced instance is called the pr oblem ker - nel , the preprocessing is called kernelization . The power of polynomial-time preprocessing can now be benchmarked in terms of the size of the kernel. Once a small kernel is obtained, we can apply any method ∗ Research funded by the ERC (COMPLEX REASON, Grand Reference 239962). 1 of choice to solve the kernel: brute-force search, heuristics, approximation, etc. (Guo and Niedermeier , 2007). Because of this flexibility a small kernel is generally preferable to a less flexible branching-based fixed-parameter algorithm. Thus, small kernels provide an additional value that goes beyond bare fixed- parameter tractability . In general the size of the kernel is exponential in the parameter, but many important NP-hard optimiza- tion problems such as Minimum V ertex Co ver , parameterized by solution size, admit polynomial kernels , see, e.g., (Bodlaender et al., 2009) for references. In previous research sev eral NP-hard AI problems ha ve been sho wn to be fixed-parameter tractable. W e list some important examples from v arious areas: • Constraint satisf action problems (CSP) o ver a fixed univ erse of values, parameterized by the induced width (Gottlob, Scarcello, and Sideri, 2002). • Consistency and generalized arc consistency for intractable global constraints, parameterized by the cardinalities of certain sets of values (Bessi ` ere et al., 2008). • Propositional satisfiability (SA T), parameterized by the size of backdoors (Nishimura, Ragde, and Szeider , 2004). • Positiv e inference in Bayesian networks with variables of bounded domain size, parameterized by size of loop cutsets (Pearl, 1988; Bidyuk and Dechter , 2007). • Nonmonotonic reasoning with normal logic programs, parameterized by feedback width (Gottlob, Scarcello, and Sideri, 2002). Howe ver , only exponential kernels are known for these fundamental AI problems. Can we hope for polynomial kernels? Results Our results are throughout negati ve. W e provide strong theoretical evidence that none of the abov e fixed-parameter tractable AI problems admits a polynomial k ernel. More specifically , we show that a polynomial kernel for any of these problems causes a collapse of the Polynomial Hierarchy to its third lev el, which is considered highly unlikely by complexity theorists. Our results are general: The kernel lower bounds are not limited to a particular preprocessing tech- nique but apply to any clever technique that could be conceiv ed in future research. Hence the results contribute to the foundations of AI. Our results suggest the inv estigation of alternati ve approaches to polynomial-time preprocessing; for instance, preprocessing that produces in polynomial time a Boolean combination of polynomially sized kernels instead of one single kernel. 2 F ormal Background A parameterized pr oblem P is a subset of Σ ∗ × N for some finite alphabet Σ . For a problem instance ( x, k ) ∈ Σ ∗ × N we call x the main part and k the parameter . W e assume the parameter is represented in unary . For the parameterized problems considered in this paper, the parameter is a function of the main part, i.e., k = π ( x ) for a function π . W e then denote the problem as P ( π ) , e.g., U - C S P (width) denotes the problem U -CSP parameterized by the width of the giv en tree decomposition. A parameterized problem P is fixed-parameter tractable if there exists an algorithm that solves any input ( x, k ) ∈ Σ ∗ × N in time O ( f ( k ) · p ( | x | ) where f is an arbitrary computable function of k and p is a polynomial in n . A k ernelization for a parameterized problem P ⊆ Σ ∗ × N is an algorithm that, gi ven ( x, k ) ∈ Σ ∗ × N , outputs in time polynomial in | x | + k a pair ( x 0 , k 0 ) ∈ Σ ∗ × N such that (i) ( x, k ) ∈ P if and only if ( x 0 , k 0 ) ∈ P and (ii) | x 0 | + k 0 ≤ g ( k ) , where g is an arbitrary computable function, called the size of the kernel. In particular , for constant k the k ernel has constant size g ( k ) . If g is a polynomial then we say that P admits a polynomial kernel . 2 Every fix ed-parameter tractable problem admits a k ernel. This can be seen by the follo wing argument due to Downey , Fellows, and Stege (1999). Assume we can decide instances ( x, k ) of problem P in time f ( k ) | n | O (1) . W e kernelize an instance ( x, k ) as follows. If | x | ≤ f ( k ) then we already have a kernel of size f ( k ) . Otherwise, if | x | > f ( k ) , then f ( k ) | x | O (1) ≤ | x | O (1) is a polynomial; hence we can decide the instance in polynomial time and replace it with a small decision-equiv alent instance ( x 0 , k 0 ) . Thus we always have a kernel of size at most f ( k ) . Howe ver , f ( k ) is super-polynomial for NP-hard problems (unless P = NP), hence this generic construction is not providing polynomial kernels. W e understand prepr ocessing for an NP-hard problem as a polynomial-time procedure that transforms an instance of the problem to a (possible smaller) solution-equiv alent instance of the same problem. Ker - nelization is such a preprocessing with a performance guarantee , i.e., we are guaranteed that the prepro- cessing yields a kernel whose size is bounded in terms of the parameter of the gi ven problem instance. In the literature also different forms of preprocessing hav e been considered. An important one is knowledge compilation , a two-phases approach to reasoning problems where in a first phase a gi ven knowledge base is (possibly in exponential time) preprocessed (“compiled”), such that in a second phase v arious queries can be answered in polynomial time (Cadoli et al., 2002). 3 T ools f or K ernel Lower Bounds In the sequel we will use recently de veloped tools to obtain kernel lower bounds. Our kernel lower bounds are subject to the widely belie ved complexity theoretic assumption NP 6⊆ co-NP / poly (or equiv alently , PH 6 = Σ 3 p ). In other words, the tools allow us to show that a parameterized problem does not admit a poly- nomial k ernel unless the Polynomial Hierarchy collapses to its third lev el (see, e.g., Papadimitriou, 1994). A composition algorithm for a parameterized problem P ⊆ Σ ∗ × N is an algorithm that recei ves as input a sequence ( x 1 , k ) , . . . , ( x t , k ) ∈ Σ ∗ × N , uses time polynomial in P t i =1 | x i | + k , and outputs ( y , k 0 ) ∈ Σ ∗ × N with (i) ( y , k 0 ) ∈ P if and only if ( x i , k ) ∈ P for some 1 ≤ i ≤ t , and (ii) k 0 is polynomial in k . A parameterized problem is compositional if it has a composition algorithm. With each parameterized problem P ⊆ Σ ∗ × N we associate a classical problem UP [ P ] = { x #1 k : ( x, k ) ∈ P } where 1 denotes an arbitrary symbol from Σ and # is a new symbol not in Σ . W e call UP [ P ] the unpa- rameterized version of P . The following result is the basis for our k ernel lower bounds. Theorem 1 (Bodlaender et al., 2009, Fortno w and Santhanam, 2008) . Let P be a parameterized pr oblem whose unparameterized version is NP -complete . If P is compositional, then it does not admit a polynomial kernel unless NP ⊆ co-NP / poly, i.e., the P olynomial Hierar chy collapses. Let P , Q ⊆ Σ ∗ × N be parameterized problems. W e say that P is polynomial parameter reducible to Q if there exists a polynomial time computable function K : Σ ∗ × N → Σ ∗ × N and a polynomial p , such that for all ( x, k ) ∈ Σ ∗ × N we have (i) ( x, k ) ∈ P if and only if K ( x, k ) = ( x 0 , k 0 ) ∈ Q , and (ii) k 0 ≤ p ( k ) . The function K is called a polynomial parameter tr ansformation . The following theorem allo ws us to transform kernel lo wer bounds from one problem to another . Theorem 2 (Bodlaender , Thomass ´ e, and Y eo, 2009) . Let P and Q be parameterized problems such that UP [ P ] is NP -complete, UP [ Q ] is in NP , and there is a polynomial par ameter transformation fr om P to Q . If Q has a polynomial kernel, then P has a polynomial kernel. 4 Constraint Networks Constraint networks hav e proven successful in modeling everyday cognitive tasks such as vision, language comprehension, default reasoning, and abduction, as well as in applications such as scheduling, design, diagnosis, and temporal and spatial reasoning (Dechter , 2010). A constraint network is a triple I = 3 ( V , U, C ) where V is a finite set of variables, U is a finite universe of values, and C = { C 1 , . . . , C m } is set of constraints. Each constraint C i is a pair ( S i , R i ) where S i is a list of v ariables of length r i called the constraint scope , and R i is an r i -ary relation ov er U , called the constraint relation . The tuples of R i indicate the allo wed combinations of simultaneous values for the variables S i . A solution is a mapping τ : V → U such that for each 1 ≤ i ≤ m and S i = ( x 1 , . . . , x r i ) , we have ( τ ( x 1 ) , . . . , τ ( x r i )) ∈ R i . A constraint network is satisfiable if it has a solution. W ith a constraint network I = ( V , U, C ) we associate its constraint graph G = ( V , E ) where E contains an edge between two variables if and only if they occur together in the scope of a constraint. A width w tr ee decomposition of a graph G is a pair ( T , λ ) where T is a tree and λ is a labeling of the nodes of T with sets of v ertices of G such that the follo wing properties are satisfied: (i) ev ery verte x of G belongs to λ ( p ) for some node p of T ; (ii) e very edge of G is is contained in λ ( p ) for some node p of T ; (iii) For each vertex v of G the set of all tree nodes p with v ∈ λ ( p ) induces a connected subtree of T ; (iv) | λ ( p ) | − 1 ≤ w holds for all tree nodes p . The tr eewidth of G is the smallest w such that G has a width w tree decomposition. The induced width of a constraint network is the treewidth of its constraint graph (Dechter and Pearl, 1989). W e note in passing that the problem of finding a tree decomposition of width w is NP-hard but fixed-parameter tractable in w . Let U be a fixed universe containing at least two elements. W e consider the following parameterized version of the constraint satisfaction problem (CSP). U - C S P (width) Instance: A constraint network I = ( V , U, C ) and a width w tree decomposition of the constraint graph of I . P arameter: The integer w . Question: Is I satisfiable? It is well kno wn that U - C S P (width) is fixed-parameter tractable over any fixed uni verse U (Dechter and Pearl, 1989; Gottlob, Scarcello, and Sideri, 2002) (for generalizations see Samer and Szeider, 2010). W e contrast this classical result and show that it is unlikely that U - C S P (width) admits a polynomial kernel, ev en in the simplest case where U = { 0 , 1 } . Theorem 3. { 0 , 1 } - C S P (width) does not admit a polynomial kernel unless the P olynomial Hierar chy collapses. Pr oof. W e show that { 0 , 1 } - C S P (width) is compositional. Let ( I i , T i ) , 1 ≤ i ≤ t , be a given sequence of instances of { 0 , 1 } - C S P (width) where I i = ( V i , U i , C i ) is a constraint network and T i is a width w tree decomposition of the constraint graph of I i . W e may assume, w .l.o.g., that V i ∩ V j = ∅ for 1 ≤ i < j ≤ t (otherwise we can simply change the names of variables). W e form a new constraint network I = ( V , { 0 , 1 } , C ) as follows. W e put V = S t i =1 V i ∪ { a 1 , . . . , a t , b 0 , . . . , b t } where a i , b i are new v ariables. W e define the set C of constraints in three groups. (1) For each 1 ≤ i ≤ t and each constraint C = (( x 1 , . . . , x r ) , R ) ∈ C i we add to C a new constraint C 0 = (( x 1 , . . . , x r , a i ) , R 0 )) where R 0 = { ( u 1 , . . . , u r , 0) : ( u 1 , . . . , u r ) ∈ R } ∪ { (1 , . . . , 1) } . (2) W e add t ternary constraints C ∗ 1 , . . . , C ∗ t where C ∗ i = (( b i − 1 , b i , a i ) , R ∗ ) and R ∗ = { (0 , 0 , 1) , (0 , 1 , 0) , (1 , 1 , 1) } . (3) Finally , we add two unary constraints C 0 = (( b 0 ) , (0)) and C 1 = (( b t ) , (1)) which force the values of b 0 and b t to 0 and 1 , respecti vely . Let G, G i be the constraint graphs of I and I i , respectiv ely . Fig. 1 shows an illustration of G for t = 4 . W e observe that a 1 , . . . , a t are cut vertices of G . Removing these vertices separates G into independent parts P , G 0 1 , . . . , G 0 t where P is the path b 0 , b 1 , . . . , b t , and G 0 i is isomorphic to G i . By standard techniques (see, e.g., Kloks, 1994), we can put the given width w tree decompositions T 1 , . . . , T t of G 0 1 , . . . , G 0 t and the tri vial width 1 tree decomposition of P together to a width w + 1 tree decomposition T of G . Clearly ( I , T ) can be obtained from ( I i , T i ) , 1 ≤ i ≤ t , in polynomial time. W e claim that I is satisfiable if and only if at least one of the I i is satisfiable. This claim can be verified by means of the following observations: The constraints in groups (2) and (3) provide that for any satisfying assignment there will be some 0 ≤ i ≤ t − 1 such that b 0 , . . . , b i are all set to 0 and 4 b 0 b 1 b 2 b 3 b 4 a 1 a 2 a 3 a 4 . . . V 1 . . . V 2 . . . V 3 . . . V 4 Figure 1: Constraint graph G . b i +1 , . . . , b t are all set to 1 ; consequently a i is set to 0 and all a j for j 6 = i are set to 1. The constraints in group (1) provide that if we set a i to 0 , then we obtain from C 0 the original constraint C ; if we set a i to 1 then we obtain a constraint that can be satisfied by setting all remaining v ariables to 1 . W e conclude that { 0 , 1 } - C S P (width) is compositional. In order to apply Theorem 1, it remains to establish that the unparameterized version of { 0 , 1 } - C S P (width) is NP-complete. Deciding whether a constraint network I ov er the univ erse { 0 , 1 } is satisfiable is well-known to be NP-complete (say by reducing 3-SA T). T o a constraint network I on n variables we can always add a tri vial width w = n − 1 tree decomposition of its constraint graph (taking a single tree node t where λ ( t ) contains all v ariables of I ). Hence UP [ { 0 , 1 } - C S P (width) ] is NP-complete. 5 Satisfiability The propositional satisfiability pr oblem (SA T) was the first problem sho wn to be NP-hard (Cook, 1971). Despite its hardness, SA T solvers are increasingly leaving their mark as a general-purpose tool in areas as diverse as software and hardware verification, automatic test pattern generation, planning, scheduling, and ev en challenging problems from algebra (Gomes et al., 2008). SA T solvers are capable of exploiting the hidden structure present in real-world problem instances. The concept of backdoors , introduced by W illiams, Gomes, and Selman (2003) pro vides a means for making the vague notion of a hidden structure explicit. Backdoors are defined with respect to a “sub-solver” which is a polynomial-time algorithm that correctly decides the satisfiability for a class C of CNF formulas. More specifically , Gomes et al. (2008) define a sub-solver to be an algorithm A that takes as input a CNF formula F and has the following properties: (i) T richotomy : A either rejects the input F , or determines F correctly as unsatisfiable or satisfiable; (ii) Efficiency : A runs in polynomial time; (iii) T rivial Solvability : A can determine if F is trivially satisfiable (has no clauses) or trivially unsatisfiable (contains only the empty clause); (i v .) Self- Reducibility : if A determines F , then for any variable x and value ε ∈ { 0 , 1 } , A determines F [ x = ε ] . F [ τ ] denotes the formula obtained from F by applying the partial assignment τ , i.e., satisfied clauses are remov ed and false literals are removed from the remaining clauses. W e identify a sub-solv er A with the class C A of CNF formulas whose satisfiability can be determined by A . A str ong A -backdoor set (or A -bac kdoor , for short) of a CNF formula F is a set B of variables such that for each possible truth assignment τ to the variables in B , the satisfiability of F [ τ ] can be determined by sub-solver A in time O ( n c ) . Hence, if we know an A -backdoor of size k , we can decide the satisfiability of F by running A on 2 k instances F [ τ ] , yielding a time bound of O (2 k n c ) . Hence SA T decision is fixed-parameter tractable in the backdoor size k for any sub-solver A . Hence the following problem is clearly fixed-parameter tractable for an y sub-solver A . S A T ( A -backdoor) Instance: A CNF formula F , and an A -backdoor B of F of size k . P arameter: The integer k . Question: Is F satisfiable? W e are concerned with the question of whether instead of trying all 2 k possible partial assignments we can reduce the instance to a polynomial kernel. W e will establish a very general result that applies to all possible sub-solvers. 5 Theorem 4. S AT ( A -backdoor) does not admit a polynomial kernel for any sub-solver A unless the P olynomial Hierar chy collapses. Pr oof. W e will devise polynomial parameter transformations from the following parameterized problem which is kno wn to be compositional (Fortno w and Santhanam, 2008) and therefore unlikely to admit a polynomial kernel. S A T (v ars) Instance: A propositional formula F in CNF on n v ariables. P arameter: The number n of v ariables. Question: Is F satisfiable? Let F be a CNF formula and V the set of all variables of F . Due to property (ii) of a sub-solv er , V is an A -backdoor set for any A . Hence, by mapping ( F , n ) (as an instance of S A T (v ars)) to ( F , V ) (as an instance of S A T ( A -backdoor)) provides a (tri vial) polynomial parameter transformation from S A T (vars) to S A T ( A -backdoor). Since the unparameterized versions of both problems are clearly NP-complete, the result follows by Theorem 2. Let 3 S AT ( π ) (where π is an arbitrary parameterization) denote the problem S A T ( π ) restricted to 3CNF formula, i.e., to CNF formulas where each clauses contains at most three literals. In contrast to S A T (v ars), the parameterized problem 3 S A T (vars) has a trivial polynomial kernel: if we remove duplicate clauses, then any 3CNF formula on n variables contains at most O ( n 3 ) clauses, and so is a polynomial kernel. Hence the easy proof of Theorem 4 does not carry over to 3 S A T ( A -backdoor). W e therefore consider the cases 3 S A T ( H O R N -backdoor) and 3 S A T (2CNF-backdoor) separately , these cases are important since the detection of H O R N and 2CNF-backdoors is fix ed-parameter tractable (Nishimura, Ragde, and Szeider , 2004). Theorem 5. Neither 3 S A T (H O R N -backdoor) nor 3 S AT (2CNF-backdoor) admit a polynomial kernel unless the P olynomial Hierar chy collapses. Pr oof. Let C ∈ { H O R N , 2CNF } . W e show that 3 S A T ( C -backdoor) is compositional. Let ( F i , B i ) , 1 ≤ i ≤ t , be a given sequence of instances of 3 S A T ( C -backdoor) where F i is a 3CNF formula and B i is a C -backdoor set of F i of size k . W e distinguish two cases. Case 1: t > 2 k . Let k F i k := P C ∈ F i | C | and n := max t i =1 k F i k . Whether F i is satisfiable or not can be decided in time O (2 k n ) since the satisfiability of a Horn or 2CNF formula can be decided in linear time. W e can check whether at least one of the formulas F 1 , . . . , F t is satisfiable in time O ( t 2 k n ) ≤ O ( t 2 n ) which is polynomial in t + n . If some F i is satisfiable, we output ( F i , B i ) ; otherwise we output ( F 1 , B 1 ) ( F 1 is unsatisfiable). Hence we have a composition algorithm. Case 2: t ≤ 2 k . This case is more inv olved. W e construct a new instance ( F , B ) of 3 S A T ( C -backdoor) as follows. Let s = d log 2 t e . Since t ≤ 2 k , s ≤ k follo ws. Let V i denote the set of variables of F i . W e may assume, w .l.o.g., that B 1 = · · · = B t and that V i ∩ V j = B 1 for all 1 ≤ i < j ≤ t since otherwise we can change names of variable accordingly . In a first step we obtain from ev ery F i a CNF formula F 0 i as follows. For each variable x ∈ V i \ B 1 we take two new variables x 0 and x 1 . W e replace each positive occurrence of a variable x ∈ V i \ B 1 in F i with the literal x 0 and each negati ve occurrence of x with the literal ¬ x s . W e add all clauses of the form ( ¬ x j − 1 ∨ x j ) for 1 ≤ j ≤ s ; we call these clauses “ connection clauses . ” Let F 0 i be the formula obtained from F i in this way . W e observe that F 0 i and F i are SA T -equi valent, since the connection clauses form an implication chain. Since the connection clauses are both Horn and 2CNF , B 1 is also a C -backdoor of F 0 i . W e take a set Y = { y 1 , . . . , y s } of new variables. Let C 1 , . . . , C 2 s be the sequence of all 2 s possible clauses (modulo permutation of literals within a clause) containing exactly s literals over the variables in Y . Consequently we can write C i as ( ` i 1 ∨ · · · ∨ ` i s ) where ` j i ∈ { y i , ¬ y i } . For 1 ≤ i ≤ t we add to each connection clause ( ¬ x j − 1 ∨ x j ) of F 0 i the literal ` i j ∈ C i . Let F 00 i denote the 3CNF formula obtained from F 0 i this way . 6 For t < i ≤ 2 s we define 3CNF formulas F 00 i as follows. If s ≤ 3 then F 00 i consists just of the clause C i . If s > 3 then we take new variables z i 2 , . . . , z i s − 2 and let F 00 i consist of the clauses ( ` i 1 ∨ ` i 2 ∨ ¬ z i 2 ) , ( ` i 3 ∨ z i 2 ∨ ¬ z i 3 ) , . . . , ( ` i s − 2 ∨ z i s − 3 ∨ ¬ z i s − 2 ) , ( ` i s − 1 ∨ ` i s ∨ z i s − 2 ) . Finally , we let F be the 3CNF formula containing all the clauses from F 00 1 , . . . , F 00 2 s . Any assignment τ to Y ∪ B 1 that satisfies C i can be extended to an assignment that satisfies F 00 i since such assignment satisfies at least one connection clause ( x j − 1 ∨ x j ∨ ` i j ) and so the chain of implications from from x o to x s is broken. It is not difficult to verify the following two claims. (i) F is satisfiable if and only if at least one of the formulas F i is satisfiable. (ii) B = Y ∪ B 1 is a C -backdoor of F . Hence we have also a composition algorithm in Case 2, and thus 3 S A T ( C -backdoor) is compositional. Clearly UP [ 3 S AT ( C -backdoor) ] is NP-complete, hence the result follows from Theorem 1. 6 Global Constraints The success of today’ s constraint solvers relies heavily on efficient algorithms for special purpose global constraints (van Hoeve and Katriel, 2006). A global constraint specifies a pattern that frequently occurs in real-world problems, for instance, it is often required that variables must all take different values (e.g., activities requiring the same resource must all be assigned different times). The A L L D I FF E R E N T global constraint efficiently encodes this requirement. More formally , a global constraint is defined for a set S of variables, each v ariable x ∈ S ranges over a finite domain dom ( x ) of values. An instantiation is an assignment α such that α ( x ) ∈ dom ( x ) for each x ∈ S . A global constraint defines which instantiations are legal and which are not. A global constraint is consistent if it has at least one leg al instantiation, and it is domain consistent (or hyper arc consistent) if for each variable x ∈ S and each value d ∈ dom ( x ) there is a legal instantiation α with α ( x ) = d . For all global constraints considered in this paper, domain consistency can be reduced to a quadratic number of consistency checks, hence we will focus on consistenc y . W e assume that the size of a representation of a global constraint is polynomial in P x ∈ S | dom ( x ) | . For several important types T of global constraints, the problem of deciding whether a constraint of type T is consistent (in symbols T - Cons ) is NP-hard. Examples for such intractable types of constraints are N V A L U E , D I S J O I N T , and U S E S (Bessi ` ere et al., 2004). An N V A L U E constraint ov er a set X of variables requires from a legal instantiation α that |{ α ( x ) : x ∈ X }| = N ; A L L D I FF E R E N T is the special case where N = | X | . The global constraints D I S J O I N T and U S E S are specified by two sets of variables X , Y ; D I S J O I N T requires that α ( x ) 6 = α ( y ) for each pair x ∈ X and y ∈ Y ; U S E S requires that for each x ∈ X there is some y ∈ Y such that α ( x ) = α ( y ) . For a set X of variables we write dom ( X ) = S x ∈ X dom ( x ) . Bessi ` ere et al. (2008) considered dx = | dom ( X ) | as parameter for N V A L U E , dxy = | dom ( X ) ∩ dom ( Y ) | as parameter for D I S J O I N T , and dy = | dom ( Y ) | as parameter for U S E S . They showed that consistency checking is fixed-parameter tractable for the constraints under the respectiv e parameteriza- tions, i.e., the problems N V A L U E - C O N S ( dx ), D I S J O I N T - C O N S ( dxy ), and U S E S - C O N S ( dy ) are fixed- parameter tractable. W e show that it is unlikely that their results can be improved in terms of polynomial kernels. Theorem 6. The pr oblems N V A L U E - C O N S ( dx ) , D I S J O I N T - C O N S ( dxy ) , U S E S - C O N S ( dy ) do not admit polynomial kernels unless the P olynomial Hierar chy collapses. Pr oof. W e devise a polynomial parameter reduction from S A T (vars). W e use a construction of Bessi ` ere et al. (2004). Let F = { C 1 , . . . , C m } be a CNF formula over v ariables x 1 , . . . , x n . W e consider the clauses and variables of F as the variables of a global constraint with domains dom ( x i ) = {− i, i } , and dom ( C j ) = { i : x i ∈ C j } ∪ { − i : ¬ x i ∈ C j } . Now F can be encoded as an N V A L U E constraint with X = { x 1 , . . . , x n , C 1 , . . . , C m } and N = n (clearly F is satisfiable if and only if the constraint is consistent). Since dx = 2 n we have a polynomial parameter reduction from S A T (vars) to N V A L U E - C O N S ( dx ). Similarly , as observed by Bessi ` ere et al. (2009), F can be encoded as a D I S J O I N T constraint with X = { x 1 , . . . , x n } and Y = { C 1 , . . . , C m } ( dxy ≤ 2 n ), or as a U S E S constraint with X = { C 1 , . . . , C m } and Y = { x 1 , . . . , x n } ( dy = 2 n ). Since the unparameterized problems are clearly NP-complete, the result follows by Theorem 2. Further results on kernels for global constraints ha ve been obtained by Gaspers and Szeider (2011). 7 7 Bayesian Reasoning Bayesian networks (BNs) have emerged as a general representation scheme for uncertain knowledge (Pearl, 2010). A BN models a set of stochastic v ariables, the independencies among these variables, and a joint probability distribution o ver these variables. For simplicity we consider the important special case where the stochastic variables are Boolean. The variables and independencies are modelled in the BN by a directed acyclic graph G = ( V , A ) , the joint probability distribution is giv en by a table T v for each node v ∈ V which defines a probability T v | U for each possible instantiation U = ( d 1 , . . . , d s ) ∈ { true , false } s of the parents v 1 , . . . , v s of v in G . The probability Pr ( U ) of a complete instantiation U of the variables of G is gi ven by the product of T v | U ov er all variables v . W e consider the problem P ositive-BN-Inference which takes as input a Boolean BN ( G, T ) and a variable v , and asks whether Pr ( v = true) > 0 . The problem is NP-complete (Cooper, 1990) and moves from NP to #P if we ask to compute Pr ( v = true ) (Roth, 1996). The problem can be solved in polynomial time if the BN is singly connected , i.e, if there is at most one undirected path between any two variables (Pearl, 1988). It is natural to parametrize the problem by the number of variables one must delete in order to make the BN singly connected (the deleted variables form a loop cutset ). In fact, P O S I T I V E - B N - I N F E R E N C E (loop cutset size) is easily seen to be fixed-parameter tractable as we can determine whether Pr ( v = true) > 0 by taking the maximum of Pr ( v = true | U ) over all 2 k possible instantiations of the k cutset variables, each of which requires processing of a singly connected network. Howe ver , although fix ed-parameter tractable, it is unlikely that the problem admits a polynomial kernel. Theorem 7. P O S I T I V E - B N - I N F E R E N C E (loop cutset size) does not admit a polynomial kernel unless the P olynomial Hierar chy collapses. Pr oof. (Sketch.) W e give a polynomial parameter transformation from S A T (vars) and apply Theorem 2. The reduction is based on the reduction from 3SA T giv en by Cooper (1990). Howe ver , we need to allow clauses with an arbitrary number of literals since, as observed abov e, 3 S A T (vars) has a polynomial k ernel. Let F be a CNF formula on n v ariables. W e construct a BN ( G, T ) such that for a variable v we hav e Pr ( v = true) > 0 if and only if F is satisfiable. Cooper uses input nodes u i for representing variables of F , clause nodes c i for representing the clauses of F , and conjunction nodes d i for representing the conjunction of the clauses. W e proceed similarly , ho wev er, we cannot represent a clause of lar ge size with a single clause node c i , as the required table T c i would be of exponential size. Therefore we split clauses containing more than 3 literals into sev eral clause nodes, as indicated in Figure 2. It remains to observe that u 1 u 2 u 3 u 4 c 1 c 2 c 3 Figure 2: BN representation of a clause on four literals. the set of input nodes E = { u 1 , . . . , u n } is a loop cutset of the constructed BN, hence we have indeed a polynomial parameter transformation from S A T (vars) to P O S I T I V E - B N - I N F E R E N C E (loop cutset size). The result follows by Theorem 2. 8 Nonmonotonic Reasoning Logic pr ogramming with negation under the stable model semantics is a well-studied form of nonmono- tonic reasoning (Gelfond and Lifschitz, 1988; Marek and Truszczy ´ nski, 1999). A (normal) logic pr o- gram P is a finite set of rules r of the form h ← − a 1 ∧ · · · ∧ a m ∧ ¬ b 1 ∧ · · · ∧ ¬ b n where h, a i , b i are atoms , where h forms the head and the a i , b i from the body of r . W e write H ( r ) = h , B + ( r ) = { a 1 , . . . , a m } , and B − ( r ) = { b 1 , . . . , b n } . Let I be a finite set of atoms. The GF reduct P I of 8 a logic program P under I is the program obtained from P by removing all rules r with B − ( r ) ∩ I 6 = ∅ , and removing from the body of each remaining rule r 0 all literals ¬ b with b ∈ I . I is a stable model of P if I is a minimal model of P I , i.e., if (i) for each rule r ∈ P I with B + ( r ) ⊆ I we ha ve H ( r ) ∈ I , and (ii) there is no proper subset of I with this property . The undirected dependency gr aph U ( P ) of P is formed as follows. W e take the atoms of P as vertices and add an edge x − y between two atoms x, y if there is a rule r ∈ P with H ( r ) = x and y ∈ B + ( r ) , and we add a path x − u − y if H ( r ) = x and y ∈ B − ( r ) ( u is a new v ertex of de gree 2). The feedback width of P is the size of a smallest set V of atoms such that ev ery cycle of U ( P ) runs through an atom in V . A fundamental computational problems is Stable Model Existence (SME) , which asks whether a giv en normal logic program has a stable model. The problem is well-known to be NP-complete (Marek and Truszczy ´ nski, 1991). Gottlob, Scarcello, and Sideri (2002) showed that S M E (feedback width) is fixed-parameter tractable (see Fichte and Szeider (2011) for generalizations). W e show that this result cannot be strengthened with respect to a polynomial kernel. Theorem 8. S M E (feedback width) does not admit a polynomial kernel unless the P olynomial Hier arc hy collapses. Pr oof. (Sketch.) W e giv e a polynomial parameter transformation from S A T (vars) to SME ( feedback width ) using a construction of Niemel ¨ a (1999). Giv en a CNF formula F on n variables, we construct a logic program P as follows. For each variable x of F we take two atoms x and ˆ x and include the rules ( ˆ x ← ¬ x ) and ( x ← ¬ ˆ x ) ; for each clause C of F we take an atom c and include for each positive literal a of C the rule ( c ← a ) , and for each negativ e literal ¬ a of C the rule ( c ← ˆ a ) ; finally , we take two atoms s and f and include the rule ( f ← ¬ f ∧ ¬ s ) and for each clause C of F the rule ( s ← c ) . F is satisfiable if and only if P has a stable model (Niemel ¨ a, 1999). It remains to observe that each cycle of U ( P ) runs through a verte x in V = { x, ˆ x : x ∈ v ars ( F ) } , hence the feedback width of P is at most 2 n . Hence we hav e a polynomial parameter transformation from S A T (vars) to S M E (feedback width). The result follows by Theorem 2. 9 Conclusion W e have established super -polynomial kernel lower bounds for a wide range of important AI problems, providing firm limitations for the power of polynomial-time preprocessing for these problems. W e con- clude from these results that in contrast to many optimization problems (see Section 1), typical AI prob- lems do not admit polynomial kernels. Our results suggest the consideration of alternati ve approaches. For example, it might still be possible that some of the considered problems admit polynomially sized T uring kernels, i.e., a polynomial-time preprocessing to a Boolean combination of a polynomial number of polynomial kernels. In the area of optimization, parameterized problems are known that do not admit polynomial kernels but admit polynomial T uring kernels (Fernau et al., 2009). This suggests a theoretical and empirical study of T uring kernels for the AI problems considered. Refer ences [ Bessi ` ere et al., 2004 ] Bessi ` ere, C.; Hebrard, E.; Hnich, B.; and W alsh, T . 2004. The complexity of global constraints. In McGuinness, D. L., and Ferguson, G., eds., Pr oceedings of the Nineteenth National Confer ence on Artificial Intelligence, J uly 25-29, 2004, San Jose, California, USA , 112–117. AAAI Press / The MIT Press. [ Bessi ` ere et al., 2008 ] Bessi ` ere, C.; Hebrard, E.; Hnich, B.; Kiziltan, Z.; Quimper , C.-G.; and W alsh, T . 2008. The parameterized complexity of global constraints. In Proceedings of the T wenty-Thir d AAAI Confer ence on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, J uly 13-17, 2008 , 235–240. AAAI Press. 9 [ Bessi ` ere et al., 2009 ] Bessi ` ere, C.; Hebrard, E.; Hnich, B.; Kiziltan, Z.; and W alsh, T . 2009. Range and roots: T wo common patterns for specifying and propagating counting and occurrence constraints. Artificial Intelligence 173(11):1054–1078. [ Bessi ` ere, 2006 ] Bessi ` ere, C. 2006. Constraint propagation. In Rossi, F .; van Beek, P .; and W alsh, T ., eds., Handbook of Constraint Pr ogramming . Else vier . chapter 3. [ Bidyuk and Dechter , 2007 ] Bidyuk, B., and Dechter, R. 2007. Cutset sampling for Bayesian networks. J. Artif . Intell. Res. 28:1–48. [ Bodlaender et al., 2009 ] Bodlaender , H. L.; Do wney , R. G.; Fello ws, M. R.; and Hermelin, D. 2009. On problems without polynomial kernels. J. of Computer and System Sciences 75(8):423–434. [ Bodlaender , Thomass ´ e, and Y eo, 2009 ] Bodlaender , H. L.; Thomass ´ e, S.; and Y eo, A. 2009. K ernel bounds for disjoint cycles and disjoint paths. In Fiat, A., and Sanders, P ., eds., Algorithms - ESA 2009, 17th Annual Eur opean Symposium, Copenhagen, Denmark, September 7-9, 2009. Pr oceedings , volume 5757 of Lectur e Notes in Computer Science , 635–646. Springer V erlag. [ Bolt and van der Gaag, 2006 ] Bolt, J., and van der Gaag, L. 2006. Preprocessing the MAP problem. In Studen ´ y, M., and V omlel, J., eds., Pr oceedings of the Thir d Eur opean W orkshop on Pr obabilistic Graphical Models, Pr ague , 51–58. [ Cadoli et al., 2002 ] Cadoli, M.; Donini, F . M.; Liberatore, P .; and Schaerf, M. 2002. Preprocessing of intractable problems. Information and Computation 176(2):89–120. [ Cook, 1971 ] Cook, S. A. 1971. The complexity of theorem-proving procedures. In Proc. 3rd Annual Symp. on Theory of Computing , 151–158. [ Cooper , 1990 ] Cooper , G. F . 1990. The computational complexity of probabilistic inference using Bayesian belief networks. Artificial Intelligence 42(2-3):393–405. [ Dechter and Pearl, 1989 ] Dechter , R., and Pearl, J. 1989. Tree clustering for constraint networks. Arti- ficial Intelligence 38(3):353–366. [ Dechter , 2010 ] Dechter , R. 2010. Constraint satisfaction. In The CogNet Library: References Collection . http://cognet.mit.edu/library/erefs/mitecs/dechter .html: MIT Press. [ Downe y , Fellows, and Ste ge, 1999 ] Downe y , R.; Fellows, M. R.; and Stege, U. 1999. Parameterized complexity: A framework for systematically confronting computational intractability . In Contem- porary T rends in Discr ete Mathematics: F r om DIMACS and DIMA TIA to the Futur e , volume 49 of AMS-DIMA CS , 49–99. American Mathematical Society . [ E ´ en and Biere, 2005 ] E ´ en, N., and Biere, A. 2005. Effecti ve preprocessing in SA T through variable and clause elimination. In Bacchus, F ., and W alsh, T ., eds., Theory and Applications of Satisfiability T esting, 8th International Confer ence, SA T 2005, St. Andr ews, UK, June 19-23, 2005, Proceedings , volume 3569 of Lectur e Notes in Computer Science , 61–75. Springer V erlag. [ Fernau et al., 2009 ] Fernau, H.; Fomin, F . V .; Lokshtanov , D.; Raible, D.; Saurabh, S.; and V illanger , Y . 2009. K ernel(s) for problems with no kernel: On out-trees with many lea ves. In Albers, S., and Marion, J.-Y ., eds., 26th International Symposium on Theor etical Aspects of Computer Science, ST A CS 2009, F ebruary 26-28, 2009, F reib urg , Germany , Pr oceedings , volume 3 of LIPIcs , 421–432. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany . [ Fichte and Szeider , 2011 ] Fichte, J. K., and Szeider , S. 2011. Backdoors to tractable answer-set pro- gramming. In Pr oceedings of the International Joint Confer ence on Artificial Intelligence, IJCAI 2011 , 863–868. AAAI Press/IJCAI. [ Fortno w and Santhanam, 2008 ] Fortno w , L., and Santhanam, R. 2008. Infeasibility of instance compres- sion and succinct PCPs for NP. In Dwork, C., ed., Pr oceedings of the 40th Annual ACM Symposium on Theory of Computing, V ictoria, British Columbia, Canada, May 17-20, 2008 , 133–142. A CM. [ Gaspers and Szeider , 2011 ] Gaspers, S., and Szeider, S. 2011. K ernels for global constraints. In Pr o- ceedings of the International Joint Confer ence on Artificial Intelligence, IJCAI 2011 , 540–545. AAAI Press/IJCAI. 10 [ Gebser et al., 2008 ] Gebser , M.; Kaufmann, B.; Neumann, A.; and Schaub, T . 2008. Adv anced prepro- cessing for answer set solving. In Ghallab, M.; Spyropoulos, C. D.; Fakotakis, N.; and A vouris, N. M., eds., ECAI 2008 - 18th Eur opean Conference on Artificial Intelligence, P atras, Gr eece, J uly 21-25, 2008, Pr oceedings , volume 178 of F r ontiers in Artificial Intelligence and Applications , 15–19. IOS Press. [ Gelfond and Lifschitz, 1988 ] Gelfond, M., and Lifschitz, V . 1988. The stable model semantics for logic programming. In K owalski, R. A., and Bowen, K. A., eds., Logic Pr ogr amming, Proceedings of the F ifth International Conference and Symposium, Seattle, W ashington, August 15-19, 1988 , 1070–1080. MIT Press. [ Gomes et al., 2008 ] Gomes, C. P .; Kautz, H.; Sabharwal, A.; and Selman, B. 2008. Satisfiability solvers. In Handbook of Knowledge Representation , volume 3 of F oundations of Artificial Intelligence . Elsevier . 89–134. [ Gottlob, Scarcello, and Sideri, 2002 ] Gottlob, G.; Scarcello, F .; and Sideri, M. 2002. Fixed-parameter complexity in AI and nonmonotonic reasoning. Artificial Intelligence 138(1-2):55–86. [ Guo and Niedermeier , 2007 ] Guo, J., and Niedermeier , R. 2007. Invitation to data reduction and prob- lem kernelization. ACM SIGA CT News 38(2):31–45. [ Kloks, 1994 ] Kloks, T . 1994. T ree width: Computations and Appr oximations . Berlin: Springer V erlag. [ Marek and T ruszczy ´ nski, 1991 ] Marek, W ., and Truszczy ´ nski, M. 1991. Autoepistemic logic. J . of the A CM 38(3):588–619. [ Marek and T ruszczy ´ nski, 1999 ] Marek, V . W ., and Truszczy ´ nski, M. 1999. Stable models and an al- ternativ e logic programming paradigm. In The Logic Pr ogramming P aradigm: a 25-Y ear P erspective . Springer . 169–181. [ Niemel ¨ a, 1999 ] Niemel ¨ a, I. 1999. Logic programs with stable model semantics as a constraint program- ming paradigm. Ann. Math. Artif. Intell. 25(3-4):241–273. Logic programming with non-monotonic semantics: representing knowledge and its computation. [ Nishimura, Ragde, and Szeider , 2004 ] Nishimura, N.; Ragde, P .; and Szeider , S. 2004. Detecting back- door sets with respect to Horn and binary clauses. In Pr oceedings of SA T 2004 (Seventh International Confer ence on Theory and Applications of Satisfiability T esting, 10–13 May , 2004, V ancouver , BC, Canada) , 96–103. [ Papadimitriou, 1994 ] Papadimitriou, C. H. 1994. Computational Complexity . Addison-W esley . [ Pearl, 1988 ] Pearl, J. 1988. Pr obabilistic reasoning in intelligent systems: networks of plausible in- fer ence . The Morgan Kaufmann Series in Representation and Reasoning. San Mateo, CA: Morgan Kaufmann. [ Pearl, 2010 ] Pearl, J. 2010. Bayesian networks. In The CogNet Library: References Collection . http: //cognet.mit.edu/library/erefs/mitecs/pearl.html: MIT Press. [ Roth, 1996 ] Roth, D. 1996. On the hardness of approximate reasoning. Artificial Intelligence 82(1- 2):273–302. [ Samer and Szeider , 2010 ] Samer , M., and Szeider , S. 2010. Constraint satisfaction with bounded treewidth re visited. J. of Computer and System Sciences 76(2):103–114. [ van Hoe ve and Katriel, 2006 ] van Hoeve, W .-J., and Katriel, I. 2006. Global constraints. In Rossi, F .; van Beek, P .; and W alsh, T ., eds., Handbook of Constraint Pr ogramming . Else vier . chapter 6. [ W illiams, Gomes, and Selman, 2003 ] W illiams, R.; Gomes, C.; and Selman, B. 2003. On the connec- tions between backdoors, restarts, and heavy-tailedness in combinatorial search. In Informal Pr oc. of the Sixth International Confer ence on Theory and Applications of Satisfiability T esting, S. Mar gherita Ligur e - P ortofino, Italy , May 5-8, 2003 (SAT 2003) , 222–230. 11