The Divide-and-Conquer Subgoal-Ordering Algorithm for Speeding up Logic Inference
It is common to view programs as a combination of logic and control: the logic part defines what the program must do, the control part -- how to do it. The Logic Programming paradigm was developed with the intention of separating the logic from the c…
Authors: O. Ledeniov, S. Markovitch
Journal of Articial In telligence Researc h 9 (1998) 37-97 Submitted 2/98; published 9/98 The Divide-and-Conquer Subgoal-Ordering Algorithm for Sp eeding up Logic Inference Oleg Ledenio v olleg@cs.technion.a c.il Shaul Mark o vitc h sha ulm@cs.technion.a c.il Computer Scienc e Dep artment T e chnion { Isr ael Institute of T e chnolo gy Haifa 32000, Isr ael Abstract It is common to view programs as a com bination of logic and con trol: the logic part denes what the program m ust do, the con trol part { how to do it. The Logic Program- ming paradigm w as dev elop ed with the in ten tion of separating the logic from the con trol. Recen tly , extensiv e researc h has b een conducted on automatic generation of con trol for logic programs. Only a few of these w orks considered the issue of automatic generation of con trol for impro ving the eciency of logic programs. In this pap er w e presen t a no v el al- gorithm for automatic nding of lo w est-cost subgoal orderings. The algorithm w orks using the divide-and-conquer strategy . The giv en set of subgoals is partitioned in to smaller sets, based on co-o ccurrence of free v ariables. The subsets are ordered recursiv ely and merged, yielding a pro v ably optimal order. W e exp erimen tally demonstrate the utilit y of the algo- rithm b y testing it in sev eral domains, and discuss the p ossibilities of its co op eration with other existing metho ds. 1. In tro duction It is common to view programs as a com bination of logic and con trol (Ko w alski, 1979). The logic part denes what the program m ust do, the con trol part { how to do it. T raditional programming languages require that the programmers supply b oth comp onen ts. The Logic Programming paradigm w as dev elop ed with the in ten tion of separating the logic from the con trol (Llo yd, 1987). The goal of the paradigm is that the programmer sp ecies the logic without b othering ab out the con trol, whic h should b e supplied b y the in terpreter. Initially , most practical logic programming languages, suc h as Prolog (Clo c ksin & Mel- lish, 1987; Sterling & Shapiro, 1994), did not include the means for automatic generation of con trol. As a result, a Prolog programmer had to implicitly dene the con trol b y the order of clauses and of subgoals within the clauses. Recen tly , extensiv e researc h has b een conducted on automatic generation of con trol for logic programs. A ma jor part of this researc h is con- cerned with con trol that aects correctness and termination of logic programs (De Sc hrey e & Decorte, 1994; Somogyi, Henderson, & Con w a y , 1996b; Cortesi, Le Charlier, & Rossi, 1997). Only a few of these w orks consider the issue of automatic generation of con trol for impro ving the eciency of logic programs. Finding a go o d ordering that leads to ecien t execution requires a deep understanding of the logic inference mec hanism. Hence, in man y cases, only exp ert programmers are able to generate ecien t programs. The problem in ten- sies with the recen t dev elopmen t of the eld of inductiv e logic programming (Muggleton c 1998 AI Access F oundation and Morgan Kaufmann Publishers. All righ ts reserv ed. Ledenio v & Mark o vitch & De Raedt, 1994). There, logic programs are automatically induced b y learning. Suc h learning algorithms are commonly built with the aim of sp eeding up the induction pro cess without considering the eciency of resulting programs. The goal of the researc h describ ed in this pap er is to design algorithms that automat- ically nd ecien t orderings of subgoal sequences. Sev eral researc hers ha v e explored the problem of automatic reordering of subgoals in logic programs (W arren, 1981; Naish, 1985b; Smith & Genesereth, 1985; Natara jan, 1987; Mark o vitc h & Scott, 1989). The general sub- goal ordering problem is kno wn to b e NP-hard (Ullman, 1982; Ullman & V ardi, 1988). Smith and Genesereth (1985) and Mark o vitc h and Scott (1989) presen t searc h algorithms for nding optimal orderings. These algorithms are general and carry exp onen tial costs for non-trivial sets of subgoals. Natara jan (1987) describ es an ecien t algorithm for the sp ecial case where subgoals in the set do not share free v ariables. In this pap er w e presen t a no v el algorithm for subgoal ordering. W e call t w o subgoals that share a free v ariable dep endent . Unlik e Natara jan's approac h, whic h can only handle subgoal sets that are completely indep enden t, our algorithm can deal with an y subgoal set, while making maximal use of the existing dep endencies for acceleration of the ordering pro cess. In the w orst case the algorithm { lik e that of Smith and Genesereth { is exp onen tial. Still, in most practical cases, our algorithm exploits subgoal dep endencies and nds optimal orderings in p olynomial time. W e start with an analysis of the ordering problem and demonstrate its imp ortance through examples. W e then sho w ho w to compute the cost of a giv en ordering based on the cost and the n um b er of solutions of the individual subgoals. W e describ e the algorithm of Natara jan and the algorithm of Smith and Genesereth and sho w ho w the t w o can b e com bined in to an algorithm that is more ecien t and general than eac h of the t w o. W e sho w dra wbac ks of the com bined algorithm and in tro duce the new algorithm, whic h a v oids these dra wbac ks. W e call it the Divide-and-Conquer algorithm ( d a c algorithm) . W e pro v e the correctness of the algorithm, discuss its complexit y and compare it to the com bined algorithm. The d a c algorithm assumes kno wledge of the cost and the n um b er of solutions of the subgoals. This kno wledge can b e obtained b y mac hine learning tec hniques suc h as those emplo y ed b y Mark o vitc h and Scott (1989). Finally , w e test the utilit y of our algorithm b y running a set of exp erimen ts on articial and real domains. The d a c algorithm for subgoal ordering can b e com bined with man y existing metho ds in logic programming, suc h as program transformation, compilation, termination con trol, correctness v erication, and others. W e discuss the p ossibilities of suc h com binations in the concluding section. Section 2 states the ordering problem. Section 3 describ es existing ordering algorithms and their com bination. Section 4 presen ts the new algorithm. Section 5 discusses the acquisition of the con trol kno wledge. Section 6 con tains exp erimen tal results. Section 7 con tains a discussion of practical issues, comparison with other w orks and conclusions. 2. Bac kground: Automatic Ordering of Subgoals W e start b y describing the con v en tions and assumptions accepted in this pap er. Then w e demonstrate the imp ortance of subgoal ordering and discuss its v alidit y . Finally , w e presen t a classication of ordering metho ds and discuss related w ork. 38 The Divide-and-Conquer Subgo al-Ordering Algorithm 2.1 Con v en tions and Assumptions All constan t, function and predicate sym b ols in programs b egin with lo w er case letters, while capital letters are reserv ed for v ariables. Braces are used to denote unordered sets (e.g., f a; b; c g ), and angle brac k ets are used for ordered sequences (e.g., h a; b; c i ). P arallel lines ( k ) denote concatenations of ordered sequences of subgoals. When sp eaking ab out abstract subgoals (and not named predicates of concrete programs), w e denote separate subgoals b y capital letters ( A; B : : : ), ordered sequences of subgoals b y capitalized v ectors ( ~ B ; ~ O S : : : ), and sets of subgoals b y calligraphic capitals ( B ; S : : : ). ( S ) denotes the set of all p erm utations of S . W e assume that the programs w e w ork with are written in pur e Pr olo g , i.e., without cut op erators, meta-logical or extra-logical predicates. Alternativ ely , w e can assume that only pure Prolog sub-sequences of subgoals are sub ject to ordering. F or example, giv en a rule of the form A B 1 ; B 2 ; B 3 ; ! ; B 4 ; B 5 ; B 6 : only its nal part f B 4 ; B 5 ; B 6 g can b e ordered (without aecting the solution set). In this w ork w e fo cus up on the task of nding al l the solutions to a set of subgoals. 2.2 Ordering of Subgoals in Logic Programs A logic program is a set of clauses: A B 1 ; B 2 ; : : : ; B n : ( n 0) where A; B 1 ; : : : ; B n are literals (predicates with argumen ts). T o use suc h a clause for pro ving a goal that matc hes A , w e m ust pro v e that al l B -s hold sim ultaneously , under consisten t bindings of the free v ariables. A solution is suc h a set of v ariable bindings. The solution set of a goal is the bag of all its solutions created b y its program. A c omputation rule denes whic h subgoal will b e pro v ed next. In Prolog, the compu- tation rule alw a ys selects the leftmost subgoal in a goal. If a subgoal fails, b acktr acking is p erformed { the pro of of the previous subgoal is re-en tered to generate another solution. F or a detailed denition of the logic inference pro cess, see Llo yd (1987). Theorem 1 The solution set of a set of sub go als do es not dep end on the or der of their exe cution. Pro of: When w e are lo oking for all solutions, the solution set do es not dep end on the computation rule c hosen (Theorems 9.2 and 10.3 in Llo yd, 1987). Since a transp osition of subgoals in an ordered sequence can b e regarded as a c hange of the computation rule (the subgoals are selected in dieren t order), suc h transp osition do es not c hange the solution set. 2 This theorem implies that w e ma y reorder subgoals during the pro of deriv ation. Y et the eciency of the deriv ation strongly dep ends on the c hosen order of subgoals. The follo wing example illustrates ho w t w o dieren t orders can lead to a large dierence in execution eciency . 39 Ledenio v & Mark o vitch parent(abraham,isaac). parent(sarah,isaac). parent(abraham,ishmael). parent(isaac,esav). parent(isaac,jakov). ... More paren t clauses ... male(abraham). male(isaac). male(ishmael). male(jakov). male(esav). ... More male clauses ... brother(X,Y) male(X), parent(W,X), parent(W,Y), X=/=Y. father(X,Y) male(X), parent(X,Y). uncle(X,Y) parent(Z,Y), brother(X,Z). ... More rules of relations ... Figure 1: A small fragmen t of a Biblical database describing family relationships. Example 1 Consider a Biblical family database suc h as the one listed in Figure 1 (a similar database app ears in the b o ok b y Sterling & Shapiro, 1994). The b o dy of the rule dening the uncle-nephew (or uncle-niece) relation can b e ordered in t w o w a ys: 1. uncle(X,Y) brother(X,Z), parent(Z,Y). 2. uncle(X,Y) parent(Z,Y), brother(X,Z). T o pro v e the goal uncle(ishmael,Y) using the rst v ersion of the rule, the in terpreter will rst lo ok for Ishmael's siblings (and nd Isaac) and then for the siblings' c hildren (Esa v and Jaco v). The left part of Figure 2 sho ws the asso ciated pro of tree with a total of 10 no des. If w e use the second v ersion of the rule, the in terpreter will create all the paren t- c hild pairs a v ailable in the database, and will test for eac h paren t whether he (or she) is Ishmael's sibling. The righ t part of Figure 2 sho ws the asso ciated pro of tree with a total of 4( N 2) + 6 2 + 2 = 4 N + 6 no des, where N is the n um b er of paren t-c hild pairs in the database. The tree con tains t w o success branc hes and N 2 failure branc hes; in the gure w e sho w one example of eac h. While the t w o v ersions of the rule yield iden tical solution sets, the rst v ersion leads to a m uc h smaller tree and to a faster execution. Note that this result is true only for the giv en mo de ( bound,free ) of the head literal; for the mo de ( free,bound ), as in uncle(X,jacov) , the outcome is the con trary: the second v ersion of the rule yields a smaller tree. 2.3 Categories of Subgoal Ordering Metho ds Assume that the curren t conjunctiv e goal (the curren t resolv en t) is f A 1 ; A 2 g . Assume that w e use the rule \ A 1 A 11 ; A 12 : " to reduce A 1 . According to Theorem 1, the pro duced resolv en t, f A 11 ; A 12 ; A 2 g , can b e executed in an y order. W e call ordering metho ds that allo w an y p erm utation of the resolv en t interle aving or dering metho ds , since they p ermit 40 The Divide-and-Conquer Subgo al-Ordering Algorithm uncle(X,Y) brother(X,Z), parent(Z,Y). uncle(X,Y) parent(Z,Y), brother(X,Z). parent(ishmael,Y) ishmael =/= ishmael, uncle(ishmael,Y) brother(ishmael,Z), parent(Z,Y) male(ishmael), parent(W,ishmael), parent(W,Z), ishmael =/= Z, parent(Z,Y) parent(W,ishmael), parent(W,Z), ishmael =/= Z, parent(Z,Y) W=abraham parent(abraham,Z), ishmael=/=Z, parent(Z,Y) Z=ishmael Z=isaac isaac =/= ishmael, parent(isaac,Y) parent(isaac,Y) Y=esav Y=jacov W=abraham W=abraham Z=isaac, Y=cain Z=adam, other parent-child pairs Y=jacov parent(Z,Y), brother(ishmael,Z) brother(ishmael,isaac) parent(W,isaac), ishmael=/=isaac male(ishmael), parent(W,ishmael), parent(W,ishmael), parent(W,isaac), ishmael =/= isaac parent(abraham,isaac), ishmael=/=isaac ishmael =/= isaac brother(ishmael,adam) parent(W,adam), ishmael =/= adam male(ishmael), parent(W,ishmael), parent(W,ishmael), parent(W,adam), ishmael =/= adam parent(abraham,adam), ishmael =/=adam uncle(ishmael,Y) Figure 2: Tw o pro of trees obtained with dieren t orderings of a single rule in Example 1. in terlea ving of subgoals from dieren t rule b o dies. When ordering is p erformed only on rule b o dies b efor e using them for reduction, the metho d is non-interle aving . In the ab o v e example, in terlea ving metho ds will consider all 6 p erm utations of the resolv en t, while non- in terlea ving metho ds will consider only t w o orderings: h A 11 ; A 12 ; A 2 i and h A 12 ; A 11 ; A 2 i . In terlea ving ordering metho ds deal with signican tly more p ossible orderings than non- in terlea ving metho ds. That means that they can nd more ecien t orderings. On the other hand, the space of p ossible orderings ma y b ecome prohibitiv ely large, requiring to o man y computational resources. Subgoal ordering can tak e place at v arious stages of the pro of pro cess. W e divide all subgoal ordering metho ds in to static, semi-dynamic and dynamic. Static ordering: The rule b o dies are ordered b efore the execution starts. No order- ing tak es place during the execution. Semi-dynamic ordering: Whenev er a rule is selected for reduction, its b o dy is ordered. The order of its subgoals do es not c hange after the reduction tak es place. Dynamic ordering: The ordering decision is made at eac h inference step. Static metho ds add no o v erhead to the execution time. Ho w ev er, the optimal ordering of a rule often dep ends on a particular binding of a v ariable, whic h can b e kno wn only at run-time. F or instance, in Example 1 w e sa w that the rst ordering of the rule is b etter for pro ving the goal uncle(ishmael,Y) . And y et, for the goal uncle(X,jacov) , it is the second ordering that yields more ecien t execution. T o handle suc h cases statically , w e m ust compute the optimal ordering for eac h p ossible binding. 41 Ledenio v & Mark o vitch Ob viously , static ordering can only b e non-in terlea ving. The dynamic metho d is more exible, since it can use more up dated kno wledge ab out v ariable bindings, but it also carries the largest run time o v erhead, since it is in v ok ed sev eral times for eac h use of a rule b o dy . The semi-dynamic metho d is a compromise b et w een the t w o: it is more p o w erful than the static metho d, b ecause it can dynamically prop ose dieren t orderings for dieren t instances of the same rule; it also carries less o v erhead than the dynamic metho d, b ecause it is in v ok ed only once for eac h use of a rule b o dy . The total time of pro ving a goal is the sum of the ordering time and the inference time. In terlea ving and dynamic metho ds ha v e the b est p oten tial for reducing the inference time, but ma y signican tly augmen t the ordering time. Static metho ds do not dev ote time to ordering (it is done o-line), but ha v e a limited p oten tial for reducing the inference time. The algorithms describ ed in this pap er can b e used for all categories of ordering metho ds, although in the exp erimen ts describ ed in Section 6 w e ha v e only implemen ted semi-dynamic, non-in terlea ving ordering metho ds: on eac h reduction, the rule b o dy is ordered and added to the left end of the resolv en t, and then the leftmost literal of the resolv en t is selected for the next reduction step. 2.4 Related W ork The problem of computational ineciency of logic inference w as the sub ject of extensiv e researc h. The most ob vious asp ect of this ineciency is the p ossible non-termination of a pro of. Sev eral researc hers dev elop ed compile-time and run-time tec hniques to detect and a v oid innite computations (De Sc hrey e & Decorte, 1994). A certain success w as ac hiev ed in pro viding more adv anced con trol through emplo ymen t of co-routining for in ter- predicate sync hronization purp oses (Clark & McCab e, 1979; P orto, 1984; Naish, 1984). Also, innite computations can b e a v oided b y pruning innite branc hes that do not con tain solutions (V asak & P otter, 1985; Smith, Genesereth, & Ginsb erg, 1986; Bol, Apt, & Klop, 1991). In the NAIL! system (Morris, 1988) subgoals are automatically reordered to a v oid non termination. Still, ev en when the pro of is nite, it is desirable to mak e it more ecien t. Sev eral researc hers studied the problem of clause ordering (Smith, 1989; Cohen, 1990; Etzioni, 1991; Laird, 1992; Mo oney & Zelle, 1993; Greiner & Orp onen, 1996). If w e are lo oking for al l the solutions of a goal, then the eciency do es not dep end on the clause order (assuming no cut s). Indeed, if some predicate has m clauses, and for some argumen t bindings these clauses pro duce all their solutions in times t 1 ; t 2 : : : t m , then all solutions of the predicate under these bindings are obtained in time t 1 + t 2 + : : : + t m , regardless of the order in whic h the clauses are applied. Dieren t clause orderings corresp ond to dieren t orders in whic h branc hes are selected in a pro of tree; if w e tra v erse the en tire tree, then the n um b er of tra v ersal steps do es not dep end on the order of branc h selection, though the order of solutions found do es dep end on it. Subgoal ordering, as w as demonstrated in Example 1, can signican tly aect the e- ciency of pro ving a goal. There are t w o ma jor approac hes to subgoal ordering. The rst approac h uses v arious heuristics to order subgoals, for example: Cho ose a subgoal whose predicate has the smallest n um b er of matc hing clauses (Mink er, 1978). 42 The Divide-and-Conquer Subgo al-Ordering Algorithm Prefer a subgoal with more constan ts (Mink er, 1978). Cho ose a subgoal with the largest size , where the size is dened as the n um b er of o ccurrences of predicate sym b ols, function sym b ols, and v ariables (Nie & Plaisted, 1990). Cho ose a subgoal with the largest mass , where the mass of a subgoal dep ends on the frequency of its argumen ts and sub-argumen ts in the en tire goal (Nie & Plaisted, 1990). Cho ose a subgoal with the least n um b er of solutions (W arren, 1981; Nie & Plaisted, 1990). Apply \tests" b efore \generators" (Naish, 1985a). Prefer calls that fail quic kly (Naish, 1985b). The heuristic metho ds usually execute quic kly , but ma y yield sub optimal orderings. The second approac h, whic h is adopted in this pap er, aims at nding optimal order- ings (Smith & Genesereth, 1985; Natara jan, 1987; Mark o vitc h & Scott, 1989). Natara jan prop osed an ecien t w a y to order a sp ecial sort of subgoal set (where all subgoals are in- dep enden t), while Smith and Genesereth prop osed a general, but inecien t algorithm. In the follo wing section w e build a unifying framew ork for dealing with subgoal ordering and describ e v ariations on Natara jan's and Smith and Genesereth's algorithms. W e also sho w ho w the t w o can b e com bined for increased eciency . 3. Algorithms for Subgoal Ordering in Logic Programs The goal of the w ork presen ted here is to order subgoals for sp eeding up logic programs. This section starts with an analysis of the cost of executing a sequence of subgoals. The resulting form ula is the basis for the subsequen t ordering algorithms. Then w e discuss dep endence of subgoals and presen t existing ordering algorithms for indep enden t and dep enden t sets of subgoals. Finally , w e com bine these algorithms in to a more general and ecien t one. 3.1 The Cost of Executing a Sequence of Subgoals In this subsection w e analyze the cost of executing a sequence of subgoals. The analysis builds mainly on the w ork of Smith and Genesereth (1985). Let S = f A 1 ; A 2 ; : : : A k g b e a set of subgoals and b b e a binding. W e denote S ol s ( S ) to b e the solution set of S , and dene S ol s ( ; ) = f;g . W e denote A i j b to b e A i whose v ariables are b ound according to b ( A i j ; = A i ). Finally , w e denote C ost ( A i j b ) to b e the amoun t of resources needed for pro ving A i j b . C ost ( A i j b ) should reect the time complexit y of pro ving A i under binding b . F or example, the n um b er of unication steps is a natural measure of complexit y for logic programs (Itai & Mak o wsky , 1987). T o obtain the cost of nding all the solutions of an ordered sequence of subgoals ~ S = h A 1 ; A 2 ; A 3 ; : : : A n i ; (1) 43 Ledenio v & Mark o vitch w e note that the pro of-tree of A 1 is tra v ersed only once, the tree of A 2 is tra v ersed once for eac h solution generated b y A 1 , the tree of A 3 { once for eac h solution of f A 1 ; A 2 g , etc. Consequen tly , the total cost of pro ving Equation 1 is C ost ( h A 1 ; : : : A n i ) = C ost ( A 1 ) + X b 2 S ols ( f A 1 g ) C ost ( A 2 j b ) + : : : + X b 2 S ols ( f A 1 ;::: A n 1 g ) C ost ( A n j b ) = = n X i =1 X b 2 S ols ( f A 1 ;:::A i 1 g ) C ost ( A i j b ) : (2) T o compute Equation 2 one m ust kno w the cost and the solution set for eac h subgoal under eac h binding. T o reduce the amoun t of information needed, w e deriv e an equiv alen t form ula, whic h uses a v erage cost and a v erage n um b er of solutions. Denition: Let B b e a set of subgoals, A a subgoal. Dene c ost ( A ) j B to b e the a v erage cost of A o v er all solutions of B and n sols ( A ) j B to b e its a v erage n um b er of solutions o v er all solutions of B : c ost ( A ) j B = 8 > > < > > : C ost ( A ) ; B = ; P b 2 S ols ( B ) C ost ( A j b ) j S ols ( B ) j ; B 6 = ; ; S ol s ( B ) 6 = ; undene d ; B 6 = ; ; S ol s ( B ) = ; n so ls ( A ) j B = 8 > > < > > : j S ol s ( f A g ) j ; B = ; P b 2 S ols ( B ) j S ols ( f A j b g ) j j S ols ( B ) j ; B 6 = ; ; S ol s ( B ) 6 = ; undene d ; B 6 = ; ; S ol s ( B ) = ; F rom the rst denition, it follo ws that: X b 2 S ols ( f A 1 ;::: A i 1 g ) C ost ( A i j b ) = j S ol s ( f A 1 ; : : : A i 1 g ) j c ost ( A i ) j f A 1 ;:::A i 1 g : (3) If w e apply the second denition recursiv ely , w e obtain j S ol s ( f A 1 ; : : : A i g ) j = X b 2 S ols ( f A 1 ;::: A i 1 g ) j S ol s ( f A i j b g ) j = j S ol s ( f A 1 ; : : : A i 1 g ) j n sols ( A i ) j f A 1 ;:::A i 1 g = : : : = i Y j =1 n sols ( A j ) j f A 1 ::: A j 1 g : (4) Note that w e dened S ol s ( ; ) = f;g ; th us, these equations hold also for i = 1. Incorp oration of Equations 3 and 4 in to Equation 2 yields C ost ( h A 1 ; A 2 ; : : : A n i ) = n X i =1 2 4 0 @ i 1 Y j =1 n so ls ( A j ) j f A 1 :::A j 1 g 1 A c ost ( A i ) j f A 1 ::: A i 1 g 3 5 : (5) 44 The Divide-and-Conquer Subgo al-Ordering Algorithm F or eac h subgoal A i , its a v erage cost is m ultiplied b y the total n um b er of solutions of all the preceding subgoals. W e can dene a v erage cost and n um b er of solutions for ev ery con tin uous sub-sequence of Equation 1: 8 k 1 ; k 2 ; 1 k 1 k 2 n , c os t ( h A k 1 ; : : : A k 2 i ) j f A 1 ;::: A k 1 1 g = c ost ( h A 1 ; : : : A k 2 i ) j ; c ost ( h A 1 ; : : : A k 1 1 i ) j ; n so ls ( h A 1 ; : : : A k 1 1 i ) j ; (6) = k 2 X i = k 1 2 4 0 @ i 1 Y j = k 1 n so ls ( A j ) j f A 1 ;:::A j 1 g 1 A c ost ( A i ) j f A 1 ;:::A i 1 g 3 5 n so ls ( h A k 1 ; : : : A k 2 i ) j f A 1 ;::: A k 1 1 g = n so ls ( h A 1 ; : : : A k 2 i ) j ; n sols ( h A 1 ; : : : A k 1 1 i ) j ; = k 2 Y i = k 1 n so ls ( A i ) j f A 1 ;::: A i 1 g (7) The v alues of c ost ( A i ) and n so ls ( A i ) dep end on the p osition of A i in the ordered se- quence. F or example, assume that w e w an t to nd Abraham's sons, using the domain of Example 1. The unordered conjunctiv e goal is f male(Y),parent(abraham,Y) g . Let there b e N males in the database (t w o of them, Isaac and Ishmael, are Abraham's sons): n sols ( male(Y) ) j ; = N n so ls ( parent(abraham,Y) ) j ; = 2 n sols ( male(Y) ) j f parent( abr ah am, Y) g = 1 n so ls ( parent(abraham,Y) ) j f male(Y) g = 2 = N Note that n sols ( h male(Y),parent(abrah am,Y) i ) = 2 = n so ls ( h parent(abraham,Y),m ale(Y) i ), exactly as Theorem 1 predicts. Ha ving dened the cost of a sequence of subgoals, w e can no w dene the ob jectiv e of our ordering algorithms: Denition: Let S b e a set of subgoals. Dene ( S ) to b e set of all p erm utations of S . ~ O S 2 ( S ) is a minimal or dering of S (denoted M in ( ~ O S ; S )), if its cost according to Equation 5 is minimal o v er all p ossible p erm utations of S : M in ( ~ O S ; S ) ( ) 8 O 0 S 2 ( S ) : C ost ( ~ O S ) C ost ( O 0 S ) : The total execution time is the sum of the time whic h is sp en t on ordering, and the inference time sp en t b y the in terpreter on the ordered sequence. In this pap er w e fo cus up on dev eloping algorithms for minimizing the inference time. Elsewhere (Ledenio v & Mark o vitc h, 1998a, 1998b) w e presen t algorithms that attempt to reduce the total execution time. The v alues of cost and n um b er of solutions can b e obtained in v arious w a ys: b y exact computation, b y estimation and b ounds, and b y learning. Let us assume at the momen t that there exists a mec hanism that returns the a v erage cost and n um b er of solutions of a subgoal in time . In Section 5 w e sho w ho w this con trol kno wledge can b e obtained b y inductiv e learning. 3.2 Ordering of Indep enden t Sets of Subgoals The general subgoal ordering problem is NP-hard (Ullman & V ardi, 1988). Ho w ev er, there is a sp ecial case where ordering can b e p erformed ecien tly: if all the subgoals in the 45 Ledenio v & Mark o vitch giv en set are indep enden t, i.e. do not share free v ariables. This section b egins with the denition of subgoal dep endence and related concepts. W e then sho w an ordering algorithm for indep enden t sets and pro v e its correctness. 3.2.1 Dependence of Subgo als Denition: Let S and B b e sets of subgoals ( B is called the binding set of S ). A pair of subgoals in S is dir e ctly dep endent under B , if they share a free v ariable not b ound b y a subgoal of B . A pair of subgoals is indir e ctly dep endent with resp ect to S and B if there exists a third subgoal in S whic h is directly dep enden t on one of them under B , and dep enden t (directly or indirectly) on the other one under B . A pair of subgoals of S is indep endent under B if it is not dep enden t under B (either directly or indirectly). A subgoal is indep endent of S under B if it is indep enden t of all mem b ers of S under B . Tw o subsets S 1 S and S 2 S are mutual ly indep endent under the binding set B if ev ery pair of subgoals ( A 1 ; A 2 ), suc h that A 1 2 S 1 and A 2 2 S 2 , is indep enden t under B . The en tire set S is called indep endent under the binding set B if all its subgoal pairs are indep enden t under B , and is called dep endent otherwise. A dep enden t set of subgoals is called indivisible if all its subgoal pairs are dep enden t under B , and divisible otherwise. A divisibility p artition of S under B , D P ar t ( S ; B ), is a partition of S in to subsets that are m utually indep enden t and indivisibl e under B , except at most one subset whic h con tains all the subgoals indep enden t of S under B . It is easy to sho w that D P ar t ( S ; B ) is unique. F or example, let S 0 = f a; b ( X ) ; c ( Y ) ; d ( X ; Y ) ; e ( Z ) ; f ( Z ; V ) ; h ( W ) g . With resp ect to S 0 and an empt y binding set, the pair f b ( X ) ; d ( X ; Y ) g is directly dep enden t, f b ( X ) ; c ( Y ) g is indirectly dep enden t and f b ( X ) ; e ( Z ) g is indep enden t. If w e represen t a set of subgoals as a graph, where subgoals are v ertices and directly dep enden t subgoals are connected b y edges, then dep endence is equiv alen t to connectivit y and indivisibl e subsets are equiv alen t to connected comp onen ts of size greater than 1. The divisibil it y partition is the partition of a graph in to connected comp onen ts, with all the \lonely" v ertices collected together, in a sp ecial comp onen t. Figure 3 sho ws an example of suc h a graph for the set S 0 and for an empt y binding set. The whole set is divisible in to four m utually indep enden t subsets. The subsets f e ( Z ) ; f ( Z ; V ) g and f b ( X ) ; c ( Y ) ; d ( X ; Y ) g are indivisibl e. Elemen ts of the divisibil it y partition D P ar t ( S 0 ; ; ) are sho wn b y dotted lines. If a subgoal is indep enden t of the set, then its a v erage cost and n um b er of solutions do not dep end on its p osition within the ordered sequence: c ost ( A ) j B = P b 2 S ols ( B ) C ost ( A j b ) j S ol s ( B ) j = j S ol s ( B ) j C ost ( A ) j S ol s ( B ) j = C ost ( A ) ; n so ls ( A ) j B = P b 2 S ols ( B ) j S ol s ( f A j b g ) j j S ol s ( B ) j = j S ol s ( B ) j j S ol s ( f A g ) j j S ol s ( B ) j = j S ol s ( f A g ) j : In this case w e can omit the binding information and write c ost ( A i ) instead of c ost ( A i ) j f A 1 :::A i 1 g , and n sols ( A i ) instead of n sols ( A i ) j f A 1 :::A i 1 g . In practice, program rule b o dies rarely feature indep enden t sets of literals. An example is the follo wing clause, whic h states that c hildren lik e candy: 46 The Divide-and-Conquer Subgo al-Ordering Algorithm f a, b(X), c(Y), d(X,Y), e(Z), f(Z,V), h(W) g ) e(Z) h(W) f(Z,V) c(Y) a b(X) d(X,Y) Figure 3: An example of a graph represen ting a set of subgoals. Directly dep enden t subgoals are connected b y edges. Indep enden t subgoals and indivisible subsets are equiv alen t to connected comp onen ts (surrounded b y dashed lines). The divisibilit y partition (under the empt y binding set) is sho wn b y dotted lines. likes(X,Y) child(X), candy(Y). More often, indep enden t rule b o dies app ear not b ecause they are written as suc h in the program text, but b ecause some v ariables are b ound in (initially dep enden t) rule b o dies, as a result of clause head unication. F or example, if the rule father(X,Y) male(X), parent(X,Y). is used to reduce father(abraham,W) , then X is b ound to abraham , and the rule b o dy b ecomes indep enden t. Rule b o dies often b ecome indep enden t after substitutions are p er- formed in the course of the inference pro cess. 3.2.2 Algorithm f or Ordering Independent Sets by Sor ting Let ~ S b e an ordered sub-sequence of subgoals, B a set of subgoals. W e denote cn ( ~ S ) j B = n so ls ( ~ S ) j B 1 c os t ( ~ S ) j B : The name \ cn " reects the participation of c ost and n so ls in the denition. When the sub- sequence ~ S is indep enden t of other subgoals, the binding information ( j B ) can b e omitted. T ogether, the a v erage cost, a v erage n um b er of solutions, and cn v alue of a subgoal will b e called the c ontr ol values of this subgoal. F or indep enden t sets, there exists an ecien t ordering algorithm, listed in Figure 4. The complexit y of this algorithm is O ( n ( + log n )): O ( n ) to obtain the con trol v alues of n subgoals, and O ( n log n ) to p erform the sorting (Kn uth, 1973). T o enable the division, w e m ust dene the cost so that c ost ( A i ) is alw a ys p ositiv e. If w e dene the cost as the n um b er of unications p erformed, then alw a ys c ost ( A i ) 1, under a reasonable assumption that predicates of all rule b o dy subgoals are dened in the program. (In this case, at least one unication is p erformed for eac h subgoal). Similar algorithms w ere prop osed b y Simon and Kadane (1975) and Natara jan (1987). Example 2 L et the set of indep endent sub go als b e f p; q ; r g , with the fol lowing c ontr ol val- ues: 47 Ledenio v & Mark o vitch Algorithm 1 L et S = f A 1 ; A 2 ; : : : A n g b e a set of sub go als. Sort S using cn ( A i ) = n sols ( A i ) 1 c ost ( A i ) as the key for A i , and r eturn the r esult. Figure 4: The algorithm for ordering subgoals b y sorting. p q r c ost 10 20 5 n sols 1 5 0 : 1 cn 0 0 : 2 0 : 18 We c ompute the c osts of al l p ossible or derings, using Equation 5: C ost ( h p; q ; r i ) = 10 + 1 20 + 1 5 5 = 55 C ost ( h p; r ; q i ) = 10 + 1 5 + 1 0 : 1 20 = 17 C ost ( h q ; p; r i ) = 20 + 5 10 + 5 1 5 = 95 C ost ( h q ; r ; p i ) = 20 + 5 5 + 5 0 : 1 10 = 50 C ost ( h r ; p; q i ) = 5 + 0 : 1 10 + 0 : 1 1 20 = 8 C ost ( h r ; q ; p i ) = 5 + 0 : 1 20 + 0 : 1 5 10 = 12 The minimal or dering is h r ; p; q i , and this is exactly the or dering which is found much mor e quickly by A lgorithm 1 for the set f p; q ; r g : r has the smal lest cn value, 0 : 18 , then go es p with cn ( p ) = 0 , and nal ly q with cn ( q ) = 0 : 2 . Note that the sorting algorithm reects a w ell-kno wn principle: The b est implemen ta- tions of generate-and-test programs are obtained with the tests placed as early as p ossible in the rule b o dy and the generations as late as p ossible (Naish, 1985a). Of course, the c heap tests should come rst, while the exp ensiv e ones should come last. If one lo oks at the cn measure, one quic kly realizes that tests should b e put in fron t (b ecause n sols < 1, so cn < 0), while generator subgoals should mo v e to w ards the end ( n so ls > 1, so cn > 0). The w eakness of the \test-rst" principle is in the fact that not ev ery subgoal can b e easily tagged as a test or a generator. If one subgoal has n sols < 1 and another one has n sols > 1, then their order is ob vious ev en without lo oking at the costs (b ecause their cn v alues ha v e dieren t signs). But if b oth subgoals ha v e n so ls < 1, or b oth ha v e n so ls > 1, then the decision is not so simple. Sorting b y cn can correctly handle all the p ossible cases. 3.2.3 Correctness Pr oof of the Sor ting Algorithm f or Independent Sets W e sa w that Algorithm 1 found a minimal ordering in Example 2. W e are no w going to pro v e that Algorithm 1 always nds a minimal ordering for indep enden t sets. First w e sho w an imp ortan t lemma whic h will also b e used in further discussion. This lemma states 48 The Divide-and-Conquer Subgo al-Ordering Algorithm that substitution of a sub-sequence b y its c heap er p erm utation mak es the en tire sequence c heap er. Lemma 1 L et ~ S = ~ A k ~ B k ~ C , ~ S 0 = ~ A k ~ B 0 k ~ C , wher e ~ B and ~ B 0 ar e p ermutations of one another, and ~ A either is empty or has n sols( ~ A ) > 0 . Then C ost ( ~ S ) < C ost ( ~ S 0 ) ( ) c ost( ~ B ) j ~ A < c os t( ~ B 0 ) j ~ A ; C ost ( ~ S ) = C ost ( ~ S 0 ) ( ) c ost( ~ B ) j ~ A = c os t( ~ B 0 ) j ~ A : Pro of: If ~ A and ~ C are not empt y , C ost ( ~ S ) C ost ( ~ S 0 ) = C ost ( ~ A k ~ B k ~ C ) C ost ( ~ A k ~ B 0 k ~ C ) = (5) = c ost ( ~ A ) j ; + n sols ( ~ A ) j ; c ost ( ~ B ) j ~ A + n sols ( ~ A k ~ B ) j ; c ost ( ~ C ) j ~ A k ~ B c ost ( ~ A ) j ; + n sols ( ~ A ) j ; c ost ( ~ B 0 ) j ~ A + n sols ( ~ A k ~ B 0 ) j ; c os t ( ~ C ) j ~ A k ~ B 0 : By Theorem 1, ~ B and ~ B 0 pro duce the same solution sets. Hence, the third terms in the paren theses ab o v e are equal, and C ost ( ~ S ) C ost ( ~ S 0 ) = n sols ( ~ A ) j ; c ost ( ~ B ) j ~ A c os t ( ~ B 0 ) j ~ A : Since n so ls ( ~ A ) > 0, the sign of C ost ( ~ S ) C ost ( ~ S 0 ) coincides with the sign of c os t ( ~ B ) j ~ A c ost ( ~ B 0 ) j ~ A . If ~ A or ~ C is empt y , the pro of is similar. 2 Denition: Let ~ S = ~ A k ~ B 1 k ~ C k ~ B 2 k ~ D b e an ordered sequence of subgoals ( ~ A , ~ C and ~ D ma y b e empt y sequences). With resp ect to ~ S , the pair h ~ B 1 ; ~ B 2 i is cn-or der e d , if cn ( ~ B 1 ) j ~ A cn ( ~ B 2 ) j ~ A [ ~ B 1 [ ~ C cn-inverte d , if cn ( ~ B 1 ) j ~ A > cn ( ~ B 2 ) j ~ A [ ~ B 1 [ ~ C W e no w sho w that t w o adjacen t mutual ly indep endent sequences of subgoals in a minimal ordering m ust b e cn-ordered. Lemma 2 L et ~ S = ~ A k ~ B 1 k ~ B 2 k ~ C , ~ S 0 = ~ A k ~ B 2 k ~ B 1 k ~ C , wher e ~ B 1 , ~ B 2 ar e mutual ly indep endent under ~ A . L et ~ A either b e empty or have n sols ( ~ A ) > 0 . Then C ost ( ~ S ) < C ost ( ~ S 0 ) ( ) cn ( ~ B 1 ) j ~ A < cn ( ~ B 2 ) j ~ A ; C ost ( ~ S ) = C ost ( ~ S 0 ) ( ) cn ( ~ B 1 ) j ~ A = cn ( ~ B 2 ) j ~ A : 49 Ledenio v & Mark o vitch Pro of: C ost ( ~ S ) < C ost ( ~ S 0 ) Lemma 1 ( ) c ost ( ~ B 1 k ~ B 2 ) j ~ A < c ost ( ~ B 2 k ~ B 1 ) j ~ A ( ) c ost ( ~ B 1 ) j ~ A + n so ls ( ~ B 1 ) j ~ A c ost ( ~ B 2 ) j ~ A [ ~ B 1 < c ost ( ~ B 2 ) j ~ A + n so ls ( ~ B 2 ) j ~ A c ost ( ~ B 1 ) j ~ A [ ~ B 2 indep. f ~ B 1 ; ~ B 2 g ( ) c ost ( ~ B 1 ) j ~ A + n so ls ( ~ B 1 ) j ~ A c ost ( ~ B 2 ) j ~ A < c ost ( ~ B 2 ) j ~ A + n so ls ( ~ B 2 ) j ~ A c ost ( ~ B 1 ) j ~ A ( ) n sols ( ~ B 1 ) j ~ A c ost ( ~ B 2 ) j ~ A c ost ( ~ B 2 ) j ~ A < n sols ( ~ B 2 ) j ~ A c ost ( ~ B 1 ) j ~ A c ost ( ~ B 1 ) j ~ A c os t ( ~ B i ) j ~ A > 0 ( ) n sols ( ~ B 1 ) j ~ A 1 c ost ( ~ B 1 ) j ~ A < n sols ( ~ B 2 ) j ~ A 1 c ost ( ~ B 2 ) j ~ A ( ) cn ( ~ B 1 ) j ~ A < cn ( ~ B 2 ) j ~ A C ost ( ~ S ) = C ost ( ~ S 0 ) ( ) cn ( ~ B 1 ) j ~ A = cn ( ~ B 2 ) j ~ A | similar. 2 In an indep endent set , all subgoal pairs are indep enden t, in particular all adjacen t pairs. So, in a minimal ordering of an indep enden t set, all adjacen t subgoal pairs m ust b e cn- ordered; otherwise, the cost of the sequence can b e reduced b y a transp osition of suc h pair. This conclusion is expressed in the follo wing theorem. Theorem 2 L et S b e an indep endent set. L et ~ S b e an or dering of S . ~ S is minimal i al l the sub go als in ~ S ar e sorte d in non-de cr e asing or der by their cn values. Pro of: 1. Let ~ S b e a minimal ordering of S . If ~ S con tains a cn-in v erted adjacen t pair of subgoals, then transp osition of this pair reduces the cost of ~ S (Lemma 2), con tradicting the minimalit y of ~ S . 2. Let ~ S b e some ordering of S , whose subgoals are sorted in non-decreasing order b y cn . Let ~ S 0 b e a minimal ordering of S . According to item 1, ~ S 0 is also sorted b y cn . The only p ossible dierence b et w een the t w o sequences is the in ternal ordering of sub-sequences with equal cn v alues. The ordering of eac h suc h sub-sequence in ~ S can b e transformed to the ordering of its coun terpart sub-sequence in ~ S 0 b y a nite n um b er of transp ositions of adjacen t subgoals. By Lemma 2, transp ositions of adjacen t indep enden t subgoals with equal cn v alues cannot c hange the cost of the sequence. Therefore, C ost ( ~ S ) = C ost ( ~ S 0 ), and ~ S is a minimal ordering of S (since ~ S 0 is minimal). 2 Corollary 1 A lgorithm 1 nds a minimal or dering of an indep endent set of sub go als. 50 The Divide-and-Conquer Subgo al-Ordering Algorithm 3.3 Ordering of Dep enden t Sets of Subgoals Algorithm 1 do es not guaran tee nding a minimal ordering when the giv en set of subgoals is dep enden t, as the follo wing prop osition sho ws. Prop osition 1 When the given set of sub go als is dep endent, then: 1. The r esult of A lgorithm 1 on it is not always dene d. 2. Even when the r esult is dene d, it is not always a minimal or dering of the set. Pro of: Both claims are pro v ed b y coun ter-examples. 1. W e sho w a set of subgoals that cannot b e ordered b y sorting. The program: a(c1). b(c1). a(c2). b(c2). Con trol v alues: a ( X ) j ; a ( X ) j f b ( X ) g b ( X ) j ; b ( X ) j f a ( X ) g c os t 2 2 2 2 n so ls 2 1 2 1 cn 1 2 0 1 2 0 The set f a(X), b(X) g has t w o p ossible orderings, h a ( X ) ; b ( X ) i and h b ( X ) ; a ( X ) i . Both orderings ha v e minimal cost, though neither one is sorted b y cn : eac h ordering has cn = 1 2 for its rst subgoal, and cn = 0 for the second one. Sorting b y cn is imp ossible here: when w e transp ose subgoals, their cn v alues are c hanged, and the pair b ecomes cn-in v erted again. 2. W e sho w a set of subgoals that can b e ordered b y sorting, but its sorted ordering is not minimal. The program: a(c1). a(c1). b(c1). b(c2) a(c1), a(c2). Con trol v alues: a ( X ) j ; a ( X ) j f b ( X ) g b ( X ) j ; b ( X ) j f a ( X ) g c os t 2 2 8 2 n so ls 2 2 1 1 cn 1 2 1 2 0 0 Let the unordered set of subgoals b e f a(X), b(X) g . Its ordering h b ( X ) ; a ( X ) i is sorted b y cn , while h a ( X ) ; b ( X ) i is not. But h a ( X ) ; b ( X ) i is c heap er than h b ( X ) ; a ( X ) i : c ost ( h a ( X ) ; b ( X ) i ) = 2 + 2 2 = 6 c os t ( h b ( X ) ; a ( X ) i ) = 8 + 1 2 = 10 2 Since sorting cannot guaran tee minimal ordering for dep enden t subgoals, w e no w con- sider alternativ e ordering algorithms. The simplest algorithm c hec ks ev ery p ossible p erm u- tation of the set and returns the one with the minimal cost. The listing for this algorithm is sho wn in Figure 5. This algorithm runs in O ( n !) time, where is the time it tak es to compute the con trol v alues for one subgoal, and n is the n um b er of subgoals. The follo wing observ ation can help to reduce the ordering time at the exp ense of addi- tional space. Ordered sequences can b e constructed incremen tally , b y adding subgoals to 51 Ledenio v & Mark o vitch Algorithm 2 F or e ach p ermutation of sub go als, nd its c ost ac c or ding to Equation 5. Stor e the curr ently che ap est p ermutation and up date it when a che ap er one is found. Final ly, r eturn the che ap est p ermutation. Figure 5: The algorithm for subgoal ordering b y an exhaustiv e c hec k of all p erm utations. Algorithm 3 Order ( S ) let P 0 f;g ; n jS j lo op for k = 1 to n P 0 k n ~ P k B ~ P 2 P k 1 ; B 2 S n ~ P o P k n ~ P 2 P 0 k 8 ~ P 0 2 P 0 k ; h per mutation ( ~ P ; ~ P 0 ) ) C ost ( ~ P ) C ost ( ~ P 0 ) i o Return the single memb er of P n . Figure 6: The ordering algorithm whic h c hec ks p erm utations of ordered prexes. the righ t ends of ordered prexes. By Lemma 1, if a c heap er p erm utation of a prex exists, then this prex cannot b elong to a minimal ordering. The ordering algorithm can build prexes with increasing lengths, at eac h step adding to the righ t end of eac h prex one of the subgoals that do not app ear in it already , and for eac h subset k eeping only its c heap est p erm utation (if sev eral p erm utations ha v e equal cost, an y one of them can b e c hosen). The listing for this algorithm is sho wn in Figure 6. A t eac h step k , P 0 k stores the set of prexes from step k 1 extended b y ev ery subgoal not app earing there already . P k P 0 k , and in P k eac h subset of subgoals is represen ted only b y its c heap est p erm utation. Ob viously , jP k j = ( n k ) (one prex is k ept for ev ery subset of S of size k ). F or eac h prex of length k 1, there are n ( k 1) p ossible con tin uations of length k . The size of P 0 k is as follo ws: jP 0 k j = ( n k 1 ) ( n ( k 1)) = n ! ( n ( k 1))!( k 1)! ( n ( k 1)) = k k n ! ( n k )!( k 1)! = k ( n k ) : F or eac h prex, w e compute its cost in time. The p erm utation test can b e completed in O ( n ) time, b y using, for example, a trie structure (Aho et al., 1987), where subgoals in prexes are sorted lexicographically . Eac h step k tak es O (( n + ) k ( n k )) time, and the 52 The Divide-and-Conquer Subgo al-Ordering Algorithm whole algorithm runs in n X k =1 O (( n + ) k ( n k )) = O ( n ( n + ) n X k =1 ( n k )) = O ( n ( n + ) 2 n ) : If = O ( n ), this mak es O ( n 2 2 n ). Smith and Genesereth (1985) and Natara jan (1987) p oin t out that in a minimal ordered sequence ev ery adjacen t pair of subgoals m ust satisfy an adjac ency r estriction . The most general form of suc h a restriction in our notation sa ys that t w o adjacen t subgoals A k and A k +1 in a minimal ordering h A 1 ; A 2 : : : A n i m ust satisfy c ost ( h A k ; A k +1 i ) j f A 1 ::: A k 1 g c ost ( h A k +1 ; A k i ) j f A 1 ::: A k 1 g : (8) The restriction follo ws immediately from Lemma 1. Ho w ev er, it can only help to nd a lo c al ly minimal ordering, i.e., an ordering that cannot b e impro v ed b y transp ositions of adjacen t subgoals. It is p ossible that all adjacen t subgoal pairs satisfy Equation 8, but the ordering is still not minimal. The follo wing example illustrates this statemen t. Example 3 L et the unor der e d set b e f p ( X ) ; q ( X ) ; r ( X ) g , wher e the pr e dic ates ar e dene d by the fol lowing pr o gr am: p ( c 1 ) : q ( c 1 ) : r ( c 1 ) : p ( c 2 ) f : q ( c 2 ) : r ( c 1 ) : q ( c 3 ) f : f fails after 50 unic ations. The or dering h p ( X ) ; q ( X ) ; r ( X ) i satises the adjac ency r estriction (Equation 8): c ost( p ( X ) ; q ( X ) ) j ; = 55 c ost( q ( X ) ; r ( X ) ) j p ( X ) = 5 c ost( q ( X ) ; p ( X ) ) j ; = 107 c ost( r ( X ) ; q ( X ) ) j p ( X ) = 8 But it is not minimal: c ost( h p ( X ) ; q ( X ) ; r ( X ) i ) = 57 c ost( h r ( X ) ; p ( X ) ; q ( X ) i ) = 12 T o nd a globally minimal ordering, it seems b enecial to com bine the prex algorithm with the adjacency restriction: if a prex do es not satisfy the adjacency restriction, then there is a c heap er p erm utation of this prex. The adjacency test can b e p erformed faster than the p erm utation test, since it m ust only consider the t w o last subgoals of eac h pre- x. Nev ertheless, the n um b er of prexes remaining after eac h step of Algorithm 3 is not reduced: if a prex is rejected due to a violation of the adjacency restriction, it w ould ha v e also b een rejected b y the p erm utation test. F urthermore, if the adjacency restriction test do es not fail, w e should still p erform the p erm utation test to a v oid lo cal minima (as in Example 3). The adjacency test succeeds in at least half of the cases: if w e examine a prex h A 1 ; : : : A k ; B 1 ; B 2 i , w e shall also examine h A 1 ; : : : A k ; B 2 ; B 1 i , and the adjacency test cannot fail in b oth. Consequen tly , addition of the adjacency test can only halv e the total running time of the ordering algorithm, lea ving it O ( n 2 2 n ) in the w orst case. 53 Ledenio v & Mark o vitch Smith and Genesereth prop ose p erforming a b est-rst searc h in the space of ordered prexes, preferring prexes with lo w er cost. The b est-rst searc h can b e com bined with the p erm utation test and the adjacency restriction. In addition, when the subgoals not in a prex are indep enden t under its binding, they can b e sorted, and the sorted result concatenated to the prex. By Lemma 1 and Corollary 1, this pro duces the c heap est completion of this prex. When w e p erform completion, there is no need to p erform the adjacency or p erm utation test: if a complete sequence is not minimal, it will nev er b e c hosen as the c heap est prex; ev en if it is added to the list of prexes, it will nev er b e extracted therefrom. The resulting algorithm is sho wn in Figure 7. Algorithm 4 Order ( S ) let pr ex-list ; , pr ex ; , r est S lo op un til empt y ( r est ) if Indep enden t ( r est j pr ex ) then let c ompletion pr ex k Sort-b y-cn ( r est j pr ex ) Insert-By-Cost ( c ompletion , pr ex-list ) else lo op for sub go al 2 r est let extension pr ex k sub go al if Adjacency-Restriction-T est ( extension ) and P erm utation-T est ( extension ) then Insert-By-Cost ( extension , pr ex-list ) pr ex Cheap est ( pr ex-list ) Remo v e-from-list ( pr ex , pr ex-list ) r est S n pr ex Return pr ex Figure 7: An algorithm for subgoal ordering, incorp orating the ideas of earlier researc hers. The adv an tage of using b est-rst searc h is that it a v oids expanding prexes whose cost is higher than the cost of the minimal ordering. The p olicy used b y the algorithm ma y , ho w ev er, b e sub optimal or ev en harmful. It often happ ens that the b est completion of a c heap er prex is m uc h more exp ensiv e than the b est completion of a more exp ensiv e prex. When the n um b er of solutions is large, it is b etter to place subgoals with high costs closer to the b eginning of the ordering to reduce the n um b er of times that their cost is m ultiplied. F or example, let the set b e f a ( X ) ; b ( X ) g , with c ost ( a ( X )) = 10, c ost ( b ( X )) = n sols ( a ( X )) = n so ls ( b ( X )) = 2. Then a minimal ordering starts with the most exp ensiv e prex: C ost ( h a ( X ) ; b ( X ) i ) = 10 + 2 2 = 14 54 The Divide-and-Conquer Subgo al-Ordering Algorithm C ost ( h b ( X ) ; a ( X ) i ) = 2 + 2 10 = 22 If there are man y prexes whose cost is higher than the cost of the minimal ordering, then b est-rst searc h sa v es time. But if the n um b er of suc h prexes is small, using b est-rst searc h can increase the total time, due to the need to p erform insertion of a prex in to a priorit y queue, according to its cost. A sample run of Algorithm 4 will b e sho wn later (in Section 4.7). 4. The Divide-and-Conquer Subgoal Ordering Algorithm Algorithm 1 presen ted in Section 3.2 is v ery ecien t, but is applicable only when the en tire set of subgoals is indep enden t. Algorithm 3 can handle a dep enden t set of subgoals but is v ery inecien t. Algorithm 4, a com bination of the t w o, can exploit indep endence of sub- goals for b etter eciency . Ho w ev er, the obtained b enet is quite limited. In this section, w e presen t the Divide-and-Conquer ( d a c ) algorithm, whic h is able to exploit subgoal inde- p endence in a more elab orate w a y . The algorithm divides the set of subgoals in to smaller subsets, orders these subsets recursiv ely and com bines the results. 4.1 Divisibilit y T rees of Subgoal Sets In this subsection w e dene a structure that represen ts all the w a ys of breaking a subgoal set in to indep enden t parts. Our algorithm will w ork b y tra v ersing this structure. Denition: Let S and B b e sets of subgoals. The divisibility tr e e of S under B , D T r ee ( S ; B ), is an AND-OR tree dened as follo ws: D T r ee ( S ; B ) = 8 > > > > < > > > > : leaf ( S ; B ) S is indep enden t under B OR ( S ; B ; f D T r ee ( S n f B i g ; B [ f B i g ) j B i 2 S g ) S is indivisible under B AND( S ; B ; f D T r ee ( S i ; B ) j S i 2 D P ar t ( S ; B ) g ) S is divisible under B Eac h no de N in the tree D T r ee ( S 0 ; B 0 ) has an asso ciated set of subgoals S ( N ) S 0 and an asso ciated binding set B ( N ) B 0 . F or the ro ot no de, S ( N ) = S 0 , B ( N ) = B 0 . If the binding set of the ro ot is not sp ecied explicitly , w e assume it to b e empt y . F or AND-no des and OR-no des w e also dene the sets of c hildren. If S ( N ) is indep endent under B ( N ), then N is a leaf. If S ( N ) is indivisible under B ( N ), then N is an OR-no de. Eac h subgoal B i in S ( N ) denes a c hild no de whose set of subgoals is S ( N ) n f B i g and the binding set is B ( N ) [ f B i g . W e call B i the binder of the generated c hild. Note that the binding set of ev ery no de in a divisibil i t y tree is the union of the binders of all its indivisibl e ancestors and of the ro ot's binding set. If S ( N ) is divisible under B ( N ), then N is an AND-no de. Eac h subset S i in the divisibili t y partition D P ar t ( S ( N ) ; B ( N )) denes a c hild no de with asso ciated set of subgoals S i and binding set B ( N ). Divisibili t y partition w as dened in Section 3.2.1. 55 Ledenio v & Mark o vitch O B(n2) = O B(n1) = O B(n3) = n4 n5 n6 n2 n1 n3 B(n5) = {d(X)} S(n2) = {a, b} S(n6) = {c(X), d(X)} B(n6) = {e(X)} S(n4) = {d(X), e(X)} B(n4) = {c(X)} S(n5) = {c(X), e(X)} S(n1) = {a, b, c(X), d(X), e(X)} S(n3) = {c(X), d(X), e(X)} Figure 8: The divisibilit y tree of f a; b; c ( X ) ; d ( X ) ; e ( X ) g under empt y initial binding set. The set asso ciated with no de n 1 is divisible, and is represen ted b y an AND-no de. Its c hildren corresp ond to its divisibilit y subsets { one indep enden t, S ( n 2) = f a; b g , and one indivis- ible, S ( n 3) = f c ( X ) ; d ( X ) ; e ( X ) g . n 3 is an OR-no de, whose c hildren corresp ond to its three subgoals (eac h subgoal serv es as a binder in one of the c hildren). The sets S ( n 2), S ( n 4), S ( n 5) and S ( n 6) are indep enden t under their resp ectiv e binding sets, and their no des are lea v es. Here w e assumed that the subgoals c ( X ), d ( X ) and e ( X ) bind X as a result of their pro of. It is easy to sho w that the divisibili t y tree of a set of subgoals is unique up to the order of c hildren of eac h no de. Figure 8 sho ws the divisibil it y tree of the set f a; b; c ( X ) ; d ( X ) ; e ( X ) g under empt y initial binding set. The asso ciated sets and binding sets are written next to the no des. The follo wing lemma expresses an imp ortan t prop ert y of divisibil it y trees: subgoals of eac h no de are indep enden t of the rest of subgoals under the binding set of the no de. Lemma 3 L et S 0 b e a set of sub go als. Then for every no de N in D T r ee ( S 0 ; ; ) , for every sub go al A 2 S ( N ) , and for every sub go al Y 2 S 0 n ( S ( N ) [ B ( N )) , A and Y ar e indep endent under B ( N ) . Pro of: b y induction on the depth of N in the divisibili t y tree. Inductiv e base: N is the ro ot no de, S 0 n S ( N ) is empt y , and no suc h Y exists. Inductiv e h yp othesis: The lemma holds for M , the paren t no de of N . Inductiv e step: Let A 2 S ( N ), Y 2 S 0 n ( S ( N ) [ B ( N )). A 2 S ( M ), and for M the lemma holds, th us either A and Y are indep enden t under B ( M ), or Y 2 S ( M ). If A and Y are indep enden t under B ( M ), then they are also indep enden t under B ( N ), since B ( M ) B ( N ). Otherwise, A and Y are dep enden t under B ( M ), and Y 2 S ( M ). 56 The Divide-and-Conquer Subgo al-Ordering Algorithm If M is an AND-no de, and A and Y are dep enden t under B ( M ), then A and Y b elong to the same elemen t of D P ar t ( S ( M ) ; B ( M )), and Y 2 S ( N ) { a con tradiction. If M is an OR-no de and Y 2 S ( M ) n S ( N ), then Y m ust b e the binder of N . But then B ( N ) = B ( M ) [ f Y g and Y 2 B ( N ) { a con tradiction again. 2 The lemma relates to subgoal indep endenc e inside divisibil it y trees. W e shall sometimes need to argue ab out indep endence inside ordered sequences of subgoals. The follo wing corollary pro vides the necessary connecting link. Corollary 2 L et S 0 b e a set of sub go als, N b e a no de in the divisibility tr e e of S 0 , ~ S an or dering of S 0 , ~ S = ~ S 1 k ~ S 2 , wher e B ( N ) ~ S 1 and S ( N ) ~ S 2 . Then S ( N ) is mutual ly indep endent of ~ S 2 n S ( N ) under ~ S 1 . Pro of: Let A 2 S ( N ), Y 2 ~ S 2 n S ( N ). A and Y are indep enden t under B ( N ), b y the preceding lemma. Since B ( N ) ~ S 1 , A and Y are indep enden t under ~ S 1 . Ev ery subgoal of S ( N ) is indep enden t of ev ery subgoal of ~ S 2 n S ( N ) under ~ S 1 ; therefore, S ( N ) and ~ S 2 n S ( N ) are m utually indep enden t under ~ S 1 . 2 4.2 V alid Orderings in Divisibilit y T rees The aim of our ordering algorithm is to nd a minimal ordering of a giv en set of subgoals. W e construct orderings follo wing a divide-and-conque r p olicy: larger sets are split in to smaller ones, and orderings of the smaller sets are com bined to pro duce an ordering of the larger set. T o implemen t this p olicy , w e p erform a p ost-order tra v ersal of the divisibil i t y tree corresp onding to the giv en set of subgoals under an empt y initial binding set. When orderings of c hild no des are com bined to pro duce an ordering of the paren t no de, the inner order of their subgoals is not c hanged: smaller orderings are c onsistent with larger orderings. Denition: Let S and G S b e sets of subgoals. An ordering ~ O G of G and an ordering ~ O S of S are c onsistent (denoted C ons ( ~ O G ; ~ O S )), if the order of subgoals of G in ~ O G and in ~ O S is the same. The divide-and-conque r pro cess describ ed ab o v e seems analogous to Merge Sort (Kn uth, 1973). There, the set of n um b ers is split in to t w o (or more) subsets, eac h subset is inde- p enden tly ordered to a sequence consisten t with the global order, and these sequences are merged. Is it p ossible to use a similar metho d for subgoal ordering? Assume that a set of subgoals is partitioned in to t w o m utually indep enden t subsets, A and B . Can w e build an algorithm that, giv en A , pro duces its ordering consisten t with a minimal ordering of A [ B , indep enden tly of B ? Unfortunately , the answ er is negativ e. An ordering of A ma y b e consisten t with a minimal ordering of A [ B 1 but at the same time not b e consisten t with a minimal ordering of A [ B 2 for some B 1 6 = B 2 . F or example, let A = f a 1( X ) ; a 2( X ) g , B 1 = f b g , B 2 = f d g and the con trol v alues b e as sp ecied in Figure 9. The single minimal ordering of A [ B 1 is h a 2( X ) ; b; a 1( X ) i , while the single minimal ordering of A [ B 2 is h d; a 1( X ) ; a 2 ( X ) i . There is no ordering of A consisten t with b oth these minimal global orderings. 57 Ledenio v & Mark o vitch The program: a1(c1). b a1(X). a1(c1). b d. a2(c1). d. a2(c1). a2(c2) a1(c2). The con trol v alues: a 1( X ) j ; a 1( X ) j f a 2( X ) g a 2( X ) j ; a 2( X ) j f a 1( X ) g b d c ost 2 2 5 3 5 1 n so ls 2 2 2 2 3 1 C ost ( b; a 1( X ) ; a 2( X )) = 5 + 3 2 + 3 2 3 = 29 C ost ( b; a 2( X ) ; a 1( X )) = 5 + 3 5 + 3 2 2 = 32 C ost ( a 1( X ) ; b; a 2( X )) = 2 + 2 5 + 2 3 3 = 30 C ost ( a 1( X ) ; a 2( X ) ; b ) = 2 + 2 3 + 2 2 5 = 28 C ost ( a 2 ( X ) ; b; a 1 ( X )) = 5 + 2 5 + 2 3 2 = 27 C ost ( a 2( X ) ; a 1( X ) ; b ) = 5 + 2 2 + 2 2 5 = 29 C ost ( d; a 1 ( X ) ; a 2 ( X )) = 1 + 1 2 + 1 2 3 = 9 C ost ( d; a 2( X ) ; a 1( X )) = 1 + 1 5 + 1 2 2 = 10 C ost ( a 1( X ) ; d; a 2( X )) = 2 + 2 1 + 2 1 3 = 10 C ost ( a 1( X ) ; a 2( X ) ; d ) = 2 + 2 3 + 2 2 1 = 12 C ost ( a 2( X ) ; d; a 1( X )) = 5 + 2 1 + 2 1 2 = 11 C ost ( a 2( X ) ; a 1( X ) ; d ) = 5 + 2 2 + 2 2 1 = 13 Figure 9: W e sho w a small program and the con trol v alues it denes. Then w e compute costs of all p erm utations of the sets f b; a 1( X ) ; a 2( X ) g and f d; a 1( X ) ; a 2( X ) g . Dieren t orderings of f a 1( X ) ; a 2( X ) g are consisten t with minimal orderings of these sets. Since, unlik e the case of Merge Sort, w e cannot alw a ys iden tify a single ordering of the subset consisten t with a minimal ordering of the whole set, our algorithm will deal with sets of c andidate or derings . Our requiremen t from suc h a set is that it con tain at least one lo cal ordering consisten t with a global minimal ordering, if suc h a lo cal ordering exists (\lo cal" ordering is an ordering of the set of the no de, \global" ordering is an ordering of the set of the ro ot). Suc h a set will b e called valid . The follo wing denition denes v alid sets formally , together with sev eral other concepts. Denition: Let S 0 b e a set of subgoals and N b e a no de in the divisibil i t y tree of S 0 . Recall that ( S ) denotes the set of all p erm utations of S . 1. ~ O S 2 ( S 0 ) is binder-c onsistent with ~ O N 2 ( S ( N )) (denoted B C N ( ~ O N ; ~ O S )), if they are consisten t, and all subgoals of B ( N ) app ear in ~ O S b efore all subgoals of ~ O N : B C N ( ~ O N ; ~ O S ) ( ) 9 ~ O B 2 ( B ( N )) : C ons ( ~ O B k ~ O N ; ~ O S ) : ~ O S 2 ( S 0 ) is binder-c onsistent with the no de N (denoted B C N ( ~ O S )), if it is binder- consisten t with some ordering of S ( N ): B C N ( ~ O S ) ( ) 9 ~ O N 2 ( S ( N )) : B C N ( ~ O N ; ~ O S ) : 2. ~ O N 2 ( S ( N )) is min-c onsistent with ~ O S 2 ( S 0 ) (denoted M C N ; S 0 ( ~ O N ; ~ O S )), if they are binder-consisten t, and ~ O S is minimal: M C N ; S 0 ( ~ O N ; ~ O S ) ( ) B C N ( ~ O N ; ~ O S ) ^ M in ( ~ O S ; S 0 ) : ~ O N 2 ( S ( N )) is min-c onsistent (denoted M C N ; S 0 ( ~ O N )), if it is min-consisten t with some ordering of S 0 : M C N ; S 0 ( ~ O N ) ( ) 9 ~ O S 2 ( S 0 ) : M C N ; S 0 ( ~ O N ; ~ O S ) : 58 The Divide-and-Conquer Subgo al-Ordering Algorithm 3. An ordering ~ O N 2 ( S ( N )) is MC-c ontr adicting , if it is not min-consisten t: MCC N ; S 0 ( ~ O N ) ( ) : M C N ; S 0 ( ~ O N ) : 4. Tw o orderings ~ O 1 ; ~ O 2 2 ( S ( N )) are MC-e quivalent , if one of them is min-consisten t i the other one is: MCE N ; S 0 ( ~ O 1 ; ~ O 2 ) ( ) [ M C N ; S 0 ( ~ O 1 ) ( ) M C N ; S 0 ( ~ O 2 )] : 5. A set of orderings C N ( S ( N )) is valid , if C N con tains a min-consisten t ordering (when at least one min-consisten t ordering of S ( N ) exists): V al id N ; S 0 ( C N ) ( ) [ 9 ~ O 0 N 2 ( S ( N )) : M C N ; S 0 ( ~ O 0 N )] ! [ 9 ~ O N 2 C N : M C N ; S 0 ( ~ O N )] : An imp ortan t prop ert y of v alid sets is that a v alid set of orderings of the ro ot of D T r ee ( S 0 ; ; ) m ust con tain a minimal ordering of S 0 . Indeed, in the ro ot S ( N ) = S 0 , and consistency b ecomes iden tit y . Also, B ( N ) = ; , so that binder-consistency b ecomes consistency , and min-consistency b ecomes minimalit y . Since there alw a ys exists a minimal ordering of S 0 , a v alid set of orderings of the ro ot m ust con tain a minimal ordering of S 0 . 4.3 The Outline of the Divide-and-Conquer Algorithm W e prop ose an algorithm that is based on pro ducing v alid sets of orderings. Eac h no de in a divisibili t y tree pro duces a v alid set for its asso ciated set of subgoals, and passes it to its paren t no de. After the v alid set of the ro ot no de is found, w e compare costs of all its mem b ers, and return the c heap est one. The set of orderings pro duced b y the algorithm for a no de N is called a c andidate set of N . Its mem b ers are called c andidate or derings of N , or simply c andidates . T o nd a candidate set of N , w e rst consider the set of all p ossible orderings of S ( N ) that are consisten t with candidates of N 's c hildren. This set is called the c onsistency set of N . Giv en the candidate sets of N 's c hildren, the consistency set of N is dened uniquely . A candidate set of N is usually not unique. Denition: Let N b e a no de in a divisibil it y tree of S 0 . The c onsistency set of N , denoted C onsS et ( N ), and the c andidate set of N , denoted C andS et ( N ), are dened recursiv ely: If N is a leaf, its consistency set con tains all p erm utations of S ( N ): C onsS et ( N ) = ( S ( N )) : If N is an AND-no de, and its c hild no des are N 1 ; N 2 ; : : : N k , w e dene the consistency set of N as the set of all p ossible orderings of S ( N ) consisten t with candidates of N 1 ; N 2 ; : : : N k : C onsS et ( N ) = n ~ O N 2 ( S ( N )) 8 i (1 i k ) ; 9 ~ O i 2 C andS et ( N i ) : C ons ( ~ O i ; ~ O N ) o : 59 Ledenio v & Mark o vitch If N is an OR-no de, and its c hild no de corresp onding to ev ery binder A 2 S ( N ) is N A , then the consistency set of N is obtained b y adding binders as the rst elemen ts to the candidates of the c hildren: C onsS et ( N ) = n A k ~ O A A 2 S ( N ) ; ~ O A 2 C andS et ( N A ) o : A candidate set of N is an y set of orderings pro duced b y remo ving MC-con tradicting and MC-equiv alen t orderings out of the consistency set of N , while k eeping at least one represen tativ e for eac h group of MC-equiv alen t orderings: C andS et ( N ) C onsS et ( N ) ; ~ O N 2 ( C onsS et ( N ) n C andS et ( N )) ) MCC N ; S 0 ( ~ O N ) _ h 9 ~ O 0 N 2 C andS et ( N ) : MCE N ; S 0 ( ~ O N ; ~ O 0 N ) i : (In other w ords, if some ordering is rejected, it is either MC-con tradicting, or MC- equiv alen t to some other ordering, whic h is not rejected.) There are t w o kinds of orderings whic h can b e remo v ed from C onsS et ( N ) while re- taining its v alidit y: MC-c ontr adicting and MC-e quivalent orderings. Remo v al of an MC- con tradicting ordering cannot c hange the n um b er of min-consisten t orderings in the set; if w e remo v e an MC-equiv alen t ordering, then ev en if it is min-consisten t, some other min- consisten t ordering is retained in the set. If there exists a min-consisten t ordering of the set of the no de, then its candidate set m ust con tain a min-consisten t ordering, and therefore the candidate set is v alid. Note that when our algorithm treats an OR-no de, the binder of eac h c hild is alw a ys placed as the rst subgoal of the pro duced ordering of this no de. On higher lev els the inner order of subgoals in the ordering do es not c hange (consistency is preserv ed). Therefore, our algorithm can only pro duce binder-consisten t orderings. This explains the c hoice of the names \binder" and \binding set" : the subgoals of B ( N ) bind some common v ariables of S ( N ), since they stand to the left of them in an y global ordering that our algorithm pro duces. In particular, if S ( N ) is indep enden t under B ( N ), then the subgoals of B ( N ) bind all the shared free v ariables of S ( N ). T o implemen t the DP art function, w e can use the Union-Find data structure (Cormen, Leiserson, & Riv est, 1991, Chapter 22), where subgoals are elemen ts, and indivisible sets are groups. In the b eginning, ev ery subgoal constitutes a group b y itself. Whenev er w e disco v er that t w o subgoals share a free v ariable not b ound b y subgoals of the binding set, w e unite their groups in to one. T o complete the pro cedure, w e need a w a y to determine whic h v ariables are b ound b y the giv en binding set. Section 7.1 con tains a discussion of this problem and prop oses some practical solutions. Finally , w e collect all the indivisibl e subgoals in to a separate group. These op erations can b e implemen ted in O ( n ( n; n )) amor- tized time, where ( n; n ) is the in v erse Ac k ermann function, whic h can b e considered O (1) for all v alues of n that can app ear in realistic logic programs. Th us, the whole pro cess of nding the divisibili t y partition of n subgoals can b e p erformed in O ( n ) a v erage time. The formal listing of the ordering algorithm discussed ab o v e is sho wn in Figure 10. The algorithm do es not sp ecify explicitly ho w candidate sets are created from consis- tency sets. T o complete this algorithm, w e m ust pro vide the three ltering pro cedures 60 The Divide-and-Conquer Subgo al-Ordering Algorithm Algorithm 5 Order ( S 0 ) RootC andS et CandidateSet ( S 0 ; ; ) Return the c heap est mem b er of RootC andS et CandidateSet ( S ; B ) case ( S under B ) indep enden t : let ConsSet N ( S ) let CandSet N V alidLeafFilter ( ConsSet N ) divisible : let fS 1 ; S 2 ; : : : S k g DP art ( S ; B ) lo op for i = 1 to k let C i CandidateSet ( S i ; B ) let ConsSet N n ~ O N 2 ( S ( N )) j 8 i = 1 : : : k ; 9 ~ O i 2 C i : C ons ( ~ O i ; ~ O N ) o let CandSet N V alidANDFilter ( ConsSet N ; fS 1 ; : : : S k g ; fC 1 ; : : : C k g ) indivisibl e : lo op for A 2 S let C ( A ) CandidateSet ( S n f A g ; B [ f A g ) let C 0 ( A ) n A k ~ O A j ~ O A 2 C ( A ) o let ConsSet N S A 2S C 0 ( A ) let CandSet N V alidORFilter ( ConsSet N ) Return CandSet N Figure 10: The sk eleton of the d a c ordering algorithm. F or eac h t yp e of no de in a divisibilit y tree, a consistency set is created and rened through v alidit y lters. The pro duced candidate set of the ro ot is v alid; hence, its c heap est mem b er is a minim al ordering of the giv en set. { V alidLeafFilter , V alidANDFilter and V alidORFilter . T rivially , w e can dene them all as n ull lters that return the sets they receiv e unc hanged. In this case the candidate set of ev ery no de will con tain all the p erm utations of its subgoals, and will surely b e v alid. This will, ho w ev er, greatly increase the ordering time. Our in ten tion is to reduce the sizes of candidate sets as far as p ossible, while k eeping them v alid. In the follo wing t w o subsections w e discuss the ltering pro cedures. Section 4.4 dis- cusses detection of MC-con tradicting orderings, and Section 4.5 discusses detection of MC- equiv alen t orderings. Finally , in Section 4.6 w e presen t the complete ordering algorithm, incorp orating the lters in to the sk eleton of Algorithm 5. 61 Ledenio v & Mark o vitch 4.4 Detection of MC-Con tradicting Orderings In this subsection w e sho w sucien t conditions for an ordering to b e MC-con tradicting. Suc h orderings can b e safely discarded, lea ving the set of orderings v alid, but reducing its size. The subsection is divided in to three parts, one for eac h t yp e of no de in a divisibil i t y tree. 4.4.1 Detection of MC-Contradicting Orderings in Lea ves The follo wing lemma sho ws that subgoals in a min-consisten t ordering of a leaf no de m ust b e sorted b y cn . Lemma 4 L et S 0 b e a set of sub go als, N b e a le af in the divisibility tr e e of S 0 . L et ~ O N b e an or dering of S ( N ) . If the sub go als of ~ O N ar e not sorte d by cn under B ( N ) , then ~ O N is MC-c ontr adicting. Pro of: Let ~ O S b e an y ordering of S 0 , binder-consisten t with ~ O N . W e sho w that ~ O S cannot b e a minimal ordering of S 0 , th us ~ O N is not min-consisten t. ~ O N is not sorted b y cn , i.e., it con tains an adjacen t cn-in v erted pair of subgoals h A 1 ; A 2 i . (Recall that a pair is cn-in v erted if the rst elemen t has a larger cn v alue than the second one { Section 3.2.3). Since ~ O S is consisten t with ~ O N , w e can write ~ O S = ~ X k A 1 k ~ Y k A 2 k ~ Z , where ~ X , ~ Y and ~ Z are (p ossibly empt y) sequences of subgoals. Since ~ O S is binder-consisten t with ~ O N , B ( N ) ~ X . If ~ Y is empt y , then A 1 and A 2 are adjacen t in ~ O S . Since B ( N ) ~ X , A 1 and A 2 are indep enden t under ~ X . Therefore, the cost of the whole ordered sequence can b e reduced b y transp osing A 1 and A 2 , according to Lemma 2 (they are adjacen t, indep enden t and cn-in v erted). If ~ Y is not empt y , then no subgoal of ~ Y b elongs to S ( N ), since otherwise it w ould app ear in ~ O N b et w een A 1 and A 2 . By Corollary 2, ~ Y is m utually indep enden t of b oth A 1 and A 2 under ~ X . If cn ( ~ Y ) j ~ X < cn ( A 1 ) j ~ X then, b y Lemma 2, a transp osition of ~ Y with A 1 pro duces an ordering with lo w er cost. Otherwise, cn ( ~ Y ) j ~ X cn ( A 1 ) j ~ X . Since the pair h A 1 ; A 2 i is cn-in v erted, cn ( A 1 ) j ~ X > cn ( A 2 ) j ~ X . Hence, cn ( ~ Y ) j ~ X > cn ( A 2 ) j ~ X , and transp osition of ~ Y with A 2 reduces the cost, b y Lemma 2. In either case, there is a w a y to reduce the cost of ~ O S . Therefore, ~ O S cannot b e minimal, and ~ O N is MC-con tradicting. 2 4.4.2 Detection of MC-Contradicting Orderings in AND-nodes Ev ery mem b er of the consistency set of an AND-no de is consisten t with some com bination of candidates of its c hild no des. If there are k c hild no des, and for eac h c hild N i the sizes of subgoal and candidate sets are jS ( N i ) j = n i and j CandSet ( N i ) j = c i , then the total n um b er of p ossible consisten t orderings is c 1 c 2 : : : c k ( n 1 + n 2 + ::: + n k )! n 1 ! n 2 ! ::: n k ! . F ortunately , most of these orderings are MC-con tradicting and can b e discarded from the candidate set. The 62 The Divide-and-Conquer Subgo al-Ordering Algorithm follo wing lemma states that it is forbidden to insert other subgoals b et w een t w o cn-in v erted sub-sequences. If suc h insertion tak es place, the ordering is MC-con tradicting and can b e safely discarded. Lemma 5 L et S 0 b e a set of sub go als, N a no de in the divisibility tr e e of S 0 , and ~ O S an or dering of S 0 , binder-c onsistent with an or dering ~ O N of S ( N ) . If ~ O N c ontains an adjac ent cn-inverte d p air of sub-se quenc es h ~ A 1 ; ~ A 2 i , ~ A 1 and ~ A 2 app e ar in ~ O S not mixe d with other sub go als, and ~ A 1 and ~ A 2 ar e not adjac ent in ~ O S , then ~ O S is not minimal. Pro of: Let ~ O S b e suc h an ordering of S 0 , binder-consisten t with ~ O N : ~ O S = ~ X k ~ A 1 k ~ Y k ~ A 2 k ~ Z ; where ~ Y is not empt y . No subgoal of ~ Y b elongs to S ( N ), since otherwise it w ould stand in ~ O N b et w een ~ A 1 and ~ A 2 . ~ O S is binder-consisten t with ~ O N ; therefore, B ( N ) ~ X . By Corollary 2, ~ Y m ust b e m utually indep enden t of b oth ~ A 1 and ~ A 2 under ~ X , and b y Lemma 2 a transp osition of ~ Y with either ~ A 1 or ~ A 2 reduces the cost { exactly as in the pro of of Lemma 4. 2 If a pair of adjacen t subgoals h A i ; A i +1 i is cn-in v erted, then b y the previous lemma an y attempt to insert subgoals inside it results in a non-minimal global ordering. Thereup on w e ma y join A i and A i +1 in to a blo ck A i;i +1 , whic h can further participate in a larger blo c k. The formal recursiv e denition of a blo c k follo ws. F or con v enience, w e consider separate subgoals to b e blo c ks of length 1. Denition: 1. A sub-sequence ~ A of an ordered sequence of subgoals is a blo ck if it is either a single subgoal, or ~ A = ~ A 1 k ~ A 2 , where h ~ A 1 ; ~ A 2 i is a cn-in v erted pair of blo c ks. 2. A blo c k is maximal ( max-blo ck ) if it is not a sub-sequence of a larger blo c k. 3. Let N b e a no de in a divisibili t y tree, M b e some descendan t of N , ~ O N 2 ( S ( N )) and ~ O M 2 ( S ( M )) b e t w o consisten t orderings of these no des. A blo c k ~ A of ~ O M is violate d in ~ O N if there are t w o adjacen t subgoals in ~ A that are not adjacen t in ~ O N (in other w ords, alien subgoals are inserted b et w een the subgoals of the blo c k). 4. Let N b e a no de, M b e its descendan t, ~ O N 2 ( S ( N )) and ~ O M 2 ( S ( M )) b e t w o consisten t orderings of these no des. ~ O M is called the pr oje ction of ~ O N on M . W e shall usually sp eak ab out pro jection of an ordering on a c hild no de. The concept of max-blo c k is similar to the maximal indivisible blo ck in tro duced b y Simon and Kadane (1975) in the con text of satiscing searc h. The follo wing corollary presen ts the result of Lemma 5 in a more con v enien t w a y . Corollary 3 L et N b e a no de in a divisibility tr e e, M b e one of its childr en, ~ O N b e an or dering of N , and ~ O M b e the pr oje ction of ~ O N on M . If ~ O M c ontains a blo ck that is violate d in ~ O N , then ~ O N is MC-c ontr adicting. 63 Ledenio v & Mark o vitch Pro of: Let ~ A b e the smallest blo c k of ~ O M violated in ~ O N . According to the denition of a blo c k, ~ A = ~ A 1 k ~ A 2 , where ~ A 1 and ~ A 2 are not violated in ~ O N , and the pair h ~ A 1 ; ~ A 2 i is cn-in v erted. Let ~ O S b e an y ordering of the ro ot no de binder-consisten t with ~ O N . ~ O S violates ~ A , since ~ O N violates ~ A . T o sho w that ~ O N is MC-con tradicting, w e m ust pro v e that ~ O S is not minimal. If ~ A 1 and ~ A 2 are not violated in ~ O S , then they are not adjacen t in ~ O S , and ~ O S is not minimal, b y Lemma 5. Otherwise, ~ A 1 or ~ A 2 is violated in ~ O S . Without loss of generalit y , let it b e ~ A 1 . Let ~ A 0 b e the smallest sub-blo c k of ~ A 1 violated in ~ O S . According to the denition of a blo c k, ~ A 0 = ~ A 0 1 k ~ A 0 2 , where the pair h ~ A 0 1 ; ~ A 0 2 i is cn-in v erted, ~ A 1 and ~ A 2 are not violated and not adjacen t in ~ O S . By Lemma 5, ~ O S is not minimal. 2 F or example, if con trol v alues of subgoals are as sho wn in Figure 9, then h a 1( X ) ; a 2( X ) i is a blo c k, since cn ( a 1( X )) j ; = 2 1 2 = 1 2 , cn ( a 2( X )) j f a 1 ( X ) g = 2 1 3 = 1 3 . As one can see from the gure, insertion of b or d inside this blo c k results in a non-minimal ordering. As w as already noted ab o v e, the consistency set of an AND-no de can b e large. In man y of its orderings, ho w ev er, blo c ks of pro jections are violated, and w e can discard these orderings as MC-con tradicting. In the remaining orderings, no blo c k of a pro jection is violated, and eac h suc h ordering can b e represen ted as a sequence of max-blo c ks of the pro jections. In eac h pro jection, its max-blo c ks stand in cn -ascending order (otherwise, there is an adjacen t cn-in v erted pair of blo c ks, and a larger blo c k can b e formed, whic h con tradicts their maximalit y). As the follo wing lemma states, in the paren t AND-no de these blo c ks m ust also b e ordered b y their cn v alues; otherwise, the ordering is MC-con tradicting. Lemma 6 If an or dering of an AND-no de c ontains an adjac ent cn-inverte d p air of max- blo cks of its pr oje ctions on the childr en, then this or dering is MC-c ontr adicting. Pro of: If these blo c ks are violated in the binder-consisten t global ordering, the global ordering is not minimal b y Corollary 3. If the blo c ks are not violated, the pro of is similar to the pro of of Lemma 4. 2 The t w o sucien t conditions for detection of MC-con tradicting orderings expressed in Corollary 3 and Lemma 6 allo w us to reduce the size of the candidate set signican tly . Assume, for example, that the set of our curren t no de N is split in to t w o m utually indep en- den t subsets whose candidates are h a 1 ; a 2 i and h b 1 ; b 2 i (one candidate for eac h c hild). There are six p ossible orderings of S ( N ), all sho wn in Figure 11. Assume that b oth h a 1 ; a 2 i and h b 1 ; b 2 i are blo c ks, and cn ( h a 1 ; a 2 i ) j B ( N ) < cn ( h b 1 ; b 2 i ) j B ( N ) . Out of six consisten t orderings, four (2{5) can b e rejected due to blo c k violation, and one of the remaining t w o (n um b er 6) puts the blo c ks in the wrong order. So, only one ordering (n um b er 1) can b e left in the can- didate set of N . Ev en if neither h a 1 ; a 2 i nor h b 1 ; b 2 i are blo c ks, Lemma 6 dictates a unique in terlea ving of their elemen ts (max-blo c ks), assuming that cn ( a 1 ) j B ( N ) 6 = cn ( a 2 ) j B ( N ) [f a 1 g 6 = cn ( b 1 ) j B ( N ) 6 = cn ( b 2 ) j B ( N ) [f b 1 g . 4.4.3 Detection of MC-Contradicting Orderings in OR-nodes The follo wing lemma states that if a blo c k has a c heap er p erm utation, then the ordering is MC-con tradicting (and can b e discarded from the candidate set). 64 The Divide-and-Conquer Subgo al-Ordering Algorithm a 1 2 a b 1 b 2 2 2 b 1 a 1 2 a b 1 b 2 b 1 2 a 1 2 a b 1 b 2 a 2 a b 1 b b 1 b 2 a a 1 2 a a 1 1. 2. 3. 4. 5. 6. Figure 11: The p ossible w a ys to com bine h a 1 , a 2 i and h b 1 , b 2 i Lemma 7 L et N b e a no de in the divisibility tr e e of ~ S 0 , ~ O N 2 ( ~ S ( N )) . L et ~ A b e a le ading blo ck of ~ O N : ~ O N = ~ A k ~ R . If ther e is a p ermutation of ~ A , ~ A 0 , such that c ost ( ~ A 0 ) j B ( N ) < c ost( ~ A ) j B ( N ) , then ~ O N is MC-c ontr adicting. Pro of: Let ~ O S 2 ( ~ S 0 ) b e binder-consisten t with ~ O N . If ~ A is violated in ~ O S , ~ O S cannot b e minimal (Corollary 3). Otherwise, ~ A o ccupies a con tin uous segmen t in ~ O S , and its replacemen t b y a c heap er p erm utation reduces the cost of the global ordering (Lemma 1). Th us, ~ O S cannot b e minimal. 2 This c hec k should b e done only for leading blo c ks of OR-no des: Ev ery ordering of a leaf no de that has not b een rejected due to Lemma 4 m ust b e sorted b y cn . Consequen tly , it con tains no cn-in v erted adjacen t pair of subgoals, and no blo c k of size 2 can b e formed. Ev ery ordering of an AND-no de that has not b een rejected due to Corollary 3 or Lemma 6 m ust ha v e its blo c ks un brok en and in cn-ascending order. Consequen tly , new blo c ks cannot b e formed here either. In OR-no des, new blo c ks can b e formed when w e add a binder as the rst elemen t of an ordering, if the cn v alue of the binder is greater than that of the subsequen t blo c k. All new blo c ks start from the binder, and w e m ust p erform the p erm utation test only on the leading max-blo c k of an ordering. 4.5 Detection of MC-Equiv alen t Orderings In the previous subsection w e presen ted sucien t conditions for detecting MC-con tradicting orderings. In this subsection w e sp ecify sucien t conditions for iden tifying MC-equiv alen t orderings. Recall that t w o orderings of a no de are MC-equiv alen t if minimal consistency of one implies minimal consistency of the other. Finding suc h sucien t conditions will allo w us to eliminate orderings without loss of v alidit y of the candidate set. W e start with dening a sp ecialization of the MC-equiv alence relation: blo ckwise e quivalenc e . W e then sho w that orderings whose max-blo c ks are sorted b y cn are blo c kwise-equiv ale n t, and therefore MC-equiv alen t. 65 Ledenio v & Mark o vitch Denition: Let S 0 b e a set of subgoals and N b e a no de in the divisibili t y tree of S 0 . Let ~ O 1 and ~ O 2 b e t w o orderings of S ( N ) with an equal n um b er of max-blo c ks. Let ~ O S b e an ordering of S 0 , binder-consisten t with ~ O 1 , where blo c ks of ~ O 1 are not violated. ~ O S j ~ O 2 ~ O 1 is the ordering obtained b y replacing in ~ O S ev ery max-blo c k of ~ O 1 with a max- blo c k of ~ O 2 , while preserving the order of max-blo c ks (the i -th max-blo c k of ~ O 1 is replaced b y the i -th max-blo c k of ~ O 2 ). ~ O 1 and ~ O 2 are blo ckwise-e quivalent if the follo wing condition holds: ~ O 1 is min-consisten t with ~ O S i ~ O 2 is min-consisten t with ~ O S j ~ O 2 ~ O 1 . As can b e easily seen, if t w o orderings are blo c kwise-equiv alen t, then they are MC- equiv alen t. No w w e sho w that a transp osition of adjacen t, m utually indep enden t cn-equal max-blo c ks in an ordering of a no de pro duces a blo c kwise-equiv alen t ordering. The pro of of the follo wing lemma is found in App endix A. Lemma 8 L et S 0 b e a set of sub go als, N b e a no de in the divisibility tr e e of S 0 , ~ O N = ~ Q k ~ A 1 k ~ A 2 k ~ R b e an or dering of S ( N ) , wher e ~ A 1 and ~ A 2 ar e max-blo cks, mutual ly indep endent and cn-e qual under the bindings of B ( N ) [ ~ Q . Then ~ O N is blo ckwise-e quivalent with ~ O 0 N = ~ Q k ~ A 2 k ~ A 1 k ~ R . Corollary 4 A l l sorte d by cn or derings of a le af no de ar e blo ckwise-e quivalent. F or example, if S ( N ) = f A; B ; C ; D g , cn ( A ) j B ( N ) = 0 : 1, cn ( B ) j B ( N ) = cn ( C ) j B ( N ) = 0 : 3, cn ( D ) j B ( N ) = 0 : 5, then the orderings h A; B ; C ; D i and h A; C ; B ; D i are blo c kwise-equiv al en t, and w e can remo v e from the candidate set an y one of them (but not b oth). Corollary 5 A l l or derings of an AND-no de, wher e blo cks of pr oje ctions ar e not violate d and adjac ent max-blo cks fr om dier ent childr en pr oje ctions ar e cn-or der e d, ar e blo ckwise- e quivalent. F or example, if the candidates of the c hildren are ~ A k ~ B and ~ C k ~ D , where ~ A; ~ B ; ~ C ; ~ D are max-blo c ks, cn ( ~ A ) j B ( N ) = 0 : 1, cn ( ~ B ) j B ( N ) [ ~ A = cn ( ~ C ) j B ( N ) = 0 : 3 and cn ( ~ D ) j B ( N ) [ ~ C = 0 : 5, then the orderings ~ A k ~ B k ~ C k ~ D and ~ A k ~ C k ~ B k ~ D are blo c kwise-equiv al en t, and w e can remo v e from the candidate set an y one of them (but not b oth). T o pro v e b oth Corollaries 4 and 5, w e note that in eac h case one of the men tioned orderings can b e obtained from the other b y a nite n um b er of transp ositions of adjacen t, m utually indep enden t and cn-equal max-blo c ks. According to Lemma 8, eac h suc h transp o- sition yields a blo c kwise-equiv alen t ordering. It is easy to sho w that blo c kwise equiv alence is transitiv e. The follo wing corollary states that subgoals within a blo c k can b e p erm uted, pro vided that the cost of the blo c k is not c hanged. Corollary 6 A l l or derings of a no de, identic al up to c ost-pr eserving p ermutations of sub- go als inside blo cks, ar e blo ckwise-e quivalent. The pro of of the corollary follo ws immediately from Lemma 1. F or example, if the set is f a ( X ) ; b ( X ) g , and the con trol v alues are as in the rst coun ter-example of Prop osition 1, 66 The Divide-and-Conquer Subgo al-Ordering Algorithm No de Set MC-con tradicting blo c kwise-equiv ale n t Leaf Indep enden t Subgoals not sorted b y cn Subgoals sorted b y cn | L emma 4 | Cor ol lary 4 Con tains violated blo c ks Max-blo c ks not violated, AND Divisible | Cor ol lary 3 sorted b y cn Max-blo c ks not sorted b y cn | Cor ol lary 5 | L emma 6 The leading max-blo c k has Cost-preserving p erm utations OR Indivisibl e a c heap er p erm utation of blo c ks | L emma 7 | Cor ol lary 6 T able 1: Summary of sucien t conditions for detection of MC-con tradicting and blo c kwise- equiv alen t orderings. i.e. cn ( a ( X ) j ; ) = cn ( b ( X ) j ; ) = 1 2 , and cn ( a ( X ) j f b ( X ) g ) = cn ( b ( X ) j f a ( X ) g ) = 0, then in b oth p ossible orderings, h a ( X ) ; b ( X ) i and h b ( X ) ; a ( X ) i , the t w o subgoals are united in to a blo c k, and these blo c ks ha v e equal cost. In an y global ordering con taining the blo c k h a ( X ) ; b ( X ) i , w e can replace this blo c k with h b ( X ) ; a ( X ) i without c hanging the total cost. Therefore h a ( X ) ; b ( X ) i is blo c kwise-equiv al en t to h b ( X ) ; a ( X ) i . The sucien t condition expressed in Corollary 6 should b e c hec k ed only in OR-no des, since in lea v es and AND-no des no new blo c ks are created, as w as argued in Section 4.4.3. 4.6 The Revised Ordering Algorithm In the t w o preceding subsections w e sa w sev eral sucien t conditions of MC-con tradiction and MC-equiv alence, summarized in T able 1. These results p ermit us to close the gaps in Algorithm 5 b y pro viding the necessary v alidit y lters. Eac h lter tests the sucien t conditions of MC-con tradiction and MC-equiv alence on ev ery ordering in the consistency set. If some of these sucien t conditions hold, the ordering is rejected. The formal listing of these pro cedures is sho wn in Figure 12. While the generate-and-test approac h describ ed ab o v e serv ed us w ell for metho dological purp oses, it is ob viously not practical b ecause of its computational limitations. F or example, for an indep enden t set of size n , the algorithm creates n ! orderings, then rejects n ! 1 and k eeps only one. This pro cess tak es O ( n ! n ) time and pro duces an ordering whic h is sorted b y cn . The same result could b e obtained in just O ( n log n ) time, b y a single sorting. So, instead of uncon trolled creation of orderings and selectiv e rejection, w e w an t to p erform a selectiv e creation of orderings. In other w ords, w e w an t to revise our algorithm to deal directly with candidate sets, instead of generating large consistency sets. The revised algorithm pro duces the candidate set of a no de N as follo ws: If N is a leaf, the subgoals of S ( N ) are sorted b y cn under the bindings of B ( N ), and the pro duced ordering is the sole candidate of N . If N is an AND-no de, then for eac h com bination of its c hildren's candidates a candi- date of N is created, where the max-blo c ks of the c hildren's candidates are ordered 67 Ledenio v & Mark o vitch V alidLeafFilter ( ConsSet N ) let CandSet N ; lo op for ~ O N 2 ConsSet N if ~ O N is sorted b y cn and there is no ~ O 0 N 2 CandSet N whic h is sorted b y cn then CandSet N CandSet N [ f ~ O N g Return CandSet N V alidANDFilter ( ConsSet N ; fS 1 ; : : : S k g ; fC 1 ; : : : C k g ) let CandSet N ; lo op for ~ O N 2 ConsSet N lo op for i = 1 to k let ~ O i b e the pro jection of ~ O N on S i if 8 i ~ O i 2 C i and max-blo c ks of ~ O i -s are not violated in ~ O N , and max-blo c ks of ~ O i -s are ordered b y cn in ~ O N , and there is no ~ O 0 N 2 CandSet N consisten t with all ~ O i -s, then CandSet N CandSet N [ f ~ O N g Return CandSet N V alidORFilter ( ConsSet N ) let CandSet N ; lo op for ~ O N 2 ConsSet N if ~ O N do es not start with a blo c k ha ving a c heap er p erm utation, and there is no ~ O 0 N 2 CandSet N , iden tical to ~ O N up to cost-preserving p erm utations in blo c ks, then CandSet N CandSet N [ f ~ O N g Return CandSet N Figure 12: The three lter pro cedures that con v ert a consistency set in to a candidate set. T ogether with Algorithm 5, they form a complete ordering algorithm. The eciency of the algorithm can b e impro v ed, as w e shall see in Algorithm 6. b y cn . The candidate is pro duced b y merging: mo ving in parallel on the candidates of the c hildren and extracting max-blo c ks that are minimal b y cn . If N is an OR-no de, then for eac h candidate of its c hild an ordering of N is created b y adding the binder to the left end of the c hild candidate. If this results in creation of a blo c k that has a c heap er p erm utation, the ordering is rejected; otherwise, it is added to the candidate set. It suces to c hec k only the leading max-blo c k. 68 The Divide-and-Conquer Subgo al-Ordering Algorithm Note that the revised algorithm do es not include a test for cost-preserving p erm utations of blo c ks in dieren t orderings (expressed in Corollary 6), b ecause of the high exp ense of suc h a test. The revised algorithm describ ed ab o v e con tains manipulations of blo c ks. F or this pur- p ose, w e need an easy and ecien t w a y to detect blo c ks in orderings. Since w e do not p ermit blo c k violation (b y Corollary 3), w e can unite all the subgoals of a max-blo c k in to one en tit y , and treat it as an ordinary subgoal. The pro cedure of joining subgoals in to blo c ks is called folding , and the resulting sequence of max-blo c ks { a folde d se quenc e . After subgoals are folded in to a blo c k, there is no need to unfold this blo c k bac k to separate subgoals: on upp er lev els of the tree, these subgoals will again b e joined in to a blo c k, unless the blo c k is violated. The unfolding op eration is carried out only once b efore returning the c heap est ordering of the set (of the ro ot no de). The candidate sets of the no des are no w dened as sets of folde d orderings. As w as already stated, new blo c ks can only b e created in the candidates of OR-no des, when the binder is added as the rst elemen t of the ordering, if the cn v alue of the binder is greater than the cn v alue of the rst max-blo c k of the c hild pro jection. Therefore, in the revised algorithm w e only build new blo c ks that start from the binder: the max-blo c ks in the rest of the ordering remain from the c hild's candidate. First w e try to mak e a blo c k out of the binder and the rst max-blo c k of the c hild's candidate. If they are cn-ordered, w e stop the folding. If they are cn-in v erted, w e unite them in to a larger blo c k, and try to unite it with the second max-blo c k of the c hild's candidate, and so on. The pro duced folded ordering con tains only maximal blo c ks: the rst blo c k is maximal, since w e could not expand it further to the righ t, and the other blo c ks are maximal, since they w ere maximal in the c hild's candidate. Lemma 7 states that an ordering whose leading max-blo c k has a c heap er p erm utation is MC-con tradicting. One w a y to detect suc h a blo c k is to exhaustiv ely test all its p erm u- tations, computing and comparing their costs. This pro cedure is v ery exp ensiv e. Instead, in our revised algorithm w e emplo y the adjacency restriction test (Equation 8). The test is applied to ev ery pair of adjacen t subgoals of a blo c k, and if some adjacen t pair has a c heap er transp osition, then the whole blo c k has a c heap er p erm utation, b y Lemma 1. Since blo c ks are created b y concatenation of smaller blo c ks, it suces to test the adjacency restriction only at the p oin ts where blo c ks are joined (for other adjacen t pairs of subgoals, the tests w ere p erformed on the lo w er lev els, when smaller blo c ks w ere formed). The adjacency re- striction test do es not guaran tee detection of all not-c heap est p erm utations (as w as sho wn in Example 3), but it detects suc h blo c ks in man y cases, and w orks in linear time. The nal v ersion of the d a c subgoal ordering algorithm is presen ted in Figure 13. The complete correctness pro of of Algorithm 6 is found in App endix B. 4.7 Sample Run and Comparison of Ordering Algorithms W e illustrate the w ork of the d a c algorithm, using the subgoal set sho wn in Figure 8, S 0 = f a; b; c ( X ) ; d ( X ) ; e ( X ) g . After pro ving c ( X ), d ( X ) or e ( X ), w e can assume that X is b ound. Let the con trol v alues for the subgoals b e as sho wn in T able 2. The column c ( f r ee ) con tains con trol v alues for the subgoal c ( X ) when X is not y et b ound b y the preceding subgoals (i.e., the binding set do es not con tain d ( X ) or e ( X )). The column c ( bound ) 69 Ledenio v & Mark o vitch Algorithm 6 : The Divide-and-Conquer A lgorithm Order ( S 0 ) let R o otCandSet CandidateSet ( S 0 ; ; ) Return Unfold (the c heap est elemen t of R o otCandSet ) CandidateSet ( S ; B ) let fS 1 ; S 2 ; : : : S k g DP art ( S ; B ) case k = 1, shar e d-vars ( S 1 ) = ; ( S is indep enden t under B ): Return f Sort-b y-cn ( S ; B ) g k = 1, shar e d-vars ( S 1 ) 6 = ; ( S is indivisibl e under B ): lo op for A 2 S let C ( A ) CandidateSet ( S n f A g ; B [ f A g ) let C 0 ( A ) n F old ( A k ~ O A ; B ) ~ O A 2 C ( A ) o Return S A 2S C 0 ( A ) k > 1 ( S is divisible under B ): lo op for i = 1 to k let C i CandidateSet ( S i ; B ) Return n Merge ( f ~ O 1 ; ~ O 2 ; : : : ~ O k g ; B ) ~ O 1 2 C 1 ; ~ O 2 2 C 2 ; : : : ~ O k 2 C k o Merge ( f ~ O 1 ; ~ O 2 ; : : : ~ O k g ; B ) let min-cn-c andidate ~ O i that minimizes cn (rst-max-blo c k( ~ O i )) j B , 1 i k let min-cn-blo ck rst-max-blo c k( min-cn-c andidate ) remo v e-rst-max-blo c k( min-cn-c andidate ) Return min-cn-blo ck k Merge ( f ~ O 1 ; ~ O 2 ; : : : ~ O k g ; B [ min-cn-blo ck ) F old ( h A 1 ; A 2 : : : A k i ; B ) if k 1 or cn ( A 1 ) j B cn ( A 2 ) j B k A 1 then Return h A 1 ; A 2 : : : A k i else if the last subgoal of A 1 and the rst subgoal of A 2 satisfy the adjacency restriction then let A 0 blo c k ( A 1 ; A 2 ) Return F old ( h A 0 ; A 3 : : : A k i ; B ) else Return ; Figure 13: The revised v ersion of the d a c algorithm. The candidate sets are built selectiv ely , without explicit creation of consistency sets. Candidate sets con tain folded orderings, and unfolding is p erformed only on the returned global ordering. The co de of the Unfold and Sort-b y-cn pro cedures is not listed, due to its straigh tforw ardness. The merging pro cedure recursiv ely extracts from the giv en folded orderings max-blo c ks that are minim al b y cn . The folding pro cedure joins t w o leading blo c ks in to a larger one, as long as they are cn-in v erted. 70 The Divide-and-Conquer Subgo al-Ordering Algorithm a b c(free) c(b ound) d(free) d(b ound) e(free) e(b ound) c os t 10 5 5 5 10 5 20 10 n so ls 0.8 2 2 0.5 4 1 0.4 0.1 cn -0.02 0.2 0.2 -0.1 0.3 0 -0.03 -0.09 T able 2: Con trol v alues for the sample runs of the ordering algorithms. con tains cost v alues of c ( X ) when d ( X ) or e ( X ) ha v e already b ound X . F or example, c ost ( c ( X )) j f a ;d ( X ) g = c ost ( c ( bound )) = 5. The d a c algorithm tra v erses the divisibili t y tree of S 0 as follo ws. (The names of the no des are as in Figure 8.) 1. The ro ot of the divisibil it y tree, n 1, has empt y binding set B ( n 1) = ; , and the asso ciated subgoal set S ( n 1) = f a; b; c ( X ) ; d ( X ) ; e ( X ) g . The set S ( n 1) is parti- tioned in to t w o subsets under B ( n 1): one indep enden t { f a; b g , and one indivisible { f c ( X ) ; d ( X ) ; e ( X ) g . These t w o subsets corresp ond to t w o c hild no des of the AND- no de n 1: n 2 and n 3, b oth with empt y binding sets. 2. S ( n 2) is indep enden t under B ( n 2). Therefore, n 2 is a leaf, and its sole candidate ordering is obtained b y sorting its subgoals b y cn under B ( n 2). cn ( a ) j ; = 0 : 02, cn ( b ) j ; = 0 : 2, th us C andS et ( n 2) = fh a; b ig . 3. S ( n 3) is indivisibl e under B ( n 3). Therefore, n 3 is an OR-no de, and its three c hildren are created { one for eac h subgoal of S ( n 3) serving as the binder. Binder c ( X ) yields the c hild no de n 4 with the asso ciated set S ( n 4) = f d ( X ) ; e ( X ) g and the binding set B ( n 4) = f c ( X ) g . S ( n 4) is indep enden t under B ( n 4). There- fore, n 4 is a leaf, and its sole candidate is obtained b y sorting its subgoals b y cn : cn ( d ( X )) j f c ( X ) g = 0 ; cn ( e ( X )) j f c ( X ) g = 0 : 09; th us, the candidate of n 4 is h e ( X ) ; d ( X ) i . Binder d ( X ) yields the c hild no de n 5 with the asso ciated set S ( n 5) = f c ( X ) ; e ( X ) g and the binding set B ( n 5) = f d ( X ) g . S ( n 5) is indep enden t under B ( n 5), and its sorting b y cn pro duces the candidate h c ( X ) ; e ( X ) i . Binder e ( X ) yields the c hild no de n 6 with the asso ciated set S ( n 6) = f c ( X ) ; d ( X ) g and the binding set B ( n 6) = f e ( X ) g . S ( n 6) is indep endent under B ( n 6), and its sorting b y cn pro duces the candidate h c ( X ) ; d ( X ) i . 4. W e no w add eac h binder to its corresp onding c hild's candidate and obtain three order- ings of the OR-no de n 3: h c ( X ) ; e ( X ) ; d ( X ) i , h d ( X ) ; c ( X ) ; e ( X ) i , h e ( X ) ; c ( X ) ; d ( X ) i . 5. W e no w p erform folding of these orderings and c hec k violations of the adjacency restriction, in order to determine whether a blo c k has a c heap er p erm utation. 71 Ledenio v & Mark o vitch First, w e p erform the folding of h c ( X ) ; e ( X ) ; d ( X ) i . The pair h c ( X ) ; e ( X ) i is cn-in v erted: cn ( c ( X )) j ; = 0 : 2, cn ( e ( X )) j f c ( X ) g = 0 : 09. W e th us unite it in to a blo c k. This blo c k do es not pass the adjacency restriction test (Equation 8): c ost ( h c ( X ) ; e ( X ) i ) j ; = 5 + 2 10 = 25 ; c ost ( h e ( X ) ; c ( X ) i ) j ; = 20 + 0 : 4 5 = 22 : Therefore, this ordering is MC-con tradicting and can b e discarded. W e p erform the folding of h d ( X ) ; c ( X ) ; e ( X ) i . cn ( d ( X )) j ; = 0 : 3, cn ( c ( X )) j f d ( X ) g = 0 : 1, the pair is cn-in v erted, and w e unite it in to a blo c k. This blo c k do es not pass the adjacency restriction test: c ost ( h d ( X ) ; c ( X ) i ) j ; = 10 + 4 5 = 30 ; c ost ( h c ( X ) ; d ( X ) i ) j ; = 5 + 2 5 = 15 : This ordering is rejected to o, ev en b efore its folding is nished. If w e con tin ue the folding pro cess, w e shall see that the subgoal e ( X ) m ust also b e added to this blo c k, since cn ( h d ( X ) ; c ( X ) i ) j ; = 4 0 : 5 1 30 = 0 : 0333, and cn ( e ( X )) j h d ( X ) ;c ( X ) i = 0 : 09. W e p erform the folding of h e ( X ) ; c ( X ) ; d ( X ) i . cn ( e ( X )) j ; = 0 : 03, cn ( c ( X )) j f e ( X ) g = 0 : 1, the pair is cn-in v erted, and w e form a blo c k ec ( X ) = h e ( X ) ; c ( X ) i , whic h passes the adjacency restriction test: c os t ( h e ( X ) ; c ( X ) i ) j ; = 20 + 0 : 4 5 = 22 ; c os t ( h c ( X ) ; e ( X ) i ) j ; = 5 + 2 10 = 25 : W e compute the con trol v alues of the new blo c k: c os t ( ec ( X )) j ; = 20 + 0 : 4 5 = 22 n so ls ( ec ( X )) j ; = 0 : 4 0 : 5 = 0 : 2 cn ( ec ( X )) j ; = 0 : 2 1 22 = 0 : 0363636 cn ( d ( X )) j f ec ( X ) g = 0, th us the pair h ec ( X ) ; d ( X ) i is cn-ordered, no more folding is needed, and w e add the folded candidate h ec ( X ) ; d ( X ) i to the candidate set of n 3. 6. W e no w p erform merging of the candidate set of n 2, fh a; b ig , with the candidate set of n 3, fh ec ( X ) ; d ( X ) ig . In the resulting sequence max-blo c ks m ust b e sorted b y cn . cn ( a ) = 0 : 02 ; cn ( b ) = 0 : 2 ; cn ( ec ( X )) j ; = 0 : 0363636 ; cn ( d ( X )) j f ec ( X ) g = 0 : The merged ordering, h ec ( X ) ; a; d ( X ) ; b i , is added to the candidate set of n 1. 7. W e compare the costs of all candidates of n 1, and output the c heap est one. In our case, there is only one candidate, h ec ( X ) ; a; d ( X ) ; b i . The algorithm returns this candidate unfolded, h e ( X ) ; c ( X ) ; a; d ( X ) ; b i . 72 The Divide-and-Conquer Subgo al-Ordering Algorithm Cheap est prex Extension/Completion Cost ; h a i 10 h b i 5 h c ( X ) i 5 h d ( X ) i 10 h e ( X ) i 20 h b i h b; a i the adjacency restriction test fails h b; c ( X ) i 5 + 2 5 = 15 h b; d ( X ) i 5 + 2 10 = 25 h b; e ( X ) i the adjacency restriction test fails h c ( X ) i h c ( X ) ; e ( X ) ; a; d ( X ) ; b i 5 + 2(10 + 0 : 1(10 + 0 : 8(5 + 1 5))) = 28 : 6 h a i h a; b i 10 + 0 : 8 5 = 14 h a; c ( X ) i 10 + 0 : 8 5 = 14 h a; d ( X ) i 10 + 0 : 8 10 = 18 h a; e ( X ) i the adjacency restriction test fails h d ( X ) i h d ( X ) ; c ( X ) ; e ( X ) ; a; b i 10 + 4(5 + 0 : 5(10 + 0 : 1(10 + 0 : 8 5))) = 52 : 8 h a; b i h a; b; c ( X ) i 14 + 0 : 8 2 5 = 22 h a; b; d ( X ) i 14 + 0 : 8 2 10 = 30 h a; b; e ( X ) i the adjacency restriction test fails h a; c ( X ) i h a; c ( X ) ; e ( X ) ; d ( X ) ; b i 14 + 0 : 8 2(10 + 0 : 1(5 + 1 5)) = 31 : 6 h b; c ( X ) i h b; c ( X ) ; e ( X ) ; a; d ( X ) i 15 + 2 2(10 + 0 : 1(10 + 0 : 8 5)) = 60 : 6 h a; d ( X ) i h a; d ( X ) ; c ( X ) ; e ( X ) ; b i 18 + 0 : 8 4(5 + 0 : 5(10 + 0 : 1 5)) = 50 : 8 h e ( X ) i h e ( X ) ; c ( X ) ; a; d ( X ) ; b i 20 + 0 : 4(5 + 0 : 5(10 + 0 : 8(5 + 1 5))) = 25 : 6 h a; b; c ( X ) i h a; b; c ( X ) ; e ( X ) ; d ( X ) i 22 + 0 : 8 2 2(10 + 0 : 1 5) = 55 : 6 h b; d ( X ) i h b; d ( X ) ; c ( X ) ; e ( X ) ; a i 25 + 2 4(5 + 0 : 5(10 + 0 : 1 10)) = 109 h e ( X ) ; c ( X ) ; a; d ( X ) ; b i complete ordering T able 3: A trace of a sample run of Algorithm 4 on the set of Figure 8. The left column sho ws the c heap est prex extracted from the list on eac h step, the middle column { its extensions or completions that are added to the list, and the righ t column { their asso ciated costs. F or comparison, w e no w sho w ho w the same task is p erformed b y Algorithm 4. The algorithm main tains a list of prexes, sorted b y their cost v alues, and whic h initially con tains an empt y sequence. On eac h step the algorithm extracts from the list its c heap est elemen t, and adds to the list the extensions or completions of this prex. Extensions are created when the set of remaining subgoals is dep enden t, b y app ending eac h of the remaining subgoals to the end of the prex. Completions are created when the set of remaining subgoals is indep enden t, b y sorting them and app ending the en tire resulting sequence to the prex. An extension is added to the list only when the adjacency restriction test succeeds on its t w o last subgoals. T o mak e the list op erations faster, w e can implemen t it as a heap structure (Cormen et al., 1991). The trace of Algorithm 4 on the set S 0 is sho wn in T able 3. The left column sho ws the c heap est prex extracted from the list on eac h step, the middle column { its extensions or completions that are added to the list, and the righ t column { their asso ciated costs. It lo oks as if the d a c algorithm orders the giv en set S 0 more ecien tly than Algorithm 4. W e can compare sev eral discrete measuremen ts to sho w this. F or example, Algorithm 6 73 Ledenio v & Mark o vitch p 5 4 X X 8 X 9 X 10 ( , , , ) p 1 1 X X 2 X 3 4 X ( , , , ) p 3 1 X X 5 X 6 X 8 ( , , , ) 2 p X 2 X 5 X 7 X 9 ( , , , ) p 4 X 7 X 10 X 6 X 3 ( , , , ) Figure 14: An example of the w orst case for ordering. When all v ariables are initially free, ev ery subset of subgoals is indivisible under the binding of the rest of subgoals, and the o v erall complexit y of ordering b y Algorithm 6 is O ( n !). p erforms 4 sorting sessions, eac h one with 2 elemen ts, while Algorithm 4 p erforms 5 sortings with 2 elemen ts, and 3 sortings with 3 elemen ts. The adjacency restriction is tested only 3 times b y Algorithm 6, and 11 times b y Algorithm 4. Algorithm 6 creates totally 8 dieren t ordered sub-sequences, with total length 22, while Algorithm 4 creates 24 ordered prexes, with total length 55. 4.8 Complexit y Analysis Both Algorithm 4 and Algorithm 6 nd a minimal ordering, and b oth sort indep enden t subsets of subgoals whenev er p ossible. Algorithm 6, ho w ev er, oers sev eral adv an tages due to its divide-and-conquer strategy . Let n b e the n um b er of subgoals in the initial set. F or con v enience, w e assume that the time of computing the con trol v alues for one subgoal is O (1); otherwise, if this time is , all the complexities b elo w m ust b e m ultiplied b y . The w orst case complexit y of Algorithm 6 is O ( n !). Figure 14 sho ws an example of suc h a case for n = 5. In this set ev ery t w o subgoals share a v ariable that do es not app ear in other subgoals. Th us, other subgoals cannot bind it. The set of the ro ot is indivisibl e, and no matter whic h binder is c hosen, the sets of the c hildren are indivisibl e. So, in eac h c hild of the ro ot, w e m ust select ev ery remaining subgoal as the binder, and so on. The o v erall complexit y of this execution is O ( n !). This is indeed the w orst-case complexit y: presence of AND-no des in the tree can only reduce it. Note that ev en when n is small, suc h a complex rule b o dy with ( n 2 ) free v ariables is v ery improbable in practical programs. Also, the w orst-case complexit y can b e reduced to O ( n 2 2 n ), if w e mo v e from divisibil it y trees to divisibility gr aphs (D A Gs), where all iden tical no des of a divisibil i t y tree (same subgoal set, same binding set) are represen ted b y a single v ertex. The equiv alence test of the tree no des can b e p erformed ecien tly with the help of trie structures (Aho et al., 1987), where subgoals are sorted lexicographically . Let there b e n subgoals, with v shared v ariables app earing in m subgoals. As w as already noted in Section 4.3, the partition of subgoals in to subsets can b e p erformed in 74 The Divide-and-Conquer Subgo al-Ordering Algorithm O ( n ) a v erage time, using a Union-Find data structure (Cormen et al., 1991, Chapter 22). In the w orst p ossible case, there are no AND-no des in the divisibili t y tree, apart from the ro ot no de (whose set is divisible in to a dep enden t set of size m and an indep enden t set of size n m ). The o v erall complexit y of the d a c algorithm in suc h a case is T ( n; m; v ) = O ( n ) | divisibili t y partition + O (( n m ) log( n m )) | ordering of indep enden t subgoals + O (( Q k i =0 ( m i )) log ( m k )) | ordering of dep enden t subgoals + O ( m Q k 1 i =0 ( m i )) | folding + O ( n Q k 1 i =0 ( m i )) | merging where k is the maximal p ossible n um b er of bindings p erformed b efore the remaining subset is indep enden t. If w e assume that ev ery subgoal binds all its free v ariables (whic h happ ens v ery frequen tly in practical logic programs), then k = min f v ; m 1 g ; otherwise k = m 1. k is equal to the maximal n um b er of OR-no des on a path from the ro ot to a leaf of the divisibil it y tree. Therefore, the heigh t of the divisibili t y tree is limited b y k + 1. Actually , the tree can b e shallo w er, since some binders can bind more than one shared v ariable eac h. This means that the n um b er of shared v ariables can decrease b y more than 1 in eac h OR- no de. Belo w w e simplify the ab o v e form ula for sev eral common cases, when k is small and when the ab o v emen tioned assumption holds (ev ery subgoal binds all its free v ariables after its pro of terminates). If v < m n : T ( n; m; v ) = O ( n m v + n log n ) If m v n : T ( n; m; v ) = O ( n m m 1 + n log n ) If v m ' n : T ( n; m; v ) = O ( n v +1 log n ) If m v ' n : T ( n; m; v ) = O ( n m ! + n log n ) Generally , for a small n um b er v of shared v ariables, the complexit y of the algorithm is roughly b ounded b y O ( n v +1 log n ). In particular, if all subgoals are indep enden t ( v = 0), the complexit y is O ( n log n ). In most practical cases, the n um b er of shared free v ariables in a rule b o dy is relativ ely small, and ev ery subgoal binds all its free v ariables; therefore, the algorithm has p olynomial complexit y . Note that ev en if a rule b o dy in the program text con tains man y free v ariables, most of them usually b ecome b ound after the rule head unication is p erformed (i.e., b efore w e start the ordering of the instan tiated b o dy). 5. Learning Con trol Kno wledge for Ordering The ordering algorithms describ ed in the previous sections assume the a v ailabilit y of correct v alues of a v erage cost and n um b er of solutions for v arious predicates under v arious argumen t bindings. In this section w e discuss ho w this con trol kno wledge can b e obtained b y learning. Instead of static exploration of the program text (Debra y & Lin, 1993; Etzioni, 1993), w e adopt the approac h of Mark o vitc h and Scott (1989) and learn the con trol kno wledge b y collecting statistics on the literals that w ere pro v ed in the past. This learning can b e p erformed on-line or o-line. In the latter case, the ordering system rst w orks with a tr aining set of queries, while collecting statistics. This training set can b e built on the 75 Ledenio v & Mark o vitch distribution of user queries seen in the past. W e assume that the distribution of queries receiv ed b y the system do es not c hange signican tly with time; hence, the past distribution directs the system to learn relev an t kno wledge for the future queries. While pro ving queries, the learning comp onen t accum ulates information ab out the con- trol v alues (a v erage cost and n um b er of solutions) of v arious literals. Storing a separate v alue for eac h literal is not practical, for t w o reasons. The rst is the large space required b y this approac h. The second is the lac k of generalization: the ordering algorithm is quite lik ely to encoun ter literals whic h ha v e not b een seen b efore, and whose con trol v alues are unkno wn. Recall that when w e transformed Equation 2 in to Equation 5, w e mo v ed from con trol v alues of single literals to a v erage con trol v alues o v er sets of liter als . T o obtain the precise a v erages for these sets, w e still needed the con trol v alues of individual literals. Here, w e tak e a dieren t approac h, that of learning and using con trol v alues for mor e gener al classes of literals. The estimated c os t ( n so ls ) v alue of a class can b e dened as the a v erage real cost (nsols) v alue of all examples of this class that w ere pro v ed in the past. The more rened the classes, the smaller the v ariance of real con trol v alues inside eac h class, the more precise the c ost and n sols estimations that the classes assign to their mem- b ers, and the b etter orderings w e obtain. One easy w a y to dene classes is b y mo des or binding p atterns (Debra y & W arren, 1988; Ullman & V ardi, 1988): for eac h argu- men t w e denote whether it is free or b ound. F or example, for the predicate father the p ossible classes are father(free,free) , father(bound,free) , father(free,bound) and father(bound,bound) . No w, if w e receiv e a literal (for example, father(abraham,X) ), w e can easily determine its binding pattern (in this case, father(bound,free) ) and re- triev e the con trol information stored for this class. Of course, to nd the binding pattern of a subgoal with a giv en binding set, w e need a metho d to determine whic h v ariables are b ound b y the subgoals of the binding set. The same problem arose in D P ar t computation (Section 4.3). W e shall discuss some practical w a ys to solv e this problem in Section 7.1. F or the purp ose of class denition w e can also use r e gr ession tr e es { a t yp e of decision tree that classies to con tin uous n umeric v alues and not to discrete classes (Breiman et al., 1984; Quinlan, 1986). Tw o separate regression trees can b e stored for ev ery program predicate, one for its c os t v alues, and one for the n sols . The tests in the tree no des can b e dened in v arious w a ys. If w e only use the test \is ar gument i b ound?" , then the classes of literals dened b y regression trees coincide with the classes dened b y binding patterns. But w e can also apply more sophisticated tests, b oth syn tactic (e.g., \is the thir d ar gument a term with functor f ?" ) and seman tic (e.g., \is the thir d ar gument female?" ), whic h leads to more rened classes and b etter estimations. A p ossible regression tree for estimating the of n um b er of solutions for predicate father is sho wn in Figure 15. Seman tic tests ab out the argumen ts require logic inference (in the example of Figure 15 { in v oking the predicate female on the rst argumen t of the literal). Therefore, they m ust b e as ecien t as p ossible. Otherwise the retriev al of con trol v alues will tak e to o m uc h time. The problem of ecien t learning of con trol v alues is further considered elsewhere (Ledenio v & Mark o vitc h, 1998a). Sev eral researc hers applied mac hine learning tec hniques for accelerating logic inference (Cohen, 1990; Dejong & Mo oney , 1986; Langley , 1985; Mark o vitc h & Scott, 1993; Min ton, 1988; Mitc hell, Keller, & Kedar-Cab elli, 1986; Mo oney & Zelle, 1993; Prieditis & Mosto w, 1987). Some of these w orks used explanation-based learning or generalized cac hing tec h- 76 The Divide-and-Conquer Subgo al-Ordering Algorithm yes no no no yes yes yes no Average: 1.0 Test: bound(arg2)? Average: 0.98 Test: bound(arg2)? Average: 0.3 Test: female(arg1)? Average: 0.0 Average: 0.0001 Average: 0.5 Average: 50 Average: 3.1416 Test: bound(arg1)? Average: 5 Figure 15: A regression tree that estimates the n um b er of solutions for father(arg1,arg2) . niques to a v oid rep eated computation. Others utilized the acquired kno wledge for the prob- lem of clause sele ction . None of these w orks, ho w ev er, dealt with the problem of subgoal reordering. 6. Exp erime n tation T o test the eectiv eness of our ordering algorithm, w e exp erimen ted with it on v arious domains, and compared its p erformance to other ordering algorithms. Most exp erimen ts w ere p erformed on randomly created articial domains. W e also tested the p erformance of the system on sev eral real domains. 6.1 Exp erimen tal Metho dology All exp erimen ts describ ed b elo w consist of a training session, follo w ed b y a testing session. T raining and testing sets of queries are randomly dra wn from a xed distribution. In the training session w e collect the con trol kno wledge for literal classes. In the testing session w e pro v e the queries of the testing set using dieren t ordering algorithms, and compare their p erformance using v arious measuremen ts. The goal of ordering is to reduce the time sp en t b y the Prolog in terpreter when it pro v es queries of the testing set. This time is the sum of the time sp en t b y the ordering pro cedure ( or dering time ) and the time sp en t b y the in terpreter ( infer enc e time ). Since the CPU time is kno wn to b e v ery sensitiv e to irrelev an t factors suc h as hardw are, soft w are and programming qualit y , w e also sho w t w o alternativ e discrete measuremen ts: the total n um b er of clause unic ations , and the total n um b er of clause r e ductions p erformed. The n um b er of reductions reects the size of the pro of tree. F or exp erimen tation w e used a new v ersion of the lassy system (Mark o vitc h & Scott, 1989), using regression trees for learning, and the ordering algorithms discussed in this pap er. 77 Ledenio v & Mark o vitch 6.2 Exp erimen ts with Articial Domains In order to ensure the statistical signicance of the results of comparing dieren t ordering algorithms, w e exp erimen ted with man y dieren t domains. F or this purp ose, w e created a set of 100 articial domains, eac h with a small xed set of predicates, but with a random n um b er of clauses in eac h predicate, and with random rule lengths. Predicates in the rule b o dies, and argumen ts in b oth rule heads and b o dies are randomly dra wn from xed distributions. Eac h domain has its o wn training and testing sets (these t w o sets do not in tersect). The more training examples are fed in to the system on the learning phase, the b etter estimations of con trol v alues it pro duces. On the other hand, the learning time m ust b e lim- ited, b ecause after seeing a certain n um b er of training examples, new examples do not bring m uc h new information, and additional learning b ecomes w asteful. W e ha v e exp erimen tally built a learning curv e whic h sho ws the dep endence of the qualit y of the con trol kno wledge on the amoun t of training. The curv e suggests that after con trol v alues w ere learned for appro ximately 400 literals, there is no signican t impro v emen t in the qualit y of ordering with new training examples. Therefore, in the subsequen t exp erimen ts w e stopp ed training after 600 cost v alues w ere learned. The training time w as alw a ys small: one learned cost v alue corresp onds to a complete pro of of a literal. Th us, if ev ery predicate in a program has four clauses that dene it, then 600 cost v alues are learned after 2400 unications, whic h is a v ery small time. The con trol v alues w ere learned b y means of regression trees (Section 5), with simple syn tactic tests that only c hec k ed whether some argumen t is b ound or whether some argu- men t is a term with a certain functor (the list of functors w as created automatically when the domain w as loaded). Ho w ev er, as w e shall see, ev en these simple tests succeeded in making go o d estimations of con trol v alues. W e tested the follo wing ordering metho ds: Random: The subgoals are p erm uted randomly and the con trol kno wledge is not used. Algorithm 3: Building ordered prexes. Out of all prexes that are p erm utation of one another, only the c heap est one is retained. Algorithm 3a: As Algorithm 3, but with b est-rst searc h metho d used to dene the next pro cessed prex. A similar algorithm w as used in the lassy system of Mark o vitc h and Scott (1989). Algorithm 3b: As Algorithm 3a, but with adjacency restriction test added. A similar algorithm w as describ ed b y Smith and Genesereth (1985). Algorithm 4: As Algorithm 3b, but whenev er all the subgoals that are not in the prex are indep enden t (under the binding of the prex), they are sorted and the result is app ended to the prex as one unit. Algorithm 6: The d a c algorithm. In our exp erimen ts w e alw a ys used the Bubble-Sort algorithm to sort literals in inde- p enden t sets. This algorithm is easy to implemen t, and it is kno wn to b e ecien t for small 78 The Divide-and-Conquer Subgo al-Ordering Algorithm Ordering Unications Reductions Ordering Inference T otal Ord.Time Metho d Time Time Time Reductions Random 86052.06 27741.52 8.1910 27.385 35.576 0.00029 Algorithm 3 2600.32 911.04 504.648 1.208 505.856 0.55 Algorithm 3.a 2829.00 978.59 347.313 1.178 348.491 0.35 Algorithm 3.b 2525.34 881.12 203.497 1.137 204.634 0.23 Algorithm 4 2822.27 976.02 40.284 1.191 41.475 0.04 Algorithm 6 2623.82 914.67 2.3620 1.102 3.464 0.0025 T able 4: The eect of ordering on the tree sizes and the CPU time (mean results o v er 100 articial domains). sets, when the elemen ts are already ordered, or nearly ordered. In practice, programmers order most program rules optimally , and the sorting stops early . Since the non-deterministic nature of the random metho d in tro duces additional noise, w e p erformed on eac h articial domain 20 exp erimen ts with this metho d, and the table presen ts the a v erage v alues of these measuremen ts. T able 4 sho ws the obtained results o v er 100 domains: the ro ws corresp ond to the ordering metho ds used, and the columns to the measuremen ts tak en. The righ tmost column sho ws the ratio of the ordering time and the n um b er of reductions p erformed, whic h reects the a v erage ordering time of one rule b o dy . The inference time w as not measured separately , but w as set as the dierence of the total time and the ordering time. Sev eral observ ations can b e made: 1. Using the d a c ordering algorithm helps to reduce the total time of pro ving the testing set of queries b y a factor of 10, compared to the random ordering. The inference time is reduced b y a factor of 25. 2. All deterministic ordering metho ds ha v e similar n um b er of unications and reductions, and similar inference time, whic h is predictable, since they all nd minimal orderings. Small uctuations of these v alues can b e explained b y the fact that some rules ha v e sev eral minimal orderings under the existing con trol kno wledge, and dieren t ordering algorithms select dieren t minimal orderings. Since the con trol kno wledge is not absolutely precise, the real execution costs of these orderings ma y b e dieren t, whic h leads to the dierences. The random ordering metho d builds m uc h larger trees, with larger inference time. 3. When w e compare the p erformance of the deterministic algorithms (3 { 6), w e see that the d a c algorithm p erforms m uc h b etter than the algorithms that build ordered prexes. In the latter ones, the ordering is exp ensiv e, and smaller inference time cannot comp ensate for the increase in ordering time. Only Algorithm 4, a com bination of sev eral ideas of previous researc hers, has total time comparable with the time of the random metho d (though still greater). 79 Ledenio v & Mark o vitch 4. It ma y seem strange that the simple random ordering metho d has larger ordering time than the sophisticated Algorithm 6. T o explain this, note that the random metho d creates m uc h larger pro of trees (on a v erage), therefore the n um b er of ordered rules increases, and ev en the c heap op erations, lik e random ordering of a rule, sum up to a considerable time. The a v erage time sp en t on ordering of one rule is sho wn in the last column of T able 4; this v alue is v ery small in the random metho d. 6.3 Exp erimen ts with Real Domains W e tested our ordering algorithm also on real domains obtained from v arious sources. These domains allo w us to compare orderings p erformed b y our algorithm with orderings p er- formed b y h uman programmers. The follo wing domains w ere used: Moral-reasoner: T ak en from the Mac hine Learning Rep ository at the Univ ersit y of California, Irvine 1 . The domain qualitativ ely sim ulates moral reasoning: whether a p erson can b e considered guilt y , giv en v arious asp ects of his c haracter and of the crime p erformed. Depth-rst planner: Program 14.11 from the b o ok \The Art of Prolog" (Sterling & Shapiro, 1994). The program implemen ts a simple planner for the blo c ks w orld. Biblical F amily Database: A database similar to that describ ed in Example 1. Appletalk: A domain describing the ph ysical la y out of a lo cal computer net w ork (Mark o vitc h, 1989). Benc hmark: A Prolog b enc hmark tak en from the CMU Articial In telligence Rep osi- tory 2 . The predicate names are not informativ e: it is an example of a program where man ual ordering is dicult. Slo w rev erse: Another b enc hmark program from the same source. Geograph y: Also a b enc hmark program from the CMU Rep ository . The domain con tains man y geographical facts ab out coun tries. T able 5 sho ws the results obtained. F or ordering w e used the d a c algorithm, with literal classes dened b y binding patterns. It can b e seen that the d a c algorithm w as able to sp eed up the logic inference in real domains as w ell. Note that in the Slo w Rev erse domain the programmer's ordering w as already optimal; th us, applying the ordering algorithm did not reduce the tree sizes. Still, the o v erhead of the ordering is not signican t. 7. Discussion In this concluding section w e discuss sev eral issues concerning the practical implemen tation of the d a c algorithm and sev eral w a ys to increase its eciency . Then w e surv ey some related areas of logic programming and prop ose the use of the d a c algorithm there. 1. URL: h ttp://www.ics.uci.edu/ ~ mlearn/MLRep osi tory .h tml 2. URL: h ttp://www.cs.cm u.edu/afs/cs.cm u.edu/pro ject/ai-rep osi tory/ ai/ h tml/ ai r.h tml 80 The Divide-and-Conquer Subgo al-Ordering Algorithm Domain Without ordering With ordering Gain ratio unications seconds unications seconds (time/time) Moral-reasoner 352180 98.39 87020 23.53 4.2 Depth-rst planner 10225 19.01 9927 18.16 1.05 Biblical F amily 347827 112.68 120701 46.08 2.5 Appletalk 5036167 1246.30 640092 221.73 5.6 Benc hmark 62012 554.31 46012 395.04 1.4 Slo w rev erse 6291 10.33 6291 11.92 0.9 Geograph y 428480 141.47 226628 82.76 1.7 T able 5: Exp erimen ts on real domains. 7.1 Practical Issues In this subsection w e w ould lik e to address sev eral issues related to implemen tation and applications of the d a c algorithm. The computation of the D P ar t function (Section 3.2.1) requires a pro cedure for com- puting the set of v ariables b ound b y a giv en binding set of subgoals. The same pro cedure is needed for computing con trol v alues (Section 5). There are sev eral p ossible w a ys to implemen t suc h a pro cedure. F or example: 1. The easiest w a y is to assume that ev ery subgoal binds all the v ariables app earing in its argumen ts. This simplistic assumption is sucien t for man y domains, esp ecially the database-orien ted ones. Ho w ev er, it is not appropriate when logic programs are used to manipulate complex data structures con taining free v ariables (suc h as dierence lists). This assumption w as used for the exp erimen ts describ ed in Section 6. 2. Some dialects of Prolog and other logic languages supp ort mo de de clar ations pro vided b y the user (Somogyi et al., 1996b). When suc h declarations are a v ailable, it is easy to infer the binding status of eac h v ariable up on exiting a subgoal. 3. Ev en when the user did not supply enough mo de declarations, they can often b e inferred from the structure of the program b y means of static analysis (Debra y & W arren, 1988). Note, ho w ev er, that as w as p oin ted out b y Somogyi et al. (1996b), no-one has y et demonstrated a mo de inference algorithm that is guaran teed to nd accurate mo de information for ev ery predicate in the program. 4. W e can le arn the sets of v ariables b ound b y classes of subgoals using metho ds similar to those describ ed in Section 5 for learning con trol v alues. Sev eral researc hers adv o cate user declarations of a v ailable (p ermitted) mo des. Suc h declarations can b e elegan tly incorp orated in to our algorithm to prune branc hes that violate a v ailable mo des. When w e x a binder in an OR-no de, w e compute the set of v ariables that b ecome b ound b y it. If this results in a violation of an a v ailable mo de for one of the subgoals of the corresp onding c hild, then the whole subtree of this c hild is pruned. Note that w e can detect violations ev en when the mo de of the subgoal is partially unkno wn 81 Ledenio v & Mark o vitch CandidateSet ( S ; B ) let fS 1 ; S 2 ; : : : S k g D P ar t ( S ; B ) case : : : k = 1, shar e d-vars ( S 1 ) 6 = ; ( S is indivisibl e under B ): lo op for A 2 S if B [ f A g do es not violate a v ailable mo des in an y subgoal of S n f A g then let C ( A ) CandidateSet ( S n f A g ; B [ f A g ) let C 0 ( A ) n F old ( A k ~ O A ; B ) ~ O A 2 C ( A ) o else let C 0 ( A ) ; (don't en ter the branc h) Return S A 2S C 0 ( A ) Figure 16: Changes to Algorithm 6 that mak e use of a v ailable mo de declarations. The rest of the algorithm remains unc hanged. at the momen t. F or example, if all the a v ailable mo des require that the rst argumen t b e un b ound, then binding of the argumen t b y the OR-no de binder will trigger pruning, ev en if the binding status of the other argumen ts is not y et kno wn. Figure 16 sho ws ho w Algorithm 6 can b e c hanged in order to incorp orate declarations of a v ailable mo des. An y other correctness requiremen t can b e treated in a similar manner: a candidate ordering will b e rejected whenev er w e see that it violates the requiremen t. The exp erimen ts describ ed in Section 6 w ere p erformed with a Prolog in terpreter. Is it p ossible to com bine the d a c algorithm with a Prolog compiler? There are sev eral w a ys to ac hiev e this goal. One w a y is to allo w the compiler to insert co de for on-line learning. The compiled co de will con tain pro cedures for accum ulating con trol v alues and for the d a c algorithm. Alternativ ely , o-line learning can b e implemen ted, with training as a part of the compilation pro cess. Another metho d for com bining our algorithm with existing Prolog compilers is to use it for program transformation, and to pro cess the transformed program b y a standard compiler. Elsewhere (Ledenio v & Mark o vitc h, 1998a) w e describ e the metho d for classifying the orderings pro duced b y the d a c algorithm. F or eac h rule w e build a classication tree, where classes are the dieren t orderings of the rule b o dy , and the tests are applied to the rule head argumen ts. These are the same t yp e of tests describ ed in Section 5 for learning con trol v alues. Figure 17 sho ws t w o examples of suc h trees. Giv en suc h a classication tree, w e can write a set of Prolog rules, where eac h rule has the same head as the original rule, and has a b o dy built of all the tests on the path from the tree ro ot to a leaf no de follo w ed b y the ordering at the leaf. F or example, the second tree in Figure 17 yields the follo wing set of rules: 82 The Divide-and-Conquer Subgo al-Ordering Algorithm A classication tree for the rule uncle(X,Y) of Example 1. [parent(Z,Y),brother(Z,X)] [brother(Z,X),parent(Z,Y)] nonvar(Y) ? no yes A p ossible classication tree for the rule head(X,Y) p1(X), p2(Y), p3(X,Y). [p2(Y),p3(X,Y),p1(X)] [p3(X,Y),p1(X),p2(Y)] [p1(X),p3(X,Y),p2(Y)] [p1(X),p2(Y),p3(X,Y)] nonvar(X) ? male(X) ? nonvar(Y) ? no no yes no yes yes Figure 17: Examples of classication trees that learn rule b o dy orderings. head(X,Y) nonvar(X), male(X), p1(X), p3(X,Y), p2(Y). head(X,Y) nonvar(X), not(male(X)), p1(X), p2(Y), p3(X,Y). head(X,Y) var(X), nonvar(Y), p2(Y), p3(X,Y), p1(X). head(X,Y) var(X), var(Y), p3(X,Y), p1(X), p2(Y). F rom T able 4 w e can see that while the d a c algorithm help ed to reduce the inference time b y a factor of 25, the total time w as reduced only b y a factor of 10. This dierence is caused b y the additional computation of the ordering pro cedure. There is a danger that the b enet obtained b y ordering will b e out w eighed b y the cost of the ordering pro cess. This is a manifestation of the so-called utility pr oblem (Min ton, 1988; Mark o vitc h & Scott, 1993). In systems that are strongly-mo ded (suc h as Mercury { Somogyi et al., 1996b) w e can emplo y the d a c algorithm statically at compilation time for eac h one of the a v ailable mo des, th us reducing the run-time ordering time to zero. The mo de-based approac h p erforms only syn tactic tests of the subgoal argumen ts. The classication tree metho d, describ ed ab o v e, is a generalization of the mo de-based approac h, allo wing seman tic tests as w ell. Due to insucien t learning exp erience or lac k of meaningful seman tic tests, it is quite p ossible that the classication trees con tain lea v es with large degrees of error. In suc h cases w e still need to p erform the ordering dynamically . T o reduce the harmfulness of the utilit y problem in the case of dynamic ordering, w e can use a cost-sensitiv e v ariation of the d a c algorithm (Ledenio v & Mark o vitc h, 1998a, 1998b). This mo died algorithm deals with the problem b y explicit reasoning ab out the econom y of the con trol pro cess. The algorithm is anytime , that is, it can b e stopp ed at an y momen t and return its curren tly b est ordering (Bo ddy & Dean, 1989). W e learn a r esour c e-investment function to compute the exp ected return in sp eedup time for additional con trol time. This function is used to determine a stopping condition for the an ytime pro cedure. W e ha v e implemen ted this framew ork and found that indeed w e ha v e succeeded in reducing ordering time, without signican t increase of inference time. 83 Ledenio v & Mark o vitch 7.2 Relationship to Other W orks The w ork describ ed in this pap er is a con tin uation of the line of researc h initiated b y Smith and Genesereth (1985) and con tin ued b y Natara jan (1987) and Mark o vitc h and Scott (1989). This line of researc h aims at nding the most ecien t ordering of a set of subgoals. The searc h for minimal-cost ordering is based on cost analysis that utilizes a v ailable information ab out the cost and n um b er-of-solutions of individual subgoals. Smith and Genesereth (1985) p erformed an exhaustiv e searc h o v er the space of all p erm utations of the giv en set of subgoals, using the adjac ency r estriction to reduce the size of the searc h space (Equation 8). This restriction w as applied on pairs of adjacen t subgoals in the global ordering of the en tire set. When applied to an indep endent set of sub go als , the adjacency restriction is easily transformed in to the sorting restriction: the subgoals in a minimal ordering m ust b e sorted b y their cn v alues. Natara jan (1987) arriv ed at this conclusion and presen ted an ecien t ordering algorithm for indep enden t sets. The d a c algorithm uses subgoal dep endence to break the set in to smaller subsets. In- dep enden t subsets are sorted. Dep enden t subsets are recursiv ely ordered, and the resulting orderings are merged using a generalization of the adjacency restriction that manipulates blo c ks of subgoals. Therefore the d a c algorithm is a generalization of b oth algorithms. During the last decade, a signican t researc h eort w en t in to static analysis (SA) of logic programs. There are three t yp es of SA that can b e exploited b y the d a c algorithm to reduce the ordering time. A ma jor part of the SA researc h deals with pr o gr am termination (De Sc hrey e & Decorte, 1994). The d a c algorithm solv es the termination problem, as a sp ecial case of the eciency problem (it alw a ys nds a terminating ordering, if suc h orderings exist). During learning, w e set limits on the computation resources a v ailable for subgoal execution. If a subgoal is non-terminating (in a certain mo de), the learning mo dule will asso ciate a v ery high cost with this particular mo de. Consequen tly , the d a c algorithm will not allo w orderings with this mo de of the subgoal. Nev ertheless, while the use of static termination analysis is not mandatory for a prop er op eration of the d a c algorithm, w e can exploit suc h analysis to increase the eciency of b oth the learning pro cess and the ordering pro cess. During learning, the limit that w e set on the computation resources dev oted to the execution of a subgoal m ust b e high, to increase the reliabilit y of the cost estimation. Ho w ev er, suc h a high limit can lead to a signican t increase in learning time when man y subgoals are non-terminating. If termination information obtained b y SA is a v ailable, w e can use it to a v oid en tering innite branc hes of pro of trees. During ordering, termination information can serv e to reduce the size of space of orderings searc hed b y the algorithm. If the termination information comes in the form of allo w ed mo des (Somogyi et al., 1996b), orderings that violate these mo des are ltered out, as in the mo died algorithm sho wn in Figure 16. If the termination information comes in the form of a partial order b et w een subgoals, orderings that violate this partial order can b e ltered out in a similar manner. The second t yp e of SA researc h that can b e com bined with the d a c algorithm is c or- r e ctness analysis , where the program is tested against sp ecications giv en b y the user. The f olon en vironmen t (Henrard & Le Charlier, 1992) w as designed to supp ort the metho dology for logic program construction that aims at reconciling the declarativ e seman- tics with an ecien t implemen tation (Deville, 1990). The construction pro cess starts with 84 The Divide-and-Conquer Subgo al-Ordering Algorithm a sp ecication, con v erts it in to a logic description and nally , in to a Prolog program. If the rules of the program are not correct with resp ect to the initial sp ecication, the sys- tem p erforms transformations suc h as reordering literals in a clause, adding t yp e c hec king literals and so on. De Bo ec k and Le Charlier (1990) men tion this reordering, but do not sp ecify an ordering algorithm dieren t from the simple generate-and-test metho d. Cortesi, Le Charlier, and Rossi (1997) presen t an analyzer for v erifying the correctness of a Prolog program relativ e to a sp ecication whic h pro vides a list of input/output annotations for the argumen ts and parameters that can b e used to establish program termination. Again, no ordering algorithm is giv en explicitly . The purp ose of the d a c algorithm is complemen tary to the purp ose of f olon , and it could serv e as an auxiliary aid to mak e the resulting Prolog program more ecien t. Recen tly , the Mercury language w as dev elop ed at the Univ ersit y of Melb ourne (Somogyi et al., 1996a, 1996b). Mercury is a strongly t yp ed and strongly mo ded language. T yp e and mo de declarations should b e supplied b y the programmer (though recen t releases of the Mercury system already supp ort partial inference of t yp es and mo des { Somogyi et al., 1996a). The compiler c hec ks that mo de declarations for all predicates are satised; if necessary , it r e or ders subgoals in the rule b o dy to ensure mo de correctness (and rejects the program if neither ordering satises the mo de declaration constrain ts). When the compiler p erforms this reordering, it do es not consider the eciency issue. It often happ ens that sev eral orderings of a rule b o dy satisfy the mo de declaration constrain ts: in suc h cases the Mercury compiler could call the static v ersion of the d a c algorithm to select the most ecien t ordering. Another alternativ e is to augmen t the d a c algorithm b y mo de declaration c hec ks, as w as sho wn in Figure 16. Note that Mercury is a purely declarativ e logic programming language, and is therefore more suitable for subgoal reordering than Prolog. It has no non-logical constructs that could destro y the declarativ e seman tics whic h giv e logic programs their p o w er; in Mercury ev en I/O is declarativ e. The third t yp e of relev an t SA is the c ost analysis of logic programs (Debra y & Lin, 1993; Braem et al., 1994; Debra y et al., 1997). Cortesi et al. (1997) describ e a cost form ula similar to Equation 5 to select a lo w est-cost ordering. Ho w ev er, they used a generate-and- test approac h whic h can sometimes b e prohibitiv ely exp ensiv e. Static analysis of cost and n um b er of solutions can b e used to obtain the con trol v alues, instead of learning them. The eciency of logic programs can also b e increased b y metho ds of pr o gr am tr ans- formation (P ettorossi & Proietti, 1994, 1996). One of the most p opular approac hes is the \rules+strategies" approac h, whic h consists in starting from an initial program and then applying one or more elemen tary tr ansformation rules . T ransformation strategies are meta- rules whic h prescrib e suitable sequences of applications of transformation rules. One of the p ossible transformation rules is the go al r e arr angement rule whic h transforms a program b y transp osing t w o adjacen t subgoals in a rule b o dy . Ob viously , an y ordering of a rule b o dy can b e transformed in to an y other ordering b y a nite n um b er of suc h transp ositions. Th us, static subgoal ordering can b e considered a sp ecial case of program transformation where only the goal rearrangemen t rule is used. On the other hand, dynamic and semi-dynamic ordering metho ds cannot b e represen ted b y simple transformation rules, since they mak e use of run-time information (expressed in bindings that rule b o dy subgoals 85 Ledenio v & Mark o vitch obtain through unications of rule heads), and ma y order the same rule b o dy dieren tly under dieren t circumstances. A program transformation tec hnique called c ompiling c ontr ol (Bruyno oghe, De Sc hrey e, & Krek els, 1989; P ettorossi & Proietti, 1994) follo ws an approac h dieren t from that of trying to impro v e the con trol strategy of logic programs. Instead of enhancing the naiv e Prolog ev aluator using a b etter (and often more complex) computation rule, the program is transformed so that the deriv ed program b eha v es under the naiv e ev aluator exactly as the initial program w ould b eha v e under an enhanced ev aluator. Most forms of compiling con trol rst translate the initial program in to some standard represen tation (for example, in to an unfolding tr e e ), while the complex computation rule is used, and then the new program is constructed from this represen tation, with the naiv e computation rule in mind. Reordering of rule b o dy subgoals can b e regarded as mo ving to a complex computation rule whic h selects subgoals in the order dictated b y the ordering algorithm. In the case of the d a c algorithm, this computation rule ma y b e to o complex for simple use of compiling con trol metho ds. Nev ertheless, it can b e easily incorp orated in to a sp ecial compiling con trol metho d. In Section 7.1 w e describ ed a metho d of program rewriting whic h rst builds classication trees based on the orderings that w ere p erformed in the past, and then uses these classication trees for constructing clauses of a deriv ed program. The deriv ed program can b e ecien tly executed under the naiv e computation rule of Prolog. This tec hnique is in fact a kind of compiling con trol. Its imp ortan t prop ert y is the use of kno wledge collected from exp erience (the orderings that w ere made in the past). One transformation metho d that can signican tly b enet from the d a c algorithm is unfolding (T amaki & Sato, 1984). During the unfolding pro cess subgoals are replaced b y their asso ciated rule b o dies. Ev en if the initial rules w ere ordered optimally b y a h uman programmer or a static ordering pro cedure, the resulting com bined sequence ma y b e far from optimal. Therefore it could b e v ery adv an tageous to use the d a c algorithm for reordering of the unfolded rule. As the rules b ecome longer, the p oten tial b enet of ordering gro ws. The danger of high complexit y of the ordering pro cedure can b e o v ercome b y using the cost-sensitiv e v ersion of the d a c algorithm (Section 7.1). 7.3 Conclusions In this w ork w e study the problem of subgoal ordering in logic programs. W e presen t b oth a theoretical base and a practical implemen tation of the ideas, and sho w empirical results that conrm our theoretical predictions. W e com bine the ideas of Smith and Genesereth (1985), Simon and Kadane (1975) and Natara jan (1987) in to a no v el algorithm for ordering of conjunctiv e goals. The algorithm is aimed at minimizing the time whic h the logic in terpreter sp ends on the pro of of the giv en conjunctiv e goal. The main algorithm describ ed in this pap er is the d a c algorithm (Algorithm 6, Sec- tion 4.6). It w orks b y dividing the sets of subgoals in to smaller sets, pro ducing candidate sets of orderings for the smaller sets, and com bining these candidate sets to obtain orderings of the larger sets. W e pro v e that the algorithm nds a minimal ordering of the giv en set of subgoals, and w e sho w its eciency under practical assumptions. The algorithm can b e emplo y ed statically (to reorder rule b o dies in the program text b efore the execution 86 The Divide-and-Conquer Subgo al-Ordering Algorithm starts), semi-dynamically (to reorder the rule b o dy b efore the reduction is p erformed) or dynamically (to reorder the resolv en t after ev ery reduction of a subgoal b y a rule b o dy). Sev eral researc hers (Mink er, 1978; W arren, 1981; Naish, 1985a, 1985b; Nie & Plaisted, 1990) prop osed v arious heuristics for subgoal ordering. Though fast, these metho ds do not guaran tee nding minimal-cost orderings. Our algorithm pro v ably nds a minimal-cost ordering, though the ordering itself ma y tak e more time than with the heuristic metho ds. In the future it seems promising to incorp orate heuristics in to the d a c algorithm. F or example, heuristics can b e used to grade binders in OR-no des: rather than exhaustiv ely trying all subgoals as binders, w e could try just one, or sev eral binders, th us reducing the ordering time. Also, the curren t v ersion of our ordering algorithm is suitable only for nding al l solutions to a conjunctiv e goal. W e w ould lik e to extend it to the problem of nding one solution, or a xed n um b er of solutions. Another in teresting issue for further researc h is the adaptation of the d a c algorithm to in terlea ving ordering metho ds (Section 2.3). There, if subgoals of a rule b o dy are added to an ordered resolv en t, it seems w asteful to start a complete ordering pro cess; w e should use the information stored in the existing ordering of the resolv en t. P erhaps the whole divisibil it y tree of the resolv en t should b e stored, and its no des up dated when subgoals of a rule b o dy are added to the resolv en t. The ordering algorithm needs con trol kno wledge for its w ork. This con trol kno wledge is the a v erage cost and n um b er of solutions of literals, and it can b e learned b y training and collecting statistics. W e mak e an assumption that the distribution of queries receiv ed b y the system do es not c hange with time; th us, if the training set is based on the distribution seen in the past, the system learns relev an t kno wledge for future queries. W e consider the issue of learning con trol v alues more thoroughly in another pap er (Ledenio v & Mark o vitc h, 1998a), together with other issues concerning the d a c algorithm (suc h as minimizing the total time, instead of minimizing the inference time only). Ullman and V ardi (1988) sho w ed that the problem of ordering subgoals to obtain ter- mination is inheren tly exp onen tial in time. The problem w e w ork with is substan tially harder: w e m ust not only nd an order whose execution terminates in nite time, but one that terminates in minimal nite time. It is imp ossible to nd an ecien t algorithm for all cases. The d a c algorithm, ho w ev er, is ecien t in most practical cases, when the graph represen ting the subgoal dep endence (Figure 3) is sparsely connected. W e ha v e implemen ted the d a c algorithm and tested it on articial and real domains. The exp erimen ts sho w a sp eedup factor of up to 10 compared with random ordering, and up to 13 compared with some alternativ e ordering algorithms. The d a c algorithm can b e useful for man y practical applications. F ormal hardw are v erication has b ecome extremely imp ortan t in the semiconductor industry . While mo del c hec king is curren tly the most widely used tec hnique, it is generally agreed that coping with the increasing complexit y of VLSI design requires metho ds based on theorem pro ving. The main obstacle prev en ting the use of automatic theorem pro ving is its high computational demands. The d a c algorithm ma y b e used for sp eeding up logic inference, making the use of automatic theorem pro v ers more practical. Logic has gained increasing p opularit y for represen tation of common-sense kno wledge. It has sev eral adv an tages, including exibilit y and w ell-understo o d seman tics. Indeed, the CYC pro ject (Lenat, 1995) has recen tly mo v ed from frame-based represen tation to logic- 87 Ledenio v & Mark o vitch based represen tation. Ho w ev er, the large scale of suc h kno wledge bases is lik ely to presen t signican t eciency problems to the inference engines. Using automatic subgoal ordering tec hniques, suc h as those describ ed here, ma y help to solv e these problems. The issue of subgoal ordering obtains a new signicance with the dev elopmen t of Induc- tiv e Logic Programming (La vrac & Dzeroski, 1994; Muggleton & De Raedt, 1994). Systems using this approac h, suc h as F OIL (Quinlan & Cameron-Jones, 1995), try to build correct programs as fast as p ossible, without considering the eciency of the pro duced programs. Com bining the d a c algorithm with Inductiv e Logic Programming and other tec hniques for the syn thesis of logic programs (suc h as the deductiv e and the constructiv e approac hes) lo oks lik e a promising direction. App endix A. Pro of of Lemma 8 In this app endix w e presen t the pro of of a lemma whic h w as omitted from the main text of the pap er for reasons of compactness. Before w e pro v e it w e sho w t w o auxiliary lemmas. Lemma 9 L et ~ A 1 and ~ A 2 b e two or der e d se quenc es of sub go als, and B a set of sub go als. The value of cn ( ~ A 1 k ~ A 2 ) j B lies b etwe en the values cn ( ~ A 1 ) j B and cn ( ~ A 2 ) j B [ ~ A 1 . Pro of: Denote c 1 = c ost ( ~ A 1 ) j B n 1 = n so ls ( ~ A 1 ) j B cn 1 = cn ( ~ A 1 ) j B c 2 = c ost ( ~ A 2 ) j B [ ~ A 1 n 2 = n so ls ( ~ A 2 ) j B [ ~ A 1 cn 2 = cn ( ~ A 2 ) j B ; [ ~ A 1 c 1 ; 2 = c os t ( ~ A 1 k ~ A 2 ) j B n 1 ; 2 = n sols ( ~ A 1 k ~ A 2 ) j B cn 1 ; 2 = cn ( ~ A 1 k ~ A 2 ) j B cn 1 ; 2 = n 1 ; 2 1 c 1 ; 2 = n 1 n 2 1 c 1 ; 2 = ( n 1 1) + n 1 ( n 2 1) c 1 ; 2 = = c 1 n 1 1 c 1 + n 1 c 2 n 2 1 c 2 c 1 ; 2 = c 1 cn 1 + n 1 c 2 cn 2 c 1 ; 2 = c 1 c 1 ; 2 cn 1 + n 1 c 2 c 1 ; 2 cn 2 So, cn 1 ; 2 alw a ys lies b et w een cn 1 and cn 2 (b ecause c 1 c 1 ; 2 and n 1 c 2 c 1 ; 2 are p ositiv e and sum to 1). More exactly , the p oin t cn 1 ; 2 divides the segmen t [ cn 1 ; cn 2 ] with ratio ( cn 1 ; 2 cn 1 ) : ( cn 2 cn 1 ; 2 ) = n 1 c 2 : c 1 : In other w ords, cn 1 ; 2 is a weighte d aver age of cn 1 and cn 2 . Note that c 1 is the amoun t of resources sp en t in the pro of-tree of ~ B 1 , n 1 c 2 { the resources sp en t in the tree of ~ B 2 , and c 1 ; 2 is their sum. So, the more time (relativ ely) w e dedicate to the pro of of ~ B 1 , the closer cn 1 ; 2 is to cn 1 . This conclusion can b e generalized to a larger n um b er of comp onen ts in a concatenation (the pro of is b y induction): cn ( ~ A 1 k ~ A 2 k : : : ~ A k ) j B = c ost ( ~ A 1 ) j B c ost ( ~ A 1 k ~ A 2 k : : : ~ A k ) j B cn ( ~ A 1 ) j B + + n so ls ( ~ A 1 ) j B c os t ( ~ A 2 ) j B [ ~ A 1 c ost ( ~ A 1 k ~ A 2 k : : : ~ A k ) j B cn ( ~ A 2 ) j B [ ~ A 1 + : : : + + n so ls ( ~ A 1 k ~ A 2 k : : : ~ A k 1 ) j B c ost ( ~ A k ) j B [ ~ A 1 [ ::: ~ A k 1 c os t ( ~ A 1 k ~ A 2 k : : : ~ A k ) j B cn ( ~ A k ) j B [ ~ A 1 [ ::: ~ A k 1 88 The Divide-and-Conquer Subgo al-Ordering Algorithm 2 Lemma 10 L et S 0 b e a set of sub go als and N b e a no de in the divisibility tr e e of S 0 . L et ~ O N = ~ Q k ~ A 1 k ~ A 2 k ~ R b e an or dering of S ( N ) , wher e ~ A 1 and ~ A 2 ar e cn-e qual max-blo cks: cn ( ~ A 1 ) j B ( N ) [ ~ Q = cn ( ~ A 2 ) j B ( N ) [ ~ Q [ ~ A 1 . L et M b e an anc estor of N and ~ O M b e an or dering of S ( M ) c onsistent with ~ O N , wher e ~ A 1 and ~ A 2 ar e not violate d. Then either ~ A 1 and ~ A 2 ar e b oth max-blo cks in ~ O M and al l max-blo cks that stand b etwe en them ar e cn-e qual to them, or ~ A 1 and ~ A 2 b elong to the same max-blo ck in ~ O M , or ~ O M is MC-c ontr adicting. Pro of: By induction on the distance b et w een N and M . If M = N , then ~ A 1 and ~ A 2 are max-blo c ks, and the lemma holds. Let M 6 = N , and let M 0 b e the c hild of M whose descendan t is N . By inductiv e h yp othesis, the lemma holds for N and M 0 . Let ~ O 0 M b e the pro jection of ~ O M on M 0 . ~ A 1 and ~ A 2 are not violated in ~ O 0 M , since they are not violated in ~ O M . If ~ A 1 and ~ A 2 are b oth max-blo c ks in ~ O 0 M , then b y the inductiv e h yp othesis all max- blo c ks that stand b et w een them are cn-equal to them. If M is an OR-no de, no new subgoals can en ter b et w een ~ A 1 and ~ A 2 . If M is an AND-no de, the insertion of new subgoals is p ossible, but if it violates blo c ks, or places max-blo c ks not ordered b y cn , then ~ O M is MC-con tradicting, b y Corollary 3 or Lemma 6. So, if ~ O M is not MC- con tradicting, then all new max-blo c ks inserted b et w een ~ A 1 and ~ A 2 m ust b e cn-equal to them b oth. Assume that ~ A 1 and ~ A 2 are not b oth max-blo c ks in ~ O M . Without loss of generalit y , let ~ A 1 b e mem b er of a larger max-blo c k in ~ O M . W e sho w that ~ A 2 m ust also participate in the same max-blo c k. Since ~ A 1 joined a larger blo c k, there m ust exist another blo c k, ~ B , adjacen t to ~ A 1 , suc h that their pair is cn-in v erted. Let ~ B stand to the left of ~ A 1 (in the opp osite case, the pro of is similar): ~ O M = ~ X k ~ B k ~ A 1 k ~ Y k ~ A 2 k ~ Z . The pair h ~ B ; ~ A 1 i is cn-in v erted, i.e., cn ( ~ B ) j B ( M ) [ ~ X > cn ( ~ A 1 ) j B ( M ) [ ~ X [ ~ B . F rom Lemma 9, cn ( ~ B k ~ A 1 ) j B ( M ) [ ~ X > cn ( ~ A 1 ) j B ( M ) [ ~ X [ ~ B , and w e m ust add to the blo c k ~ B k ~ A 1 all blo c ks from ~ Y , b ecause they are all cn-equal to ~ A 1 . Also, cn ( ~ A 1 ) j B ( M ) [ ~ X [ ~ B = cn ( ~ A 2 ) j B ( M ) [ ~ X [ ~ B [ ~ A 1 , and ~ A 2 m ust also b e added to the blo c k. Th us, ~ A 1 and ~ A 2 b elong to the same max-blo c k in ~ O M . If ~ A 1 and ~ A 2 b elong to the same max-blo c k in ~ O 0 M , then this blo c k is either violated in ~ O M , or not. In the former case, ~ O M is MC-con tradicting, b y Corollary 3. In the latter case, ~ A 1 and ~ A 2 b elong to the same max-blo c k in ~ O M . If ~ O 0 M is MC-con tradicting, then ~ O M is MC-con tradicting to o (the pro of is easy). 2 No w w e can pro v e Lemma 8: Lemma 8 Let S 0 b e a set of subgoals, N b e a no de in the divisibili t y tree of S 0 and ~ O N = ~ Q k ~ A 1 k ~ A 2 k ~ R 89 Ledenio v & Mark o vitch b e an ordering of S ( N ), where ~ A 1 and ~ A 2 are max-blo c ks, m utually indep enden t and cn-equal under the bindings of B ( N ) [ ~ Q . Then ~ O N is blo c kwise-equiv al en t with ~ O 0 N = ~ Q k ~ A 2 k ~ A 1 k ~ R . Pro of: Let ~ S b e a minimal ordering of S 0 binder-consisten t with ~ O N . By Corollary 3, ~ S do es not violate the blo c ks of ~ O N , in particular ~ A 1 and ~ A 2 : ~ S = ~ X k ~ A 1 k ~ Y k ~ A 2 k ~ Z . Let ~ S 0 = ~ S j ~ O 0 N ~ O N = ~ X k ~ A 2 k ~ Y k ~ A 1 k ~ Z . W e m ust sho w that ~ S 0 is minimal, whic h implies blo c kwise equiv alence of ~ O N and ~ O 0 N . If ~ Y is empt y , then C ost ( ~ S ) = C ost ( ~ S 0 ) b y Lemma 2 ( ~ A 1 and ~ A 2 are adjacen t, m utually indep enden t and cn-equal; th us, their transp osition do es not c hange the cost). If ~ Y is not empt y , then b y Corollary 2 ~ Y is m utually indep enden t of b oth ~ A 1 and ~ A 2 ( ~ S is binder-consisten t with ~ O N , therefore B ( N ) ~ X , and consequen tly ~ Y \ B ( N ) = ; ). ~ Y can b e divided in to sev eral blo c ks, eac h one of them cn-equal to ~ A 1 and ~ A 2 : since ~ S is minimal, ~ O N cannot b e MC-con tradicting, and the claim follo ws from Lemma 10. By Lemma 9, cn ( ~ Y ) j ~ X = cn ( ~ A 1 ) j ~ X = cn ( ~ A 2 ) j ~ X . By Lemma 2: C ost ( ~ S ) = C ost ( ~ X k ~ A 1 k ~ Y k ~ A 2 k ~ Z ) = == sw ap ( Y ; A 2 ) = C ost ( ~ X k ~ A 1 k ~ A 2 k ~ Y k ~ Z ) = == sw ap ( A 1 ; A 2 ) = C ost ( ~ X k ~ A 2 k ~ A 1 k ~ Y k ~ Z ) = == sw ap ( A 1 ; Y ) = C ost ( ~ X k ~ A 2 k ~ Y k ~ A 1 k ~ Z ) = C ost ( ~ S 0 ) Minimalit y of ~ S 0 implies blo c kwise equiv alence of ~ O N and ~ O 0 N . 2 App endix B. Correctness of the d a c Algorithm In this section w e sho w that the d a c algorithm is correct, i.e., giv en a set of subgoals S 0 , it returns its minimal ordering. It suces to sho w that the candidate set of the ro ot no de of D T r ee ( S 0 ; ; ) is v alid. In suc h a case, as follo ws from the denition of v alid sets, it m ust con tain a minimal ordering. The algorithm returns one of the c heap est candidates of the ro ot. Therefore, if the candidate set of the ro ot is v alid, the d a c algorithm m ust return a minimal ordering of S 0 . W e start b y dening str ong validity of sets of orderings. W e then pro v e that strong v alidit y implies v alidit y . Finally , w e use induction to pro v e a theorem, sho wing that the candidate set pro duced for eac h no de in the divisibil i t y tree is strongly v alid. Denition: Let S 0 b e a set of subgoals, N b e a no de in the divisibil it y tree of S 0 . The set C N ( S ( N )) is str ongly valid , if ev ery ordering in ( S ( N )) n C N is either MC-con tradicting or blo c kwise-equiv alen t to some mem b er of C N , unless no ordering of S ( N ) is min-consisten t. S tr ong l y V al id N ; S 0 ( C N ) ( ) [ 9 ~ O 0 N 2 ( S ( N )) : M C N ; S 0 ( ~ O 0 N )] ! [ ~ O N 2 ( S ( N )) n C N ! M C C N ; S 0 ( ~ O N ) _ ( 9 ~ O 00 N 2 C N ^ M C E N ; S 0 ( ~ O N ; ~ O 00 N ))] Lemma 11 A str ongly valid set of or derings is valid. 90 The Divide-and-Conquer Subgo al-Ordering Algorithm Pro of: Let S 0 b e a set of subgoals, N b e a no de in the divisibil it y tree of S 0 , C ( N ) b e a strongly v alid set of orderings of N . If there is no min-consisten t ordering of N , then C ( N ) is v alid, b y the denition of a v alid set (Section 4.2). Otherwise, there exists at least one minimal ordering of S 0 , binder-consisten t with N . Ev ery ordering in ( S ( N )) n C ( N ) is either MC-con tradicting or blo c kwise-equiv alen t to some mem b er of C ( N ). T o pro v e that C ( N ) is v alid, w e m ust sho w that it con tains an ordering ~ O N , whic h is binder-consisten t with some minimal ordering ~ S of S 0 . Let ~ S 0 b e a minimal ordering of S 0 , binder-consisten t with N . Let ~ O 0 N b e the pro jection of ~ S 0 on N . If ~ O 0 N 2 C ( N ), w e are done ( ~ O N = ~ O 0 N , ~ S = ~ S 0 ). Otherwise, ~ O 0 N 2 ( S ( N )) n C ( N ). ~ O 0 N cannot b e MC-con tradicting (it is min-consisten t to ~ S 0 ), therefore it m ust b e blo c kwise-equiv ale n t to some ~ O 00 N 2 C ( N ). Blo c ks of ~ O 0 N are not violated in ~ S 0 , since ~ S 0 is minimal (Corollary 3). Therefore the substitution ~ S 00 = ~ S 0 j ~ O 00 N ~ O 0 N is w ell dened. ~ S 00 is minimal, since ~ S 0 is minimal and ~ O 0 N and ~ O 00 N are blo c kwise-equiv alen t. ~ S 00 is binder-consisten t with ~ O 00 N , since ~ S 0 w as binder-consisten t with ~ O 0 N . Thereup on ~ S 00 and ~ O 00 N satisfy the requiremen ts of v alidit y ( ~ O N = ~ O 00 N , ~ S = ~ S 00 ). 2 Theorem 3 L et S 0 b e a set of sub go als. F or e ach no de N of the divisibility tr e e of S 0 , A lgorithm 6 cr e ates a str ongly valid c andidate set of or derings. Pro of: By induction on the heigh t of N 's subtree. Inductiv e base: N is a leaf no de, whic h means that S ( N ) is indep enden t under B ( N ). The candidate set of N con tains one elemen t, whose subgoals are sorted b y cn . All orderings that b elong to ( S ( N )) n C andS et ( N ) are either not sorted b y cn , and hence are MC-con tradicting (Lemma 4), or are sorted b y cn , and hence are blo c kwise- equiv alen t to the candidate (Corollary 4). Consequen tly , C andS et ( N ) is strongly v alid. Inductiv e h yp othesis: F or all c hildren of N , Algorithm 6 pro duces strongly v alid candi- date sets. Inductiv e step: An in ternal no de in a divisibili t y tree is either an AND-no de or an OR- no de. 1. N is an AND-no de. Let N 1 ; N 2 ; : : : N k b e the c hildren of N . First w e sho w that C onsS et ( N ) is strongly v alid. Let ~ O N 2 ( S ( N )) n C onsS et ( N ). F or all 1 i k , let ~ O i b e the pro jection of ~ O N on N i . The set of pro jections f ~ O 1 ; ~ O 2 ; : : : ~ O k g can b elong to one of the three follo wing t yp es, with regard to ~ O N . (a) The sets of the rst t yp e con tain at least one MC-con tradicting pro jection. In suc h a case ~ O N is MC-con tradicting to o. Assume the con trary: there exists a minimal ordering ~ S of S 0 , binder-consisten t with ~ O N . Let ~ O i b e an MC- con tradicting pro jection. Since ~ O i is consisten t with ~ O N , it is also consisten t 91 Ledenio v & Mark o vitch with ~ S . Since B ( N i ) = B ( N ), all subgoals of B ( N i ) app ear in ~ S b efore subgoals of S ( N i ). Therefore, ~ O i is binder-consisten t with ~ S , and since ~ S is minimal, ~ O i is min-consisten t and not MC-con tradicting { a con tradiction. (b) The sets of the second t yp e do not con tain MC-con tradicting pro jections, but in ~ O N some blo c k of some pro jection is violated, or max-blo c ks from dieren t pro jections are not ordered b y cn . In suc h a case, ~ O N is MC-con tradicting, b y Corollary 3 and Lemma 6. (c) The sets of the third t yp e do not con tain MC-con tradicting pro jections, and max-blo c ks of the pro jections are not violated in ~ O N and are sorted b y cn . Ev ery pro jection ~ O i either b elongs to C andS et ( N i ), or not. If ~ O i 62 C andS et ( N i ), then there exists ~ O 0 i 2 C andS et ( N i ) suc h that ~ O i is blo c kwise-equiv ale n t to ~ O 0 i (b ecause C andS et ( N i ) is strongly v alid b y the inductiv e h yp othesis, and ~ O i is not MC-con tradicting). If ~ O i 2 C andS et ( N i ), w e can set ~ O 0 i = ~ O i . Let ~ O 0 N = ~ O N j ~ O 0 1 ~ O 1 j ~ O 0 2 ~ O 2 : : : j ~ O 0 k ~ O k . This substitution is w ell dened, since eac h ~ O i has the same n um b er of max-blo c ks as ~ O 0 i , and max-blo c ks of the pro jections are not violated in ~ O N . Let ~ S b e a minimal ordering of S 0 , binder-consisten t with ~ O N . Since ~ S is minimal, blo c ks of ~ O 1 are not violated in ~ S . Since ~ O 1 is blo c kwise-equiv al en t to ~ O 0 1 , the ordering ~ S 1 = ~ S j ~ O 0 1 ~ O 1 is w ell-dened and minimal. In ~ S 1 the p ositions of the subgoals from B ( N ) did not c hange; th us, ~ O 2 is min-consisten t with ~ S 1 , and blo c kwise equiv alence of ~ O 2 and ~ O 0 2 en tails minimalit y of the ordering ~ S 2 = ~ S 1 j ~ O 0 2 ~ O 2 = ~ S j ~ O 0 1 ~ O 1 j ~ O 0 2 ~ O 2 . W e con tin ue with other ~ O i -s, and nally obtain that ~ S 0 = ~ S j ~ O 0 1 ~ O 1 j ~ O 0 2 ~ O 2 : : : j ~ O 0 k ~ O k is minimal. F rom the denition of ~ O 0 N , ~ S 0 = ~ S j ~ O 0 N ~ O N (note that w e in tro duced blo c kwise equiv alence and strong v alidit y only to b e able to p erform this transition). ~ S 0 is minimal, therefore ~ O N is blo c kwise-equiv al en t to ~ O 0 N . ~ O 0 N 2 C onsS et ( N ), since all its pro jections are candidates of the c hild no des. Thereup on, ~ O N is blo c kwise- equiv alen t to a mem b er of C onsS et ( N ). So, C onsS et ( N ) is strongly v alid. T o pro v e that C andS et ( N ) is strongly v alid, it suces to sho w that all the mem b ers of C onsS et ( N ) that are not included in C andS et ( N ) b y Algorithm 6, are either MC-con tradicting or blo c kwise-equiv ale n t to mem b ers of C andS et ( N ). Suc h orderings can b e of three t yp es: (a) Orderings that violate blo c ks of the c hildren pro jections. They are MC- con tradicting b y Corollary 3. (b) Orderings that do not violate blo c ks, but where max-blo c ks of c hildren pro- jections are not ordered b y cn . They are MC-con tradicting b y Lemma 6. (c) Orderings that do not violate blo c ks and ha v e them sorted b y cn . F or eac h com bination of pro jections, one consisten t ordering of N is retained in the candidate set, and all the other are rejected. By Corollary 5, the rejected orderings are blo c kwise-equiv ale n t to the retained candidate. Consequen tly , C andS et ( N ) is strongly v alid. 92 The Divide-and-Conquer Subgo al-Ordering Algorithm 2. N is an OR-no de. Again, w e start with sho wing that C onsS et ( N ) is strongly v alid. Let ~ O N 2 ( S ( N )) n C onsS et ( N ) . ~ O N is constructed from a binder H and a \tail" sequence ~ T : ~ O N = H k ~ T . Let N H b e the c hild of N that corresp onds to the binder H . By the inductiv e h yp othesis, C andS et ( N H ) is strongly v alid. ~ T 62 C andS et ( N H ), since otherwise ~ O N 2 C onsS et ( N ). Therefore, ~ T is ei- ther MC-con tradicting, or blo c kwise-equiv ale n t to some ~ T 0 2 C andS et ( N H ). If ~ T is MC-con tradicting, ~ O N is MC-con tradicting to o (pro of b y con tradic- tion, as for AND-no des). If ~ T is blo c kwise-equiv al en t to ~ T 0 , then ~ O N = H k ~ T is blo c kwise-equiv al en t to H k ~ T 0 2 C onsS et ( N ) (the pro of is easy). Hence, C onsS et ( N ) is strongly v alid. The only orderings of C onsS et ( N ) that are not in- cluded in C andS et ( N ) b y the d a c algorithm ha v e c heap er p erm utations of their leading max-blo c ks, and therefore are MC-con tradicting, b y Lemma 7. Hence, C andS et ( N ) is strongly v alid. 2 Corollary 7 The c andidate set found by A lgorithm 6 for the r o ot no de is valid. Corollary 8 A lgorithm 6 nds a minimal or dering of the given set of sub go als. References Aho, A. V., Hop croft, J. E., & Ullman, J. D. (1987). Data Structur es and A lgorithms . Addison-W esley . Bo ddy , M., & Dean, T. (1989). Solving time-dep enden t planning problems. In Sridha- ran, N. S. (Ed.), Pr o c e e dings of the 11th International Joint Confer enc e on A rticial Intel ligenc e , pp. 979{984, Detroit, MI, USA. Morgan Kaufmann. Bol, R. N., Apt, K. R., & Klop, J. W. (1991). An analysis of lo op c hec king mec hanisms for logic programs. The or etic al Computer Scienc e , 86 (1), 35{79. Braem, C., Le Charlier, B., Mo dar, S., & V an Hen tenryc k, P . (1994). Cardinalit y Analysis of Prolog. In Bruyno oghe, M. (Ed.), L o gic Pr o gr amming - Pr o c e e dings of the 1994 International Symp osium , pp. 457{471, Massac h usetts Institute of T ec hnology . The MIT Press. Breiman, L., F riedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classic ation and R e gr ession T r e es . W adsw orth In ternational Group, Belmon t, CA. Bruyno oghe, M., De Sc hrey e, D., & Krek els, B. (1989). Compiling con trol. The Journal of L o gic Pr o gr amming , 6 , 135{162. Clark, K. L., & McCab e, F. (1979). The con trol facilities of IC-Prolog. In Mic hie, D. (Ed.), Exp ert Systems in The Micr o ele ctr onic A ge. , pp. 122{149. Univ ersit y of Edin burgh, Scotland. Clo c ksin, W. F., & Mellish, C. S. (1987). Pr o gr amming in Pr olo g (Third edition). Springer- V erlag, New Y ork. 93 Ledenio v & Mark o vitch Cohen, W. W. (1990). Learning appro ximate con trol rules of high utilit y . In Pr o c e e dings of the Seventh International Machine L e arning Workshop , pp. 268{276, Austin, T exas. Morgan Kaufmann. Cormen, T. H., Leiserson, C. E., & Riv est, R. L. (1991). Intr o duction T o A lgorithms . MIT Press, Cam bridge, Mass. Cortesi, A., Le Charlier, B., & Rossi, S. (1997). Sp ecication-based automatic v erication of Prolog programs. In Gallagher, J. (Ed.), Pr o c e e dings of the 6th International Work- shop on L o gic Pr o gr am Synthesis and T r ansformation , V ol. 1207 of LNCS , pp. 38{57, Sto c kholm, Sw eden. Springer-V erlag. De Bo ec k, P ., & Le Charlier, B. (1990). Static t yp e analysis of Prolog pro cedures for ensuring correctness. In Deransart, P ., & Ma luszy nski, J. (Eds.), Pr o gr amming L anguages Implementation and L o gic Pr o gr amming , V ol. 456 of LNCS , pp. 222{237, Link oping, Sw eden. Springer-V erlag. De Sc hrey e, D., & Decorte, S. (1994). T ermination of logic programs: The nev er-ending story . The Journal of L o gic Pr o gr amming , 19 & 20 , 199{260. Debra y , S., L op ez-Garc a, P ., Hermenegildo, M., & Lin, N.-W. (1997). Lo w er b ound cost estimation for logic programs. In Ma luszy nski, J. (Ed.), Pr o c e e dings of the Interna- tional Symp osium on L o gic Pr o gr amming (ILPS-97) , pp. 291{306, Cam bridge. MIT Press. Debra y , S. K., & Lin, N.-W. (1993). Cost analysis of logic programs. A CM T r ansactions on Pr o gr amming L anguages and Systems , 15 (5), 826{875. Debra y , S. K., & W arren, D. S. (1988). Automatic mo de inference for logic programs. The Journal of L o gic Pr o gr amming , 5 , 207{229. Dejong, G., & Mo oney , R. (1986). Explanation-based learning: An alternativ e view. Ma- chine L e arning , 1 , 145{176. Deville, Y. (1990). L o gic Pr o gr amming: Systematic Pr o gr am Development . In ternational Series in Logic Programming, Addison-W esley . Etzioni, O. (1991). ST A TIC: A problem-space compiler for PR ODIGY. In Dean, Thomas L.; McKeo wn, K. (Ed.), Pr o c e e dings of the 9th National Confer enc e on A rticial Intel ligenc e , pp. 533{540, Anaheim, California. MIT Press. Etzioni, O. (1993). Acquiring searc h-con trol kno wledge via static analysis. A rticial Intel- ligenc e , 62 , 255{301. Greiner, R., & Orp onen, P . (1996). Probably appro ximately optimal satiscing strategies. A rticial Intel ligenc e , 82 (1-2), 21{44. Henrard, J., & Le Charlier, B. (1992). F OLON: An en vironmen t for declarativ e construction of logic programs. In Bruyno oghe, M., & Wirsing, M. (Eds.), Pr o c e e dings of the F ourth International Symp osium on Pr o gr amming L anguage Implementation and L o gic Pr o- gr amming , V ol. 631 of LNCS , pp. 217{231, Leuv en, Belgium. Springer-V erlag. 94 The Divide-and-Conquer Subgo al-Ordering Algorithm Itai, A., & Mak o wsky , J. A. (1987). Unication as a complexit y measure for logic program- ming. The Journal of L o gic Pr o gr amming , 4 , 105{117. Kn uth, D. E. (1973). The A rt Of Computer Pr o gr amming , V ol. 3. Addison-W esley , Reading, Mass. Ko w alski, R. A. (1979). Algorithm = Logic + Con trol. Communic ations of the A CM , 22(7) , 424{436. Laird, P . D. (1992). Ecien t dynamic optimization of logic programs. In Pr o c e e dings of the ML92 Workshop on Know le dge Compilation and Sp e e dup L e arning Ab erdeen, Scotland. Langley , P . (1985). Learning to searc h: F rom w eak metho ds to domain-sp ecic heuristics. Co gnitive Scienc e , 9 , 217{260. La vrac, N., & Dzeroski, S. (1994). Inductive L o gic Pr o gr amming: T e chniques and Applic a- tions . Articial In telligence. Ellis Harw o o d, New Y ork. Ledenio v, O., & Mark o vitc h, S. (1998a). Con trolled utilization of con trol kno wledge for sp eeding up logic inference. T ec h. rep. CIS9812, T ec hnion, Haifa, Israel. Ledenio v, O., & Mark o vitc h, S. (1998b). Learning in v estmen t functions for con trolling the utilit y of con trol kno wledge. In Pr o c e e dings of the Fifte enth National Confer enc e on A rticial Intel ligenc e , pp. 463{468, Madison, Wisconsin. Morgan Kaufmann. Lenat, D. B. (1995). CYC: A large-scale in v estmen t in kno wledge infrastructure. Commu- nic ations of the A CM , 38 (11), 33{38. Llo yd, J. W. (1987). F oundations of L o gic Pr o gr amming (Second edition). Springer-V erlag, Berlin. Mark o vitc h, S., & Scott, P . D. (1989). Automatic ordering of subgoals | a mac hine learning approac h. In Lusk, E. L., & Ov erb eek, R. A. (Eds.), Pr o c e e dings of the North A meric an Confer enc e on L o gic Pr o gr amming , pp. 224{242, Clev eland, Ohio. MIT Press. Mark o vitc h, S. (1989). Information Filtering: Sele ction Me chanisms in L e arning Systems . Ph.D. thesis, EECS Departmen t, Univ ersit y of Mic higan. Mark o vitc h, S., & Scott, P . D. (1993). Information ltering: Selection mec hanisms in learning systems. Machine L e arning , 10 , 113{151. Mink er, J. (1978). Searc h strategy and selection function for an inferen tial relational system. In A CM T r ansactions on Datab ase Systems , V ol. 3, pp. 1{31. Min ton, S. (1988). L e arning Se ar ch Contr ol Know le dge: A n Explanation-Base d Appr o ach . Klu w er, Boston, MA. Mitc hell, T. M., Keller, R. M., & Kedar-Cab elli, S. T. (1986). Explanation-based general- ization: A unifying view. Machine L e arning , 1 , 47{80. 95 Ledenio v & Mark o vitch Mo oney , R. J., & Zelle, J. M. (1993). Com bining F OIL and EBG to sp eed-up logic programs. In Ba jcsy , R. (Ed.), Pr o c e e dings of The Thirte enth International Joint Confer enc e for A rticial Intel ligenc e , pp. 1106{1111, Cham b ery , F rance. Morgan Kaufmann. Morris, K. A. (1988). An algorithm for ordering subgoals in NAIL!. In Pr o c e e dings of the Seventh A CM SIGA CT-SIGMOD Symp osium on Principles of Datab ase Systems , pp. 82{88, Austin, TX. A CM Press, New Y ork. Muggleton, S., & De Raedt, L. (1994). Inductiv e logic programming: Theory and metho ds. The Journal of L o gic Pr o gr amming , 19 & 20 , 629{680. Naish, L. (1984). MU-Pr olo g 3.1db R efer enc e Manual . Dept. of Computer Science, Univ. of Melb ourne. Naish, L. (1985a). Automatic con trol for logic programs. The Journal of L o gic Pr o gr amming , 3 , 167{183. Naish, L. (1985b). Prolog con trol rules. In Joshi, A. (Ed.), Pr o c e e dings of the 9th Inter- national Joint Confer enc e on A rticial Intel ligenc e , pp. 720{723, Los Angeles, CA. Morgan Kaufmann. Natara jan, K. S. (1987). Optimizing bac ktrac k searc h for all solutions to conjunctiv e prob- lems. In McDermott, J. (Ed.), Pr o c e e dings of the 10th International Joint Confer enc e on A rticial Intel ligenc e , pp. 955{958, Milan, Italy . Morgan Kaufmann. Nie, X., & Plaisted, D. A. (1990). Exp erimen tal results on subgoal ordering. In IEEE T r ansactions On Computers , V ol. 39, pp. 845{848. P ettorossi, A., & Proietti, M. (1994). T ransformation of logic programs: F oundations and tec hniques. The Journal of L o gic Pr o gr amming , 19 & 20 , 261{320. P ettorossi, A., & Proietti, M. (1996). Rules and strategies for transforming functional and logic programs. A CM Computing Surveys , 28 (2), 360{414. P orto, A. (1984). Epilog: A language for extended programming. In Campb ell, J. (Ed.), Implementations of Pr olo g . Ellis Harw o o d. Prieditis, A. E., & Mosto w, J. (1987). PR OLEARN: T o w ards a prolog in terpreter that learns. In F orbus, Kenneth; Shrob e, H. (Ed.), Pr o c e e dings of the 6th National Con- fer enc e on A rticial Intel ligenc e , pp. 494{498, Seattle, W A. Morgan Kaufmann. Quinlan, J. R. (1986). Induction of decision trees. Machine L e arning , 1 , 81{106. Quinlan, J. R., & Cameron-Jones, R. M. (1995). Induction of logic programs: F OIL and related systems. New Gener ation Computing, Sp e cial Issue on Inductive L o gic Pr o- gr amming , 13 (3-4), 287{312. Simon, H. A., & Kadane, J. B. (1975). Optimal problem-solving searc h: All-or-none solu- tions. A rticial Intel ligenc e , 6 , 235{247. Smith, D. E. (1989). Con trolling bac kw ard inference. A rticial Intel ligenc e , 39 (1), 145{208. 96 The Divide-and-Conquer Subgo al-Ordering Algorithm Smith, D. E., & Genesereth, M. R. (1985). Ordering conjunctiv e queries. A rticial Intel li- genc e , 26 , 171{215. Smith, D. E., Genesereth, M. R., & Ginsb erg, M. L. (1986). Con trolling recursiv e inference. A rticial Intel ligenc e , 30 (3), 343{389. Somogyi, Z., Henderson, F., Con w a y , T., Bromage, A., Do wd, T., Jeery , D., & al. (1996a). Status of the Mercury system. In Pr o c. of the JICSLP '96 Workshop on Par al lelism and Implementation T e chnolo gy for (Constr aint) L o gic Pr o gr amming L anguages , pp. 207{218, Bonn, German y . Somogyi, Z., Henderson, F., & Con w a y , T. (1996b). The execution algorithm of Mercury , an ecien t purely declarativ e logic programming language. Journal of L o gic Pr o gr am- ming , 29 (1{3), 17{64. Sterling, L., & Shapiro, E. (1994). The A rt of Pr olo g (Second edition). MIT Press, Cam- bridge, MA. T amaki, H., & Sato, T. (1984). Unfold/fold transformation of logic programs. In T arnlund, S.- A. (Ed.), Pr o c e e dings of the Se c ond International Confer enc e on L o gic Pr o gr am- ming , pp. 127{138, Uppsala, Sw eden. Ullman, J. D., & V ardi, M. Y. (1988). The complexit y of ordering subgoals. In Pr o c e e dings of the Seventh A CM SIGA CT-SIGMOD Symp osium on Principles of Datab ase Systems , pp. 74{81, Austin, TX. A CM Press, New Y ork. Ullman, J. D. (1982). Principles of Datab ase Systems . Computer Science Press, Ro c kville, MD. V asak, T., & P otter, J. (1985). Metalogical con trol for logic programs. Journal of L o gic Pr o gr amming , 2 (3), 203{220. W arren, D. H. D. (1981). Ecien t pro cessing of in teractiv e relational database queries expressed in logic. In Zaniola, & Delob el (Eds.), Pr o c e e dings of the 7th International Confer enc e on V ery L ar ge Data Bases , pp. 272{281, Cannes, F rance. IEEE Computer So ciet y Press. 97
Original Paper
Loading high-quality paper...
Comments & Academic Discussion
Loading comments...
Leave a Comment