Breaking Instance-Independent Symmetries In Exact Graph Coloring

Journal of Artiﬁcial In telligence Research 26 ( 2006) 247-280 Submitted 10/04; published 07/06 Breaking Instance-Indep enden t Symmetries in Exact Graph Coloring Arathi Ramani ramania@umich.edu Igor L. Mark o v imarko v@eecs.umich.edu Karem A. Sak allah karem@eecs.umich.edu Dep artment of Ele ctric al Engine ering and Computer Scienc e University of Michi gan, Ann Arb or, USA F adi A. Aloul f aloul@umich.edu Dep artment of Computer Engine ering Americ an Un iversity in Sharjah, UAE Abstract Co de optimization and hig h level synt hesis can b e po sed as c onstraint satisfaction and optimization problems, suc h as graph coloring used in register allo cation. Graph color ing is also used to mo del mo re traditional CSPs r e lev a nt to AI, such as planning, time-tabling a nd scheduling. Prov ably optimal solutions may b e desira ble for commercial and defense a p- plications. Additionally , for applications such as register allo cation and co de optimization, naturally-o c c urring instances of graph colo ring a r e o ften small a nd ca n b e so lved optimally . A recent w av e o f improvemen ts in alg orithms for Bo olea n satisﬁability (SA T) and 0-1 In- teger Linear Prog ramming (ILP) sugges ts generic problem- r eduction methods , rather than problem-sp eciﬁc heuristics, bec ause (1) heuristics ma y b e upset b y new constr a ints, (2) heuristics tend to ignor e structure, and (3) many relev a nt problems ar e prov ably inapprox- imable. Problem r eductions often lead to highly s y mmetric SA T instances, a nd symmetries are known to slo w down SA T solvers. In this work, w e co mpare several av enues for symme- try breaking, in particular when certain kinds of symmetry are prese nt in all generated instances. Our fo c us on reducing CSPs to SA T allows us to leverage r ecent dramatic improv ement in SA T solvers and automa tica lly b eneﬁt fro m future pr ogress . W e ca n use a v ariety of bla ck-box SA T solv ers without mo difying their source co de because our symmetry-brea king techniques are static, i.e., we detect symmetr ies and add symmetry breaking predica tes (SBPs) during pre-pro ces s ing. An impo r tant result of our work is that among the types of instance-indep endent SBPs we s tudied and their combinations, the simplest and least complete constructio ns are the most eﬀective. Our exp eriments a lso clearly indicate that instance-indep endent symmetries should mostly be pro cessed together with instance-sp eciﬁc symmetries ra ther than at the sp eciﬁcation level, contrary to what has bee n sugg ested in the literature. 1. In tro duction Detecti ng and using pr ob lem structur e, such as symmetries, can often b e v ery useful in accele rating the searc h for solutions of constrain t s atisfactio n pr oblems (CSPs). This is particularly true for algorithms whic h p erf orm exhaustive searc hes and b eneﬁt from prun - ing the searc h tree. This work condu cts a theoretical and emp irical study of the imp act of breaking stru ctural symmetries in 0- 1 IL P reductions of the exact graph coloring problem c  2006 AI Access F oundation. All rights reserved. Ramani, Aloul, Mark ov, & Sakal lah whic h has applications in a num b er of ﬁelds. F or example, in compiler design, many tech- niques for co de optimization and high-lev el synt hesis op erate with relativ ely f ew ob j ects at a time. Graph colo ring used for register allocation d uring program compilati on (Ch aitin, Auslander, Ch andra, Co c k e, Hopkins, & Markstein, 1981) is limite d b y small n umb ers of registers in embed d ed p ro cessors as well as by the num b er of lo cal v ariables and virtual registers. Graph co loring is also r elev ant to AI applications suc h as p lanning, sc hedulin g, and map coloring. Rec ent w ork on graph coloring in AI h as included algorithms based on neural n et w orks (Jagota, 1996), evolutio nary algorithms (Galinier & Hao, 1999) , scatter searc h (J.-P . Hamiez, 2001) and sev eral other appr oac hes d iscussed in Section 2. While man y of these searc h pro cedures are heur istic, our w ork fo cus es on exact graph coloring, whic h is closely related to seve ral usefu l com binatorial p roblems suc h as maximal inde- p endent set and vertex c over . W e seek pr o v ably optimal solutions b ecause they ma y b e desirable in commercial and d efense applications for comp etitiv e reasons, and can often b e found. Our wo rk f o cuses on solving exact graph coloring by reduction to 0-1 ILP . While the idea of solving N P − complete problems by reduction is w ell-kno wn, it is rarely used in practice b ecause algorithms dev elop ed for “ standard ” problems, su c h as SA T, ma y n ot b e comp etitiv e with domain-sp eciﬁc tec hniques that are a w are of pr oblem s tructure. How ev er, man y applications imply problem-sp eciﬁc constraints and n on-trivial ob jectiv e f unctions. These extensions ma y upset heuristics for standard problems. Heuristics, p articularly those based on lo cal searc h, often fail to use s tr ucture in pr ob lem in stances (Pr estwic h, 2002) and are ineﬃcien t when used with pr ob lem reductions. In con trast, exact solv ers based on branc h-and-b ound and bac k-trac king tend to adapt to new constraints and can b e applied through pr ob lem r eduction. There is a gro wing literature on handling structure in optimal solv ers (Aloul, Ramani, Mark o v, & S ak allah, 2003; Cra wford , Ginsb er g, Luks, & Ro y , 1996; Huang & Darwic he, 2003) , a nd our w ork falls in to this category as w ell. The NP-sp ec pr o ject (Cadoli, Pa lop oli, S c haerf, & V asileet, 1999) oﬀers a f ramew ork for formulat ing a wide range of combinato rial p roblems and automatical ly reducing their instances to instances of Boolean satisﬁabilit y . T his app r oac h is attractiv e b ecause it circum - v en ts problem-sp eciﬁc solve rs and lev erage s recent br eakthroughs in Bo olean satisﬁabilit y (Mosk ewicz, Madigan, Zhao, Z hang, & Malik, 2001). Ho w ev er, this appr oac h remains unex- plored in practice, p ossibly b ecause the eﬃciency of problem-solving ma y b e reduced when domain-sp eciﬁc s tr ucture is lost durin g problem r eductions. This dr awbac k is add r essed b y recen t w ork on the detection of s tr ucture, particularly symmetry , in SA T and 0-1 ILP instances in order to accelerate exact solv ers (Crawford et al., 1996; Aloul et al., 2003; Aloul, Ramani, Mark o v, & Sak alla h, 20 04). In these pap er s , symm etries in a SA T /0-1 ILP in s tance are d etected by reduction to gr aph automorphism , i.e. the formula is rep- resen ted by a grap h an d th e automorphism problem for that graph is solv ed using graph automorphism soft w are pac k age s (McKa y , 1990; Darga, Liﬃton, Sak allah, & Mark o v, 2004). Un til recen tly , this t yp e of symmetry d etectio n w as frequentl y ineﬃcien t b ecause solving the automorphism problem f or large graphs can b e time-consuming. Ho w ev er, more r e- cen t au tomorp h ism soft wa re (Darga et al., 2004) h as remov ed th is b ottlenec k to a large exten t. Moreo v er, adding simple s y m metry br eaking predicates as new constrain ts sig niﬁ- can tly sp eeds up exact SA T s olv ers (Aloul et al., 2003). This w ork can b e viewed as a case study of sym metry breaking in problem reductions, as we focus on gr aph c oloring and it s v ariants th at can b e redu ced to Bo olean satisﬁabilit y and 0-1 ILP . Our main goals are to (i) 248 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring accele rate optimal solving of graph coloring instances, and (ii) compare diﬀeren t strategies for b reaking instance-indep endent sym metries. T here are t w o distinct sources of symmetries in graph -coloring instances: (i) colors can b e arbitrarily p ermuted (instance-indep end ent symmetries), and (ii) some graphs may b e in v arian t un der certain p er mutations of v ertices (instance-dep endent symmetries). Pr evious w ork (Crawford et al., 1996; Aloul et al., 2003, 2004) deals on ly with instance-dep end en t sym m etries in SA T and 0-1 ILP instances. Sy m - metries are ﬁrst detected by redu ction to grap h automorphism and th en br ok en by adding symmetry b reaking predicates (SBPs) to the form ulation. The adv ant age of such a strat- egy is that ev ery instance-indep endent s y m metry is also instance-dep endent, whereas the rev erse does not hold. Symmetries that exist due to problem formula tion app ear in ev ery instance of the problem, in add ition to s y m metries that exist d ue to sp eciﬁc parameter v alues for an ins tance. Giv en that there may b e many instance-sp eciﬁc sy m metries, one ma y pro cess all symmetries at once u sing pu blicly a v aila ble symm etry pr o cessing pac k age s suc h as Sh a tter (Aloul et al., 2003; Aloul, Mark o v, & Sak allah, 2003). Alternativ ely , one ma y add symm etry br eaking predicates for in stance-indep endent s ymmetries early , hoping to sp eed-u p th e p ro cessing of r emaining symmetries. This type of symmetry breaking has not b een discus sed in earlier w ork (Aloul et al., 2003, 2003), and in this pap er we study its utilit y for the graph coloring problem. Our w ork deals with symmetries of problem and instance descriptions; we distinguish (i) symmetries of generic problem sp eciﬁcations from (ii) symmetries of problem-instance data. The former s ymmetries translate to the latter bu t n ot the other wa y around — an example from graph coloring is give n by color p erm utations versus automorphism s of sp eciﬁc graphs. While b oth t yp es of symmetries can b e detected by solving th e graph au to- morphism p roblem, symmetries in sp eciﬁcations can often b e captured man ually , whereas capturing symmetries in problem instances ma y require la rge-scale computation and non- trivial soft w are. Indeed, when sp eciﬁcation-lev el sym m etries are in stan tiated, the size of their sup p ort (the num b er of ob jects mo v ed) t ypically increases dramatically . F or example, color p erm utations in graph coloring should b e sim ultaneously applied to ev ery v ertex of a graph in question. Detecting symmetries w ith larger supp ort seems lik e a w aste of compu - tational eﬀo rt. T o this end, recen t wo rk on breaking s ymmetries in sp eciﬁcations (Cadol i & Mancini, 2003 ) pr efers instance-indep enden t tec h niques and breaks symmetries only at the sp eciﬁcation lev el. This appr oac h is p articularly relev an t with constrain t solv ers and languages that pro cess p roblem sp eciﬁcations p rior to s eeing actual problem ins tances and can amortize the symm etry-d etectio n eﬀort. Also, in a more general setting, u sing instance- indep end en t symmetry breaking do es not ru le out applyin g redun dan t (or complementa ry) instance-sp eciﬁc tec hniques at a later stage. Un til recen tly automatic symmetry detection had b een a serious b ottlenec k in handling symmetries. F or example, if graph automorphism is solv ed using the pr ogram Nauty (McKa y , 1990), detecting sym metries often can take longer th an constrain t solving without symmetry breaking. This w as observed for micropro cessor veriﬁcatio n SA T instances b y Aloul et. al. in 2002 (Aloul et al., 2003 ). Therefore, detec ting symmetries early and represent ing them in a more structur ed w a y app ears at tractiv e, esp ecially giv en that this ma y p oten tially in crease the eﬃciency of symmetry-b reaking. Ho w ev er, the symmetry- detection b ottlenec k has recen tly b een eliminated in man y applications with the softw are to ol Saucy (Darga et al., 2004) that often ﬁnds symmetries of practical graphs many times 249 Ramani, Aloul, Mark ov, & Sakal lah faster than Nauty . Th is dev elopmen t und er m ines, to some exten t, the p oten tial b eneﬁts of symmetry pro cessing at the sp eciﬁcation lev el and puts th e sp otligh t on symmetry-br eaking. T o that end, SBPs added in diﬀerent circumstances ma y h a v e d iﬀeren t eﬃciency , and while it is unclear a priori whic h approac h is more successful, the d iﬀerences in p erform ance ma y b e signiﬁ can t. Since S BPs app ear to the solve r as add itional constraint s, they ma y either sp eed up or frustrate the solv er (the latter eﬀect is clearly visible in our exp eriments with CPLEX). Outcomes of practical exp eriment s are also aﬀected b y recent dramatic impro ve ments in the eﬃciency of symmetry-br eaking p r edicates (Aloul et al., 2003, 2004). While it seems diﬃcult to justify an y particular exp ectatio n for emp irical p erformance, w e are f ortu nate to observ e clear tr ends in exp erimenta l data presen ted in Section 4 and summarize them with simple ru les. While w e fo cus on graph coloring instances, our tec hniques are immediately applicable to related CSP pr oblems, e.g., those p ro duced b y ad d ing n ew t yp es of constrain ts that can b e easily exp ressed in S A T or 0-1 ILP when graph coloring is con v erted to those generic problems. W e also exp ect that our conclusions ab out symmetry-breaking carry ov er to other CSPs that can b e economically reduced to SA T and 0- 1 ILP , e.g ., m axim um indep endent set, min im um dominating set, etc. Another adv an tage of our approac h is b eing able to use a v ariety of existing and future SA T and 0-1 ILP solv ers without mo d ifying their source co d e. Unfortunately , this p recludes the use of dynamic symm etry-b reaking th at would requir e mo difying the source co de and ma y adve rsely aﬀec t p erformance by disturbing the fragile balance b et w een the amount of reasoning and searching p erform ed b y mo d ern SA T solv ers. Sp eciﬁcally , heuristics for v ariable ordering and decision selectio n ma y b e aﬀected, as well as th e recording of learned conﬂict clauses (nogo o ds). The main con tributions of this wo rk are listed b elow. • Using the sym metry br eaking ﬂo w for p seudo-Bo olean (PB) formulas d escrib ed by Aloul et. al in 2004 (Aloul et al., 2004), w e detect and b reak sym metries in DIMACS graph coloring b enc hmarks exp ressed as instances of 0-1 ILP . W e sho w that instance- dep end en t symmetry breaking enables man y m ed ium-sized instances to b e optimally solv ed in r easonable time on commo dity PCs • W e p rop ose in stance-indep endent tec hniques for b reaking sy m metries d uring prob- lem form ulation, assess their relati ve s trength and completeness, and ev aluate them empirically using well-kno w n academic and commercial tools • W e sho w empirically that instance-depen d en t techniques are, in general, more eﬀectiv e than instance-indep end en t symmetry breaking for the b enc hmarks in question. In fact, only the simp lest and least complex instance-indep endent SBPs are comp etitiv e The r emaining p art of the pap er is organized as follo ws. Section 2 co v ers backg roun d on graph coloring, SA T and 0-1 ILP , as wel l as pr evious w ork on symmetry b reaking. Instance- indep end en t symmetry b reaking predicates are discussed in S ection 3. S ection 4 presen ts our emp irical r esults and Section 5 concludes th e pap er. The App endix giv es detailed results for the queens family of instances. 250 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring 2. Bac kground and Previous W ork This sectio n d iscusses p roblem deﬁnitions and applications of some existing algorithms for exact graph coloring. W e also d iscuss previous w ork on s ymmetry breaking for SA T and 0-1 I LP in some detail. 2.1 Graph Coloring Giv en an undirected graph G ( V , E ), a vertex coloring of the graph is an assignmen t of a lab el (color) to eac h no de su c h th at th e lab els on ad j acen t no des are diﬀeren t. A minimum coloring uses the smallest p ossible num b er of colors, known as the chr omatic numb er of a graph. The de cisi on version of graph coloring ( K − coloring) asks whether ve rtices in a graph can b e colored us in g ≤ K colors for a giv en K . A clique of an u ndirected graph G ( V , E ) is a set of m utually adj acen t v ertices in the graph. The maxim um clique problem consists of seeking a clique of maximal size, i.e., a clique w ith at least as many v ertices as an y other clique in the graph. T he maximum clique and graph coloring problems are closely related. Sp eciﬁcally , the max-clique size is a lo we r b ound on the c hromatic num b er of the graph. Over the y ears, a num b er of diﬀeren t algorithms for solving graph coloring ha v e b een d ev elop ed, b ecause of its f unda- men tal imp ortance in compu ter science. T hese algo rithms fall in to three broad categ ories: p olynomial-time app r o ximation schemes, optimal algorithms, and heuristics. W e brieﬂy discuss work in eac h of these categories b elo w. Th ere are a num b er of online resources on graph coloring (T r ic k, 1996; Cu lb erson, 2004) that oﬀer more detailed bibliographies. As far as app ro ximation s chemes are concerned, the most common tec hnique used is suc c essive augmentation . In this appr oac h a p artial coloring is found on a small n umb er of v ertices and this is extended v ertex b y v ertex until the en tire graph is colored. Examp les include the algorithms b y Leighto n (Leigh ton, 1979) for large sc heduling problems, and b y W elsh and P o wel l (W elsh & P o wel l, 1967) for time-tabling. More recen t work h as atte mpted to tighte n the wo rst-case b ounds on the c hromatic num b er of the graph . Th e algorithm pro viding the curr en tly b est w orst-case ratio (n umb er of colors used divided by optimal n umb er) is due to Haldorsson (Haldorsson, 19 90), and guaran tees a r atio of no more than O  n (log log n ) 2 (log n ) 3  , w here n is the num b er of ve rtices. General h euristic metho ds that ha v e b een tried include sim ulated annealing (Chams, Hertz, & W erra, 1987; Aragon, Joh n son, McGeoc h, & Schev on, 199 1) and tabu searc h (Hertz & W erra, 1987). A w ell-kno wn heuristic that is still widely used is the DSA TUR algorithm by Brelaz (Brela z, 1979) which colors v ertices according to their satur ation de gr e e . The saturation degree of a v ertex is the num b er of diﬀerent colors to which it is adjacen t. Th e DSA TUR heuristic rep eatedly pic ks a vertex with maximal saturation degree and colors it with the lo w est-n umb ered color p ossible. This heuristic is optimal for bipartite graphs. Algorithms for ﬁndin g optimal colorings are frequently based on implicit enumeratio n, and are d iscussed in more detail later in this s ection. Both the graph coloring and max-clique p r oblems are N P -complete (Garey & Johnson, 1979 ) and even ﬁndin g n ear-optimal solutions with go o d appro ximation guarantees is N P -hard (F eige, Goldwa sser, Lo v asz, Safra, & Szege, 1991). The in appro ximabilit y of graph coloring suggests that it ma y b e more diﬃcult to solv e heuristically than, say , the T rav eling Salesman Problem for whic h Pol ynomial-Time Approxi mation Sc hemes (PT AS) 251 Ramani, Aloul, Mark ov, & Sakal lah are known for Eu clidean and Manhattan graphs. F or this and a n umber of other reasons, w e stud y optimal graph coloring and many app lication-deriv ed ins tances that are solv able in reasonable time. S everal applications are outlined next. Time-T abling and S cheduling problems inv olve placing pairwise restrictions on jobs that cannot b e p erformed simulta neously . F or example, t wo classes taught b y the same fac- ult y mem b er cannot b e scheduled in the same time slot. The problem has b een studied in previous w ork b y Leigh ton (Leigh ton, 1979) and De W erra (W erra, 1985). More generally , graph coloring is an imp ortant problem in Artiﬁcial In telligence b ecause of its close rela- tionship to p lanning and s cheduling. Seve ral traditional AI tec hniques hav e b een applied to this p r oblem, including parallel algorithms using n eural netw orks (Jagota, 1996). Genetic and h ybrid ev olutionary algorithms ha v e also b een dev elop ed, n otably b y Galinier et. al. in 1999 (Galinier & Hao, 1999), in add ition to more traditional optimization metho dology , suc h as scatte r searc h (J.-P . Hamiez, 2001). Th ere ha v e also b een stud ies of b enc hmarking mo dels for graph coloring, such as the recen t w ork b y W alsh (W alsh, 2001), which shows that graph s with high vertex degrees are more lik ely to o ccur in real-w orld app lications. Register Alloca tion is a v ery activ e application of graph coloring. This problem seeks to assign v ariables to a limited n um b er of hard w are registers during program execution. Accessing v ariables in registers is muc h faster than fetc hing them from memory . Ho we ve r, the num b er of registers is limited and is t ypically m uc h smaller th an the num b er of v ariables. Therefore, m ultiple v ariables must b e assigned to the same r egister. There are restrictions on these assignmen ts. T wo v ariables conﬂict with eac h other if they are live at the same time, i.e. on e is used b oth b efore and after the other within a short p erio d of time (for instance, with in a subr ou tin e). T he goal is to assign v ariables that do not conﬂ ict so as to minimize the use of n on-register memory . T o formalize this, one creates a graph where no des repr esen t v ariables and edges repr esen t conﬂicts b et w een v ariables. A coloring maps to a conﬂict-free assignment, and if the num b er of registers exceeds the c hromatic num b er, a conﬂict-free register assignmen t exists (Chaitin et al., 1981). Printed Cir cuit Board Testing (Garey & John son, 1979) inv olve s the problem of testing printed circuit b oards (PC Bs) for unintended short circuits (caused by stra y lines of solder). T his giv es rise to a graph coloring problem in whic h the v ertices corresp on d to the nets on b oard and there is an edge b et w een tw o vertice s if th ere is a p otent ial for a s hort circuit b et we en the corresp ond ing nets. Coloring the grap h corresp onds to partitioning the nets into “sup ernets,” where the nets in eac h su p ern et can b e simultaneo usly tested for shorts against all other nets, thereb y sp eeding up the testing pr o cess. Radio freq uency as signment for broadcast services in geographic regio ns (includ- ing commercial radio stations, taxi disp atc h, p olice and emergency services). T he list of all p ossible frequ encies is ﬁxed b y go v ernment agencies, b ut adjacen t geo graphic regions can- not use o v erlapping f requencies. T o r educe frequency assignment to graph coloring, eac h geographic region n eeding K frequ en cies is represented with a K − clique, and all N × K p ossible bip artite edges are in tro du ced b et w een tw o geographicall y adj acent regions needing N and K frequen cies resp ectiv ely . Other applications of graph coloring in circuit design and la y out includ e circuit cluster- ing, sc heduling for signal ﬂow graphs, and many others. Benc hmarks fr om these applications are not p ublicly a v aila ble, and therefore d o not app ear in this pap er. Ho we ve r, all the s ym- metry breaking tec hniques describ ed here extend to in stances from any applicat ion. The 252 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring b enchmarks we u s e h ere do include register allocation, n − queens, and sev eral other appli- cations discus s ed in more detail in S ection 4. Emp irically , we observ e that many of the instances in this p ap er can b e optimally solv ed in r easonable time, esp ecially when sym- metry breaking is emplo y ed. Since this w ork deals with ﬁn ding optimal s olutions f or graph coloring, we discuss previous work on ﬁn ding exact algorithms for this p roblem in some detail. The literature on exact graph coloring includes generic algorithms (Kubale & J ack owski, 1985) and s p ecialized algorithms for a particular application, suc h as Chaitin’s register al- lo cation algorithm (Chaitin et al., 1981). A t the moment , there do es not app ear to b e a comprehensiv e survey of tec hniqu es for this pr ob lem. Ho w ev er, onlin e sur v eys (T r ic k, 1996; Culb er s on, 2004) con tain r easonably large b ibliographies and ev en do wnloadable source co de for coloring algorithms in some cases. Pu blished algorithms for ﬁ nding optimal graph colorings are m ainly b ased on implicit enumeration. The algorithm p r op osed by Bro wn (Bro wn, 1972 ) enumerates solutions for a giv en instance of graph coloring an d c hec ks eac h solution for correctness and optimalit y . The al gorithm introd u ces a sp ecial tr e e c onstruc- tion to av oid redun dancy in enumerating solutions. The work b y Brelaz (Brelaz, 1979) impro ve s up on this algorithm by creating an initial coloring based on some clique in the graph and then considering assignmen ts in d uced by this coloring. The work b y Ku b ale and Kusz (K u bale & Kusz, 1983) discusses the empirical p erformance of implicit en umeration algorithms, and later work by Kubale and Jac k o wski (Kubale & Jac k o wski, 1985) augmen ts traditional implicit enumeration tec hniques with more sophisticated bac ktrac king metho ds. Our work deals with s olving graph coloring b y r e duction to another pr oblem, in this case 0-1 ILP . T his t yp e of reduction has b een discussed in the past, n otably in the recen t w ork by Mehrotra and T rick (Mehrotra & T ric k, 1996), whic h p rop oses an optimal coloring algorithm whic h expresses graph coloring using I LP-lik e constrain ts. It relies on an auxiliary indep end en t set form ulation, where eac h ind ep endent set in a graph is r epresen ted b y a v ariable. T here can b e prohibitivel y man y v ariables bu t in p r actical cases this n umb er may b e reduced b y c olumn gener ation , a metho d that ﬁrst tries to solv e a linear relaxation using a su bset of v ariables a nd then adds more where n eeded. This approac h inherently br eaks problem symmetries, and th us rules out the u se o f SBPs as a wa y to sp eed up the searc h pro cess. Our ILP co nstru ction diﬀers consid erably from the one describ ed ab ov e, since it do es n ot rely on an indep endent set formulation, b u t assigns colors to individual v ertices by using indicator v ariables. Th e construction is describ ed in m ore detail later in this section. Solving graph coloring b y reduction allo ws exact solutions to be found b y using SA T /0-1 ILP solv ers as b lack b o xes. Earlier w ork b y Cou d ert (Coudert, 1997) demonstrated that ﬁnding exact solutions for app lication-deriv ed graph coloring b enchmarks often take s no longer than heu ristic approac hes, and that heuristic s olutions ma y diﬀer fr om the optimal v alue by as m uc h as 100%. C oudert (Coudert, 1997) pr op oses an algorithm th at ﬁnds exact graph coloring solutions b y solving the max-clique problem. The algorithm u ses a tec hnique called “ q − color prun ing”, whic h assigns colors to vertic es and systematically remo v es v ertices that can b e colo red by q col ors, wh ere q is greater than a sp eciﬁed limit. 253 Ramani, Aloul, Mark ov, & Sakal lah 2.2 Breaking Symmetries in CSPs Sev eral earlier wo rks hav e add r essed the imp ortance of sy m metry br eaking in the searc h for solutions of CS Ps. It has b een s h o wn (Krishnam urthy , 1985) that symm etry facilitates short pro ofs of p rop ositions suc h as the p igeonhole principle, whereas pur e-resolution pro ofs are necessarily exp onen tial in size. Finding such pro ofs is, of course, a v ery diﬃcult problem, but the p erformance of man y CS P techniques can b e lo we r-b ounded by the b est-case pro of size. A t ypical app roac h to use symm etries is to pr ev en t a CSP solv er fr om considering redund an t symm etric solutions. This is called symmetry-breaking and can b e accomplished b y adding constrain ts, often called symmetry-b r eaking p redicates (SBPs). Static sym m etry- breaking, such as the instance-indep end en t constructions prop osed in this w ork and th e instance-dep endent p redicates from the literature (Aloul et al., 2003; Crawford et al., 1996 ), detects symm etries and add s SBPs during p re-pro cessing and not when b ranc hing to wa rd p ossible solutions. T he S y m metry Breaking b y Dominance Detection (SBDD) pro cedur e describ ed b y F ah le in 2001 (F ahle, Sc ham b erger, & S ellmann, 20 01) detects symmetric c hoice p oint s dur ing searc h. Eac h c hoice p oin t generated by the search algorithm is c hec k ed against pr eviously expanded search no des. If the same or an equiv alen t c hoice p oin t has b een pr eviously expanded , the c h oice p oin t is not visited again. The global cut algorithm prop osed by F o cacci and Milano (F o cacci & Milano, 2001) r ecords all nogo o ds found during searc h whose sym m etric images should b e prun ed. This set of n ogoo d s, called the “global cut seed” is used to generate global cut constraints that p rune s ymmetric images for the ent ire searc h tree, w hile ensurin g that correctness of the original constraints is not violated. Later w ork (Pu get, 2002) has prop osed impr ov ed metho ds for nogo o d r ecording. These works do not oﬀer a s y s tematic strategy for symmetry detection - they either require symmetries to b e known or declared in adv ance, or record information d uring searc h that en ables symmetry detection. O ur work outlines and implements a complete strategy to detect and break symm etries automatica lly du ring pre-pro cessing, so th at a blac k-b o x solver can b e used du r ing searc h. This con text is br oader than those that justify th e d ev elopmen t of sp ecializ ed solv ers. On th e other h and, our tec hniqu es d o not conﬂict with dyn amic symmetry-breaking and some of our results can p otent ially b e reused in that con text. A promising new partially-dynamic approac h to symmetry-br eaking, called Group Equiv- alence (GE) trees is prop osed b y Roney et. al. (Roney-Do ugal, Gent, Kelsey , & Lint on, 2004) . This w ork aims to r educe the p er-no d e ov erhead asso ciated with dynamic approac hes. A GE tree is constructed from a CSP with a symm etry group G suc h that the no des of the tree represent equiv alence classes of p artial assignmen ts u nder the group. Th is app roac h is illustrated by tr acking value symmetries, i.e., simultaneous p ermutatio ns of v alues in CSP v ariables. T he wo rk also sho ws that GE tr ees empirically outp erform sev eral well-kno w n symmetry-breaking metho dologies, suc h as SBDDs. In comparison, our w ork compares dif- feren t wa ys to hand le arbitrary comp ositions of v ariable and v alue symmetries (in graph coloring, v alue symmetries are seen at the sp eciﬁcatio n leve l, whereas v ariable symmetries can only b e seen in p roblem instances). T o this en d , our static techniques app ear compat- ible rather than comp eting with the use of GE trees. There ha v e also b een many symme- try br eaking app roac hes with particular r elev ance to graph coloring. Recent w ork by Gent (Gen t, 2001) prop oses constraints that b reak symmetry b et we en “indistinguishable v alues”, but do es n ot ev aluate them emp irically . L ik e th e lo w est-index ordering (LI) constrain ts pro- 254 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring p osed b y us in Section 3, these constrain ts also use the pre-existing sequentia l n umberin g of v ertices in an in stance of graph coloring to enforce distin ctions b et we en sym metric v ertices. The constru ction app ears complex compared to alternative S BPs and not as eﬀectiv e in our exp eriments as simp ler constructions. An other r elated work is (Hen tenryc k, Agren, Flener, & Pea rson, 2003), w hic h p rop oses a constan t-time, constan t-space algorithm for detect- ing a nd breaking v alue symmetries in a cla ss of CS Ps th at includes graph coloring. More recen tly , Benhamou (Benhamou, 2004) discu s ses sym m etry b reaking for CSPs mo deled us- ing not-equals constraints (NECSP), and u ses graph coloring as an illustrativ e example. The pap er deﬁnes a suﬃcien t cond ition for symmetry such that certain symmetries can b e detected in linear time. Th e remov al of these symmetries leads to considerable gains in bac ktrac king s earch algorithms for NECS Ps. In general, our empirical results, rep orted in Section 4, app ear comp etitiv e with those for state-of-the-art d y n amic approac hes. Ho w ev er, designing the w orld’s b est graph-colorer is not th e goal of our r esearc h. Instead, w e fo cus on more eﬃcien t prob lem reductions to SA T and 0-1 ILP b y imp ro ving symmetry-breaking. T o en sure a b road applicabilit y of our results, we treat SA T solvers as b lac k b o xes, and p erform a compr ehensiv e comparison of s tatic SBPs and rep ort empirical trend s . While a more comprehensive comparison against existing graph coloring literature w ould b e of great v alue, making it rigorous, conclusiv e and rev ealing requires that the b est static and the b est dyn amic symmetry-breaking tec hniques are kno wn. T o this end , we sp eculate that a more likely winner wo uld b e a hybrid. Additional ma jor issues to b e r esolv ed include the tuning of solvers to sp eciﬁc b enc hmarks (noted in the work b y Kiro vski and P otk on- jak (Kirovski & Potk onjak, 1998), diﬀerences in exp erimental setup, diﬀerent soft wa re and hardware platforms, etc. Giv en that suc h a co mparison is not complete ly in the scop e of our wo rk, it is b etter delegated to a dedicated publication. How ev er, to demonstrate that our tec h niques are co mp etitiv e with r elated w ork, w e provide a comparison with the b est results fr om recent literature (Benhamou, 2004; Coudert, 1997) in Section 4.3. 2.3 SA T and 0-1 ILP One can solv e the decision version of graph coloring by reducing it to Bo olean satisﬁabilit y , and the optimizatio n v ersion by reduction to 0-1 ILP . The Bo olean satisﬁabilit y (SA T) problem in v olv es ﬁnding an a ssignment to a set of 0-1 v ariables that satisﬁes a set of constrain ts, called clauses , expr essed in conjunctiv e normal form (CNF). A CNF form ula on n b inary v ariables, x 1 , . . . , x n consists of a conju nction of clauses, ω 1 , . . . , ω m . A clause consists of a d isjunction of liter als . A literal l is an o ccurr ence of a Bo olean v ariable or its complement. The 0-1 ILP p roblem is closely related to SA T , and allo ws the u se of pseudo-Bo ole an (PB) constrain ts, whic h are linear inequalities with in teger co eﬃcients that can b e expressed in th e normalized form (Aloul, R amani, Marko v, & Sak all ah, 2002) of: a 1 x 1 + a 2 x 2 + . . . a n x n ≤ b wh ere a i , b ∈ Z + and x i are literals of Bo olean v ariables. 1 In some cases a single PB constrain t can replace an exp onen tial num b er of CNF clauses (Aloul et al., 2002). In general, the eﬃciency of CNF reductions is enco ding-dep end en t. Earlier work by W arners (W arners, 1998) shows that a linear-o ve rhead conv ersion exists from linear inequalities with in teger coeﬃcien ts and 0-1 v ariables to CNF. Ho wev er, CNF 1. Using the relations ( Ax ≥ b ) ⇔ ( − Ax ≤ − b ) and x i = (1 − x i ), any arbitrary PB constrain t can b e expressed in normalized form with only p ositive co eﬃcients . 255 Ramani, Aloul, Mark ov, & Sakal lah enco dings whic h d o not use this con ve rsion m a y b e less eﬃcien t. When con v erting CNF to PB, a single C NF constraint can alw a ys b e expressed as a single 0-1 ILP constraint (by replacing disju n ctions b et we en literals in the constrain t with ‘+’ and setting the right-hand- side v alue as ≥ 1). Ho w ev er, this ma y not alw a ys b e suitable since certain op erations, such as disjunction, implication and inequ alit y are more intuitiv ely expressed as CNF, and can b e eﬃcien tly pro cessed b y S A T solv ers su c h as Ch aﬀ (Mosk ewicz et al., 2001). A con v ersion to 0-1 ILP is more desirable for arithm etic op erations, or “counting constraints”, whose CNF equiv alen t requir es p olynomially many clauses (and exp onen tially many f or some con v ersions). T o maximize the adv an tages of b oth CNF and PB formats, most recent 0-1 ILP solv ers such as PBS (Aloul et al., 2002) and Galena (Chai & Kuehlmann, 2003) allo w a formula to p ossess CNF and PB comp onent s. Additionally , 0-1 ILP solv ers also pro vide for the solutio n of optimization pr oblems . Sub j ect to giv en co nstraints, one ma y request the minimization (or maximization) of an ob jectiv e function whic h must b e a linear com bination of the problem v ariables. Exact SA T solv ers (Goldb erg & Novik o v, 2002; Mosk ewicz et al., 2001; S ilv a & S ak allah, 1999) are t ypically based on th e original Da vis-Logemann-Lo v eland (DLL) bac ktrac k searc h algorithm (Da vis, Logemann, & Lo v eland, 1962). Recen tly , several p o w erful metho ds ha v e b een pr op osed to exp edite the bac ktrac k searc h algorithm, such as conﬂict diagnosis (Silv a & Sak allah, 1999) and watc hed literal Bo olean constrain t propagation (BCP) (Mosk ewicz et al., 2001). With these improv ements, m o dern SA T solv ers (Mosk ewicz et al., 2001; Goldb erg & No vik o v, 2002) are capable of solving instances with several million v ariables and clauses in r easonable time. Th is increase in scalabilit y and scop e has enabled a n umber of SA T-based app lications in v arious domains, includ ing circuit la y out (Aloul et al., 2003), micropro cessor ve riﬁcation, symb olic mo del c hec king, and many others. More recen t work has fo cused on extend ing adv ances in SA T to 0-1 ILP (Aloul et al., 200 2; Chai & Kuehlmann , 2003) . In this work, we fo cus on solving instances of exact graph coloring by reduction to 0-1 I LP and the u s e of S BPs. Our c hoice of 0-1 I L P is motiv ated by th e follo wing reasons. Firstly , 0-1 ILP p ermits the use of a more general inp ut format than CNF, allo wing greater eﬃciency in problem enco ding, but at the same time is similar enough to SA T to allo w imp ro v ed metho ds for SA T-solving to b e used without pa ying a p enalt y for generalit y . The sp ecialized 0-1 ILP solv ers PBS (Aloul et al., 2002) and Galena (Chai & Kuehlmann, 2003) b oth p rop ose sophisticated n ew techniques f or 0-1 ILP that are based on r ecen t decision h euristics (Mosk ewicz et al., 2001), conﬂict diagnosis and bac ktrac king tec hniques (Silv a & S ak allah, 1999) for S A T solv ers. As a result, they emp irically p erform b etter than b oth the generic IL P solv er CPLE X (ILOG, 2000) and the leading-edge SA T solv er zChaﬀ on sev eral DIMACS SA T b enc hmarks and application-deriv ed instances such as FPGA r ou tin g instances from circu it la y out. Also, since 0-1 ILP is an optimization problem, un lik e SA T whic h is a decision problem, 0-1 ILP solv ers p ossess th e abilit y to maximize/minimize an ob jectiv e function. They can, therefore, b e directly applied to the optimiz ation v ersion of exact graph coloring, u nlik e p ure CNF-SA T solv ers th at can only b e used on the k − coloring decision v arian t. It is p ossib le to solv e the optimization version by rep eatedly solving instances of th e k − coloring u sing a SA T solver, with the v alue of k b eing up d ated after eac h call. How ev er, 0-1 ILP solv ers do not require this extra step, and moreo v er tend to p ro vide b etter p erformance th an rep eated calls to a S A T solv er on m an y Bo olean optimization problems (Aloul et al., 2002). 256 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring It is p ossib le to use a generic ILP solv er, suc h as th e commercial solv er CPLEX (ILOG, 2000) instead of a sp ecialize d 0-1 ILP solv er without an y c hanges in pr oblem form ulation. Ho w ev er, Aloul et al. (Aloul et al., 2002) show that th is generalization is not alwa ys desirable, particularly in th e case of Bo olean optimizatio n p r oblems su ch as Max-SA T. 0-1 ILP is also esp ecially useful for ev aluating the eﬀectiv eness of symmetry b reaking for graph coloring, the primary pur p ose of this w ork. Detecting and breaking symm etries in SA T form ulas has b een sho wn to sp eed u p the problem-solving pro cess (Cra wford et al., 1996; Aloul et al., 2003). Recen tly , symmetry breaking tec hn iques for SA T ha v e b een exte nded to 0-1 ILP (Aloul et al., 2004), and ha ve b een shown to pro duce searc h sp eedups in this domain as w ell. Ho we ve r, a similar extension for non-bin ary v ariables f or generic ILP do es not present ly exist. There is evidence (Aloul et al., 2002) that the adv an tages of symmetry breaking may dep end on the actual algorithm used in the searc h. Sp eciﬁcally , results in the cited wo rk su ggest that the generic IL P solve r CPLEX is actually slowe d down by the addition of SBPs. Sin ce CPLEX is a commercial to ol and the algorithms used b y it are not publicly kno wn, it is diﬃcult to pinp oin t a reason for this disparity . Ho w ev er, our empirical results in Section 4 do b ear out these observ ations. The remainder of this section discusses the reduction of graph coloring to 0-1 ILP and explains previous w ork in symmetry breaking in some detail. 2.4 Detecting and Breaking Symmetries in 0-1 IL Ps Previous w ork (Crawford et al., 1996; Aloul et al., 2003) has sho wn that br eaking symmetries in CNF formulas eﬀectiv ely p runes the searc h space and can lead to signiﬁcant runtime sp eedup s. Breaking symmetries pr ev en ts s ymmetric images of searc h paths from b eing searc hed, thus pru ning the search tree. The p ap ers cited in this work all use v arian ts of the approac h ﬁ rst describ ed by Cr awford et al. (Crawford et al., 19 96), whic h detects symmetries in a CNF formula u sing graph automorphism. The formula is exp ressed as an undirected graph suc h that the symmetry group of the graph is isomorphic to the symmetry group of th e CNF formula. Sym m etries indu ce equiv alence relations on the set of tru th assignmen ts of the CNF form ula. All assignment s in an equiv alence class result in the same truth v alue for the form ula (sa tisfying or not). Th erefore, it is only n ecessary to consider one assignment from eac h su c h class. T ec hniqu es for s ymmetry breaking prop osed in the literature follo w the follo wing s teps: (i) construction of a colored graph fr om a CNF form ula (ii) detection of sym m etries in the graph using graph automorphism soft ware (iii) use of the detected symmetries to con- struct s y m metry b reaking pr edicates (SBPs) that can b e app ended as additional clauses to the CNF form ula (iv) solution of the new CNF formula th us created using a SA T solv er. Cra wford’s constru ction (Crawford, 1992) uses 3 colors for v ertices, one for p ositiv e lit- erals, one for negativ e literals and a third for clauses. Edges are added b et w een literals in a cla use and the corresp on d ing clause ve rtex, and b et w een p ositive an d negati ve literal v ertices for Bo olean consistency . As an optimization, bin ary clauses (with just tw o literals) are r epresen ted b y adding an edge b et w een the t w o inv olv ed literals, so an extra vertex is not needed. T his is useful b ecause the runtime of graph automorph ism programs suc h as Nauty (McKa y , 1990) generally increases with the num b er of v ertices in the graph . Ho w ev er, with this optimizat ion Bo olean consistency is n ot enforced, since binary clausal 257 Ramani, Aloul, Mark ov, & Sakal lah edges could b e confused with Bo olean consistency edges b et w een p ositive and n egativ e lit- erals of the same v ariable. This ma y b e improv ed by r epresen ting binary clausal edges as double e dges (Crawford et al., 1996), th us distin gu ish ing b et w een the tw o edge types. Ho w ev er, Nauty (and other graph automorphism programs) do not sup p ort the u s es of double edges, so this construction is not very u seful in p r actice. F urtherm ore, the cited constructions (Cra wford, 1992; Cra wford et al., 1996) do not allo w detectio n of phase-shift symmetries, wh en a v ariable’s p ositive literal is mapp ed to its negativ e literal and vice versa, since they color p ositiv e and negativ e literals diﬀeren tly . Our previous wo rk (Alo ul et al., 2003) imp ro v es u p on these constru ctions by giving p ositiv e and negativ e literal vertic es the same color, and allo wing b inary clauses and Bo olean consistency edges to b e represented the same wa y , i.e. a single edge b et w een tw o literal v ertices. Although this construction ma y allo w s p urious symmetries - when clause edges are mapp ed in to consistency edges - this can o ccur only when a formula cont ains cir cular chains o f implic ations o v er a subs et of its v ariables. F or example, giv en a sub set of v ariables x 1 . . . x n , suc h a c hain is a collection of clauses ( y 1 ⇒ y 2 )( y 2 ⇒ y 3 ) . . . ( y n − 1 ⇒ y n ), where eac h y i is a p ositiv e or negativ e literal of x i . T hese circular chains rarely o ccur in practice, and can b e easily c hec ke d for. Therefore, the eﬃcien t graph construction d escrib ed ab o v e can b e used in most practical cases. Graph automorphism s are detected in Cr a wford’s work (Cra wford et al. , 1996 ) as wel l as our previous w ork (Aloul et al., 2003) usin g the program Nauty (McKa y , 199 0), wh ic h is part of the GAP (G roup s, Algebra and Programming) pack age. Nauty accepts graphs in the GAP input format and returns a list of gener ato rs for the automorphism group (the term “generators” is u sed in a mathematical sense, the symmetry group partitions the set of v ertex p ermutations for the graph int o equiv alence classes suc h that all p ermutatio ns in the same class are equiv alen t. Na uty return s the set of ge nerators for this symmetry group ). More recen t w ork ((Aloul et al., 2003, 200 4)) uses the automorphism program Saucy (Darga et al., 2004), whic h is more eﬃci ent than Na uty and can also pro cess larger graphs with more vertic es. After generators of the symmetry group are detected, sym metry breaking predicates are added to the instance in a pre-pro cessing step. Crawford et al. (Cra wford et al., 1996) pr op ose the addition of SBPs that c ho ose lexico graphically smallest assignmen ts (lex-leaders) fr om eac h equiv alence class. W e refer to suc h S BPs as instance-dep end en t SBPs, since th e symmetries are ﬁrst detected an d then broken, and th erefore the exact n umb er and nature of SBPs added alw a ys dep ends on the connectivit y of the graph itself. Although detecting symmetries is n on-trivial, using mo dern soft w are suc h as Nauty and Saucy th e detection time is frequently insigniﬁcan t when compared with SA T -solving time. Cra wford et. al. (Crawford et al., 199 6) construct lex-leader SBPs for the entire symmetry group, using the group generators returned b y Nauty . Th is t yp e of s y m metry breaking is c omplete . Ho w ev er, the approac h used b y Aloul et al. in TC AD 2003 (Aloul et al., 2003) sho ws that inc omplete s ymmetry breaking, which br eaks symmetries only b et w een generators, is often eﬀectiv e in practice and muc h more eﬃcien t sin ce it do es not requ ire the whole group to b e r econstructed. The S BP construction pr op osed in the cited work (Aloul et al., 2003) is quadr atic in the n umb er of problem v ariables, compared with th e earlier construction (Crawford et al., 1996), which could ru n to exp onen tial size. T his construction is further impro v ed in the 2003 w ork b y Alo ul, S ak allah and Mark o v (Aloul et al., 2003), whic h describ es eﬃcien t, tautology-free SBP construction, whose size is linear in the n um b er of pr oblem v ariables. Empir ical results from b oth C ra wford’s work (Crawford et al., 1996) as 258 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring w ell as the w ork in TCAD 2003 (Aloul et al., 2003) show that breaking symmetries pro d uces large sea rch sp eedups on a num b er of CNF b enc hmark families, including pigeonhole and Urquhart b enc hmarks, micropro cessor veriﬁcati on, FPGA routing and ASI C global routing b enchmarks f r om the VLSI domain. Our wo rk on symmetry b r eaking in SA T (Aloul et al., 2003) has also b een extend ed to to o ptimization problems that include b oth CNF and PB constraints, and an ob jectiv e function (Aloul et al., 2004). As b efore, s ymmetries are detected b y reduction to graph automorphism. A P B form ula for an optimization p roblem is represen ted b y an undirected graph. Graph symmetries are d etected u sing the graph automorphism to ol Saucy (Darga et al., 2004). Eﬃcient symmetry breaking predicates (Alo ul et al., 2003) are app end ed to the formula as CNF clauses. The empir ical results for our work on s ymmetry breaking in 0-1 ILP (Aloul et al., 2004) show that the addition of symmetry b reaking predicates to PB f orm ulas results in considerab le searc h sp eedup s for the sp ecialize d 0-1 I L P solv er PBS (Aloul et al., 2 002). In this w ork, we use the ab ov e metho dology (Aloul et al., 200 4) for detecting and breaking instanc e-dep endent symmetries in instances of graph coloring expressed as 0-1 ILP . These instance-dep endent S BPs a re co mpared w ith a n um b er of instance-indep endent SBP constru ctions d escrib ed in the next section. Detecti ng and br eaking sym m etries in application-deriv ed SA T instances amoun ts to a reco v ery of structure from th e original application. Th e loss of stru cture dur ing problem reductions is one reason wh y redu ction-based tec hniques are often not comp etitiv e with domain-sp eciﬁc algorithms, and recent work on symmetry breaking is useful in this con text. Other types of structure includ e clusters (Huang & Darwic he, 2003 ; Aloul, Mark o v, & Sak allah, 2004). Huang et al. (Huang & Darwiche, 2003) p rop ose an algorithm that detects clusters in SA T instances and u ses them to p ro du ce v ariable orderin gs, and these structure- a w are orderin gs r esult in considerable empirical impro ve ments with the SA T solv er zChaﬀ (Mosk ewicz et al., 2001) . 2.5 Reducing Graph Coloring to 0-1 I LP W e express an instance of the minimal graph coloring problem as a 0-1 ILP op timization problem, consisting of (i) CNF and PB constraints that m o del the graph (ii) an ob jectiv e function to minimize the num b er of colors used. Consider a graph G ( V , E ). Let n = | V | b e th e n umber of v ertices in G , and m = | E | b e the num b er of edges. An instance of the K − coloring problem for G (i.e., can the vertice s in V b e colored with K colors) is formulated as follo w s. • F or eac h v ertex v i , K indic ator variables x i, 1 , . . . , x i,K , denote possib le color assign- men ts to v i . V ariable x i,j is set to 1 to indicate that v ertex v i is colored with color j , and 0 otherwise • F or eac h v ertex v i , a PB constrain t of the form P K j =1 x i,j = 1 ensur es that eac h vertex is colo red with exactly one color. • Eac h ed ge e i in E connects t w o vertic es ( v a , v b ). F or eac h edge e i , we deﬁ ne CNF constrain ts of the form V K j =1 ( x a,j ∨ x b,j ) to sp ecify that no tw o ve rtices connected b y an edge can b e give n the same color. 259 Ramani, Aloul, Mark ov, & Sakal lah • T o track u sed colors, we deﬁn e K new v ariables, y 1 , . . . , y K . V ariable y i is tru e if and only if at least one v ertex uses color i . Th is is exp ressed using the f ollo wing CNF constrain ts: V K j =1 ( y j ⇔ ( W n i =1 x i,j )). • The optimization ob jectiv e is to m inimize the num b er of y i v ariables set to true, i.e. MIN P K i =1 y i The total n umb er of v ariables in the formula is n K + K . Th e tota l num b er of constraints is computed as follo ws. There are total ly n 0-1 ILP constrain ts (one p er v ertex) to ensu re that eac h v ertex u ses exactly one color. F or eac h edge, there are K CNF clauses s p ecifying that the t wo v ertices connected by that edge cannot hav e th e same color, giving a total of mK CNF cla uses. There are an additional nK CNF clauses ( K p er v ertex) for setting indicator v ariables, and K C NF clauses, one p er color, to complete the iﬀ condition for in di- cator v ariables. This giv es a total of K · ( m + n + 1) C NF clauses and n 0-1 ILP constraints, plus one ob jectiv e fun ction, in the con v erted form ula. F or dense graph s, where | E | ≈ | V | 2 , the resulting form ula s ize is quadr atic in the num b er of v ertices of the graph, but for sparser graphs it ma y b e linear. A k ey observ at ion is th at instance-dep end en t s ymmetries in graph coloring surviv e the ab ov e r eduction to 0-1 IL P . F or instance-indep endent symmetries (i.e. p ermutatio ns of colors) this is easy to see, since the ord ering of colors can b e c hanged without ha ving an y eﬀect on the form ula and pro du cing the same set of constraints. F or instance-dep endent symmetries, consider tw o ve rtices v a and v b that are symmetric to eac h other and can b e swapp ed in the original graph . C learly , the constraints that sp ecify that v a and v b m ust use exactly one color are interc h angeable, as are the constraints that d eter- mine co lor usage based on the colors assigned to v a and v b . I t on ly remains to show that the c onne ctivity c onstr aints that cont rol colors of v ertices adj acen t to v a and v b are also symmetric. This is clear from the fact that for ev ery edge E i inciden t on v a , there m ust b e a corresp onding edge E j inciden t on v b for the t wo v ertices to b e s y m metric ( E i and E j can b e the same edge). Therefore, for the set of K CNF clauses added to the form ula to represent E i , there must b e a symmetric set of clauses added f or E j , and thus connectivit y is pr eserv ed. It is also clear that the 0-1 ILP formulation d o es not introd uce spurious sy m metries, i.e. an y sym metry in th e form ula is a sym metry in th e graph. A s purious symmetry arises wh en (i) v ariables of d iﬀeren t t yp es can b e mapp ed into eac h other, e.g. v ertex color v ariables are mapp ed to color usage indicator v ariables and (ii) v ariables of the same type are m app ed in to eac h other wh en the corresp onding vertic es are not actually symm etric. F rom the construction of the 0-1 ILP formula, it is clear that all K v ariables p er v ertex that ind icate a v ertex’s color can b e p ermuted, as can the K color usage v ariables, since these all app ear in exactly the same constrain ts. This corresp onds to the ins tance-indep endent sy m metry - colors in an instance of graph colo ring can b e arbitrarily p ermuted. Ho w ev er, v ertex color v ariables app ear in constrain ts restricting the num b er of colors a v ertex can use and also in constrain ts that describ e the conn ectivit y of the graph, whereas colo r u sage v ariables app ear only in constraints that sp ecify when th ey are set. Ther efore, the tw o t yp es of v ariables cannot map to one another. Since all constrain ts regarding color and connectivit y of a v ertex are wr itten usin g al l K color v ariables for that v ertex, these v ariables are symm etric to eac h other only in groups of K , i.e. if one su c h v ariable for a giv en v ertex v 1 is symmetric to a v ariable for another v ertex v 2 , then all K v ariables for v 1 and v 2 are corresp ondingly 260 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring symmetric. Additionally , this symmetry b et w een v ariables in dicates a corresp onden ce of clauses in which they o ccur. This is only p ossible if th e ve rtices v 1 and v 2 are symm etric in terms of connectivit y (instance-dep end ent symmetry). Thus, b oth t yp es of symmetries are preserve d dur ing con v ersion to 0-1 ILP , a nd no false sym m etries are added. Therefore, we can app ly known tec hniques for symmetry detection in 0-1 ILP . 3. Instance-Independen t SB Ps The question add ressed in this work is whether instanc e-i ndep endent SBPs added dur in g the r eduction can provide eve n greater sp eedups, p ossibly by accelerating the detection of in stance-dep endent symmetries. T o answer this question, we p r op ose thr ee pro v ably correct SBP constructions of v arying s trength, and one heuristic that is inte nded to break a small num b er of symm etries w ith minimal o v erhead. Eac h construction is imp lemen ted and emp irical resu lts are r ep orted in Section 4. W e us e the follo wing n otatio n. Consider an instance of the K − coloring problem, whic h asks whether a graph G ( V , E ) can b e colored using ≤ K colors and minim izes the n umber of colors. Assume the colo rs are num b ered 1 . . . K . W e denote a v alid color assignmen t b y ( n 1 , n 2 , . . . , n K ) where n i is the num b er of ve rtices colored with color i , and | V | = P K i =1 n i . Eac h n i in th e color assignment denotes th e cardinalit y of the indep endent set colored with color i . W e are n ot concerned with the actual c omp osition of the indep end en t sets here, since that is an instance-dep end ent issue. Instance-indep endent symmetries are only the arbitrary p ermuta tions of colors b et we en diﬀerent indep end en t sets. The eﬀects of eac h prop osed construction are illustrated using the example in Figure 1. The ﬁgure is an examp le of the 4-coloring problem on a g raph with four v ertices. P art (a) of the ﬁgur e sho ws the graph to b e colored. F or visual clarit y , part (b) sh o ws color patterns corresp onding to the diﬀeren t color n umbers . It is clear from the ﬁgure that the v ertices V 1 , V 2 and V 3 form a clique, and m ust us e diﬀeren t colors. How ev er, V 4 can b e giv en the same color as either V 1 or V 2 , and therefore only 3 colors are n eeded for this instance. The in stance can b e partitioned into indep endent sets in t wo w a ys: {{ V 1 , V 4 } , { V 2 } , { V 3 }} and {{ V 1 } , { V 2 , V 4 } , { V 3 }} . O ur SBPs do not actually addr ess ho w the indep en d en t sets are comp osed, b ecause this is an instance-dep endent issue. Ho w ev er, giv en any partition of indep en d en t sets, co lors ca n b e arbitrarily p ermuted b etw een sets in the partition. T he instance-indep endent SBPs prop osed here restrict this p erm utation. In the examples b elo w, w e assume the ﬁr st partition of indep en d en t s ets i.e. {{ V 1 , V 4 } , { V 2 } , { V 3 }} . Results are pro v ed with resp ect to the p ermuta tion of colors for this partition. 3.1 Null-Color Elimination (NU) Consider a K − colo ring p r oblem with col ors 1 . . . K f or a graph G ( V , E ). Assume that G can b e minimally colored with K − 1 colors. Consider an optimal solution wh ere color i is not used: ( n 1 , n 2 , ..n i − 1 , 0 , n i +1 , . . . , n K ). This assignmen t is equiv alent to another assignmen t, ( n ′ 1 , n ′ 2 , ..n ′ j − 1 , 0 , n ′ j +1 ...n ′ K ) where i 6 = j and n ′ i = n j . F or example, the assignment (1 , 0 , 2 , 3) is equiv alent to (1 , 3 , 2 , 0), (0 , 1 , 2 , 3), (1 , 2 , 0 , 3). This is d ue to the existence of nul l colors, whic h cr eate symmetries in 261 Ramani, Aloul, Mark ov, & Sakal lah 1: 2: 3: 4: (b) (a) V 1 V 2 V 3 V 4 V 1 V 2 V 3 V 4 (c) V 1 V 2 V 3 V 4 V 1 V 2 V 3 V 4 V 1 V 2 V 3 V 4 (d) V 1 V 2 V 3 V 4 V 1 V 2 V 3 V 4 (e) (1,0,2,1) (1,2,1,0) (1,1,2,0) (2,1,1,0) (2,1,1,0) (1,1,2,0) Figure 1: Instance-indep enden t symmetry breaking predicates (SBP s). P art (a) shows the original graph with no vertices colored. P art (b) sho ws the color k ey . P art (c) sho ws ho w null- color SBPs preven t color 4 from b eing used. P art (d) sho ws ho w cardinality based SBPs assign colors in the order of in- dep endent set sizes, a llowing few er assignmen ts than null- color SBP s. P art (e) demonstrates ho w low est-index ordered SBPs break symmetries that are undetected b y other types of SBPs. 262 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring an instance of K − coloring b ecause an y color can b e swa pp ed with a n ull color. Null colors are extraneous b ecause they are not actually r equ ired to color an y vertic es, and so can b e inserted anywhere in a solution, as seen ab ov e. W e pr op ose a constru ction that enforces an ordering on null colo rs: null colors ma y app ear only at the end of a color assignmen t, after all n on-n ull colors. T his is implement ed by adding K − 1 CNF constrain ts of the form: y k +1 ⇒ y k for 1 ≤ k ≤ K − 1, to the original f orm ulation. In the example ab o v e, only one of the four symmetric assignmen ts (1 , 3 , 2 , 0) w ould b e allo wed und er this construction. Since our IL P formulat ion deﬁn es and s ets the K indicator v ariables that trac k colo r u sage, it is extremely easy to enf orce n ull color elimination as d escrib ed ab o v e. Th e SBPs r equire the addition of no extra v ariables and only K − 1 new CNF clauses. W e pro ve that the p rop osed constru ction is correct. Assume th at under the original form ulation, an optimal solution for graph G ( V , E ) uses m colors. Assume that this solution con tains null colors and n on-n ull colors, and with n ull-color elimination, there is a diﬀer e nt optimal solution that us es m ′ colors, wh ere m 6 = m ′ . The only colors us ed in this solution are 1 . . . m ′ , since n ull colors cannot o ccur b efore n on -null colors. Since our construction adds SBPs without c hanging the original constrain ts, any lega l s olution that satisﬁes the SBPs will satisfy all constraints in the original formulatio n. The solution to the original satisﬁes all constraints in the n ew form ulation except th e SBPs. If m < m ′ , w e can r e-order the solution so that all n ull colors are placed last. This will satisfy all SBPs and use m colors, where m < m ′ , violating the assumption that the m ′ -color solution w as optimal. If m ′ < m , we already hav e a solution that satisﬁes all the original constrain ts and uses few er colors, whic h again violates assum ptions of optimalit y . An illustration of the use of NU pr edicates f or the example in Figure 1 (a) is sho wn in Figure 1 (c). Th e ﬁgure shows tw o v alid minimal-color assignments to the graph v ertices in the example. The assignmen t on the left u ses colo rs 1, 3 and 4, while the one on the righ t uses colors 1, 2 and 3. The assignments are sym m etric b ut un der NU predicate s only th e righ t-hand side assignment is p ermissible. 3.2 Cardinality-Based Color Ordering (CA) Null-color elimination is useful only in cases wh ere null colors exist. F or a K − coloring problem where all colors are needed, the construction b reaks no symmetries. Eve n w hen n ull colors exist, sev eral symmetries go undetected. In the ﬁ rst example fr om ab o ve , n ull- color eliminatio n p ermits s ix symmetric color assignments (1 , 2 , 3 , 0), (1 , 3 , 2 , 0), (2 , 1 , 3 , 0), (2 , 3 , 1 , 0) (3 , 2 , 1 , 0 ) and (3 , 1 , 2 , 0). Th is is b ecause restrictions are placed on null colors, but the ordering of non-null colors is unrestricted. A stronger construction wo uld distin- guish b et we en the indep enden t sets themselv es. W e prop ose an alternativ e constru ction, whic h assigns colors based on the cardinalit y of ind ep endent sets. Th is subs u mes n ull-color elimination, since null colors can b e viewe d as coloring sets of cardinalit y 0. The cardi- nalit y ru le is implemen ted as follo ws: the largest indep en den t set is assigned the color 1, the second-largest the colo r 2, etc. In the example ab o ve , only the assignment (3 , 2 , 1 , 0) is v alid. This is enforced by adding K − 1 PB constraint s of the form: P n i =1 x i,k ≥ P n i =1 x i,k +1 , where 1 ≤ k ≤ K − 1. Again, this construction is fairly simple to implement , requirin g only K − 1 additional constrain ts. Ho w ev er, th ese are 0-1 I LP constraints with m ultiple 263 Ramani, Aloul, Mark ov, & Sakal lah v ariables, unlike the simple CNF implication clauses b et w een tw o v ariables used for the NU predicates. Thus, there is some o v erhead for greater completeness. W e pr o v e the CA constru ction corr ect as follo ws. Assume an optimal solution u nder this construction uses m < K colors: ( n 1 , n 2 , . . . , n m ), where ( n 1 ≥ n 2 . . . ≥ n m ). Colors > m are not u sed on an y v ertex, Assu m e there exists an optimal solution to the original form ulation that u s es m ′ colors: ( n ′ 1 , n ′ 2 , . . . , n ′ m ′ ), (where n ′ 1 , etc. are n ot arranged in descending order). Without loss of generalit y , assume that m ′ < m . W e can s ort the n umb ers n ′ 1 , . . . , n ′ m ′ and reassign colors in descending order. W e would ha v e a solution with m ′ colors satisfying cardinalit y constrain ts. Ho w ev er, m ′ < m , w h ic h is not p ossible if the m − color solution w as optimal. A similar argument applies when m < m ′ . F or the example from Figure 1 (a), only the largest indep en den t s et un d er the partition w e are considering, i.e. { V 1 , V 4 } can b e giv en color 1. Th erefore, th e assignmen t on the right of Figure 1 (c), wh ic h assigns the largest set color 2 and is correct under NU pr edicates, is incorrect un der the CA constru ction. The left-hand side of Figure 1 (d) sho ws another assignmen t that is correct un der NU predicates but in correct und er CA predicates, since it assigns the set { V 1 , V 4 } colo r 3. A correct assignmen t, sho wn on the right-hand side of Figure 1 (d), giv es th e largest set color 1 and since b oth the other sets h a v e one elemen t eac h, they can ea c h b e assigned either color 2 or co lor 3. Thus, sev eral symmetric assignmen ts whic h surviv e NU predicates are pr ohibited u n der this construction. 3.3 Lo west Index Color Ordering (LI) While more complete than NU pr edicates, C A predicates do n ot break sy m metries when diﬀeren t indep endent sets h av e the same card inalit y . C onsider a graph G where V = { v 1 , . . . , v 8 } , and an optimal solution, satisfying card inalit y-based ordering, that partitions V into 4 indep endent sets: S 1 = { v 4 , v 6 , v 7 } , S 2 = { v 1 , v 5 } , S 3 = { v 3 , v 8 } , S 4 = { v 2 } . A solution that assigns colors 2 and 3 to S 2 and S 3 is symmetric to one that assigns colors 2 and 3 to S 3 and S 2 . Both are legal und er cardinalit y-based orderin g. In order to completely break symmetries, it is not adequate to distinguish b et w een s ets solely on the basis of cardinalit y (unless no t wo sets ha v e the same cardinalit y). It is necessary to construct SBPs b ased on the actual c omp osition of sets in a partition, wh ic h is uniqu e. Ho w ev er, the d istinctions that we m ak e on the b asis of comp osition are not to b e confused with instance-dep endent SBPs, since our construction is implemen ted b efor e the symmetries in an instance are kno wn, and regardless of its actual comp osition. The S BPs here s p ecify broad guidelines for the coloring of indep end en t sets that are applicable to all graphs. T o impro ve u p on cardinalit y-based orderin g, w e p r op ose a set of predicates to enforce th e lowest-index or dering (LI). Consider all vertic es with color i , and ﬁnd the low est index j i among those. W e require that the lo w est ind ices for eac h color b e ordered. This constraint can b e enforced b y adding inequalities for colors with adjacen t n um b ers. Note that eac h color has a unique lo w est-index v ertex — otherwise some v ertex would ha v e to b e colored with t wo colors. In the ab ov e example, the only color assignment compatible with the partitioning of v ertices into ind ep endent sets is: color 1 to S 1 , 2 to S 3 , 3 to S 4 , an d 4 to S 2 . T o ev aluate the stren gth of this sym metry-breaking tec hnique, consid er an arbitrary coloring and a color p ermutatio n that remains a symmetry after th e LI constrain ts are 264 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring imp osed. If an y colors are permuted s im ultaneously on all v ertices, this will p erm ute the lo w est in dices for those colors. S ince all lo w est indices are d iﬀeren t, their ord ering is com- pletely determined b y the ord ering of colors, and thus the color p ermuta tion we c hose must b e the ident it y p erm utation. In ot her w ords, no instance -indep endent symmetries remain after sy m metry-breaking with LI. W e implemen t lo west-index color ord ering as follo ws. F or eac h vertex v i , w e d eclare a new set of K v ariables, V i, 1 , . . . V i,K . V ariable V i,k b eing set implies that v ertex v i is the lo w est-index v ertex colored with color k . This is enforced by the follo wing CNF constraint s: V i,k ⇒  V i − 1 j =1 V j,k  . Also, exactly one V i,j v ariable must b e true for ev ery color used. Therefore, w e add the co nstraints: y k ⇒ W n i =1 V i,k , where 1 ≤ k ≤ K , y k are the v ariables that ind icate color k is u s ed, and n = | V | from Section 2. Finally , the f ollo wing CNF clause is added for eac h V i,k to ensure lo w est-index ordering: V i,k ⇒  W n j = i +1 V j,k − 1  , Since the LI ordering completely breaks symmetries b et w een indep enden t sets, it su bsumes earlier constructions. Ho we ve r, it do es come at an added cost. While the NU and CA constructions required n o new v ariables an d only K − 1 constraint s, the LI construction requires nK n ew v ariables and an ad d itional 2 nK CNF clauses, w hic h is almost double the size of the original form ula. The LI construction can b e p ro v ed correct b y the same means as the CA constru ction. Giv en an optimal assignm ent of colors to in dep end en t sets, we can sort the indep enden t sets in ord er of lo w est-index v ertex and assign colors from 1 to K acc ordingly , without aﬀecting correctness. Figure 1 (e) illustrates the eﬀect of LI SBPs on the example in Figure 1 (a). The graph on the left, whic h is sho wn as b eing correct for CA p redicates in Figure 1 (d) is incorrect under the L I construction, b ecause the lo west- index verte x with color 2 ( V 3 ) do es n ot ha ve a higher ind ex than th e lo w est-index vertex with color 3, whic h is V 2 . T he graph on the righ t sho ws the correct assignment, which under LI pred icates is the only p er m issible assignmen t for the partition {{ V 3 } , { V 2 } , { V 1 , V 4 }} . In addition to b eing ve ry complex, LI predicates are so rigid that they obscure symme- tries of th e original in stance. F or example, in Figure 1 (a), it is easily seen that the ve rtices V 1 and V 2 are sy m metric and can b e p ermuted with no eﬀect on th e r esulting grap h . This symmetry is instanc e- dep endent - it is decided b y the wa y V 1 and V 2 are conn ected. With- out the add ition of any SBPs, it is app aren t that u nder any legal coloring of th e graph, th e colors given to V 1 and V 2 can b e swa pp ed regardless of ho w V 3 and V 4 are colored. Th e NU predicates preserve this symm etry , sin ce they are only concerned with n ull colors whic h by deﬁnition could not b e used on V 1 and V 2 . T he CA predicates also p reserv e the symmetry since V 1 and V 2 can b e in terc hangeably used in an y indep enden t set, and sw apping them b et wee n sets w ould not ha v e any eﬀect on the card in alit y of the sets. Ho w ev er, un der the LI predicates, an indep enden t s et conta ining V 1 m ust always b e giv en a higher-n um b ered color than a set con taining V 2 , and the tw o cannot b e in terc hanged. If V 1 w as giv en an y color other than th e h ighest color in u se, th ere w ould exist s ome ind ep endent set whose color index was 1 grea ter than the color assigned to V 1 , and for this set, the lo w est-index predicate wo uld n ot b e s atisﬁed. T h us, LI predicates actually destroy any v ertex p erm uta- tions in the graph . This is seen in our empirical results in Section 4, where the addition of 265 Ramani, Aloul, Mark ov, & Sakal lah LI S BPs lea v es no symmetries in an y of the b enchmarks. T his is unusual b ecause ord inarily b enchmarks of reasonable size would con tain at least some v ertex p ermuta tions. 3.4 Selectiv e Coloring (SC) It is noticeable that the ILP formulation and constraints can b e very complex for more complete SBPs, su c h as the LI predicates ab o v e, which in tro du ce sev eral additional v ariables and clauses. This raises the question of whether su c h a complex constru ction is actually coun terpro d uctiv e - it ma y b reak symmetries, but require so muc h eﬀort dur ing searc h th at the b eneﬁt of complete symmetry b reaking is lost. T o inv estigate this, we also p r op ose a simp le “heu ristic” construction to break some symmetries b et wee n vertice s wh ile add ing almost no additional constraints. T o impact as many v ertices as p ossible, we ﬁnd the ve rtex v l with the largest degree of all v ertices in the graph. W e then color v l with color 1. This is ac hiev ed by simply adding the unary clause x l, 1 . W e searc h v l ’s neigh b ors to ﬁnd the v ertex v l ′ with the h ighest d egree out of all vertice s adjace nt to v l . W e color v l ′ with color 2, by add ing the u n ary clause x l ′ , 2 . This construction has the eﬀect of simplifying color assignmen t for all v ertices adjacen t to v l and v l ′ . No v ertex adjacen t to v l can b e colored color 1, and no v ertex adjacen t to v l ′ can b e co lored color 2. Moreo v er, all v ertices in an indep end en t set with v l ( v l ′ ) must b e colored color 1 (color 2). I f v l and v l ′ ha v e suﬃcientl y large d egree, this construction can r estrict many ve rtex assignments. An even stronger construction w ould b e to ﬁnd a triangular cli que and ﬁ x co lors for all three vertices in it; ho w ev er, clique ﬁn ding is complicated and some graphs ma y not p ossess any suc h cliques. W e refer to this construction as sele ctive c oloring . The exten t to wh ic h s electiv e coloring breaks symmetries is instance-dep enden t. It f ails to completely break sym metries for almost all graphs. How ev er, it is a simple construction, adding jus t t w o constrain ts as un ary clauses. These are easily r esolv ed in pre-pro cessing by most SA T solve rs, so an y symmetry br eaking ac hiev ed b y this constru ction h as virtually no o v erhead. W e note that all instance-indep endent pred icates d eﬁned here are only concerned with symmetries b et w een co lors, whic h exist in an y instance of graph coloring. Ho wev er, addi- tional instance- indep endent symmetries m a y be in tro duced during the reduction to graph coloring for certain app licatio ns. F or example, in the radio frequency assignmen t app lica- tion from Section 2, add ing all p ossible bipartite edges b et w een cliques f or adjacen t regions will result in symm etries b et w een ve rtices in these cliques. Additional p redicates can b e added to instances from this application to break these symm etries. 4. Empirical Results This section describ es our exp erimen tal setup, emp irical results, and p erform an ce compared with r elated w ork. 4.1 Exp erimental Setup W e used 20 medium -sized instances fr om the DIMA CS graph coloring b enc hmark suite. W e brieﬂy describ e eac h family of b en c hmarks used b elo w. 266 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring • Random graphs. Benc hmarks with rand omly created connectio ns b et w een ve rtices, named DS J • Bo ok graphs. Edges represent int eraction b et w een c haracters in a b o ok. Th ere are four s uc h b enc hmarks: ann a, david, huck, jean • Mileage graphs. These repr esen t distances b et w een cities on a map, and are named miles • F o otball game graphs. Indicate relationships b et w een teams that must pla y eac h other in college fo otball games. In the tables th ese are referred to as gam es • n − queens graphs. In s tances of the n − queens pr ob lem, named qu een • Register allo cation graphs. Repr esen t the register allo cation problem f or diﬀeren t systems. W e us e t wo families in this w ork, named mul sol, zeroin • Mycielski graphs. Instances of triangle-free graph s b ased on the Mycielski (My- cielski, 1955) transform ation, called myciel T able 1 give s the name, size (n um b er of v ertices and edges) and the chromatic num b er for eac h b en c hmark. W e use a maxim um v alue of K = 20 for K − coloring. F or b enc hmarks with chromatic num b er > 20, w e do not rep ort the c hromatic n umb er. Our problem form ulation with a ﬁ x ed K is app licatio n-dr iv en. Indeed, in many do- mains it is only us efu l to ﬁnd th e exact c hromatic num b er when it is b elo w a w ell-kno wn threshold. F or example, in graph coloring in stances fr om register allocation, there ca nnot b e more colors than pro cessor r egisters. PC pro cessors often ha v e 32 registers, and high-end CPUs ma y ha v e more. Ho w ev er, realistic graphs are relativ ely sparse and ha v e lo w c hro- matic n umbers. On the ot her hand , pro cessors em b edded in cellular ph ones, automobiles and p oint- of-sale terminals may ha v e very few registers, leading to tight er constraints on acceptable chromatic num b ers. The v alue K = 20 used in ou r exp eriments is in no wa y sp ecial, bu t th e results ac hiev ed with it are representat iv e of other results. Also, while we apply the K = 20 b ou n d to all in s tances h ere to study trend s, more reasonable b ounds can b e determined on a p er-instance basis using the follo wing simple p ro cedure. 1. Apply an y heuristic for min-coloring to determine a feasible upp er b ound 2. If the v alue is r elativ ely small, p erform linear searc h by increment ally tigh tening the color constrain t, otherwise p erf orm binary search Benc hmark graphs are transformed in to instances of 0-1 ILP using the conv er s ion de- scrib ed in Section 2. T o s olv e instances of 0-1 ILP , we used the academic 0-1 ILP solv ers PBS (Aloul et al., 2002), Galena (C h ai & Kuehlmann, 2003), and Pueblo (S h eini, 2004), and also the commercial ILP solv er CPLEX ve rsion 7.0 . Pueblo is more recent than PBS and Galena, and in corp orates Pseud o-Boolean (PB) learning b ased on I LP cutting-plane tec hniques. W e u se a la ter version of PBS, PBS I I, that enhances the original PBS algo- rithms (Aloul et al., 2002 ) with learnin g tec hniques from the Pueblo solv er (Sh eini, 2004). W e do n ot include the results with the original v ersion of PBS that are rep orted in (Ramani, 267 Ramani, Aloul, Mark ov, & Sakal lah Instance #V #E K anna 138 986 11 david 87 812 11 DSJC125.1 125 1472 5 DSJC125.9 125 13922 > 20 games120 120 127 6 9 huc k 74 602 11 jean 80 508 10 miles250 128 774 8 muls ol.i.2 188 388 5 > 20 muls ol.i.4 185 394 6 > 20 myciel 3 11 20 4 myciel 4 23 71 5 myciel 5 47 236 6 queen5 5 25 320 5 queen6 6 36 580 7 queen7 7 49 952 7 queen8 12 96 2736 12 zeroin.i.1 211 410 0 > 20 zeroin.i.2 211 354 1 > 20 zeroin.i.3 206 354 0 > 20 T able 1: DIMA CS graph coloring b e nc h- marks Aloul, Mark o v, & S ak allah, 2004), since it has b een retired b y the newer version. How ev er, in the Ap p end ix w e rep ort detailed results for n − queens in s tances using the older version of PBS along with r esults for the other s olvers, for the sak e of a more d etailed stud y . PBS I I is implemen ted in C+ + and compiled using g++. Galena and Pu eblo binaries we re pr o vided b y the authors. PBS was r un using the v ariable state indep endent deca ying s um (VSIDS) decision heu r istic option (Mosk ewicz et al., 2001). Galena w as r un using its default options of linear searc h with cardinalit y redu ction (CARD) learning. All exp erimen ts are ru n on Sun-Blade-1000 workstatio ns with 2GB RAM, CPUs clo c k ed at 750MHz and th e Solaris op erating system. Time-out limits for all solv ers are set at 1000 seconds. W e use the symmetry breaking ﬂow ﬁ rst p rop osed in our earlier w ork (Aloul et al., 2004) to d etect and br eak symmetries in our original ILP form ulation from Section 2. This ﬂo w uses the to ol Sha tter (Aloul et al., 2003), wh ic h uses the Saucy (Darga et al., 200 4) graph automorphism program and the eﬃcient SBP construction from (Aloul et al., 2003) . W e also chec k for un brok en symmetries in form ulations p r o duced b y eac h of the instance- indep end en t constru ctions describ ed in Section 3. O ur ru n times for symmetry detection and f or solving th e r educed 0-1 ILP p roblems are rep orted in the n ext section. 4.2 Run times for Symmetry Detection and 0-1 I LP Solving T able 2 shows symmetry detectio n results and runtime s. The num b ers rep orted in the table are su ms of individu al results for all 20 b enc hmarks us ed . W e rep ort statistics as sums b ecause rep orting results for all of SBPs on all b enc hmarks would b e space-consuming, and 268 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring SBP CNF Stats Sym. Stats (SAUCY) T yp e #V #CL # PB #S #G Time no SBPs 4 37K 777505 3193 1.1e+168 994 18 5 NU 437K 777885 3193 5.0e+149 614 49 CA 437K 777505 3630 5.0e+149 614 49 LI 870K 4 019980 3193 2.0e+01 0 84 SC 437K 777545 3193 3.0e+164 941 167 NU+SC 437K 777925 3193 5.0e+148 597 47 T able 2: CNF formula s izes, symm etry detection res ults and run times, totaled for 20 b enchmark s from T able 1, w ith K = 20 . NU = n ull- color el imina- tion; CA = cardinality-based; LI = low es t-index; SC = sel ectiv e coloring . F or the LI SBPs, o ne in- stance o f the “do-nothing ” symmetry is coun ted in eac h case, giving a total of 20 symmetries and 0 generators. Saucy is run on an Intel Xeo n dual pro cessor at 2 GHz running R edHat Lin ux 9.0. w ould also not illustrate trend s as clearly . Th is w ork is concerned with c haracterizing the broad impact of symmetry breaking. Ho w ev er, w e sh o w detailed results for the que ens instances in the App end ix. The ﬁ rst column in the table indicates th e typ e of construction: w e use no SBPs for the basic formulatio n, NU for null-c olor elimination, CA for cardinalit y-based ordering, LI for lo we st-index ordering, and SC for selectiv e coloring (the last ro w sho ws NU and SC in com bination). The next three column s sh o w the num b er of v ariables, CNF clauses, and PB c onstraints in the problems. The last three columns sho w th e num b er of symmetries, n umb er of symmetry generators, and symmetry detection runtimes for Saucy . Henceforth, w e will refer to instance-dep endent S BPs as e xternal , b ecause they are add ed to an in- stance after symmetries are detected and are not p art of the p roblem form ulation. The top ro w is separated from the b ottom 5 r o ws b ecause it repr esen ts statistics without instance- indep end en t S BPs. W e observe that adding ins tance-indep endent SBPs dur ing pr ob lem form ulation d o es cut do wn the sym m etry detection ru n time considerably . Saucy has a to- tal runtime of 185 seconds when no instance-indep end en t SBPs are added, bu t its runtimes with NU, CA, LI and NU + SC constr u ctions are muc h smaller. Only the SC constru ction has a co mparab le r unti me b ecause it is a heuristic and breaks very few symmetries. The columns sho wing num b ers of sym metries and generators supp ort this observ ation: the NU, CA, LI and NU + S C constru ctions all hav e far few er symmetries than the top ro w, bu t the SC construction has almost the same n umber. F or these b enc hmarks, th e LI constru ction, breaks al l symm etries, ev en ins tance-dep endent v ertex p erm utations that ma y exist in a graph. Sa ucy rep orts ﬁnding no symmetries for this construction (except one instance of the do-nothing sym metry for eac h graph, wh ich is trivial). Ho w ev er, Saucy runtimes for this construction are larger than for the NU, CA and NU + S C constructions (85 seconds to appro ximately 49 seconds) ev en though ther e are no symmetries in the instances after LI 269 Ramani, Aloul, Mark ov, & Sakal lah SBP PBS II, PB Learning CPLEX Galena Pueblo T yp e Orig. w/i.-d. SBPs Orig. w/i.-d. SBPs Orig. w/i.-d. S BPs Orig. w/i.-d. SBPs Tm. #S Tm. #S Tm. #S Tm. #S Tm. #S Tm. #S Tm. #S Tm. #S no SBPs 17K 3 4.2K 16 6.3K 14 13K 7 1.7K 2 3K 17 18K 3 1.6K 19 NU 8. 2K 13 7.5K 13 5.9K 15 6.5K 15 8.3K 11 6.7K 11 9.1K 12 8.3K 13 CA 13K 6 12K 8 11K 11 11K 10 1 9K 1 17K 3 9K 12 10K 12 LI 15 K 6 15K 6 16K 4 16K 4 15K 5 15K 5 16K 5 16K 5 SC 14K 6 6 5 20 5.3K 15 12K 8 16K 4 94.4 20 15k 5 2.1K 18 NU+SC 6 .9K 14 6.8K 14 4.5 K 1 6 6.4K 14 6. 1K 14 6.1K 14 7.3K 13 7.1K 13 T able 3: Runt im e s and n um b er of solutions found b efore and after SBPs are added for all constructions usin g PBS I I (with PB learning), CPLEX, Galena and Pueblo; all exp erim e n ts are run on SunBlade 1000 worksta tions . Tim e outs for all solvers w ere set at 1000s. The maximum color limi t is set at 20, instances with k > 20 are uns atisﬁable un de r these formulations. This i s not a comparison of so lv ers. W e sol ve ILP formulations with e qual optimal v alues using diﬀ eren t solv ers to weed out sol v er-sp eciﬁc iss u e s. Best results for a given sol v er are shown in b ol dface. In the en tries, K deno tes multiples of 1000s seconds rounded to the ne arest in teger. predicates are added. A lik ely reason for this is the sharp increase in instance size caused b y the LI construction. In general, the SC construction has v ery little eﬀect on the num b er of s ymmetries - when used by itself, it lea ve s most symmetries in tact, and when used with the NU construction, th e imp ro v emen t o ve r the NU constru ction alone is ve ry small. T able 3 sho ws the eﬀect of symm etry breaking on run times of PBS I I (Aloul et al., 2002), CPLEX (ILOG, 2000), Galena (Chai & Kuehlmann, 2003) and Pueblo (Sheini, 2004). The ﬁrst co lumn in the table sp eciﬁes the construction type, follo w ed b y the total run time for eac h solv er (with and without the add ition of instance-indep endent SBPs) and the n umber of instances solv ed for the construction. F or eac h solv er, the b est p erformance among a ll conﬁgurations (largest num b er of instances solved and corresp onding runtime) is b oldfaced. Results are give n ﬁ rst for the new version of PBS, PBS I I based on (S heini, 2004), follo wed b y CPL E X, Galena and Pu eb lo. Runt imes for the older version of PBS can b e obtained from our earlier work (Ramani et al., 2004). T o compare p erformance of an ind ividual s olver for diﬀeren t constructions, obser ve th e r unt ime and solution en tries for diﬀerent rows in the same column , and to compare p erformance for diﬀeren t solv ers on the same constru ctions, observ e n umbers for the same ro w across all columns. W e obser ve the follo wing tr en ds. 1. All b enc hmarks p ossess a large n umber of symmetries. Diﬀerent instance-indep endent SBPs ac hiev e v arying degrees of completeness: the lo west- index ord ering (LI) br eaks all symm etries in the b enc hmarks u sed, while the selectiv e coloring (SC) S BP breaks the few est symmetries. Saucy runtimes f or residu al sym metry detection after the addition of instance-indep endent SBPs are highest for the no SBPs construction and the SC construction, since they p ossess th e largest num b ers of symmetries 2. F or the case w here no SBPs of an y kind are added, C PLEX p erforms wel l, s olving 14 out of 20 instances with in th e time limit. Ho w ev er, PBS I I, Galena and Pueblo p erform p o orly - Galena s olves only 2 instances and PBS I I and Pueb lo eac h solv e 3 270 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring 3. PBS II, Gale na and Pu eb lo b eneﬁ t considerably fr om instance-dep en d en t sy m metry breaking. When instance-dep enden t SBPs are used without an y of the instance- indep end en t constructions we prop ose, PBS I I solve s 16 instances w ithin the time limit, while Galena and Pueblo solve 17 and 19 instances r esp ectiv ely . Ho w ev er, CPLEX is hamp ered b y th e addition of instance-dep end ent SBPs, and solv es only 7 instances in this case 4. Adding only instance-indep endent SBPs im p ro v es p erformance for all sp ecialized 0- 1 IL P solv ers o v er the no-SBP version. The b est p erf orm ance for PBS I I, Galena and Pueb lo is seen for the NU + SC constru ction - PBS I I and Galena solv e 14 instances, and Pueblo solv es 13. F or CPLEX, the NU + S C constru ction sh ows marginal improv emen t o v er the n o-SBPs case (16 instances are solv ed), b ut the more complex constru ctions, CA and L I , actually undermin e p erformance - CPLEX solve s only 4 instances with the LI construction. In general, complex SBP constructions p erform muc h worse than simple ones. PBS I I, Pu eblo and Galena also p erform p o orly with the CA and LI constructions - Galena solv es only 1 in stance with th e CA construction with no help f r om instance-dep endent SBPs, and v ery few ins tances are solv ed with th e LI construction for an y solv er 5. Adding instance-indep endent S BPs alone do es n ot s olve as many instances as adding instance-dep endent S BPs to th e SBP-free form ulation. The b est p erformance seen with instance-indep enden t SBPs is 14 instances solv ed, by Ga lena and PBS I I, and 16 instances solv ed by CPLEX, with the NU + S C construction. When instance- dep end en t SBPs are added PBS I I and Galena solv e all 20 instances with the SC construction. T he CA and LI constructions lea v e v ery few (or n one at all) s ymmetries to b e b r ok en by instance-dep endent SBPs. C onsequen tly , there is almost no diﬀerence in results with and without instance-dep endent SBPs for these constructions. How- ev er, they d o not ac hiev e the same p erformance impro ve ments as in stance-dep endent SBPs, d u e to their size and complexit y 6. Using instance-dep enden t S BPs in conjun ction with the S C construction is useful. With this combinatio n, PBS I I and Galena solve all 20 instances with in the time limit, and Pu eblo solv es 18. Runtime is also considerably impro v ed for PBS I I and Galena – PBS I I solves all 20 instances in a tota l of 65 seconds, and Galena in 94.4 seconds. The b est o v erall p erformance, in terms of num b er of solutions and run time, is seen with this combinatio n. In general, how ev er , the SC constru ction is not dominant on its o wn. Results for the SC construction alone are v ery similar to results with no SBPs, an d results for th e NU + SC combinatio n are ve ry similar to those ac hiev ed b y using only NU SBPs. The SC constr u ction is eﬀectiv e at “b o osting” the p erformance of other constructions 7. The three sp ecialized 0-1 ILP solv ers - PBS I I, Galena and Pueblo, exhibit the same p erformance trend s w ith resp ect to the constructions used, and their p erformances are all comparable, in terms of b oth the num b er of solutions foun d and runtime Th is indicates that the v ariations in p erformance are due to the diﬀerent SBPs, n ot due to diﬀering solve r implemen tations. All solv ers are indep endent implement ations based 271 Ramani, Aloul, Mark ov, & Sakal lah SBP PBS I I, PB Learnin g CPLEX Galena Pueblo T yp e O rig. w/i.- d. S BPs Orig. w/i.-d. SBPs Orig. w/i.-d. SBPs Orig. w/i.-d. S BPs Tm. #S Tm. #S Tm. #S Tm. #S Tm. #S Tm. #S Tm. # S Tm. #S No SBPs 18K 2 6.2K 14 11K 9 8.2K 12 19K 1 9.1K 11 1 9K 1 7.5K 13 NU 9.2K 12 7.9K 13 11K 9 12K 8 10K 10 7.6K 13 11K 11 9.5K 11 CA 13K 7 13K 9 13K 9 14K 8 19K 1 17K 4 11K 11 13K 8 LI 15K 5 15K 5 19K 2 19K 2 16K 5 16K 5 17K 3 17K 3 SC 15K 5 5.3K 15 10K 10 12K 9 16K 4 5.3K 15 16K 4 6.0K 15 NU + SC 7 .1K 13 7.0K 13 9. 7K 11 9.9K 11 9.2K 12 6.9K 14 8.0K 13 7.4K 13 T able 4: T otal runtimes and n um b er o f solutions found b efo re and after SBPs are added for all constructions using PBS I I (wi th PB learning), CPLEX, Galena and Pueblo. The exp eri men tal setup is the same as that used in T able 3 but with a colo r lim it of K = 30 . Best results for a sol v er are b ol d- faced. F ewer instances are solved than i n T able 3 b e cause the hi gher color limit results in larger and p oten tially more diﬃcult instances. on the same algorithmic framew ork (the Da vis-Logemann-Lo v eland bac ktrac k searc h pro cedur e), but PBS I I and Galena also ha ve learning capabilitie s 8. Adding instance-dep endent S BPs to any construction us u ally adv ersely aﬀects the p erformance of CPLEX. This h as b een previously n oted in other w ork (Aloul et al., 2004) . Since the CPLEX algorithms and implementat ion are n ot a v ailable in the public domain, it is diﬃcult to accoun t for this eﬀect. How ev er, PBS and Galena with symmetry b reaking s igniﬁcan tly outp erform CPLEX without symmetry b r eaking 9. W e rep ort results as the sum of runtimes for all instances to illustrate trends. On a p er -in s tance basis, the same trend s are d ispla y ed. F or example, for the no-SBPs case in the top ro w, PBS I I solv es 3 instances and Galena solve s 2, but the tw o instances solv ed b y Galena are among th ose solv ed b y PBS I I. In general, the same instances tend to b e “easy” or “diﬃcult” for th e 0-1 ILP solv ers, although C P LEX b ehav es d iﬀeren tly . An example of this b eha vior for th e que ens family of in stances is illustrated in the App end ix Ov erall, the results suggest that for graph coloring, addin g instance-indep enden t SBPs alone is not comp etitiv e with the use of ins tance-dep enden t S BPs alone. The b est results are ac hiev ed usin g a com bination of b oth typ es, and even here, the instance-indep endent SBPs used are the most simple v ariet y . This is true ev en when sym m etry detection runtimes are tak en into consideratio n. W e attribu te this result to the complexit y of instance-indep end ent SBPs w e use, and also to th e fact that impr o v emen ts in graph automorphism soft wa re (Darga et al., 2004) ha v e greatly reduced the o ve rhead of detecting symm etries b y reduction to graph automorphism. P reviously , f or static appr oac hes that require symmetries to b e detected and broken in adv ance, the task of symmetry detection was often a b ottlenec k that could actually tak e longer than the search itself. With this b ottlenec k remov ed, the adv antag es of static symmetry breaking - simple pr ed icates th at address sp eciﬁc symmetries rather than complex constructions that alter the problem sp eciﬁcation considerably - are more clearly illustrated. Ev en among instance-indep endent pr edicates, simple constructions are m ore eﬀectiv e than complex ones. As we hav e noted in Section 3, simple constructions 272 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring lik e NU and S C add v ery few add itional constrain ts and d o not alter the original p roblem greatly . How ev er, the CA and LI constru ctions ad d many more constrain ts, which ma y confuse the sp ecializ ed 0-1 ILP solv ers. It is imp ortan t to note that color p erm utations, wh ile instance-indep endent , d o ap- p ear at the instance-sp eciﬁc lev el. Thus, the symmetries targeted b y in stance-indep endent predicates are a sub set of th ose targeted by instance-dep end en t p r edicates. Our ins tance- indep end en t constr u ctions are not in tended to co v er a diﬀeren t set of symmetries, b ut rather to break s ome of the same symmetries durin g problem form ulation, th us reducing or elim- inating th e o ve rhead of any instance-dep end ent metho ds th at may follo w. The fact that this strategy is not successful su ggests that, for the same set of symmetries, the instance- dep end en t predicates w e u se are more eﬃcien t and easier for solv ers to tac kle. T o verify our claims ab out p er f ormance trends, we sh o w resu lts for an add itional set of exp eriments w ith increased color limit K = 30 in T able 4. T he instances are re-form ulated with K = 30 and with diﬀeren t SBP co nstru ctions. This exp erimen t is in tended to v erify trends from the K = 20 case, and to in v estigate whether instances with chromatic num b er > 20, that are un satisﬁable in the ﬁrst case, can b e colored with ≤ 30 colors. Resu lts from T ab le 4 v alidate our observ ations from T able 3 – the b est results for PBS I I, Galena and Pueblo are again ac hiev ed with the NU + SC (with no in stance-dep endent S BPs) and SC (with in stance-dep endent SBPs) constructions. Ho we v er, with this formulation fewer instances are solv ed than for the K = 20 case, p ossibly because the K = 30 limit results in larger instances. Al so, for instances whose c hromatic num b er is muc h closer to 30 than 20, it ma y b e hard er to pro ve optimalit y , whereas pro ving unsatisﬁabilit y for th e K = 20 exp eriments m a y b e simp ler. 4.3 Comparison with Related W ork Here, we discuss the emp irical p erformance of our approac h when compared with related w ork (Cou d ert, 1997 ; Benhamou, 2004). W e n ote that b oth cited w orks d escrib e algorithms sp eciﬁcally deve lop ed for graph coloring, and the searc h p ro cedures cannot b e used to solv e other problems. Our app roac h, on the other hand, solv es h ard p roblems b y red uction to generic problems s u c h as SA T or 0-1 ILP , and this work on graph colo ring can b e viewed as a case study . Consequently , w e use problem-sp eciﬁc kno wledge only during the actual problem form ulation (instance-indep endent S BPs are also add ed dur in g r eduction), b ut not dur ing searc h itself. This ma y b e useful for applications wher e problem-sp eciﬁc s olvers cann ot b e dev elop ed or acquired due to limited resour ces. Our goal is to determine whether symm etry breaking can imp ro v e the p er f ormance of redu ction-based metho d s, w h ic h are traditionally not comp etitiv e with prob lem-sp eciﬁc metho ds . Thus, while our techniques may n ot b e sup er ior to all problem-sp eciﬁc solve rs on all instances, w e hop e to sho w reasonably s tr ong p erformance o v er a broad sp ectrum of instances. Common data p oints b et w een our wo rk and Coud ert’s (Coudert, 1997) include instances of q ueens , myciel and DSCJ125. 1 . Referrin g to our detailed resu lts for queens ins tances in the App endix, w e note th at our runtimes are comp etitiv e with those of Coudert’s algorithm - for example, on que en5 5 , b oth algorithms ha v e a runtime of 0.01s. On larger ins tances, ho w ev er, our run times are somewhat slow er. On the myciel instances, we obtain the b est results with the Pueblo solve r and th e SC predicates, with ru n times of 0.01, 0.06, and 1.80s 273 Ramani, Aloul, Mark ov, & Sakal lah on my ciel 3, 4, and 5, compared with 0.0 1, 0.02 and 4.17 for C oudert’s algorithm. Therefore, it app ears that our app roac h is comp etitiv e on these common data p oin ts. Moreo v er, other studies (Kiro vski & Po tko njak, 19 98) h a v e observed that Couder t’s wo rk d o es not pro vide results for sev eral hard r eal-wo rld p roblem classes, particularly those where m o deling results in dense graphs. Our w ork is more general, and cannot b e b iased to fa v or certain t yp es of graphs. The algorithm describ ed by Benhamou (Benhamou, 2004 ) shows very comp etitiv e run- times on a num b er of DIMA CS b en chmarks, particularly instances of register allocation. F or example, the DSJC 125.1 instance is solved by Benh amou’s a lgorithm in 0.01 seconds, while the b est time achiev ed b y us is 1.12 seconds, usin g the Pueblo solv er with only instance-dep endent SBPs. Ho w ev er, we note that Benh amou’s algorithm d etermin es the upp er limit for the c hromatic num b er K using more instance-sp eciﬁc kno wledge, for exam- ple, for DS CJ125.1 , it is s et at K = 5. W e solv e all instances with K = 20, w h ic h ma y be to o large a limit in some cases. The v alue of K aﬀec ts the size of th e resulting 0-1 ILP reductions and S BPs, w hic h is like ly to aﬀect runtime. W e also n ote that the DIMA CS b enchmarks used in the cited w ork (Benhamou, 2004) are primarily r egister allocation and randomly generated instances, whereas we ac hieve reasonably goo d p erformance on a w ide v ariety of b enc hmark applications. Moreo v er, Benhamou’s approac h relies on mo d eling graph colo ring as a n ot-equals CSP , wh ic h does not bo de well for generalit y . Many C SPs cannot b e mo deled using on ly not-equals constraints. Additionally , the sym metry d etec- tion, b reaking and searc h pro cedur es describ ed in that w ork are sp eciﬁc to graph coloring, whereas our work can b e extended to several other problems, only requiring a reduction to SA T/0-1 ILP . 5. Conclusions Our w ork sh ows that problem reduction to 0-1 IL P is a viable metho d for optimally solving com binatorial pr oblems without inv esting in sp ecializ ed solv ers. Th is approac h is lik ely to b e eve n more successful as the eﬃciency of 0-1 ILP solv ers impro v es in the futur e, and as they are able to b etter hand le problem stru ctur e. In particular, p roblem redu ctions ma y pro du ce highly-structur ed in stances making the abilit y to automatically detect and exploit structure ve ry imp ortan t. In the case of graph coloring we d emonstrate that a generic, publicly-a v ail able s y m metry breaking ﬂo w from our earlier w ork (Aloul et al., 2004) signiﬁ- can tly improv es empirical r esults in conjunction with the academic 0-1 IL P solve rs PBS I I, a n ew v ersion of the solv er PBS (Aloul et al., 2002), Galena (Chai & Kuehlmann, 2003) and Pueblo (Sheini, 2004). All sp ecialized 0-1 ILP solv ers signiﬁcan tly outp erform the commer- cial g eneric ILP solv er CPLEX 7.0 when symmetry-breaking is used. The p erformance of CPLEX actually d eteriorates w h en S BPs are added, and on the original instances with no SBPs, CPLEX is able to solv e m ore in stances than the 0-1 ILP solv ers. How ev er, the b est p erformance ov erall is obtained with the 0-1 ILP solv ers on instances with S BPs added. Although our tec hniqu es are tested on stand ard DIMA CS b enc hmarks instances, w e note that the symmetry-breaking ﬂo w d escrib ed h ere can b e applied to graph coloring instances from any application. W e are particularly in terested in comparing strategies for b reaking sym metries that are present in ev ery ILP instance pro du ced b y problem reduction (instance-indep enden t sym- 274 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring metries). Suc h sym m etries may b e kn o wn ev en b efore the ﬁ rst in stances of the original problem are delive red (i.e., symmetries ma y b e detected at the sp eciﬁcatio n lev el), and one has the option to use them d uring problem redu ction. Int uitive ly , this may prev ent disco v ering these symm etries in every instance and thus imp ro v e the o v erall CPU time. T o this end , we p r op ose four constructions for instance-indep endent symmetry b r eaking pred- icates (S BPs). These constructions v ary in terms of strength and completeness. Our goal in exp er im ents wa s to compare the p erformance of the four instance-indep endent S BP con- structions relativ e to eac h other, as well as to assess their p erformance w h en compared with instance-dep endent SBPs. In stance-indep endent SBPs h a v e the adv an tage of not r equiring the add itional step of symmetry detection, since th ey are part of the pr oblem sp eciﬁcation. Additionally , they are designed with more inf ormation ab out the p roblem itself, and their eﬀect on solutions is clear - for example, we know that n ull-color elimination will force all th e lo w er-n umb ered colors to b e used in a solution. Instance-dep end en t SBPs are d etected and added automatically on the 0-1 ILP r eduction of an instance without an y un derstanding of their signiﬁcance. On the other h and, in stance-dep endent constructions are less com- plex and resu lt in more compact predicates. Our empirical data indicate that simplicit y of construction is a more p ow erful factor in determining p erformance - instance-dep endent SBPs consisten tly outp erform instance-indep enden t SBPs, and the most complete and com- plex instance-indep end en t constructions (LI) are actually the weak est in p erformance. It is clear from our results that symmetry br eaking itself is usefu l in graph coloring: adding instance-dep endent SBPs alw a ys sp eeds up searc h o v er the no-SBPs case. It is lik ely that instance-indep endent SBPs are less successful du e to their complex construction. Simpler instance-indep endent constr u ctions (NU, SC) outp erform the more complex ones (C A, LI). It is w ell kno wn that the s yn tactic structure of CNF and PB constraints ma y d ramatically aﬀect the eﬃciency of SA T and ILP solv ers. Sh orter clauses an d PB constraints are m uc h preferable as they are easier to resolv e against other constrain ts, and are more useful to the learning strategie s employ ed by exact SA T solvers. Another factor that giv es instance- dep end en t SBPs the adv an tage is the ease of symmetry d etection, wh ic h wa s previously a b ottlenec k. Due to improv ed soft w are (Darga et al ., 2004), the o v erhead of symmetry de- tection via redu ction to graph automorphism in SA T/0-1 ILP instances is almost n egligible. W e also show that the three sp ecialized 0-1 ILP solv ers, PBS I I, Galena and Pu eblo, all exhib it similar p erformance trend s for d iﬀeren t constructions. This indicates that p er- formance is not decided by s olv er-sp eciﬁc issu es, but by the d iﬃcult y of the instances a nd the SBPs added to them. CP LEX d o es not disp la y the same b eha vior as the other solv ers, and is in fact slo w ed down by the addition of instance-dep end en t SBPs and by sev eral instance-indep endent constructions. CPLEX is a commercial sol ve r f or ge neric ILP prob- lems, and its algorithms and decision heuristics are lik ely to b e v ery diﬀerent than those used b y academic solv ers. How ev er, since details ab out CP L EX are not publicly a v ailable, it is not p ossible to accurately exp lain its b ehavio r. W e do note th at while CPLEX does not app ear to b eneﬁt from symmetry breaking, its p erformance on the reduced instances with no SBPs of any kin d is sup er ior to the 0-1 ILP solvers. How ever, once SBPs are added the sp ecialized solv ers solv e more instances than CPLEX in less time. In the con text of generic searc h and combinato rial optimization problems d eﬁ ned in the NP-sp ec language (Cadoli et al., 1999), our empirical data suggest th at new theoretical breakthroughs are r equ ired to mak e use of instance-indep endent s y m metries during p roblem 275 Ramani, Aloul, Mark ov, & Sakal lah reductions to SA T or 0-1 ILP . At our current lev el of understand ing, the simple strategy of pro cessing instance-i nd ep endent and instance-dep endent symmetries together pro du ces smallest ru n times f or graph co loring b enc hmarks. Our curr en t and future w ork is fo cused on develo ping more eﬀectiv e S BPs for th is pr ob lem, and also in ve stigating the utilit y of symmetry breaking for other hard search prob lems. Moreo ve r, w hile our w ork uses in stance- indep end en t p redicates only for color symmetries, our results and analysis ma y ha v e broader scop e, for example, in app lications suc h as r ad io f r equency assignment (Sec tion 2) where symmetries are introd uced during the reduction to graph coloring and are lik ely to b e preserve d du ring f u ture r eductions. The issues inv olv ed in u sing instance-dep endent vs. instance-indep endent SBPs are v ery relev ant to suc h app lications. 6. Ac kno wledgmen ts This w ork w as fu nded in part b y NSF ITR Grant #0205288 . Also, w e thank Donald Ch ai and Andr eas Kuehlmann from UC Berkele y for providing us with binaries of the Galena solv er, and Hossein Sheini f or pr o viding us with th e b inaries for Pueblo. References Aloul, F. A., Mark o v, I. L., & Sak alla h, K. A. (2003). S h atter: Eﬃcient s y m metry-breaking for b o olean satisﬁabilit y . In International Joint Confer enc e on A rtiﬁcial Intel ligenc e , pp. 271 –282. Aloul, F. A., Marko v, I. L., & Sak allah, K. A. (2004). MINCE: A s tatic global v ariable- ordering heur istic for s at search and b dd manipulation. Journal of U niversal Computer Scienc e (JUCS) , 10 , 1562– 1596. Aloul, F. A., Ramani, A., Marko v , I. L., & S ak allah, K . A. (2002). Generic ILP versus sp ecialized 0-1 ILP : An up date. In International Confer e nc e on Computer-Aide d Design , p p. 450–457. Aloul, F. A., Ramani, A., Marko v, I. L., & S ak allah, K . A. (2003). Solving diﬃcult instances of b o olean satisﬁabilit y in the p resence of symmetry . IEEE T r ansactions on CAD , 22 , 111 7–1137. Aloul, F. A., Ramani, A., Mark o v, I. L ., & Sak allah, K. A. (2004). Symmetry-breaking for pseudo-b o olean formula s. In Asia-Paciﬁc D esign Automat ion Confer e nc e , pp. 884– 887. Aragon, C. R., John son, D. S., McGeo c h, L. A., & Schev on , C. (1991). O ptimization by sim ulated ann ealing: An exp erimen tal ev aluatio n; part ii, graph coloring and num b er partitioning. Op er ations R ese ar ch , 39 , 378– 406. Benhamou, B. (2 004). S ymmetry in n ot-equals binary constrain t net w orks. In Workshop on Symmetry i n CSPs , pp. 2–8. Brelaz, D. (1979 ). New metho ds to color ve rtices of a graph. Communic ations of the ACM , 22 , 251 –256. Bro wn, R. J. (1972 ). Ch romatic sc hedulin g and the c hromatic num b er problem. Manage- ment Scienc e , 19 , 451–4 63. 276 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring Cadoli, M., & Mancini, T . (2003). Detec ting and breaking symmetries on sp eciﬁcations. In The Thir d Annual Workshop on Symmetry in Constr aint Satisfaction Pr oblems (SymCon) , p p. 13–26. Cadoli, M., Pa lop oli, L., Schaerf, A., & V asileet, D. (1999). NP-SPEC: An executable sp ec- iﬁcation language for solving all problems in NP. In Pr actic al Asp e cts of D e c lar ative L anguages , pp . 16–30. Chai, D., & Kuehlmann, A. (2003). A fast p seudo-b o olean constraint solv er. In D esign Autom ation Confer e nc e , pp. 830–835. Chaitin, G. J., Auslander, M., Chandr a, A., Co c k e, J., Hopkins, M., & Markstein, P . (1981) . Register allocation via coloring. Computer L anguages , 6 , 47–57. Chams, M., Hertz, A., & W erra, D. D. (1987 ). S ome exp erimen ts w ith simulated annealing for coloring graphs. Eur op e an J ournal of Op er ations R ese ar ch , 32 , 260 –266. Coudert, O. (1997). Coloring of real-life graphs is easy . I n Design Automat ion Confer enc e , pp. 121 –126. Cra wford, J. (1992). A theoretical analysis of reasoning b y symmetry in ﬁrst-order logic . In AAAI Workshop on T r actable R e asoning at the T enth National Co nfer enc e on Artiﬁcial Intel lig enc e . Cra wford, J., Ginsb erg, M., Luks, E., & Ro y , A. (1996 ). Symm etry-breaking pred icates for s earch problems. In 5th International Confer e nc e on Principles of Know le dge R epr esentation and R e asoning , p p. 148–159 . Culb er s on, J. (2004). Graph coloring page. htt p://web.cs.ualb erta.ca /˜jo e/Co lo ring/index.html . Darga, P . T., Liﬃton, M. H., S ak allah, K. A., & Mark o v, I. L. (2004). Exploiting structure in symmetry generation f or cnf. In 41st Internation Design Automation Confer enc e , pp. 530 –534. Da vis, M., L ogemann, G., & Lo v eland, D. (1962). A machine program for theorem p r o ving. Communic ations of the ACM , 5 , 394–3 97. F ahle, T., Sc ham b er ger, S ., & Sellmann , M. (2001). Symmetry breaking. In 7th International Confer enc e on Principles and Pr actic e of Constr aint Pr o g r amming , pp. 93–1 07. F eige, U., Goldwa sser, S., Lo v asz, L., Safra, S., & Szege, M. (199 1). Appro ximating clique is almost NP-complete. In IEEE Symp osium on F oundations of Computer Scienc e , pp. 2–1 2. F o cacci, F., & Milano, M. (2001). Global cut f r amew ork for removi ng sy m metries. In Principles and Pr actic e of Constr aints Pr o gr amming , pp. 77–82. Galinier, P ., & Hao, J. (1999) . Hybrid evolutio nary algorithms for graph coloring. J ournal of Combinatorial O ptimization , 3 , 379–397. Garey , M. R., & Johnson, D. S. (1979). Computers and Intr actability: A Guide to the The ory of NP- c ompleteness . W. H. F reeman and Company . Gen t, I. P . (200 1). A symmetry-breaking constrain t for ind istinguishable v alues. In Work- shop on Symmetry in Constr aint Satisfaction Pr oblems . 277 Ramani, Aloul, Mark ov, & Sakal lah Goldb erg, E., & Novik ov, Y. (2002). Berkmin : A fast and robu st SA T -solv er. In Design Autom ation and T est in Eur op e , p p. 142–149 . Haldorsson, M. M. (1990). A still b etter p erformance guarant ee for appr o ximate graph coloring.. Hen tenryc k, P . V., Agren, M., Flener, P ., & Pearson, J. (20 03). T ractable s y m metry breaking for CSPs with in terc hangeable v alues. In The Interna tional Joint Confer e nc e on Artiﬁcial Intel lig enc e (IJCAI) . Hertz, A., & W erra, D. D. (1987). Using tabu searc h tec hn iques for graph coloring. Com- puting , 39 , 345–351. Huang, J., & Darwic he, A. (2003) . A structure-based v ariable o rd er in g heuristic for SA T. In The Internatio nal Joint Confer e nc e on Artiﬁcial Intel ligenc e , p p . 1167–117 2. ILOG (2000). ILOG CPL EX ILP solv er, version 7.0. h ttp://www.ilo g.com/pro ducts / cplex/ . J.-P . Hamiez, J.-K. H. (2001). Scatte r searc h for graph coloring. In The 5th Eur op e an Confer enc e on A rtiﬁcial Evolution , pp. 168–179. Jagota, A. (1 996). An adaptiv e, m ultiple r estarts neural net w ork algorithm for grap h col- oring. Eur op e an Journal of Op er atio nal R ese ar ch , 93 , 257–270 . Kiro vski, D., & P otk onjak, M. (1998 ). Eﬃcien t coloring of a large sp ectrum of graph. In Design Automatio n Confer enc e . Krishnamurth y , B. (1985). S hort pro ofs f or tric ky form ulas. A cta Informatic a , 22 , 327–3 37. Kubale, M., & Jac k o wski, B. (1985). A generalized implicit en umeration alg orithm for graph coloring. Communic ations of the A CM , 28 , 412–4 18. Kubale, M., & Kusz, E. (1983). Compu tational exp erience w ith implicit en umeration algo- rithms for graph colo ring. In Pr o c e e dings of the WG’83 International Workshop on Gr aph The or etic Conc e pts in Computer Scienc e , p p. 167–176. Leigh ton, F. (1979 ). A graph coloring algorithm for large sc hedu ling p roblems. Journal of R ese ar ch of the N ational Bur e au o f Stand ar ds , 84 , 489–50 6. McKa y , B. D. (1990). Nauty us er ’s guide (version 1.5). http://cs.anu.edu.au/˜bdm/na ut y/ . Mehrotra, A., & T ric k, M. A. (1996). A column generation approac h for graph coloring. INFORMS J ournal on Computing , 8 , 344– 354. Mosk ewicz, M., Madigan, C., Zh ao, Y., Z hang, L., & Malik, S . (2001). Ch aﬀ: Engineering an eﬃcien t sat solv er. In Design Automation Confer enc e , pp. 530–535 . Mycielski, J . (1955). Sur le coloriage des graphs. Col lo qium Mathematicum , 3 , 161–162. Prest wic h, S. (2002). Su p ersymm etric mo delling for lo cal searc h. In SymCon: Workshop on Symmetries in CSPs , pp . 21–28 . Puget, J . (200 2). Symmetry br eaking revisited. In Principles and Pr actic e of Constr aints Pr o gr amming , p p . 446–461. Ramani, A., Aloul, F. A., Mark o v, I. L ., & Sak alla h, K. A. (2004). Breaking instance- indep end en t symmetries in exact graph coloring. In D esign A utomation and T est in Eur op e , p p. 324–329. 278 Breaking Inst ance-Independent Symmetries in Exact G raph Co loring Roney-Dougal, C. M., Gent , I. P ., K elsey , T., & Linto n, S. (2004). T ractable symm etry breaking using restricted searc h trees. In E ur op e an Confer enc e on Artiﬁcial Intel li- genc e , pp. 211–215. Sheini, H. (2004). Pueb lo 0-1 ILP solv er. h ttp://www.eec s .umich .edu/˜ hs heini/pueblo/ . Silv a, J . P . M., & Sak allah, K . A. (1999 ). GRASP: A new searc h algorithm for satisﬁabilit y . IEEE T r ansactio ns On Computers , 48 , 506–521. T rick, M. (1996). Net w ork resources for coloring a graph . http://mat.gsia.cmu.edu/COLOR/color.html . W alsh, T. (2001). Search on high degree graph s. In 17th International Joint Confer enc e on Artiﬁcial Intel lig enc e , pp. 266– 271. W arners, J. P . (1998). A linear-time transform ation of linear inequalities in to conjunctive normal form. Inf ormation Pr o c essing L etters , 68 , 63–69. W elsh, D. J. A., & Po well, M. B. (19 67). An upp er bou n d on the chromatic num b er of a graph and its application to timetabling p roblems. Computer Journal , 10 , 85–86. W erra, D. D. (1985). An in tro du ction to timetabling. Eur op e an Journal of Op er ations R ese ar ch , 19 , 151–162 . App endix A: Performa nce Analysis on Queens Instances This section provides a more detailed discussion of our results on ind ividual b en chmarks in the q ueens family of instances. T he problem p osed b y queen s instances is whether queens can be placed on an n × m c hessb oard without conﬂicts. The instances w e use in our exp erimen ts are queen s 5 × 5, 6 × 6, 7 × 7 and 8 × 12. T able 5 s ho ws results for the queens f amily . Results are sho wn for ev ery instance with no SBPs, with eac h of the four constructions NU, C A, L I and SC, and with th e NU + SC com bination. All constructions are tested with and without in s tance-dep endent SBPs as b efore. W e rep ort results f or the original version of PBS, from (Aloul et al., 2002), and for PBS I I, C PLEX, Galena and Pueblo as in Section 4. Exp erimen ts are r un on Sun Blade 1000 w orkstations as b efore. In the table, w e r ep ort solv er runtime if an ins tance is solv ed, and T/O for a timeout at 1000 seconds. The b est results for a solv er on a particular instance are b oldfaced. While there is greater v ariation wh en considering p erformance on a p er-instance basis, the table largely reﬂects the same trends r ep orted in Section 4. F or example, when no instance-dep endent SBPs are used, PBS, PBS I I, Galena and Pueblo all largely p erform b est w ith the NU + SC construction. Wh en instance-dep endent SBPs are added, the b est p erformance is seen with the SC construction in most cases. CPLEX do es not disp la y the same b eha vior as the other solv ers, and its p erformance clearly d eteriorates when ins tance- dep end en t SBPs are add ed to any construction. A similar eﬀect has b een observ ed in related w ork (Aloul et al., 2004). R esu lts for the original version of PBS (Aloul et al., 2002), which could not b e included in Section 4, hav e b een added in this section. It can b e seen that PBS f ollo ws the same trends as PBS I I, Galena and P ueblo, reinforcing our claim that this b ehavio r is n ot solv er-dep end en t. 279 Ramani, Aloul, Mark ov, & Sakal lah PBS PBS II CPLEX Galena Pueblo Inst.-dep. Inst.-dep. Inst.-dep. Inst.-dep. Inst.-dep. Inst. SBP SBPs used? SBPs used? SBPs used? SBPs used? SBPs used? Name Type No Y es No Y es N o Y es No Y es No Y es no SBPs T/O 0.19 3 4.52 0.04 1.11 643.93 83.06 0.35 203.09 0.01 NU 1.84 T/O 0.01 0.02 1.38 23.67 0.21 0.27 0.08 0.1 queen5 5 CA T/O T/O 0.31 0.24 39.2 2.76 T/O T/O 0.14 0.52 LI 135 134.71 1.48 1.48 262.96 217.21 5.4 5.4 8.48 8.48 SC 15.99 0.19 0.15 0.07 0.45 229.79 0.29 0.29 0.25 0.19 NU + SC 8.63 12.34 0 0.01 0 .83 0.88 0. 3 1 0.06 0.07 no SBPs T/O 3.61 T/O 0.21 T/O T/O T/O 0.87 T/O 0.49 NU 331.63 521.12 56.63 13.59 T/O T/O 192.17 19.11 123.99 18.88 queen6 6 CA T/O T/O 50.6 780.57 T/O T/O T/O T/O 196.94 80.53 LI T /O T/O T/O T/O T/O T/O T/O T/O T/O T/O SC T/O 0.5 8 T/O 0.1 242.79 T/O T/O 1.0 T/O 0.32 NU + SC 2.89 1.72 1.4 0.63 95.91 T/O 11 .19 1 .05 4. 85 2. 64 no SBPs T/O 36.56 T/O 1.79 243.3 T/O T/O T/O T/O 1.13 NU 0.45 3 .29 36.31 24.74 119.16 459.44 56.6 147.52 9.59 15.49 queen7 7 CA T/O T/O T/O T/O 271.2 T/O T/O T/O 692.67 150.86 LI T /O T/O 53.3 53.4 T/O T/O 78.85 78.8 212.18 213.8 SC T/O 8.42 38.57 0 .85 38.04 T/O T/O 1.33 217.82 1.23 NU + SC 5.65 38.07 4.37 5 .73 119.7 T/O 17.46 5.16 25.73 14.04 no SBPs T/O 1.31 T/O 0.52 T/O T/O T/O T /O T/O T/O NU T/O T/O T/O T/O T/O T/O T/O 138.61 T/O T/O queen8 12 CA T/O T/O T/O T/O T/O T/O T/O T/O T/O T/O LI T /O T/O T/O T/O T/O T/O T/O T/O T/O T/O SC T/O 1.0 5 T/O 0.47 T/O T/O T/O 1.9 T/O 0.98 NU + SC T/O T/O 787.26 780.14 T/O T/O 52.1 53.63 T/O T/O T able 5: Detailed results for qu eens instances. F or each ins tance, w e show resul ts for the sol v ers PBS, PBS I I, CPLEX, Galena and Pueblo. All solv ers are run o n SunBlade 1000 workstat ions . Instances are te s ted with no instance-indep e nden t SBPs, w i th eac h of the four prop os e d constructions in Se ctio n 3 and with a combination of the NU and SC constructions. All instance-indep e nden t SB Ps are tested alone and with instance-dep enden t SBPs added. The table sho ws the run time for a giv en instance under dif- feren t construction. T/O indicates a timeout at 1000 seconds. Best results for a giv en sol v er on eac h instance are shown in b oldface. 280 ... 1

Breaking Instance-Independent Symmetries In Exact Graph Coloring

Original Paper

Comments & Academic Discussion

Leave a Comment

Original Paper

Related Papers

Comments & Academic Discussion

Leave a Comment