Analysis of Estimation of Distribution Algorithms and Genetic Algorithms on NK Landscapes

Analysis of Estimati on of Distribution Algo rithms and Genetic Algo rithms on NK Land- scap es Ma rtin Pelik an MED AL Rep o rt No. 20080 01 January 20 08 Abstract This study analyzes performance of several genetic and evolutionary algorithms on randomly g enerated NK fitness landscap es with va rious values of n and k . A large n u mber of N K p roblem instances ar e first generated for each n and k , and the g lobal optimum of e a c h instance is obtained using the branch-and-b o und alg o rithm. Next, the hierarchical Ba y esian optimizatio n algorithm (hBO A), the uni variate marginal d istribution alg o rithm (UMDA), and the simple genetic algorithm (GA) with uniform and tw o -po int crossover operators a re applied to all generated instances. Perf ormance o f all algorithms i s then analyzed and compared, and the results a re discus s ed. Keyw o rds NK fitne s s lands cape, hiera rchica l BO A, genetic algorithm, branch and bound, pe rfo rmance analysis, scalabil i t y , lo cal search, cross o ver. Missouri Estimatio n of Distribution Algorithms Lab or atory (MEDAL) Department of Mathematics and Co m p uter Science University o f M i ssouri–St. Louis One University Blvd., St. Louis, MO 6312 1 E-mail: medal @cs.umsl .edu WWW: htt p://meda l.cs.umsl.edu/ Analysis of Estimation of Distribution Algorithms and Genetic Algorithms on NK Landscap es Martin Pe lik an Missouri Estimation of Distribution Algorithms Lab oratory (MED AL) Dept. of Math and Computer Science, 320 CCB Univ ersit y of Missouri at St. Louis One Univ ersit y Blvd., St. Louis, MO 63121 pelikan@cs. umsl.edu Abstract This study ana lyzes p erformanc e of several genetic and evolutionary algo rithms on ra ndo mly generated NK fitness la ndscapes with v arious v alues of n and k . A large num ber of NK pro b- lem instances ar e first g enerated for ea ch n and k , and the globa l optimum o f each insta nce is obtained using the branc h-and-b ound algo rithm. Next, the hierarchical Bay es ia n o ptimization algorithm (hBOA), the univ ariate marg ina l distribution algorithm (UMD A), and the s imple genetic a lgorithm (GA) with unifor m and tw o-p oint cro ssov er op erators are applied to all gen- erated instances. P e rformance of all algorithms is then analyzed and c o mpared, and the r esults are discuss e d. Keyw ords: NK fitness landscap e, hierarc hical BO A, genetic algorithm, branc h and b ound , p er- formance analysis, scalabilit y , lo cal searc h , crosso ver. 1 In tro duc tion NK fitness landscap es [14, 15] w er e in tro duced b y Kauffman as tunable models of r u gged fi tness landscap e. An NK la ndscap e i s a function defin ed on binary strings o f fixed length and is c har- acterized b y tw o parameters: (1) n for the o v erall num b er of bits and (2) k for the neighborh oo d size. F or eac h bit, k neighbors are sp ecified and a fu n ction is giv en that determines th e fitn ess con- tribution of the b it and its neigh b ors. Usually , the fun ction for eac h bit is giv en as a lo okup table of size 2 k +1 (one v a lue for eac h com bination of the bit and its neighbors ), and b oth the n eigh b ors as well as the s u bfunction lo okup tables are initialize d randomly in some w ay . NK landscap es are NP-complete for k > 1, alt hough some v arian ts of NK landscap es are p oly- nomially solv able and there exist appro ximation algorithms for other ca ses [23, 8 , 6]. Nonetheless, NK lands capes remain a c hallenge for any optimization algorithm and they are also int eresting f rom the p ersp ectiv e of complexit y theory and compu tatio nal biology; that is wh y sin ce their inception NK landscap es h av e attrac ted researc hers in all these area s [15, 2, 23, 8, 1, 6]. This pap er presents an in-depth empirical p erformance analysis of v arious genetic and evolu- tionary algorithms on NK landscap es with v arying n and k . F or eac h v alue of n and k , a large n um b er of p roblem instances are first generated. Then, the branch-and-b ound algorithm is applied to eac h of these instances to provide a guarant eed global optim um of this in stance. Although the application of branch and b ound limits the size of pr oblems that we can stud y , one of the p rimary 1 goals w as to ensu re that w e are able to v erify the global optim um of eac h instance for ev ery algo- rithm considered in this study . Several genetic and ev olutionary algorithms are then applied to all generated pr oblem instances and their p erformance is analyzed and compared. More sp ecifically , w e consider the hierarc hical Bay esian optimization algorithm (h BO A), the un iv ariate marginal dis- tribution algorithm (UMD A), and the simple genetic algorithm (GA) with bit-flip m utation, and uniform or t wo-point crosso v er op erator. Additionally , GA without any crosso ver is considered. The results pr o vide insight in to the difficulty of NK landscap es with r esp ect to the parameters n and k and p erformance differences b et ween all compared algorithms. Sev eral interesting av en u es for futu r e work are outlined. The pap er starts b y describing NK landscap es and the branch-a nd-b ound algorithm used to v erify the global optima of generated NK landscap es in section 2. Section 3 outlines compared algorithms. Section 4 pr esen ts experimental resu lts. Section 5 discu sses futu re w ork . Finally , section 6 summarizes an d concludes the paper . 2 NK Landscap es This section describ es NK landscap es and a metho d to ge nerate random problem in s tances of NK landscap es. Additionally , th e section describ es the b ranc h-and-b ou n d alg orithm, whic h w as u sed to obtain global optima of all NK problem instances considered in this p ap er. Branc h and b ound is a complete algorithm and it is thus guarante ed to fi nd the true global optim um ; this wa s esp ecially useful for scalabilit y exp erimen ts and p erformance analyses of differen t ev olutionary algorithms. Nonetheless, branc h and b oun d requires exp onent ial time and th us the size of instances it can solv e in pr actical time is severely limited. 2.1 Problem Definition An NK fi tness landscap e [14, 15] is fully defin ed by the f ollo wing co mp onent s: • T h e num b er of bits, n . • T h e num b er of neigh b ors p er bit, k . • A set of k neigh b ors Π( X i ) for the i -th bit, X i , for ev ery i ∈ { 0 , . . . , n − 1 } . • A subfun ctio n f i defining a real v alue for eac h com bination of v alues of X i and Π( X i ) for ev ery i ∈ { 0 , . . . , n − 1 } . T yp ically , eac h su bfunction is defined as a lo okup table with 2 k +1 v alues. The ob jectiv e function f nk to maximize is then defin ed as f nk ( X 0 , X 1 , . . . , X n − 1 ) = n − 1 X i =0 f i ( X i , Π( X i )) . The difficult y of optimizing NK landscap es dep ends on all of the four comp onen ts definin g an NK problem instance. One us efu l approac h to analyzing complexit y of NK landscap es is to f ocus on the influence of k on problem complexit y . F or k = 0, NK land scapes are simple unimo dal functions similar to onemax or bin in t, whic h can b e solv ed in linear time and should b e easy for practically an y genetic and evolutio nary algorithm. Th e global optim um of NK landscap es can b e obtained in p olynomial time [23] eve n for k = 1; on the other hand, for k > 1, the problem of finding the global 2 Figure 1: Branc h an d b oun d tra verses the recursion tree w here eac h eac h lev el sets the v alue of one b it and eac h lea f thus corresp onds to one instance of all n bits. Su btrees that lead to solutions that cannot impro ve th e b est-so-far solution are cut to impro v e efficiency . optim um of un restricted NK landscap es is NP-co mplete [23]. The problem b ecomes p olynomially solv able w ith dyn amic programming ev en f or k > 1 if the neighbors are restricted to only adjacent string p ositions (usin g circular strings) [2 3] or if the s ubfunctions are ge nerated according to some distributions [8]. F or unr estricted NK landscap es with k > 1, a p olynomial-time approximat ion algorithm exists with the app r o ximation thr eshold 1 − 1 / 2 k +1 [23]. 2.2 Generating Random NK P roblem Instances T yp ically , b oth the n eighb ors as we ll as the lookup tables defining the subfunctions are generate d randomly . In this p ap er , for eac h string p osition X i , w e first generate a random set of k neighbors where eac h s tring p osition except for X i is selected with equal probability . Then, the lo okup table defining f i is generated using the u niform distribution ov er [0 , 1). Consequent ly , the studied class of NK landscap es is NP-complete for any k > 1. Since the case for k = 1 is extremely s im p le to solv e, we only considered k > 1; sp ecifically , we considered k = 2 to 6 with step 1. T o stud y scalabilit y of v arious ev olutionary algorithms, for eac h k , we considered a r an ge of v alues of n with th e minimum v alue of n = 20 and the maxim u m v alue b ound ed m ainly b y the a v ailable compu tatio nal resources an d the scop e of th e empirical analysis. 2.3 Branc h and Bound The b asic idea of br anc h and b ound is to recurs iv ely explore all p ossible binary strings of n bits using a recursion tree where eac h lev el corresp onds to one of the bits and the su b trees b elo w eac h lev el corresp ond to th e different v alues of the bit corresp ond ing to this lev el. T o mak e the algorithm more efficien t, some su btrees are cut if they can b e p ro ve n to not lead to an y solution that is b etter than the b est-so-far solution found . While this cannot eliminate the exp onen tial complexit y , wh ic h can b e exp ected due to the NP-completeness for NK landscap es with k > 1, it significant ly impro v es the p erformance of the algorithm and allo ws it to solv e muc h larger p roblem instances than if a complete r ecursion tree had to b e explored. The br anc h-and-b ound pro cedure is illustrated in figure 1. Before r unning the branch-and-b ound algorithm, w e first use a simple hill climber based on 3 bit-flip m utation with seve ral r an d om restarts to lo cate high-quality lo cal optima. Th e b est of the disco vered optima is then used as the b est-so-far solution wh en the branc h-and-b ound algorithm is started. In the br anc h-and-b oun d app roac h us ed in this p ap er, the bits are assigned sequentia lly from X 0 to X n (there are t w o su btrees of eac h no de at lev el i , eac h corresp onding to one v alue of X i ), although reordering the bits m ight improv e p erforman ce u nder some conditions. When pro cessing a no de at lev el i , th e b est v alue we can obtain b y setting the remaining n − i bits is giv en by max x i ,...,x n ∈{ 0 , 1 } n − i f nk ( X 1 = x 1 , . . . , X n = x n ) where bits x 0 to x i − 1 are assumed to b e fixed to the v alues defined by the path from the ro ot of th e recursion tree to the cu r ren t no de. If a solution has b een found already that has a higher fitness than this maximum p ossible v alue, the pro cessing b elo w the currently pro cessed no de do es not h a v e to con tin ue and the remaining unexplored p arts of the r ecur sion tree can b e explored with the exception of those p arts th at ha ve already b een cut. W e also tried another v arian t of the br anc h-and-b oun d algorithm, in w hic h the b est v alue of the ob j ective function is co mputed incremen tally for s ubsets con taining only the fi rst i bits with i = 2 to n . While this appr oac h has b een very efficient in solving instances of the Sherr ington-Kirkpatric k spin glass mo del [11], f or NK landscap es, the algorithm d escrib ed earlier p erformed m ore efficien tly . The aforedescrib ed branch-and-b ound alg orithm is complete and it is thus guaran teed to find the global optim um of an y p roblem in stance. Nonetheless, the complexit y of branch and b ound can b e exp ected to gro w exp onentia lly fast and solving large NK instances b ecomes intrac table with this algorithm. F or example, f or k = 2, the p r op osed branch-and-b ound algorithm was fast enough to solv e ten thousand unique instances of n ≤ 52; for k = 6, the algorithm w as fast enough to deal with instances of size n ≤ 36. While the ev olutionary algorithms present ed in the n ext section should b e capable of reliably solving larger instances, their con vergence to th e global optim u m cannot b e guaran teed; nonetheless, section 5 discusses how to extend this stud y to deal with larger NK problem instances, whic h are in tractable w ith the b r anc h-and-b oun d algorithm. 3 Compared Algorithms This section outlines the optimization algorithms discussed in this p ap er: (1) the hierarc h ical Ba y esian optimizatio n algorithm (hBOA ) [19, 18], (2) the univ ariate marginal distrib u tion algo- rithm (UMD A) [17], and (3) the genetic algorithm (GA) [13, 9]. Add itional ly , the section describ es the deterministic hill climber (DHC) [20 ], which is incorp orated into all compared algorithms to impro v e th eir p erformance. In all compared alg orithms, ca ndidate solutions are repr esented by b i- nary strings of n b its and a n ic hing tec hn ique called restricted tournamen t replacemen t (R TR) [10] is used for effectiv e div ersity mainte nance. 3.1 Genetic Algorithm The genetic algorithm (GA) [13, 9] ev olv es a p opulation of candidate solutions t yp icall y r epresen ted b y fixed-length b inary strin gs. The fi rst p opu latio n is ge nerated at r andom. Eac h iteration starts b y selecting p r omising solutions from the current p opulation. W e use binary tourn amen t selection. New solutions are created by applying v ariation op er ators to the p opulation of selected solutions. Sp ecifically , crossov er is used to exc h ange bits and pieces b et ween pairs of candidate solutions and m u tatio n is used to p erturb th e resulting solutions. Here we use u niform or t w o-p oint crosso v er, and bit-flip mutati on [9]. T o ensur e effectiv e diversit y main tenance, the new candidate solutions 4 are in corp orated into the original p opu lati on using r estricted to urnament replacemen t (R TR) [10]. The ru n is terminated when termination criteria are m et. 3.2 Univ a riate Marginal Distribution A lgor ithm ( UMD A) The un iv ariate marginal distribution algorithm (UMD A) [17] also ev olv es a p opulation of candi- date solutions represente d by fixed-length bin ary strings w ith the initial p opulation generated at random. Eac h ite ration starts by selecting a p opulation of promising solutions using an y common selection metho d of genetic and evol utionary algorithms; we use b inary tourn amen t selection. T hen, the probability v ector is learned that stores the prop ortion of 1s in eac h p osition of the selected p opulation. Eac h bit of a new cand idate solution is then set to 1 w ith th e p robabilit y equal to the prop ortion of 1s in this p osition; o therwise, the b it is set to 0. Consequently , the v ariation op erator of UMD A preserve s the prop ortions of 1s in eac h p osition w hile decorrelating different string p ositions. The new candidate solutions are incorp orated in to the original p opu lati on using R TR. Th e run is terminated when termination criteria are met. UMD A is an estimation of distribution algorithm (EDA) [3, 17, 16, 21]. ED As—also called probabilistic mo del-building genetic algorithms (PMBGAs) [21] and iterated dens it y estimation al- gorithms (IDEAs) [4]—replace standard v ariatio n op erators of genetic algorithms su c h as crossov er and mutation b y b uilding a probabilistic mo del of promisin g solutions and sampling the built m odel to generate n ew candidate solutions. The only difference b et w een the GA and UMD A is in the wa y the selected solutio ns are pro cessed to generate new solutions. 3.3 Hierarc hical BO A (hBO A) The hierarc hical Ba yesian optimization algorithm (hBOA) [19, 18] is also an EDA and the basic pro cedure of hBOA is sim ilar to that of the UMD A v arian t describ ed earlier. How ev er, to mo del promising solutions and sample new solutions, Bay esian netw orks with lo cal structur es [5, 7] are used in s tea d of the simple probabilit y vecto r of UMD A. Sim ilarly as in the considered GA and UMD A v arian ts, the new candid ate solutions are incorp orated into the original p opu latio n u s ing R TR and the ru n is terminated when termination criteria are m et. 3.4 Deterministic Hill Clim b er (DHC) The deterministic hill clim b er (DHC) is incorp orated into GA, UMD A and hBOA to impro v e their p erformance. DHC tak es a candidate solution represen ted by an n -bit binary string on inpu t. Then, it p erform s one-bit c hanges on the solution that lead to the maximum impro vemen t of solution qualit y . DHC is term in ated wh en no single-bit flip impr o v es solution qualit y and the solution is th us lo cally optimal. Here, DHC is u sed to imp ro v e eve ry solution in the p opulation b efore the ev aluation is p erformed. 4 Exp erimen ts This section d escrib es exp eriments and presen ts exp erimental results. First, pr oblem instances and exp erimen tal setup are discussed. Next, the an alysis of hBOA, UMD A and sev eral GA v arian ts is present ed. Finally , all algorithms are compared and th e results of the comparisons are d iscussed. 5 4.1 Problem Instances NK in stances for k = 2 to k = 6 with step 1 w er e stu died. Th e only restriction on problem size wa s the efficiency of the branc h-and -b ound algorithm, the complexit y of whic h grew v ery f ast with n . F or k = 2, we considered n = 20 to n = 52 with step 2; f or k = 3, we consid ered n = 20 to n = 48 with step 2; for k = 4, w e considered n = 20 to n = 40 w ith step 2; for k = 5, we considered n = 20 to n = 38 with step 2; fi nally , for k = 6, we considered n = 20 to n = 32 with step 2. F or eac h co m b ination of n and k , w e ge nerated 10,0 00 random problem instances and for eac h instance we used the b ranc h-and-b oun d algorithm to lo cate the global optim u m. T hen, we app lied hBO A, UMD A and several GA v arian ts to ea c h of these instances and collected empirical results, whic h we re subs equen tly analyzed. That means that o verall 600,000 u nique p roblem instances were generated and all of them we re tested with ev ery algorithm in cluded in this study . 4.2 Compared Algorithms The follo w ing list summarizes the algorithms included in this study: (i) Hierarc hical BO A (hBO A). (ii) Univ ariate marginal distrib ution algorithm (UMD A). (iii) Genetic algorithm w ith uniform crosso v er and b it-flip m utation. (iv) Genetic algorithm with t wo- p oin t crosso ver and bit-flip mutat ion. (v) Genetic algorithm with b it-flip m u tati on and no crossov er. (vi) Hill clim bing (r esu lts omitted d u e to inferior p erformance and in feasible computation). 4.3 Exp erimen tal Setup T o select promisin g solutions, bin ary tournamen t s election is used . New solutions (offspring) are incorp orated in to the old p opulation using R TR with windo w size w = min { n, N / 5 } as s uggeste d in ref. [18]. In hBOA, Ba ye sian net wo rks with decision trees [5, 7, 18] are used and the mo dels are ev aluated us in g the Ba y esian-Diric h let metric with lik eliho od equiv alence [12, 5] and a p enalty for mo del complexity [7, 18]. All GA v arian ts use bit-flip m utation with the probabilit y of fl ipping eac h bit p m = 1 /n . Two common crosso ver op erators are considered in a GA: tw o-p oin t and uniform crosso ve r. F or b oth crossov er o p erators, the p robabilit y of applying crosso ve r is set to 0 . 6. T o emphasize the imp ortance of using crosso v er, the results for GA without any crosso ver are also included, wh ere only bit-flip m utation is u sed. A sto c hastic hill clim b er with b it-flip m u tati on h as also b een considered in the in itial stage, but th e p erformance of this algorithm w as far inf erior compared to any other algorithm included in the co mparison and it wa s in tractable to solv e most problem instances included in the comparison; that is why the r esults f or th is algorithm are omitted. F or eac h problem instance and eac h algorithm, an adequ ate p op u latio n size is approxi mated with the bisectio n metho d [22, 18], which estimates the minim u m p opu latio n size r equired f or reliable con vergence to th e optim um. Here, the bisection metho d finds an adequate p opu latio n size for the algorithms to find the optim um in 10 ou t of 10 indep endent runs. Each run is terminated when the global optim um has b een foun d. The results for eac h problem instance comprise of th e follo wing statisti cs: (1) the p opulation size, (2) the num b er of iterations (generat ions), (3) the n um b er of ev aluations, and (4) the num b er of flips of DHC. F or eac h v alue of n and k , all observ ed 6 20 30 40 50 20 40 80 160 320 640 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of evaluations (hBOA) 20 30 40 50 160 320 640 1280 2560 5120 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of flips (hBOA) 20 30 40 50 20 40 80 160 k=6 k=5 k=4 k=3 k=2 Problem size, n Population size (hBOA) 20 30 40 50 1 1.5 2 2.5 3 3.5 4 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of iterations (hBOA) Figure 2: Performance of hBO A on NK landscap es with k = 2 to 6 (log-log scale). statistics w ere a verage d ov er the 10,000 r andom instances. Since for eac h instance, 10 successful runs we re p erformed, for eac h n and k and eac h algorithm the results are av eraged o ver 100,000 successful runs . Ove rall, for ea c h alg orithm, the resu lts corresp ond to 6,000,00 0 successful run s on a total of 600,000 unique problem instances. 4.4 P er formance Analysis Figure 2 shows the a v erage p erformance statistics f or hBOA on NK p roblem instances for k = 2 to k = 6. As exp ected, p erformance of hBOA gets wo rse with increasing k . More sp ecifically , the p opulation size, the num b er of iterations, the n u m b er of ev aluations, and th e num b er of DHC flips app ear all to grow exp onen tially with k . F or a fi x ed k , the time complexit y app ears to grow with n sligh tly faster than p olynomially regardless of whether it is measured by the n u mb er of ev aluations or the n umb er of flips. Figure 3 shows the av erage p erf ormance statistics for UMD A. Similarly as with hBOA , time complexit y of UMD A gro ws exp onen tially fast with k and its gro w th with n f or a fixed k app ears to b e slightly faster than p olynomial. Figures 4, 5 and 6 sho w the a v erage p erformance statistics for all three GA v arian ts. Similarly as with hBO A and UMD A, time complexit y of all GA v ariant s gro ws exp onent ially fast with k and its growth with n for a fixed k is slightl y faster th an p olynomial. 7 20 30 40 50 60 10 20 40 80 160 320 640 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of evaluations (UMDA) 20 30 40 50 60 80 160 320 640 1280 2560 5120 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of flips (UMDA) 20 30 40 50 60 10 20 40 80 160 k=6 k=5 k=4 k=3 k=2 Problem size, n Population size (UMDA) 20 30 40 50 1 2 3 4 5 6 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of iterations (UMDA) Figure 3: Performance of UMD A on NK landscap es with k = 2 to 6 (lo g-log scale) . 8 20 30 40 50 60 10 20 40 80 160 320 640 1280 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of evaluations (GA, 2−point) 20 30 40 50 60 80 160 320 640 1280 2560 5120 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of flips (GA, 2−point) 20 30 40 50 60 10 20 40 80 160 320 k=6 k=5 k=4 k=3 k=2 Problem size, n Population size (GA, 2−point) 20 30 40 50 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of iterations (GA, 2−point) Figure 4: P erf orm ance of GA with tw o-p oin t crosso v er on NK landscap es with k = 2 to 6 (log-log scale). 9 20 30 40 50 60 10 20 40 80 160 320 640 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of evaluations (GA, uniform) 20 30 40 50 60 80 160 320 640 1280 2560 5120 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of flips (GA, uniform) 20 30 40 50 60 10 20 40 80 160 k=6 k=5 k=4 k=3 k=2 Problem size, n Population size (GA, uniform) 20 30 40 50 1 2 3 4 5 6 7 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of iterations (GA, uniform) Figure 5: Performance of GA w ith uniform crossov er on NK landscap es with k = 2 to 6 (log-log scale). 10 20 30 40 50 60 10 20 40 80 160 320 1280 2560 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of evaluations (GA, no crossover) 20 30 40 50 60 80 160 320 640 1280 2560 5120 10240 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of flips (GA, no crossover) 20 30 40 50 60 10 20 40 80 160 320 640 1280 k=6 k=5 k=4 k=3 k=2 Problem size, n Population size (GA, no crossover) 20 30 40 50 1 2 3 4 5 6 k=6 k=5 k=4 k=3 k=2 Problem size, n Number of iterations (GA, no crossover) Figure 6: Pe rformance of GA with bit-flip mutatio n but without crosso v er on NK land scap es with k = 2 to 6 (log-log scale) . 11 4.5 Comparison of All Algorit hms T o compare p erformance of algorithms A and B , for eac h p roblem instance, we can compute the ratio of the num b er of ev aluations r equired by A and the num b er of ev aluations required by B ; analogica lly , w e can compu te the ratio of the num b er of flips requir ed by A and the n u m b er of flips requir ed b y B . Then, the r atios can b e av eraged o ver all instances with sp ecific n and k . If A p erforms b etter than B , the compu ted ratios should b e smaller than 1; if A p erforms the same as B , the r atio s should b e ab out 1; fin ally , if the A p erforms worse than B , the ratios should b e greater than 1. A comparison based on the aforemen tioned ratio w as compu ted for eac h pair of algorithms studied in this wo rk. T o mak e the results ea sier to read, the sup erior algorithm was t ypically u sed as the second algorithm in the comparison (in the denominator of the ratios), so that the ratios should b e exp ected to b e greater than 1. Figure 7 compares p erformance of GA with tw o-p oin t crosso v er an d that of h BOA. Figure 8 compares p erformance of GA with u niform crosso ver and that of hBO A. Figure 12 compares p erfor- mance of GA with t w o-p oint crossov er and that of GA with uniform crosso v er. Finally , figures 13 and 14 co mpare perf ormance of GAs with and without crosso ve r. One of the imp ortan t trends to ob s erv e in th e results of th e comparisons is the c hange in the t wo ratios with p r oblem s ize. In most cases, wh en one algorithm outp erforms another one, the differences b ecome more significant as the pr oblem size in creases. In some cases, although one algorithm outperf orms another one on small problems, b ecause of the observ ed dynamics w ith problem size, w e can exp ect the situation to rev erse for large problems. The comparisons based on the num b er of ev aluations and the num b er of flips can b e summarized as follo w s: hBO A. While for small v alues of k , h BO A is outp erform ed b y other algorithms included in the comparison, as k increases, the situatio n c hanges rapidly . More sp ecifically , for larger k , hBO A outp erforms all other algorithms and its relativ e p erform ance with resp ect to other algorithms imp ro ve s with increasing p roblem size. The larger the k , the more fa vorably hBOA compares to other algorithms. GA w ith uniform crosso ver. GA with uniform crosso ver p erforms b etter than GA with t w o- p oin t crosso ver and UMD A regardless of k and its relativ e p erformance with resp ect to these algorithms impr o v es with problem size. Ho wev er, as mentio ned ab ov e, for larger v alues of k , GA with uniform crosso v er is outp erformed b y hBO A and the factor by wh ich hBOA outp erforms GA w ith uniform crosso v er gro w s with p roblem size. GA w ith tw o-p oin t crosso ver. GA with tw o-p oin t crosso ver p erf orm s w orse than h BOA and GA with u niform crosso v er for larger v alues of k , but it s till outp erforms UMD A w ith resp ect to the n umb er of flips, whic h is the m ost imp ortan t p erformance measur e. UMD A. UMD A p erforms w orst of all recombinatio n-based algorithms included in th e comparison except f or a few cases with small v alues of k . Crosso ver versus m utat ion. Crosso ver h as pr o v en to outp erf orm p ure mutat ion, wh ic h is clear from all the resu lts. First of all, for the most difficult instances, hBO A—which is a pure selectorec om b in ativ e ev olutionary algorithm w ith no explicit m u tatio n—outp erforms other algorithms with increasing n . Second, eliminating crosso ve r from GA significan tly decreases its efficiency and the mutation-based approac hes p erf orm worst of all compared algorithms. 12 20 25 30 35 40 45 50 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 Problem size Num. GA (2P) evals / num. hBOA evals k=6 k=5 k=4 k=3 k=2 20 25 30 35 40 45 50 0.6 0.7 0.8 0.9 1 1.1 1.2 Problem size Num. GA (2P) flips / num. hBOA flips k=6 k=5 k=4 k=3 k=2 Figure 7: Comparison o f hBO A and GA with tw o-p oin t crosso ver with resp ect to the num b er of ev aluations and th e num b er of flips. The comparison is visualized by the a v erage ratio of the n um b er of ev aluations (n u m b er of flips) r equired by GA with t w o-p oint crossov er and the n um b er of ev aluatio ns (num b er of flips) required b y hBOA. T h e greater the ratio, the b etter the p erformance of h BOA compared to GA. Sp ecifically , GA with no crosso ver is outp erformed by all other v arian ts of GA, and the sto c hastic h ill climbing is not ev en capable of solving many p roblem instances in pr act ical time. 5 F uture W ork There are s ev eral int eresting wa ys of ext ending the w ork p resen ted in this pap er. First of all, the problem instances generated in this wo rk can b e used for analyzing p erformance of other optimiza- tion algorithms and comparing d ifferen t optimization algorithms on a br oad class of p roblems with tunable difficult y . S eco nd, th e class of problems considered in this stu d y can b e extended sub - stan tially using genetic and ev olutionary algorithms with adequate settings for solving instances unsolv able with b ranc h and b oun d. Although the global op timum w ould no longer b e guaran teed, metho ds can b e devised that still guaran tee that the global optimum is f ound reliably . Finally , other probabilit y d istributions for generating NK problem instances can b e considered to pro vide further in s igh ts in to the difficulty of v arious classes of NK landscap es and the b enefits an d costs of using alternativ e optimization strategi es in eac h of these classes. 6 Summary and Conclusions This pap er presented an in-depth empirical p erforman ce study of sev eral genetic and evol utionary algorithms on NK landscap es with v arious v alues of n and k . Sp ecifically , the algorithms considered in this wo rk included the hierarc h ical Ba yesian optimiza tion algorithm (hBO A), the un iv ariate marginal distribution algorithm (UMD A), and the simp le genetic algorithm (GA) with bit-flip m u tatio n, and tw o-p oin t or uniform crosso v er. Additionally , GA with bit-flip m utation b ut n o crosso ve r w as considered. F or eac h v alue of n and k , a large n u m b er of NK instances were generated and solve d with th e b ranc h-and-b ou n d algorithm, wh ic h is a complete algorithm that is guaran teed 13 20 25 30 35 40 45 50 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 Problem size Num. GA (U) evals / num. hBOA evals k=6 k=5 k=4 k=3 k=2 20 25 30 35 40 45 50 0.5 0.6 0.7 0.8 0.9 1 1.1 Problem size Num. GA (U) flips / num. hBOA flips k=6 k=5 k=4 k=3 k=2 Figure 8: Comparison of hBOA and GA with uniform crosso ve r with resp ect to the n u mb er of ev aluatio ns and the num b er of flips. The co mparison is visualized b y the a v er age ratio of th e n um b er of ev aluations (n um b er of flips) required by GA with uniform cr ossov er and the num b er of ev aluatio ns (num b er of flips) required b y hBOA. T h e greater the ratio, the b etter the p erformance of h BOA compared to GA. 20 25 30 35 40 45 50 0.6 0.7 0.8 0.9 1 1.1 1.2 Problem size Num. UMDA evals / num. hBOA evals k=6 k=5 k=4 k=3 k=2 20 25 30 35 40 45 50 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 Problem size Num. UMDA flips / num. hBOA flips k=6 k=5 k=4 k=3 k=2 Figure 9: Comparison of h BOA and UMD A with resp ect to the n u m b er of ev aluations and the n um b er of flips. The comparison is visualized b y the a verage ratio of the n um b er of ev aluations (n u m b er of flips) required by UMD A and the num b er of ev aluations (n umber of flips) required by hBO A. The great er the ratio, the b etter the p erform ance of h BO A compared to UMD A. 14 20 25 30 35 40 45 50 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 Problem size Num. UMDA evals / num. GA (2P) evals k=6 k=5 k=4 k=3 k=2 20 25 30 35 40 45 50 1 1.1 1.2 1.3 1.4 1.5 Problem size Num. UMDA flips / num. GA (2P) flips k=6 k=5 k=4 k=3 k=2 Figure 10: Comparison of GA with t w o-p oint crosso ver and UMD A with resp ect to the n um b er of ev aluations and th e num b er of flips. The comparison is visualized by the a v erage ratio of the n um b er of ev aluati ons (n umb er of fl ips) r equ ired by UMDA and the num b er of ev aluations (num b er of flips) r equired by GA with t w o-p oint crosso ve r. The greater the r atio , the b etter the p erform ance of GA compared to UMD A. 20 25 30 35 40 45 50 0.75 0.8 0.85 0.9 0.95 1 1.05 1.1 Problem size Num. UMDA evals / num. GA (U) evals k=6 k=5 k=4 k=3 k=2 20 25 30 35 40 45 50 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 Problem size Num. UMDA flips / num. GA (U) flips k=6 k=5 k=4 k=3 k=2 Figure 11: Comparison of GA with u niform crosso ver and UM D A with resp ect to the n u m b er of ev aluations and th e num b er of flips. The comparison is visualized by the a v erage ratio of the n um b er of ev aluati ons (n umb er of fl ips) r equ ired by UMDA and the num b er of ev aluations (num b er of fl ips) required by GA with un iform crosso ve r. The greater th e ratio, the b etter the p erform an ce of GA compared to UMD A. 15 20 25 30 35 40 45 50 1.1 1.2 1.3 1.4 1.5 Problem size Num. GA (2P) evals / num. GA (U) evals k=6 k=5 k=4 k=3 k=2 20 25 30 35 40 45 50 1.1 1.15 1.2 1.25 1.3 Problem size Num. GA (2P) flips / num. GA (U) flips k=6 k=5 k=4 k=3 k=2 Figure 12: Comparison of GA with un iform and t wo-point crosso v er with resp ect to the num b er of ev aluations and th e num b er of flips. The comparison is visualized by the a v erage ratio of the n um b er of ev aluations (n umb er of flip s) required by GA with t wo- p oin t crosso ver and the num b er of ev aluations (n umber of flips) r equired by GA with un iform crosso v er. T he greater the ratio, the b etter the p erformance of unif orm cr ossov er compared to t w o-p oint crosso ver. 20 25 30 35 40 45 50 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 Problem size Num. GA (NC) evals / num. GA (U) evals k=6 k=5 k=4 k=3 k=2 20 25 30 35 40 45 50 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 Problem size Num. GA (NC) flips / num. GA (U) flips k=6 k=5 k=4 k=3 k=2 Figure 13: Comparison of GA with un iform cr ossov er and GA with no crosso v er (mutatio n only) with resp ect to the n um b er of ev aluations and the n u m b er of flips. The comparison is visualized b y the a verage ratio of the num b er of ev aluatio ns (n umb er of flip s) requir ed by GA w ithout crosso v er and the n u m b er of ev aluati ons (n umb er of flips) required b y GA with uniform crosso ve r. The greater the ratio , the b etter the p erform an ce of uniform crosso v er compared to m utation only . 16 20 25 30 35 40 45 50 1 1.5 2 2.5 3 3.5 4 4.5 Problem size Num. GA (NC) evals / num. GA (2P) evals k=6 k=5 k=4 k=3 k=2 20 25 30 35 40 45 50 1.2 1.4 1.6 1.8 2 2.2 2.4 Problem size Num. GA (NC) flips / num. GA (2P) flips k=6 k=5 k=4 k=3 k=2 Figure 14: C omp arison of GA with t wo-point crosso v er and GA with no crosso v er (m utation only) with resp ect to the n um b er of ev aluations and the n u m b er of flips. The comparison is visualized b y the a verage ratio of the num b er of ev aluatio ns (n umb er of flip s) requir ed by GA w ithout crosso v er and the n um b er of ev aluations (n umb er of flip s ) required by GA with tw o-p oin t crosso ver. T he greater the ratio , the b etter the p erform an ce of t wo- p oin t crosso ver compared to m u tatio n only . to find the global optim u m. Performance of all algorithms was analyzed and compared, and the results were discussed. The m ain contributions of this work are summarized in what follo ws. First of all, NK landscap es represent an imp ortan t class of test problems and d espite that there h as b een pr actic ally no work on u sing adv anced estimation of distribution algorithms (ED As) on NK landscap es. This work pro vides man y exp erimenta l results on one adv anced and one sim p le ED A, and it shows that adv anced EDAs ca n significan tly outp erform o ther genetic and ev olutionary algorithms on NK landscap es for larger v alues of k . Second, most studies concerned with NK lands cap es do not verify the global optim u m of the considered pr oblem instances and it is th u s often difficult to in terpr et th e results and ev aluate their imp ortance. In this stud y , the global optim um of eac h instance is v erified with the co mplete branch-and-b ound algorithm. Third , w hile the diffi cu lt y of NK land scap es can b e exp ected to v ary sub s tan tially f rom instance to in stance, most studies presen ted in the p ast used only a limited sample of pr ob lem instances; here we pro vid e an in-depth study where ab out 600,00 0 un ique p roblem instances are considered. Finally , the results in this pap er are n ot based on only one ev olutionary algorithm; instead, w e consider sev eral qualitativ ely d ifferen t evo lutionary algorithms, pro vidin g insigh t in to the comparison of ge netic algorithms and ED As, as w ell as in to the comparison of the mutat ion-based and r eco m b ination-based evol utionary algo rithms. Ac kno wledgmen ts This pro ject was sp onsored b y the National Science F ound atio n under CAREER gran t ECS- 05470 13, b y the Air F orce Office of Scientific Resea rc h, Air F orce Materiel Command, USAF, under gran t F A955 0-06-1-0096 , and by the Universit y of Missouri in St. Louis thr ough the High P erf orm ance Comp u ting C olla b oratory sp onsored by Information T e c hn olog y Services, and the Researc h Aw ard and Researc h Board programs. The U.S. Go vernmen t is authorized to repro du ce and d istribute r eprin ts for go vernmen t pur - p oses not w ithstanding an y co p yrigh t notation thereon. Any opinions, findings, and conclusions or 17 recommendations expressed in this mate rial are those of the authors and d o not necessarily refl ect the views of the National S cience F ound atio n, the Air F o rce Offi ce of Scient ific R esearch, or the U.S. Gov ernmen t. Some exp eriments we re d one u sing the h BO A soft wa re dev elop ed by Martin P elik an and Da vid E. Goldb erg at the Univ ersity of Illinois at Urbana-Champaign and most exp er- imen ts we re p erformed on the Beo wu lf clus ter main tained by ITS at the Universit y of Missour i in St. Louis. References [1] H. E. Aguirre and K. T anak a. Genetic algorithms on nk-land scapes: Effects of selection, drift, m u tatio n, and recom bination. In G. R. Rai dl et al., ed itors, Applic ations of Evolutionary Computing: EvoWorksh ops 2003 , p age s 131–142, 20 03. [2] L. Alten b erg. NK landscap es. In T. B¨ ac k, D. B. F ogel, and Z. Mic halewicz, editors, Handb o o k of E v olutiona ry Com putation , pages B2.7:5– 10. Institute of Physics Pub lishing and Oxford Univ ers ity Press, Bristol, New Y ork, 1997. [3] S. Baluja. P opulation-based incrementa l learning: A metho d for int egrating genetic searc h based fu nction optimization and comp etitiv e learning. T ec h . Rep. No. CMU-CS-94-163, Carnegie Mellon Univ ersit y , P ittsbu rgh, P A, 199 4. [4] P . A. N. Bosman and D. Th ierens. Con tin u ous iterated densit y estimatio n ev olutionary algo - rithms within the IDEA fr amew ork. Workshop Pr o c e e dings of the Genetic and Ev olutionary Computation Confer enc e (GECCO-2000) , pages 197–200, 2000 . [5] D. M. C hic k ering, D. Heck erman, and C. Meek. A Ba y esian app roac h to learning Ba ye sian n et- w orks with lo cal structur e. T ec h nical R ep ort MSR-TR-97-07, Microsoft Resea rc h , Redmond, W A, 1997. [6] S.-S. C hoi, K. Jung, and J. H. Kim. Phase transition in a random NK landscap e mo del. pages 1241– 1248, 2005. [7] N. F riedman and M. Goldszmidt. Learn in g Ba y esian n et w orks with lo cal structure. In M. I. Jordan, editor, Gr a phic al mo dels , page s 421–459. MIT Press, C am bridge, MA, 1999. [8] Y. Gao and J. C. Culb ers on. An analysis of phase transition in NK landscap es. Journal of Artificial Intel ligenc e R ese ar c h (JA IR) , 17 :309– 332, 2002. [9] D. E . Goldb erg. Genetic algorithms in se ar ch, optimization, and machine le arning . Addison- W esley , Reading, MA, 19 89. [10] G. R. Harik. Findin g m ultimo dal solutions u sing restricted tournamen t selection. Pr o c e e dings of the Internationa l Confer enc e on Genetic Algorithms (ICGA-95) , p ages 24–31 , 1995 . [11] A. Hart wig, F. Dask e, and S. Kob e. A recursive branc h-and-b ound algorithm for the exact ground state of Ising s pin-glass mo dels. Computer Physics Communic ations , 32:133–138 , 1984. [12] D. Hec k erm an , D. Ge iger, an d D. M. Chick ering. L earn ing Bay esian net works: The com b ina- tion of knowle dge and s tatistical data. T ec hnical Rep ort MSR-TR-94-09, Mic rosoft Researc h, Redmond, W A, 1994. 18 [13] J . H. Holland. A daptat ion in natur al and artificial systems . Univ ers ity of Mic higan Press, Ann Arb or, MI, 1975. [14] S . K au ffm an. Adaptation on rugged fitness land scap es. In D. L. S tein, editor, L e ctur e Notes in the Scienc es of Comp lexity , page s 527–618. Addison W esley , 1989. [15] S . K auffman. The origins of Or der: Self- or gan ization and sele ction in evolution . Oxford Univ ers ity Press, 199 3. [16] P . Larra˜ naga and J. A. Loza no, editors. Estimation of Distribution Algorithms : A New T o o l for Evolutionary Computa tion . Klu we r, Boston, MA, 2002. [17] H. M ¨ uhlenbein and G. P aaß. F rom recombinatio n of genes to the estimati on of distrib utions I. Binary parameters. Par al lel Pr oblem Solving fr om Natur e , pages 178–187, 1996 . [18] M. P elik an. H i er ar chic al Bayesian optimization algorith m: T owar d a new gener ation of ev olu- tionary algorithm s . Springer, 2005. [19] M. Pelik an and D. E . Goldb erg. Escaping hierarchica l traps with comp etent genetic algorithms. Pr o c e e d ings of the Genetic and Evolutionary Computat ion Confer enc e (GECCO -2001 ) , pages 511–5 18, 2001. Also IlliGAL Report No . 200 0020. [20] M. P elik an an d D. E. Goldb erg. Hierarc h ical BOA solv es Ising spin glasses and maxsat. Pr o- c e e dings of the Genetic and Evolutionary Computation Confer enc e (GECCO- 200 3) , I I:1275– 1286, 2003. Also IlliGAL Rep ort No. 2003001. [21] M. P elik an, D. E. Goldb erg, and F. Lob o. A sur v ey of optimization b y bu ilding and using probabilistic m odels. Computational O ptimizat ion and Applic ations , 21(1):5–20 , 2002. Also IlliGAL Rep ort No. 99 018. [22] K . S astry . Ev aluation-relaxation schemes for genetic an d evo lutionary algorithms. Master’s thesis, Un iv ersit y of Illinois at Urb ana-Champaign, Department of General Engineering, Ur- bana, IL, 200 1. Also IlliGA L Rep ort No. 2002004. [23] A. H. W righ t, R. K. Th ompson, and J. Zhan g. The computational complexit y of N-K fi tness functions. IEEE T r ansactions Evolutionary Computation , 4(4):37 3–379, 2000. 19