On the Application of Hierarchical Coevolutionary Genetic Algorithms: Recombination and Evaluation Partners

1 On the Application of Hierar chical Coevolutionary Genetic Algorithms: Recom bination and Evaluation Partners Special issue on "Real Life Application s of Natur e Inspired Com binatoria l Heuristics" , J ournal of Applied Syst em Scienc es, 4(2), pp 2-17, 200 3. Uwe Aickelin School of Computer Sci ence University of Nottingham NG8 1BB UK uxa@cs.nott.ac.uk Larry Bull Faculty of Comp uting, Engineering & Mathem atical Science s University of the West of E ngland Bristol BS16 1QY, U. K. Abstract This paper examin es the use of a hierarchical coevol utionary genet ic algorithm und er differ ent p artnering strategies. Cascading clust ers of sub -populations are built from the bottom up, wit h higher-level sub-popul ations optimising larger parts of the probl em. Hence higher-level s ub-populations potenti ally sear ch a l arger searc h sp ace with a lower resol ution whilst lower-le vel sub-populati ons s earch a s maller s earch space with a higher r esolution. Th e ef fects of di fferent p artner sel ection schemes amongst the sub-populat ions on solution qualit y are exami ned for two constrained optimisation problems . W e ex am ine a number of recombinati on partnering st rategies in the construction of higher-level indi viduals and a n umber of related schemes for evalu ating su b-solutions. It is s hown that partnering strategies that exploit probl em- specific knowledge ar e superior and can counter inap propriate (sub -) fitness measur ements. Keywords: Genetic A lgorithm s, Coevolution, Scheduling . 1 INTRODUCTION The use of coevolutionary c omputation for optim isation raises a number of new questions, one of which is addressed in this paper: the issue of intelligently selecting partners for both mating and evaluation from other evolving populati ons. This paper will l ook at a number of dif ferent partnering strategies when combined with a hierarchical scheme that uses a co-operative sub-population structure. We will evaluate the different strategies according to their optimisation perfo rmance on two const rained scheduling problem s. Within the hierarchical structure, all sub-populations follow different (sub-) fitness functions, so in effect they are only searching specific parts of the solution space. Following s pecial crossover- operators, t hese parts ar e then gradually merged to f ull s olutions. The advantage of such a divide and conquer approach is a reduced 2 search space size and/or epis tasis within the lower- l evel sub-populations which (potent ially) mak es the optimisation task easier for the genetic algo rithm (GA) [H olland 1975]. The paper is arranged as follows: the following s ection describes the nurse scheduling and tenant selection problems. Pyramidal genetic algorithms a nd their appl ication to these two pr oblem s are detailed in section 3. Section 4 explains the se ven part nering strategies exam ined in the paper and section 5 describes their use an d computational results. T he final section discusses all findings and draws conclusio ns. 2 THE NURSE SCHEDULING AND TENANT SEL ECTION PROBLEMS Two optimisation problems are cons idered in this paper , the nurse s cheduling problem and the tenant selection problem. Both h ave a number of characteristics that make the m an i deal testbed for the e nhanced genetic algorithm using partneri ng strategies. Firstly, they are bot h in the class of NP -comp lete problems [e.g. Martello & Tot h 1 990]; h ence, they are c hallengin g p roblems. Secon dly, they have proved resistant t o optimisation by a standard gene tic algorithm, with good solutions only found by using a novel strategy of indirectly optimising the problem with a decoder based genetic algor ithm [Aickelin & Dowsland 2001 ]. Finally, both problems are similar multiple-choice allocation p roblem s. For nur se scheduling, the choice is to allocate a shift-pattern to each nurse, whilst for the tenan t selection it i s to a llocate an area of the mall to a shop. However, as the f ollowing more deta iled explanation of the two wi ll show, th e two pro blems als o have some very dis tinct characteristics m aking them different yet sim ilar enough for an interesting comparison of results. The nurse-scheduling problem is that of creating weekly schedules for wards of up to 30 nurses at a major UK hospital. These schedules have to satisfy working contracts and meet the demand for given numbers of nu rses of different grades on each s hift, whilst at the s ame time being seen to be fair by the staff concerned. The latter objective is a chieved by m eet ing as m any of the nurses’ requests as p ossible and by considering historical inform at ion to ensure that unsatisfied requests a nd u npopular shifts are evenly distributed. Due to various hospital poli cies, a nurse can n ormally only work a sub-set of the i n total 411 theoretica lly pos sible shift-patterns. For instance, a nurse should work either days or nights in a given week, but not both. The interested reader is dire cted to [Aick elin & Dowsland 200 0] for further details of this problem. For our purposes, the problem can be modelled as follows. Nurses are scheduled weekly on a ward ba sis such that they work a feasible pattern with regards t o their contract and that the dem and for all days and nights and for all qualification levels is covered. I n total t hree qualification lev els with corresponding demand e xists. It is hospital policy that more qualified nurses are allowed to cover for less qualified one. Infeasible solutions with respect t o c over are not acceptable. A solution to the problem would be a str ing, with the number of elements equal to the numb er of nurses. Each element would then indicate the shift-patt ern worked by a particular nurse. Depending on t he nurses’ preferences, the recent hi story of patterns worked and t he overall attractiveness of the patter n, a pen alty cost is the n allocated to each nurse-shift-pattern pair. T hese values were set in close cons ultation with the hospital and rang e from 0 (perfect) to 1 00 (unacceptable), with a bia s to lower values. The su m o f the se value s gives the quality of the schedule. 52 data sets are available, with an average problem size of 30 nurses per ward and u p to 411 possible sh ift-patterns pe r nurse. The problem can be form ula ted as an integer linea r program as follows. 3 Indices: i = 1... n nurse inde x. j = 1... m shift patte rn index. k = 1...14 day and nigh t index (1...7 a re days and 8...14 are nigh ts). s = 1... p grade index. Decision variables:    = else 0 pattern shift works nurse 1 j i x ij Parameters: n = Number of nurse s. m = Number of shif t patterns. p = Number of grad es.    = else 0 night day / covers pattern shift 1 k j a jk    = else 0 higher or grade of is nurse 1 s i q is p ij = Preference cost o f nurse i working shift patte rn j . N i = Working shif ts per week of nurse i if night shifts are worked. D i = Working shift s per week of nurse i if day shifts are worked . B i = Working shifts per week of nurse i if both day a nd night shifts are worked. R ks = Deman d of nurses with grade s o n day respectiv ely night k . F(i) = Set of feasibl e shift patterns for nurse i , where F(i) is defined as i shifts combined j B a or shifts night j N a or shifts day j D a i F i k jk i k jk i k jk ∀                           ∈ ∀ = ∈ ∀ = ∈ ∀ = = ∑ ∑ ∑ = = = 14 1 14 8 7 1 ) ( 4 Target function: ! min 1 ) ( → ∑ ∑ = ∈ n i m i F j ij ij x p Subject to: 1. Every nurse works ex actly one feasible sh ift pattern: i x i F j ij ∀ = ∑ ∈ 1 ) ( (1) 2. The demand for nurse s is fulfilled for every grade on every day and night: s k R x a q ks i F j n i ij jk is , ) ( 1 ∀ ≥ ∑ ∑ ∈ = (2) Constraint set (1) ensures th at every nurse works exactly one shift pattern from his/ her feasible set, and constraint set (2) ensures that t he demand for nurses is covered for e very grade on every day an d night. Not e that the definition of q is is such that hi gher graded nurses can substituted those at lower g rades if ne cessary. Typical problem d imensions are 30 nurses of three grades and 411 shift patterns. Thus, t he Integer Programming formulation has about 12000 bina ry variable s and 100 constraints. Finally for all decoders, the fitness of completed solutions has to be calculated. Unfortunately, feasibility cannot be guaranteed, as otherwise an unlimited supply of nurses, respectively o vertime, would be nece ssary. This is a problem-specific issue and cannot be changed. Therefore, we still need a pe nalty function approach. Since the chosen encoding automatically satisf ies constraint set (1) of the integer programming formulati on, we can use the foll owing for mula, where w demand is th e penalty weight, to calculate the fitness of solutions. Hence the penalty is prop ortional t o t he number of uncovered shifts and the f itness o f a solution i s calculate d as follows. min! 0 ; max 14 1 1 1 1 1 1 →       − + ∑ ∑ ∑ ∑ ∑ ∑ = = = = = = k p s n i m j ij jk is ks demand n i m j ij ij x a q R w x p Here we use an encoding t hat follows directly from the Integ er P rogramm ing f ormulation. Each individual represents a full one-week schedule, i.e. it is a string of n elements with n being the n umber of nurses. The ith element of the string is the ind ex of the shift pattern worked by nurse i . For example, if we have 5 nurses, the string (1,17,56,67,3) re presents the schedule i n which nurse 1 w orks pattern 1, nu rse 2 pattern 17 etc. For comparison, all data sets were attempted using a standard Integer Programm ing package [Fuller 1998]. However, some remained unsolved after each bein g all owed 15 hour s run-time on a Pentium II 200 . Experiments with a number of de scen t methods using di fferent neighbourhoo ds, an d a standard simulated annealing im plementation, were even less successfu l and frequ ently failed to find f easible solutions. A straightforward genetic alg orithm approach fai led to solve the prob lem [Aickelin & Dowsland 2000]. The bes t evolutionary results to dat e have be en achieved with an i ndirect genetic approach e mploying a decoder 5 function [Aickelin & Dowsland 2001]. However, we b elieve that there is further leverage in direct evolutionary approaches to this problem. Hence, we p ropose to u se an enhanced pyramidal genetic algorithm in this paper. The secon d problem is a m all layout and tenant selection prob lem; ter med the mall problem here. The mall problem arises both in the pla nning phase of a new shopping centre and on co mpletion when the type and number of shops occ upying the mall has to be decide d. To m aximise revenue a good mixtur e of shops that is both heterogeneous and ho mog eneous has to be achieved. Due to the difficulty of obtaining re al-life data because of confidentiality , the problem an d data used in this research are constructed artificially , but closely modelled after the actual real-life problem as described for instance in Bean et al. [1988]. In the following, we will briefly outline our model. The objective of the mall problem is to maximise the rent revenue of the mall. A lthough there is a small f ixed rent per shop, a large part of a shop’s rent depends on the sales revenue generated by it. Therefore, it is important to select the right n umb er, size and type of tenants and t o place them into the ri ght l ocatio ns to maximise revenue. As ou tlined in Bean et a l. [1988], the ren t of a shop depends on the following factors: • The attrac tiveness of the area in wh ich the shop is located. • The total num be r of shops of the sam e type in the m a ll. • The size of the sh op. • Possible s ynergy effects with ne ighbouring similar shops, i .e. shops i n the same group (not used by Bean et al.). • A fixed am ount of rent based on the type of the shop and the area in which it is l ocated. This problem ca n be modelled as fo llows : Before placing shops, the mall is divide d into a discrete number of locations, each big enough to hold the smallest shop s ize. Larger si zes can be created by placing a shop of the same type in adj acent locations. Hence, the problem is t hat of placing i shop- types ( e.g. menswear) i nto j locations, where each shop- type can belong to one or mo re of l groups (e. g. clothes shops) and each location is situated in one of k areas. For each type of shop there w ill be a minimum , i deal and maximu m number allowed in the mall, as con sum ers are drawn to a mall by a balance of variety and ho mogeneity of shops. The size of sho ps is determined by how many locations they occupy within the same area. For the purpose of this study, shops are grouped into three s ize classes, na mely small, medium and large, occupying one, two a nd three locations in one area of the mall respectively . For instance, if there are two locations to be filled with the same shop-ty pe within on e area, then this will be a shop of medium size. If there are five locations with t he same sho p-type ass igned in the same area, then they will form one large and one m edium shop et c. Usually, there will also be a m aximum total nu mber o f small, medium and large shops allow ed in the mall. To te st the robustness and performance of our algorithms thoroughly on t his problem, 50 pro blem instance s were created. All problem instances have 100 location s grouped into f ive areas. However, the sets diffe r in the number of shop-types a vailable (between 50 and 20 ) an d i n the tightness of the constraints regarding the minimum and maxim um number of shops of a certain type or size. Full details of the m o del, how the data was created, its dim ensions and the differences betwee n the sets can be found in [Aicke lin 1999]. 6 3 PYRAMIDAL GENETIC ALGORITHMS Both problem s failed to be optimised w ith a standard genetic algorithm [Ai ckelin & Dowslan d 2000, 2001]. Our previous research showed that the d ifficulties were attributable to epistasis created by t he co nstraine d nature of the optimisation. Briefly, epistasis re fers to the ‘non-lineari ty’ o f the solution string [Davidor 1991], i.e. individual variable values which were good in their own right, e.g. a particular shift / location fo r a particular nurse / shop fo rmed low quality solution s once combined. This effect w as created by those constraints that could only be incorporated into the genetic a lgorithm via a penalty f unction approach. For instance, most nurs es preferr ed working days; thus, parti al solutions with many ‘day’ shift-patterns have a higher fitness. However, combining these shift-patterns leads to shortages at nigh t a nd t herefore infeasibl e solutions. T he situation for the mall problem is similar yet more complex, as t wo types of constraints have to be dealt with: size con straints and num ber constraints. In [Aickelin & Dowsland 2000] we presented a simple, and on it s own unsuccessful, pyramidal gene tic algorithm for the nurse-scheduling problem. A pyramidal approach can best be described as a hierarchica l coevolutionary genetic algori thm where cascading clusters of sub-populations are built from botto m u p, wi th higher-level sub-populations optimising larger parts of t he p roblem. Thus, the hierarchy is not within one representation of the p roblem bu t ra ther between sub-populations which optimise different portions of the global problem. Hence, h igher- level s ub-populations search a l arger search space with a l ower resolution whilst lower-level sub-populations search a smalle r search s pace with a higher resolution. A related hierarchical framew ork was presented using Genetic Programming [Ko za 1991] w hereby main program trees coevolve with successively lower level functions [e.g. Ahluwalia & Bull 1998]. The pyramidal GA can be applied to the nurse- sc heduling problem in the fol lowing way : • Solutions in s ub-population s 1, 2 and 3 have t heir fitness based on cover and requests only for grade 1, 2 and 3 respectively. • Solutions in sub-populations 4, 5 and 6 have t heir fitness based on cover and requests for grades (1+ 2), (2+3) and (3+1). • Solutions in su b-population 7 opt imise cover and requests for (1+2+3). • Solutions in su b-population 8 solve the o riginal (all) pr oblem, i.e. cover for 1, for (1+2) and for (1+2+3). The full structure is illust rated in figure 1. Sub- so lution strings from lo wer populations are cascaded upwards using suitable crossover and selection mechanisms. For instance, fixed crossover points are used such that a solution from sub-population (1) combined with one from (1+ 2) f orms a new sol ution in sub-populatio n (1+2). Each sub-population performs 50% of crossove rs uniform w ith two parents from itself. The other 50% are done b y ta king one parent from itself and the other from a suitable lo wer level population (picked at random) and then performing a fi xed-point crossover. Bottom le vel sub-populations use only un iform crossover. The top level (all) population randomly chooses t he second parent f rom all other po pulations. Although the full p roblem is as epistatic as before, the sub-problems are less so a s the i nteracti on between nurse grades is (partially) ignored. Compatibility problems of combining t he parts are reduced by the pyramidal structure wi th its hierarchical and gradua l com bining. Using t his ap proach improved solution quality in comparison to a standard genetic algorithm was recorded. Initially roulette wheel selection based on fitness r ank had been used t o choose parents. The fitness of each sub-string is calculated using a substitute f itness m easure based on the re quests and cover as detaile d above, i.e. the possibility of more qualified nurses covering for less- qualified ones is partially ignored. Unsatisfied constraints are still included via a penalty fun ction. This paper will in vestigate various partnering strategies between the agents of the sub- populations to improve upon these re sults. 7 Figure 1: Nurse Prob lem Pyram i dal Structure. Similar to the nurse problem, a solutio n t o the mall problem can be represented by a string with as many elements as locations in the mall. Each element then indicates what shop-type is to be located there. The mall is geographically split i nto di fferent re gions, for instance north, east, south, west and central. Some of the objectives are regional; e.g. the size of a shop, the synergy effects, t he attractivene ss of an area to a shop-type, whereas others are global, e.g. the total num ber of shops of a certain ty pe or size. The application of t he pyramidal structure to the mall prob lem foll ows along si milar lines to that o f the nurse problem. In line with decomposing partitions into t hose with nurses of the same grade, th e proble m is now split into the areas of the m all. Thus, we will have sub- strings with all the shops in one area in them . These can then be combined to create larger ‘parts ’ of the mall a nd finally full solutions. However, the question ar ises how to calculate the substi tute fitness measure of the partia l strings. The solutio n chosen here w ill be a pseudo measure based on area dependant components only, i.e. global aspects are not taken into ac count when a substitute fitness f or a partial string is c alculated. Thus, sub-fitness will be a measure o f the rent reve nue cre ated b y par ts of t he m all, taking into account those constraints that are area based. All other cons traints are ignored. A pen alty function is used to account for un satisfied constraints. Due to the com plexity of the fitness calculat ions and the li mited overal l population si ze, we r efrained from using several levels in the hierarchical design as we did with the nurse scheduling. I nstead a simpler two-level hierarchy is used a s shown in f igure 2: Five s ub- populations opti mising the f ive areas separately (1,2,3,4,5) and one m ai n p opulation opt imising the original pr oblem (all ). Wit hin the s ub-populations 1-5 uniform crossover is used. The top -lev el pop ulation uses uniform crossover between two members of t he population half the time and for the remainder a special crossover that selects one solution from a random sub-populatio n that then perform s a fixed-point crossover w ith a member of the top popula tion. 1+2+3 1+2 all 3+1 2+3 1 3 2 8 Figure 2: Mall Problem Pyram idal Structure. The remainder of t his paper will i nvestigate ways to try to improve on previously found poor results by suggesting wa ys of com bining partial strin gs more intelligently. An alternative, particularly f or t he mall problem, would be a more gradual build-up of s ub-popula tions. Without increasing the overall population size, this would lead to more and hence smaller sub- populations. However, t his more gradual approach might have enabled the algorithm to find good feasible solutions by m ore slowly j oining together promising building blocks. This is in contrast to the relatively harsh two-level a nd three-level design where building blocks had to ‘succeed’ imm ediatel y. Exploring t he exa ct benefits of a gradual build-up of sub-solutions would mak e for another challenging area of poss ible future research. 4 PARTNERING STRATEGIES The problem of h ow to pick partners has been noted in both competitive and co -op erative coevolutiona ry algorithms. Many strategies have b een presented in the literature as su mm ar ised for instance in [Bull 1997]. In this paper, the following st rategies are compare d for t heir effectiveness in fighting epistasis i n the pyramidal genetic algorithm op timising th e nurse scheduling a nd the mall problem s. As indicated, some strategies will be used for fitness evaluation, som e for m ating selection and some for both. Rank-Selection (S) [Bo th]: T his is the method used so far i n our algorithms. Solutions are assigned a sub - fitness sc ore b ased as closely as p ossible on the contribution of their partial string to full solutions. All solutions are then ran ked within each sub-population and select ion follows a roulette wheel scheme based on the ranks [e.g. Aickelin & Dowsland 2000]. Random (R) [ Both]: Solutions choose their mating partners randomly f rom amongst all those in the sub- population their sub- population is paired w ith [e.g. Bull & Foga rty 1993]. Best (B) [Bo t h]: I n this s trategy, each agent i s pai red with t he current b est solution of the o ther sub- population(s). I n case of a tie , t he solutio n with the lower population index is c hosen [e.g . Po tter & De J ong 1994]. 2 1 all 3 4 5 9 Distributed (D) [Both]: Here each sub-population i s spaced out evenly across a sin gle t oroidal gri d [e.g. Ackley & Litt man 1987] . Subsequently, solutions are pa ired with others on the same grid location in the appropriate other sub-populations. Children created in this way are inserted in an adjacent grid locati on. This is said to be be neficial to the search process because a consistent coevolu tionary p ressure emerges since all offspring appear in their parents’ neighbourhoods and so there is potentially less variance in partners between generations [Husbands 1994]. In our algorithms, we use local mating with the neighbourhood set t o the eight agents surrounding the chosen location. Joined (J) [Mating only]: In nat ure, some species carry others internally with the relationship propagated from generation to generation. Thus, each i ndiv idual represents a complete solution; i. e. all the parts have been joined together [e.g. Iba 1996]. In our case, this results in all sub- populations solving the original problem , i.e. we h ave a tr aditional parallel genetic algorithm. T his means that all sub -po pulations use the full fitness function for evaluation and rank- proportional selection. Attractiveness (A) [Mating only]: The five st rategies described so far are general and do not mak e use of problem spe cific knowledge. However, there is a grow ing body of rese arch [e.g. St anley et al. 199 4, Wolpert & Mac ready 1997], as well as our own previous work [Aickelin & Dowsland 2 000], which suggests that approaches that exploit p roblem specifi c knowledge achieve be tter results. Here pairing is done as fo r the rank-selection strategy (S). However, the pair is only accepted with a probabil ity p roportional to their fitness or substitute fitness once combined. The probabilities are scaled such that if the (substitute) fitness f comb is equal or greater to the best-kn own fitness f best t he pa iring i s automatically ac cepted . Ot herwise the probability is f comb / f best for the m all problem and the inverse for the nurse schedul ing. Partner Choice (C) [Mating only]: This approach again exploits pro blem specific knowledge and was inspired by an idea presented by Ronald [1995]. He solves Royal Roads and multi-objective optimisation problems using a genetic algorithm where t he first parent is chosen following standard rules, i.e. proportional to its fitness. However, the second parent is not cho sen according to its fitness, but d epe nding on it s ‘attractiveness’ to the first paren t, which is m easured on a d ifferent sca le. Our approach will be slightly different. T he first parent is still chosen according to its ran k. But rat her than picking one individual from the appropriate sub- population as the second parent, ten candidates are chosen at r andom. T he secon d pare nt will th en be chose n as the one that creates th e fittest children wi th the first parent. Best / Random (S R) [Evaluation only]: A s olution is paired twice: with t he best of the other sub-population(s ) and with a random par tner(s). The better of th e two fitness values is recorded. Rank-based / Random (SR) [Evaluation on ly]: A s olution is paired t wice: w ith roulette w heel se lected solution(s) and with (a) ran do m partner(s). The better of the two fitne ss values is recorded. Random / Random ( RR) [Ev aluation only ]: A s olution is paired twice with random partner(s). The bett er of the two fitness values is recorded. 5 EXPERIMENTAL RESULTS 5.1 T HE MODEL To allo w for fair comparison, the parameters and strategies used f or both problems are kept as similar as possible. Both have a total population of 1000 individuals. These are split into sub- populations o f size 100 for 10 the lower-levels and a main population of size 300 for the nurse schedul ing and respectively of size 500 f or the mall problem. In principle, two t ypes of crossover take place: wit hin sub-populations, a two- parent-two- children param eterised uniform crossover w ith p =0.66 for genes com i ng from one parent tak es place. Each new solution created undergoes mutation with a 1% bit m utation probability, where a mutation would re-initialise the bit in the feasible range. T he a lgorithm is run in generational mode to accommodate th e sub - population structure better. In every generation, the worst 90% of parents of all sub-po pulations a re r eplaced. For all fitness and sub-fitness function calculat ions a fitness sc ore as described before i s used. Constraint violations are penalised with a dynamic p enalty parameter, which a djusts itself depending on the (sub)-fitness difference between t he best and the best f easible agent in each (sub-) p opulation. Fu ll details on this type of weight and how it was calculated can be fo und in [Smith & Tate 1993] and [Aickelin & Dowsland 2000]. The stopping criterion is the to p sub- population showing no im provement for 50 genera tions. To obta in statistically sound results all experiments were conduc ted as 20 runs over all problem instances. All experiments were started with the same set of random seeds, i.e. with the same initial pop ulations. The results are presented in feasibility and c ost respectively rent f ormat. Feasibility denotes the probability of fin ding a feasible s olution averaged ov er all problem instances. Cost / Rent refer to the objectiv e function value of the best feasible solution for ea ch problem instance averaged over the number of instances for whi ch at least one feasible solution was found . Should t he algorithm fa il to fi nd a single feasible solution f or all 20 runs on one problem instance, a ce nsored observation of one hundred in the nurse case and zero for the mall problem is made instead. As we are minimising the c ost for the nurses and maximising the rent of the mall, thi s is equi valent to a very poor solution. For t he nur se- scheduling problem, the c ost represents the sum of unfulfilled nurses’ requests and unfavourable shift- patterns worked. For the m all, the v alues for the rent are in tho usands of pounds per yea r. 5.2 R ESULTS: PART NERING FOR R ECOMBINATION Table 1 sh ows t he results found b y o ur algorithm s for t he two problems (N = Nurse problem, M = Mall problem) using th e seven different partnering strategies i n combination with the pyramidal str uctu re. The results are compared to those found by the standard genetic algor ithm (SGA) [Aick elin & Dowsland 2000 and 2001] and t he Integer Programming results [Fuller 1998] for the n urse problem and t heoretical bounds for the mall problem (both refer red to as ‘bound’). A num ber of interesting obs ervations c an be made. In t he nurse scheduling cas e, the SGA approach failed to find good or e ven feasi ble solutions for many data sets. T his can be e xplained by the high degree o f epistasis present and the i nabi lity of the unmodified genetic algorithm to deal with it. Once the pyram idal structure with r ank- based selection (S) is i ntroduc ed, results improve significantly, however there is still room for improvement. For the mall problem, the situation is different. Results found by the SGA are good with high f easibility. T his indicates the higher number of feasible solutions for th is problem. Solution quality seems rea sonably good, too. However, the add ition of the pyramidal structure (S ) results in a marke d deterioration of result s. How can these different results be explained? With the nurse scheduling , the objective function value o f a partial solution was obtained by s umming the cost values of the n urses and shift-patterns involved. Furthermore, we were able to de fine relativ ely mean ingful sub- fitness scores by exploiting the ‘ cum ulative’ nature of the covering constraints due to t he grade structure. Hence, the substitute fitness scores calculated allowed f or an effective recombination of partial solutions f or the nurse-schedu ling probl em. T hus, there i s a good corr elation between the su b-fitness of an agent (and hence i ts r ank and its chance of b eing selected) and 11 the l ikelihood that it will form part of a good solution. T his also explains why t he random (R) sc heme produces worse r esults. T he best (B) strategy althou gh giving be tter results than th e random selection fails to solve many problems. How ever, closer observation of experime nts showed that it solved some single data sets well. This indicates th at genetic variety is as important as fitn ess in the evolution of good solutions. Both t he distributed (D) and j oint (J) strategies again fa il to provide better solutions than the rank-b ased selection. The d i stributed st rategy is sim ilar to the random strategy as it too ignores fitne ss scores for selection. Choosing from a fixed pool does h ave so me benefi ts, as the results are better than for complete random choi ce. The joint strategy works almost as well a s the rank-selection. T his shows t hat the principle of the ‘dividing an d co nquering ’ works wel l with the n urse problem split along the grade boundaries. The slightly poorer r esults can be explained by t he ‘full’ evaluation of all sub-strings although only ‘p arts’ a re passed on. Thus, some of the correlation descr ibed abov e is lost. The two best strategies, both outperforming (S), are p artner selection based on attractiveness (A) and choice (C). Again, this further confirms t hat the partial s ub-fitness scores are a good criterion of selection for the pyramidal algorithm. Overall , (C) i s better than (A), which corresponds to (C) having a h igher selection pressure than (A), which in turn has a higher selection pressure than (S). To conclude, it seems that for this problem a good correla tion between agents’ sub-fitness, the pyramidal structure and good full solutions exist. Hence, the scheme w ith the highest selection pr essure using m ost problem specific information sco res best. With t he Mall Problem, the situatio n i s more complicated since unlike for the nurse problem a large part of the objective function is a source o f epistasis, which the proposed partitioning of the string wil l not eliminate fully. The constraints are a second source for epistasis. In contrast to the objective function, th ese depend largely o n the who le string, a s for i nstance th e total nu mber of s hops of a particular size allowed. Only after adding up the shops and sizes for all areas is it known if a solution is feasible or not. So unsurprising ly, a combination of these par tial solutions is often unsuccessful be cause it usually v i olates the overall cons traints. On their own, solutions of the sub-populations are extrem ely unlikely to be feasible for the overall pr oblem, as they covered only one fifth of the string. It is equally unlikely for those solutions in the main popula tion, which are formed fr om the f ive sub-populations, to be feasible. A lthough t hese solutions are of high r ent, because the su b-p opulations i gnore the main constraints, their combination is unlik ely to produce an overall feasible solution. The situation is only slightly better with those solutio ns f ormed by an i ndiv idual of th e sub-populations and an individual of t he main population. Usual ly, even if the in dividual of t he main population is feasible, the children were not. Again, althoug h the partial string from the sub-population agent was of high rent, it was usually incompatible w ith the rest of the string, resul ting in too many or too f ew shops of som e types. Thus, in contrast to t he nurse-scheduling problem, t heir s ub- fitness scores are a far poorer predictor for the compatibility of the par ts to form com plete solutions. This is c onfirm ed by t he above average performance of the random strategy (R ) and t he extremely poor results found b y the best str ategy (B). Similarly to before, t he distributed strategy (D) performs well again giving credit to the idea of even selection pressure without relying on fitness scores, whereas the joint strategy (J) performs poorly s uffering both fr om the unsuitable sub-fitness s core s and the now hindering pyramidal structure. 12 Overall, the real w inners are again the mo re comp lex strategies o f choice (C) and attraction (A). At first, this seems con tradictory as these rely heavily upon the su b-fitness scores. However, ap art f rom the rank- based initial selection of the first parent, sub sequent fitness calculations are made after combining the agents. Since the mall pyramid only has two l ayers, these combinations are always full solution and hence the full fitness score is used. Thus, the direct link between high fitness and good solutions is r e-established. O f the two, (A) performs better th an (C). This se ems to show that a certain amount o f randomness i s still i mp ortant here, which again mig ht be an indication for the lower predictive qu ality of the sub- fitness scores. N Cost N Feasibility M Rent M Feasibility Bound 8.8 100% 2640 100% SGA 54.2 33% 1850 94% S 17.6 75% 1540 78% R 37.4 54% 1790 86% B 27.1 57% 1490 70% D 26 .5 61% 1770 84% J 19.9 71% 1590 78% A 12 .2 83% 1950 98% C 11.1 87% 1910 94% Table 1: Partnering Strategies for Recom bination Results (N = N urse, M = Mall). 5.3 R ESULTS: PART NERING FOR FI TNESS EVALU ATION Table 2 shows the results for a variet y of fitness evaluation strategies used a nd again c omp ares these to the theoretic bounds (Bound) and the st andard genetic algorithm approach (SGA). For the Nurse Scheduling Problem a ll strategies used give better results than those found by the SGA. However, as expl ained above, most credit for this is att ributed to the pyram idal structure reducing epistasis. On closer examination, r ank- bas ed (S), random (R) and distributed ( D) p erform alm ost equally well, with the rank-based method b eing sli ghtly better than t he o ther two. All three methods have in common that t hey contain a stochastic element in the choice of pa rtner. The benefit of this is apparent when compared to the best (B) method. Here the resu lts are far worse which we attributed to t he inherently restricted sampling. Interestingly, using the double sc hemes improves resu lts across t he b oard, w hich again strengthens our hypothesis how important good s amp ling is. The overall best results a re found by the double random (RR) method. These resul ts correspond to those reported in [Bull 1997]. The results for t he M all problem are s imilar to t hose found for the nurse problem : Double st rategies work better than single ones and the Best strategy does particularly poorly. However, unlike for the nurse scheduling none of the single strategies significantly improves r esults over the SGA approac h. Reasons for this have already b een outlined in t he previous sections, i.e. mainly the nature of splitting the problem into sub-problems being con trary to many of the problem’s constraints. On the o ther hand, e ven for t he simpl e strategies results are f ar improved over those found by using t he p artnering strategies for mating, whilst those found by t he double strateg i es even outperform the SGA . W e believ e that this can be explained as fol lows: The main downfall of the part nering fo r mating strategies for the mall p roblem was outside those strategies. It 13 lies in t he fact that the sub-fitness scores are not a goo d predictor for the succe ss of sub-solu tions. Howeve r, as these results show, i f the original fitness measu res are used combined with go od partnering methods the pyramidal structure does work. This confirms our suspicion that the previous ‘failure’ of the pyramidal idea for t he mall problem was r ooted within our c hoice of su b-fitness measure rather than in the hierarchical sub- population idea itself. N Cost N Feasibility M Rent M Feasibility Bound 8.8 100% 2640 10 0% SGA 54.2 33% 1850 94% S 13.3 79% 1860 90% R 14.5 77% 1915 94% B 35.9 44% 1550 72% D 14.6 77% 1820 88% SR 12.7 84% 1950 99% BR 14.2 81% 1897 86% RR 12.1 83% 1955 99% Table 2: Partnering Strat egies for Fitness Evaluat ion Results (N = Nurse, M = Mall). 5.4 R ESULTS: NURS E SCHEDULING WITH A HILLC LIMBER The results presented so far show that even w ith t he best algo rithm for the nurse schedu ling problem some data instances were unsolvable. I n order t o overcome this, a special hil lclimber has been developed wh ich is fully described in [Aickelin & Dowsla nd 2001]. T he use of local search to refine solutions produced via th e GA f or co mplex pro blem domains i s well est ablished – often termed memetic algorithms [ e.g. Mosc ato 1999]. Briefly, the hill-clim ber is l ocal search based algorithm that iteratively tries to improve soluti ons by (chain-) swapping shift p atterns between nurses or alternatively assigns a strictly sol ution improving pattern to a nurse. As the hill climber is co mputationally expen sive, it is only used on those sol utions showing favourable characteristics for it to exploit. T hose solutions are r eferred to as ‘balanced’ and one example is a nurse surplus one on day shift and a shortag e on another day shift. The last set of experiments presen ted in table 3 shows what impact the best partnering schemes for mating (Choice) and ev aluation (RR ) have once the previ ously excluded h illclimber is at tached to th e genetic algorithm. The res ults reveal that the improvem ents made by the partnering fo r mat ing strategies are equalled by the SGA once both ha ve access to th e hill climber. The best solutions are found with the double r andom fitness evaluation approach coupled with the hill climber. One possible explanation for this effect can be found by having a cl oser l ook at the choice mating operator, where an individual pi cks the best fitting partner from a set. So effectively, a crossover-h ill climber strategy is at work here. I n the RR c ase, all gains are also made due to better sampling . However, as mentioned before t here is a large stochastic elem ent i nvolved in this case. J udging fr om these results i t seems that again t his is beneficial as it leads to a bigg er variety of solutions in turn leav ing more for the hill cl imber to exploit. 14 N Cost N Feasibility SGA & Hillclimber SGA&H 10.8 91% Choice & Hillclim ber C& H 10.7 90% RR & H illclimber RR&H 9.9 95% Table 3: Results for Algo rithm s combined with a Hillclim ber for the Nurse Schedu ling Problem . 6 CONCLUSIONS This paper has s hown the effect d ifferent partner strategies h ave on a pyramidal coevolutionary genet ic algorithm solving two di fferent optimisation problems from t he a rea of m ultiple-choice scheduling. The recombination results for the five simple strategies ( S, R, B, D and J) differ for both problems. T his i s a reflection of the ac curateness o f the sub-fitness measure in the sense of it s predictive power for sub-solutions to form f ull solutions following the pyramidal recom bination strategies. Therefore, in the ca se of the nurse problem wit h a g ood m atch between sub -fitness and usef ulness f or recombin ation the simple strategies worked well, whereas for the mall problem with its poorer corre lation between the two it d id not. The t wo more advanced strategies ( A) and (C) use most problem sp ecific knowledge and work well for both problems. T hey worked well for the nurse problem b ecause the sub-fitness scores are m eaningful. They also worked well f or the mall problem because the partners are chosen based on a fit ness score after recombination, which in this case equals th e full origina l fitness score. T hus, choosing parents ‘post- birth’ after evaluating possib le children can overcom e possible shortcomings in the su b-fitness m easure. Using the partnering st rategies for evaluation purposes yi elds results in acc ordance with t hose reported in [Bull 1997]. For both p roblem the simple str ategies worked equally well a part from the restricting ‘best’ choice. Comb ining two p artnering sche mes i mp roved results further wit h the overall best solutions found by the double random strategy. Interestingly, the improveme nts of results seemed to be of a different nature than those found by the recom bination s trategies. There the best method was a hill-clim ber crossov er t ype approach; here the improvement seems to be based on better sampling and more diversity. Thus for the latter approach an additiona l hillclimber is able to im prove solutions beyo nd the prev iously best ones. REFERENCES Ackley, D.H . & Littman, M.L. (1994) Altruism in t he Evolution of Commu nicat ion. I n R Brooks & P Maes (eds.) Artificial Life I V, MIT Press, Mass., pp 40-48. Ahluwalia, M . & Bull, L. (19 98) C oevolving Funct ion s in Genetic Programm ing: Dynamic ADF C reation using GLiB. In V .W. Porto, N. Saravanan, D. Wagen & A .E. Eibe n (eds.) Proceedings of t he Seventh Annual Conference on Evolu tionary Program ming. Springer Verlag, pp 809 -818. Aickelin, U. ( 1999).Genetic Algorithms for Mu ltiple- Choice Optimisation Problems. PhD Disse rtation, University of Wales, Swan sea, United K ingdom. Aickelin, U. & Dowsland, K. (2000). Exploiting pr obl em struct ure in a genetic algorithm appr oach to a nurse rostering problem. Journal of Scheduling 3, pp 13 9- 153. 15 Aickelin, U. & Dowsland, K . (2 001). An indirect genetic algorithm a pproach t o a nurse scheduling problem. Under review by the Journal of Computi ng and Operation al Research. Bean J, Noon C, Ry a n S, Salton G (1988) Selecting Tenants in a Shopp ing Mall. Interfaces 1 8, pp 1-9. Bull, L. (1997) Evolutionary Computing in Multi-Agent Environments: Partners. In T. Baeck (ed.) Proceedings of the Seve nth I nternational Conference on Genetic Algorithm s, Morgan Kaufmann, pp 370- 3 77. Bull, L. & Foga rty, T.C. (1993) Coevolving Comm unicating Classifier System s for Tracking. In R F Albrecht, C R Reeves & N C Ste ele (eds.) Artificial Neural Networks an d Genetic Algorithms, Springer- Verlag, New York, pp 522- 5 27. Davidor, Y. (1991) , Epistasis V ariance: A Viewpoint on GA-Hardness. In G Rawl ins (ed) Foundations of Genetic Algorithms, Mo rgan Kaufmann, San Mateo, pp23- 35. Davis, L. (1991) Handbo ok of Genetic Algorithm s. Van Nostrand Reinhold Fuller, E. (1998) T ackling Scheduling Problems Using Integer Programm ing. Master Thesis, University of Wales Swansea, United Kingdom. Holland, J. (1975), Adapta tion in Natura l and Artificial Systems. Ann A rbor: Univ ersity of Michigan Pr ess. Husbands, P. (1994) Distributed Coevolut ionary Genetic Algorithms fo r Multi-Criteria and Multi-Constraint Optimisation. In T.C, Foga rty (ed.) Ev olutionary Computing, Springer-Verlag, pp 150-165. Iba, H. (1 996) Emergent Co-operation for Multiple Agents Using Genetic Programm i ng. In H-M Voigt, W Ebeling, I Rechenb erg & H-P Schwef el (eds) Parallel Problem Solving from Nature - PPSN IV, Springer, Berlin, pp 32- 41. Koza, J.R. (1991) Genetic Programm ing. MIT Press. Martello, S. & Toth, P. (1990), Knapsack Problem s, Wiley, Chichester. Moscato, P. ( 1999), Memetic Algorithms: A Short Introduc tion. In D. Corne, M. Dori go & F. Glover (eds), New Ideas in Optim ization, Mc Graw Hil l, pp 219-234. Potter, M. & De Jong, K. ( 1994) A Co-o perative Coevolutionary Approach to F unction Optimisation. I n Y Davidor, H-P S chwefel & R Manne r (eds) Parallel Problem Solving Fro m Na ture - PPSN III, Sprin ger- Verlag, Berlin, pp 249- 259 . Ronald, E. (1995) W hen Selection Meets Seduction. In L. Eshelman (ed) Proceedings of the S event h International Conference on Genetic A lgorithms, Morg an Kaufmann, San Francis co, pp 167- 173. Smith, A. & T ate, D. (1993) Genetic Optimisation Using a Penalty. In S. F orrest (ed) Proceedings of the Fifth International Conference on Genetic A lgorithms, Morg an Kaufmann, San Francis co, pp 499- 505. Stanley, A. E., Ashlock, D. & T estatsion, L. (1994) Iterated P risoner’s Dilemma with Cho ice and Refusal of Partners. In C G Langton (ed) Artificial Li fe III, Addison-Wesley, Redwood Ci ty, pp 131-146. Wolpert, D. & M acready, W. (1997) No Fr ee Lunch Theorem for Search. IEEE T ransaction s on Evolutionary Com putati on 1(1), pp 6 7-82.