A theoretical framework for conducting multi-level studies of complex social systems with agent-based models and empirical data

A theoretical framew ork for conducting m ulti-lev el studies of complex so cial systems with agen t-based mo dels and empirical data Chih-Ch un Chen 1 Abstract A formal but in tuitiv e framework is in tro duced to bridge the gap b et w een data obtained from empirical studies and that generated b y agen t-based mo dels. This is based on three k ey tenets. Firstly , a sim ulation can b e given m ultiple formal descriptions corresp onding to static and dynamic properties at differen t levels of observ ation. These can b e easily mapped to empirically observed phenomena and data ob- tained from them. Secondly , an agent-based mo del generates a set of closed systems, and computational sim ulation is the means by which w e sample from this set. Thirdly , prop erties at different levels and statistical relationships b et w een them can be used to classify sim ula- tions as those that instan tiate a more sophisticated set of constrain ts. These can b e v alidated with mo dels obtained from statistical mo d- els of empirical data (for example, structural equation or multi-lev el mo dels) and hence pro vide more stringen t criteria for v alidating the agen t-based mo del itself. ABM framew ork for m ulti-level studies of complex so cial systems 3 1 In tro duction Man y so cial and economic phenomena can be characterised in terms of ‘com- plex systems’ . Within this c haracterisation, entities and patterns of b eha viour emerge at differen t lev els, and interact with one another in non-linear wa ys. Agen t-based mo delling (ABM) is a computational metho d for mo delling and sim ulating such systems. The complex systems p ersp ectiv e has tw o ma jor strands. F rom a more Statistical Mec hanics-orien ted view, the study of complex systems has fo- cused mainly on the wa ys in which lo wer level micro-prop erties and inter- actions give rise to higher level macro-prop erties (F eldman and Crutchfield, 2003), (Ellis, 2005). The more biologically-based approac h tends to focus more on relating properties at different levels, suc h as functional modules in the brain or bio c hemical path wa ys and netw orks (in some cases, suc h as feedbac k, the emergen t phenomenon ma y ev en be at the micro-lev el) (V arela, 1979), (T ononi et al., 1994), (Hart well et al., 1999). Both these strands should b e lev eraged in the so cial sciences. F rom a policy point of view, it is imp ortan t to understand how changes in rules at the micro-level (whic h migh t represent the interaction b et ween psyc hology and p olicy) affect more macro-level behaviours (whic h might in- clude those asso ciated with family , organisational, or geographical units). A t the same time, w e often ha v e imp ortan t information ab out the w a y decisions or b eha viours of units at differen t lev els relate to eac h other (for example, ABM framew ork for m ulti-level studies of complex so cial systems 4 if several commercial organisations dominate a sector, this can affect b oth other organisations within the sector and other sectors). Curren tly , the complex systems approac h is largely mo del-dominated and/or mo del-driv en (whether mo dels are informal, formal, mathematical, or statistical). While this allows theories to b e specified with precision, there is a risk of alienating those conducting empirical research and hence mo d- els b ecoming irrelev ant or to o idealized for real w orld application. There is therefore an urgen t need to establish robust tec hniques for analysing and v al- idating mo dels with resp ect to empirical data, particularly as, unlike in the ph ysical sciences, idealizations of mo dels do not alw a ys hav e clear isomor- phism with empirically based studies (Henrickson and McKelv ey, 2002). As ABM is maturing and b ecoming more widely adopted in the so cial sciences (Bonab eau, 2002), (Sawy er, 2001), (Gilb ert and T rioitzsc h, 2005), (F o cus, 2010), it is crucial that the appropriate metho ds of analysis are applied and that the conclusions we draw from these analyses are v alid. This requires an understanding of their theoretical basis and rationale. F urthermore, a rigorously grounded theory allows us to defend the con- clusions w e dra w from v alidating mo dels against empirical data and av oid the doubts often cast up on the utility and v alidity of agent-based mo dels (see, for example (McCauley, 2006)). Related to this are questions regarding the in terpretation and analysis of simulations , for example: • Ho w many sim ulations do we need to run to dra w a conclusion? ABM framew ork for m ulti-level studies of complex so cial systems 5 • Ho w do w e interpret differences betw een simulations? • Ho w do we use empirical data and sim ulation-generated data to v alidate a mo del? (Discussions of different aspects of ABM empirical v alidation issues can b e found in (Kleijnen, 1995)„ (Axtell et al., 1996), (Kleijnen and Kleijnen, 2001), (F agiolo, 2003), (T roitzsc h, 2004), (Brenner and W erker, 2007), (Marks, 2007), (Windrum et al., 2007), (Moss, 2008).) • Ho w do w e c ho ose b et ween different agen t-based mo dels and parameter configurations when they are all able to generate empirically v alid data? This article introduces a simple theory of t yp es for describing agen t-based sim ulations at differen t levels and relates this to the application of differen t established analytical tec hniques. The theory is based on three fundamental tenets: 1. Theoretically , an agen t-based mo del generates a finite set of formally describable closed complex systems, and simulations are the means by whic h w e sample from this set. In other w ords, each sim ulation is an instan tiation of a p ossible system generated b y the mo del; 2. A sim ulation can b e formally described in terms of prop erties or phe- nomena at different levels, with micro-level prop erties corresp onding to computational states end ev ents, and higher level prop erties corre- sp onding to sets and/or structures of these states and even ts. (Higher lev el prop erties suc h as p opulation behaviour can also b e expressed in ABM framew ork for m ulti-level studies of complex so cial systems 6 terms of macro-v ariables and c hanges in macro-v ariables, which would define the sets of ev en t structures; see Section ?? ) 3. Prop ert y descriptions at different lev els can b e used (either in isolation or in com bination) to classify sim ulations. These tenets allow us to build more stringently constrained mo dels relat- ing phenomena at different lev els, which pro vide stricter criteria for v alida- tion with empirical data. Instead of simply requiring that some phenomenon ‘emerges’ at the systemic leve l in simulations, structur es of r elate d phenom- ena (p ossibly at different scales and/or levels of abstraction) need to b e repro duced with appr opriate fr e quencies or pr ob abilities . Before commencing, we wish to emphasize that the so cial sciences cov er a v ast landscap e of disciplines and domains, and that eac h domain (and sub domain) will hav e its own set of issues to address when b oth developing and v alidating agent-based mo dels. The hop e is that each sp ecific domain will b e able to adapt, extend and apply our framework for their sp ecific purp oses. 2 Bac kground and motiv ation: The applica- tion of ABM in the study of so cial systems Quan titative c haracterization of dynamic so cial and economic systems is of- ten problematic b ecause suc h systems are complex. By complex, it is mean t ABM framew ork for m ulti-level studies of complex so cial systems 7 that the b eha viour of these systems arises as the result of in teractions b e- t ween multiple factors at different lev els (w e will formalise the notion of levels in Section 4). In the Complex Systems literature (particularly from Statis- tical Ph ysics) the terms ‘non-equilibrium’, ‘non-linear’ and ‘non-ergo dic’ are often used to refer to describ e suc h system. The difficult y p osed by suc h systems is that knowledge of micro-b eha viour do es not guaran tee kno wledge of the macro-b eha viour, and vice versa. There are tw o asp ects to this. Firstly , the macro-lev el b eha viour by definition can not b e descriptively or logically reduced to micro-level b eha viour; language used to describ e the micro-lev el is therefore logically distinct from that used to c haracterise the macro-lev el (Darley, 1994), (Bonab eau and Dessalles, 1997), (Kubik, 2003), (Deguet et al., 2006). This follo ws from the fact that micro- and macro-level phenomena require different levels of observ ation to be manifest (Crutc hfield, 1994), (Crutchfield and F eldman, 2003), (Sasai and Gunji, 2008), (R yan, 2007), (Prok op enk o et al., 2009). Secondly , in contrast to systems in equilibrium in whic h differences at the micro-lev el mak e little difference to macro-lev el observ ations and hence for whic h we can predict macro-lev el b eha viour from micro-lev el observ ations, complex systems are sensitiv e to relatively small p erturbations. This sensi- tivit y means that p erturbations at the micro-lev el can ha v e non-linear effects at the macro-level (Kauffman, 1993), (Holland, 2000), (Y am, 2003), (Ellis, 2005). The motiv ation for ABM comes from b oth these asp ects. In ABM, the ABM framew ork for m ulti-level studies of complex so cial systems 8 micro-lev el is sp ecified computationally in the form of state transition rules ( S T R s) gov erning the b eha viour of computational agen ts and the macro-level b eha viour is usually represented by system-lev el global state v ariables which aggregate in some wa y the states or b eha viours of the agents in the system. These t wo mo des of representation can b e seen to resp ectiv ely represent the logically distinct micro- and macro-lev el languages. At the same time, ABM is used to study the effects of p erturbations at the micro-level, which are in tro duced as differences in initial conditions and/or parameters. The types of questions that ABM practice typically try to address are: • Ho w differen t are the b eha viours of sim ulations generated from different initial conditions? • Whic h parameters is the mo del most sensitiv e to? • Under which parameter configurations and/or ranges is the b eha viour most sensitiv e or stable? Ho wev er, the idea that complex, non-linear relationships exist b et ween phenomena at differen t lev els is in fact extremely perv asiv e in empirical studies. The key difference b et ween suc h studies and more mo del-cen tric approac hes to complexit y lies in the metho ds used to analyse and represen t this complexit y . In empirical studies, the techniques tend to fo cus more on interactiv e statistical asso ciations b etw een phenomena e.g. (Pearl, 1998), (Krull and MacKinnon, 2001), whic h tend to b e represented in netw ork-based or hierarc hical mo dels, suc h as Bay esian net w orks, structural equations, or ABM framew ork for m ulti-level studies of complex so cial systems 9 m ulti-level mo dels. In such represen tations, the relations b et ween ‘lev els’ are not formal, but descriptive (based only on our understanding of the phe- nomena) or statistical (as in the case of multi-lev el (Gelman, 2006), (Gelman and Hill, 2006) or mo dular (Seth, 2008) mo dels). Model-driven studies on the other hand, tend to consider associations in terms of their fundamental statistical mechanics ( ? ) or emergent netw ork dynamics (Barab´ asi, 2002), (Dorogo vtsev and Mendes, 2003). This pap er seeks to explicitly relate these tw o p ersp ectiv es using an ex- tended ABM framework that p ermits the represen tation of prop erties and b eha viours at an y lev el of abstraction and the relationships b et ween them, going b eyond simple tw o-level micro-macro/macro-micro relationships. 2.1 Hyp otheses, empirical data, mo dels and sim ula- tions Empirical v alidation is a significan t c hallenge that needs to b e o v ercome in order for ABM to b ecome more seriously adopted in the Social sciences. W e can classify v alidation techniques according to the types of h yp otheses they supp ort. T o date, the motiv ation for applying agent-based mo delling tends to b e motiv ated b y the follo wing tw o hypotheses classes: • Hyp otheses concerning the abilit y of mechanisms and in teractions at the micro-level to give rise to phenomena at the systemic lev el, for example, attraction/repulsion -¿ regional segregation. In these cases, ABM framew ork for m ulti-level studies of complex so cial systems 10 qualitativ e data can b e used to (weakly) v alidate the mo del (i.e. show that it is not false). At the micro-level, these migh t b e based on findings from Psyc hology or on enforced p olicies. A t the systemic level, they migh t b e anecdotal or ev ent-based observ ations. This is illustrated in Figure 1. • Hyp otheses concerning the conditions under whic h mec hanisms and in- teractions at the micro-lev el are able to giv e rise to phenomena at the systemic level, for example, attraction/repulsion -¿ regional segrega- tion when the initial div ersity of agen ts exceeds a particular threshold. With these t yp es of h yp otheses, v alidation would require empirical data ab out both the initial configuration and the observ ed phenomena (e.g. regional distribution of different ethnic bac kgrounds at t 1 and t 2 ). This is illustrated in Figure 2. Ho wev er, w e can also analyse agent-based mo dels with empirical data to address the follo wing: • Hyp otheses relating micro-lev el mechanisms to r elationships b etw een phenomena at differ ent lev els, including ho w they might interact to giv e rise to global systemic phenomena. F or example, w e could for- m ulate and v alidate a mo del that describ es the relationship b etw een individuals’ psychology , p olicy decisions, regional migration, lo cal un- emplo yment, and the coun try’s economy . This would require empirical data relating to each of the phenomena at the different lev els. If the ABM framew ork for m ulti-level studies of complex so cial systems 11 totalit y of qualitativ e effects are observ ed, then we can sa y the mo del is v alid. This can be expressed as a graph or net work. If the data we ha v e is quan titativ e, the edges of the graph c an also b e w eigh ted to represen t the strength of the relationships. This is illustrated in Figures 3 and 5. • Hyp otheses ab out the conditions under whic h relationships b etw een phenomena at differen t levels hold. This w ould require data from differ- en t instances of the related phenomena, including their non-occurrence. F or example, w e w ould need to ensure that the cases in which the re- lationships hold ha ve the same features (or feature com binations) as h yp othesised, and that these features combinations are not found in the cases in which the relationships do not hold. It is also p ossible that this a matter of degree e.g. factor X reduces the strength of association b et w een phenomena A , B , and C . This is illustrated in Figure 4. 2.2 A general c haracterisation of ABM T o ensure w e ha v e as general a characterisation of ABM as p ossible, w e do not base our framew ork on an y sp ecific mo delling language or softw are framew ork, but instead giv e an abstract definition that can be easily mapp ed to existing ABM framew orks. W e define an ABM as a set of agent t yp es A 0 , ..., A n (global state v ari- ables, e.g. represen ting resource a v ailabilit y , and dynamic spatial represen- tations can also be represen ted as agen t t yp es in this abstract formulation) ABM framew ork for m ulti-level studies of complex so cial systems 12 and constraints C determining how agents are able to in teract in the system (for example, whether they can communicate directly , sync hronously , asyn- c hronously , symmetrically , asymmetrically or via some sp ecified proto col or top ology; this also determines the updating or execution order). Eac h agen t t yp e A i consists of a set of v ariables with defined v alue ranges and a set of state transition rules ( S T R i ). S T R s can b e seen to represent the range of p ossible b eha viours for agen ts (instantiations) of the particular t yp e A i and therefore enco de the knowledge w e hav e ab out individual- or micro-level b eha viour. The set of v ariables and v alue ranges define the set of states that agen ts of the t yp e are able to realise. W e define a state transition rule S T R Ai to b e a function that maps (i) a source subsystem state ( ϕ source ) represented by the v alues of some subset of the system’s state v ariables (which might b e encapsulated in the agent itself or b elong to other agents and/or elements in the system) to (ii) a target subsystem state ( ϕ targ et ) represented by some new set of v alues for the set of v ariables when a particular condition cn is satisfied. The mapping ϕ source → ϕ targ et is the state transition, as defined b elow: State transition A state transition is a transformation of one subsystem state to another subsystem state. The state b efore the transformation is applied is called the source state and is denoted ϕ source , while the state after the transformation has b een applied is called the target state and denoted ϕ targ et . (The definition for subsystem state is given in Definition ?? ). ABM framew ork for m ulti-level studies of complex so cial systems 13 State transition rule ( S T R ) S T R Ai ( cn ) = ϕ source → ϕ targ et , (1) where cn ∈ C N , and C N denotes the set of conditions that can b e distin- guished b y agen ts of the type A i . * The condition cn under whic h an S T R is executed might b e dep endent on the agent’s o wn state q a , the state q e of its environmen t or neighbourho o d e (which might itself b e made up of other agents’ states), or b oth. State transition rules migh t also b e expressed implicitly in terms of constrain ts on p ermissible action as w ell as explicitly in terms of conditional state changes, but these are formally equiv alen t. In the most general terms therefore (abstracting aw a y from particular formal languages or mo delling framew orks), an agent-based mo del is a set of agen t t yp es with a set of constraints gov erning the interactions b et w een agen ts. 3 Agen t-based mo dels as b oth generators and classifiers of system t yp es T o truly understand what w e are doing when w e run simulations of agen t- based mo dels, it is necessary to delve a little into some of the tec hnical ABM framew ork for m ulti-level studies of complex so cial systems 14 computational details of simulation. Although the practice of agent-based mo delling should b e seen as abstracted from computational matters (just as programming languages are seen as distinct from machine co de), when running sim ulations, the realisation of computations can hav e certain impli- cations. The follo wing are esp ecially imp ortant to note: Execution order Different orders of execution and state up dating can lead to radically different outcomes, ev en with the same initial conditions and parameter settings (Garg et al., 2008), (Blok et al., 1999). In fact, we can see differen t up dating rules as an extension of the agent-based model itself, since the set of systems generated b y one set of up dating rules (e.g. asynchronous) is different to (and may not ev en o verlap with) the set generated by another (e.g. synchronous). Set of systems generated The set of p ossible systems (distinguishable sim ulations) that can b e generated from an agen t-based mo del can b e ar- bitrarily limited b y the nature of the platform on whic h it is run. This is particularly p ertinen t in cases where real (rather than in teger) v alues are included in the mo del or where sto c hasticit y features. In the case of real v al- ues, the memory limitations mean that accuracy is limited. In other words, the set of p ossible simulations only includes systems in which w e are able to measure a v ariable to n decimal places. While this might at first seem trivial, the implication is theoretically significant, since it means that the ABM framew ork for m ulti-level studies of complex so cial systems 15 set of systems w e are able to study computationally (if we w ere to sim ulate ev ery p ossible system) is only a subset of the p ossible systems that could theoretically b e generated by our agen t-based mo del. More generally , if w e see each distinguishable simulation 1 as a computa- tional representation of a system that the agent-based mo del can generate, it is clear that how ever man y simulations w e run, the set of systems that w e can study is finite, ev en if the agent-based mo del is theoretically able to generate an infinite set of systems. (W e will formalise this later in terms of complex ev ent t yp es.) 2 In the remainder of this section we will prob e more deeply into the impli- cations of this for three important asp ects of simulation: (i) mo del concreti- sation for v alidating predicted b ehaviour; (ii) sampling to determine ‘t ypical’ b eha viour; (iii) probing to ev aluate parameter sensitivit y . 3.1 Sim ulation as mo del concretization In the practice of agent-based mo delling, the most basic function of simula- tion is to establish whether or not the mo del defined at the agen t lev el is able to generate some phenomenon at a higher systemic lev el. This is t ypically represen ted by one or more state v ariables that aggregate individual agen ts’ state v ariable v alues. In many cases, mo dels are also parametised so as to capture some features of the system b eing modelled, so that the higher level phenomenon is h yp othesised to o ccur within some defined v alue range(s). ABM framew ork for m ulti-level studies of complex so cial systems 16 Sim ulation is therefore treated as a means of determining what happ ens when the statically represented agent-based mo del (expressed in terms of agen t state transition rules) is concretised and dynamically executed under particular conditions (represen ted b y parameters and initial conditions). Returning to the fact that the set of distinguishable sim ulations is finite, the implication is that ev en if w e were to run every p ossible simulation, we nev er observe the desired systemic phenomenon ev en though the agen t-based mo del is theoretically able to generate it. In other words, we are only able to concretise part of the agent-based mo del (this is equiv alen t to saying we can only sample a subset of the p ossible systems the mo del can generate; see b elo w). This is esp ecially problematic when the phenomenon we are trying to understand itself a one-off or rare ev ent. In this case, we hav e no informa- tion ab out how probable the phenomenon is under the conditions we ha v e represen ted in the concretised mo del. Hence, ev en if the concretised model (sim ulation) do es emulate the phenomenon, w e are not really en titled to dra w any strong conclusions (unless w e hav e extremely detailed information ab out the initial conditions and the phenomenon is only repro duced in simu- lations where these initial conditions are realised; this is the rationale b ehind ‘history-friendly’ v alidation (W erker and Brenner, 2004)). ABM framew ork for m ulti-level studies of complex so cial systems 17 3.2 Sim ulation as sampling Another widely adopted approac h to simulation is to treat it as sampling (see Figure 6). In terms of data ab out the system b eing mo delled, this requires us to hav e information ab out the distribution or probabilit y with whic h the desired phenomenon o ccurs. When simulating therefore, it is not sufficien t simply to repro duce the phenomenon, but to repro duce it to the correct degree. F or example, if our real w orld data tell us that phenomenon X o ccurs in 50% of the cases, only around 50% of our simulations should exhibit the phenomenon (assuming that we ha ve represen ted in our agen t- based mo del ev erything w e know ab out the system and that the fact that X is only observ ed in 50% of the empircal cases is due to the incompleteness of our kno wledge of the conditions necessary for it to o ccur). The issue with sampling from only a subset of systems implied by the agen t-based mo del is that neither our knowledge nor our ignorance is com- pletely represen ted. Hence the resulting distribution of simulations sampled is not strictly sp eaking a reflection of the information (or lack of information) w e hav e included in the agent-based model. 3.3 Sim ulation as probing Y et another approac h to agent-based simulation is to use it as a means of understanding the fundamental nature of the phenomenon b eing studied. This is strongly link ed to other complex systems modelling techniques, suc h ABM framew ork for m ulti-level studies of complex so cial systems 18 as equations or iterative maps. The type of mo del features that we are in terested in within this approach include for example, whether or not a phenomenon is sensitive to scale (scale in v ariance) or ho w the degree to whic h it o ccurs alters under differen t conditions (parameter sensitivit y). In other words, sim ulation is used as a means to b etter understand the mo del and its set of systems. The issue that arises here generalises that whic h arises when simulation is used as a means of sampling. If w e are using simulation as a means of understanding the shap e of the space of systems defined b y our mo del, the fact that we may only able to include a subset of the p ossible systems means that only a region of the p ossible lo cations in the space of systems will b e accessible to us, leading to a mis-representation of the shap e of this space. More concretely , our resp onse to the result that out of 1000 sim ulations, all exp ect one sho w sufficient agreemen t with our empirical data migh t b e very differen t dep ending on the type of study . W e could conclude that w e ha ve captured the essential mechanisms underlying the phenomenon describ ed by our empirical data and that our agen t-based mo del has b een v alidated. On the other hand, w e migh t wish to further in v estigate the differences b et ween the anomalous simulation and the others b y identifying the k ey differences (for example, differen t initial conditions, subsystem behaviours or global sub- tra jectories). Empirical data asso ciated with these distinguishing attributes could then b e sough t to provide further supp ort for the mo del (in the b est ABM framew ork for m ulti-level studies of complex so cial systems 19 case, the differences in the anomalous simulation w ould map directly onto an anomalous case in the real world with the same distinguishing attributes). On the other hand, even if the distinguishing attributes in the anomalous sim ulation are implausible (for example, they refute what we believe should b e p ossible in h uman interactions), w e might still accept the model as having sufficien t explanatory and predictive v alidity since the v ast ma jority of sim- ulations manage to repro duce what has been observ ed in the real world (of course, differen t domains will ha ve different tolerances to such discrepancies). F rom a theoretical p ersp ective, an agen t-based model can b e seen as both a generator and a classifier of systems. The totality of the set of systems that can p ossibly b e generated computationally is determined by (i) the agen t- based mo del; (ii) the up dating rules (whic h can b e seen as an extension of the mo del)the up dating rules (which can b e seen as an extension of the mo del); (iii) the set of parameter v alue com binations that can b e represented, includ- ing the initial conditions and the set of possible v alues for random generator seeds for sto c hastic mo dels (e.g. x 1 = [ 0 . 00000000000 , 0 . 9999999999] × x 2 = [0 . 00000000000 , 0 . 9999999999] × x 3 = [0 . 00000000000 , 0 . 9999999999]). Corresp ondingly , the abstractly defined unparametised agent-based mo del can be seen as defining a set of systems, with subsets defined b y sp ecific com- binations of (i), (ii) and (iii). Ev en more generally , any feature that can b e represen ted computationally in terms of the mo del, either as simulation in- put (as in the case of (i), (ii) and (iii)) or as some prop erty or b ehaviour ‘observ ed’ in the sim ulation (see Section 4 b elow), can b e seen to define a ABM framew ork for m ulti-level studies of complex so cial systems 20 subset of distinguishable systems and hence be used to classify sim ulations (see Figure 6). 4 ‘Lev els’ and ‘observ ations’ within sim ula- tions Although agent-based models w ere initially motiv ated by the desire to un- derstand how phenomena observ ed at one lev el can giv e rise to phenomena observ ed at another level. surprisingly little work has fo cused on formally defining lev els or observ ations in agen t-based sim ulations. This section ad- dresses this issue b y showing ho w to formally represent observ ations at dif- feren t levels in agen t-based mo delling terms. In order to do this, we b egin b y first defining what w e mean in general b y observing a system at differen t levels, and what it means to sa y that a prop ert y exists at a particular lev el. An imp ortan t p oin t to note is that the notion of level is b y its very nature a relative one; it only mak es sense to to sa y that some prop erty exists at a higher level than some other prop erty . Essen tially there are tw o types of relation that link low er lev el prop erties to higher lev el ones: 1. Comp osition, where low er level prop erties are the constituen ts of the higher lev el prop erty in some structured relation (e.g. Na + Cl -¿ NaCl) 3 ; ABM framew ork for m ulti-level studies of complex so cial systems 21 2. Set membership, where lo w er lev el prop erties b elong to a set defined b y the higher lev el prop ert y (e.g. dog -¿ mammal). (See Figure 7.) In man y cases, these tw o t yp es of relations are combined. F or example, in the case of ‘marriage’, not only do es the prop erty require the participation of tw o individuals in some structured relation, but it is also blind to whic h particular individuals participate in this structured relation. This can b e formally represen ted as a hypernetw ork (Johnson, 2006), (Johnson, 2007) or ‘heterarch y’ (Gunji and Kamiura, 2004). F urthermore, when sp eaking of levels, it is imp ossible to separate a prop ert y’s existence at a particular lev el from the observ ation or description of the prop erty at this lev el. The resolution or precision of observ ation is equiv alen t to set mem b ership (since a lo wer resolution implies more mem b ers b elonging to the set), while the scop e of observ ation is related to composition (since a greater scope implies more constituen ts) (Ry an, 2007), ( ? ). 4.1 Static and dynamic prop erties in sim ulations Prop erties in agent-based sim ulations can b e either static or dynamic. In terms of computational representation, static prop erties are subsystem states, whic h are represented b y the v alues of a subset of the v ariables (which migh t also cut across agent b oundaries, as in the example of group states, whic h take an aggregate of only a subset of the v ariable v alues within each agent mem- b er). Dynamic prop erties (or b ehaviours) are represen ted computationally ABM framew ork for m ulti-level studies of complex so cial systems 22 b y (p ossibly temp orally extended) structures of state transition rule execu- tions and state transitions. Indeed, every distinguishable system generated b y an agen t-based mo del can b e describ ed formally as a unique structure of S T R executions and their state transitions. 4.1.1 Static prop erties as v ariable v alues and their configurations A t an y giv en p oin t in time during the sim ulation, we can formally describ e the curren t state of the system as a structured set of v ariable v alues. F ur- thermore, w e can giv e descriptions of this structured set at different levels. F or example, from a single-agent lev el, the curren t state is describ ed simply as the set of state v ariable v alues encapsulated in the agen t. On the other hand, we can give descriptions that cut across agen t b oundaries, for example taking only a subset of differen t agen ts’ v ariable v alues (returning to the ex- ample of a marriage, w e do not necessarily need to kno w the colour of agen ts’ hair to obtain the n umber of married couples in the system at a given time, only the marital status). T o capture the observ ations or description of properties, we in tro duce the notion of typ es . A type is a sp ecification for a class of ob jects such that ob jects satisfying the sp ecification b elong to the set defined b y the class. T o formalise the observ ation of prop erties in sim ulation using the tw o notions of hierarc h y (comp ositional and subset, as defined ab ov e), w e define a subsystem state t yp e ( S S T ) using a h yp ergraph represen tation where the h yp eredges can b e either comp ositional or set relations (as defined by ab o ve). A hypergraph ABM framew ork for m ulti-level studies of complex so cial systems 23 is a generalisation of a graph, where instead of the edges b eing limited to binary relations b etw een t w o no des, they can b e n-ary b etw een an y n umber of no des. An S S T is then recursiv ely defined b y the h yp ergraph: S S T :: ( { S S T } , { R } ) | ( V AR, [ RG ]) , (2) where: • R is a comp ositional or subset relation connecting n S S T s • V AR is a v ariable; • [ RG ] is the range of v alues that the v ariable m ust fall within (to rep- resen t the prop ert y). • ( | stands for O R ) So for example, to observ e marriage, w e migh t define the S S T : sst M arr iag e = ( { ( sst M 1 ) , sst M 2 , S S T M 3 , sst M 4 } , { ( sst M 1 ∧ sst M 2 ∧ sst M 3 ∧ sst M 4 ) } ) , where • sst M 1 = ( husbI D , N otN ul l ) (an agen t has a h usband); • sst M 2 = ( w if eI D, N otN ul l ) (an agent has a wife); • sst M 3 = ( ag entI D , husbI D ) (identifies whic h agent the h usband is); ABM framew ork for m ulti-level studies of complex so cial systems 24 • sst M 4 = ( ag entI D , w if eI D ) (iden tifies whic h agen t the wife is); • w edg e stands for AND and is a comp ositional relationship. 4.1.2 Beha viours as ev en ts and structured ev ent executions Giv en that an imp ortant motiv ation for agent-based mo delling is often to b etter understand the relationship b etw een micro-level mec hanisms (repre- sen ted b y S T R s) and higher level phenomena, w e further distinguish b et ween b eha viours arising from the execution of a single S T R and those arising from an execution structure of S T R s. In general, a structure of S T R executions and their state transitions is called a complex even t. When a state transition results from only a single S T R execution, w e call it a simple event (a simple ev ent is also a complex ev ent, alb eit one whic h results from only one S T R execution). Each simulation is therefore a complex ev ent. As with states, observation of b ehaviour is formally represented using ev ent types, where an even t t yp e is a sp ecification defining a set of even ts (state transitions). T o resp ect the distinction b et ween even ts arising from the execution of a single S T R and those arising from more than one S T R execution, simple even t t yp es ( S E T s) are those ev ent classes where the re- quiremen t for class membership is determined at least in part by whic h S T R is executed. How ev er, for a giv en S T R execution, different observ ations (descriptions) are p ossible. F or example, an str i that results in the state transition ( v ar 1 , v ar 2) → ( v ar 1 0 , v ar 2 0 ) can be describ ed with three distinct S E T s (or observ ed at three differen t ‘lev els’): ABM framew ork for m ulti-level studies of complex so cial systems 25 1. { str i : [( v ar 1 , v ar 2) → ( v ar 1 0 , v ar 2 0 )] } ; 2. { str i : [ v ar 1 → v ar 1 0 ] } ; 3. { str i : [ v ar 2 → v ar 2 0 ] } ; F urthermore, executions of differen t S T R s might give rise to different state transitions, but should b e defined by distinct S E T s (i.e. str i : [ v ar 1 → v ar 1 0 ] 6 = S T R j : [ v ar 1 → v ar 1 0 ]). F ormally therefore, an S E T is defined b oth by a set of t wo-tuple: S E T :: ( S T R, S T ) , (3) where • S T R is a state transition rule, and • S T :: S S T → S S T 0 is a constrain t that the descriptio n (or observ ation) of the resulting state transition S S T → S S T 0 m ust satisfy . 4 So, for example, the S E T { S T R i : [ v ar 1 → v ar 1 0 ] , S T R j : [ v ar 2 → v ar 2 0 ] , S T R k : [ v ar 3 → v ar 3 0 ] } would b e the set of ev ents resulting from either S T R i , S T R j or S T R k observ ed at the one v ariable level whic h satisfy the constrain ts satisfied (e.g. v ar 1 > x, v ar 1 0 < y ... ). Complex even t t yp es ( C E T s) are even t classes defined by a structure or set of structures of state transitions resulting from a set of structured S T R executions (this w ould include S E T s, since S E T s are simply classes of ev ents ABM framew ork for m ulti-level studies of complex so cial systems 26 where the structure of S T R execution is a single execution). As with S S T s, this can b e defined as a h yp ergraph of C E T s, where each h yp eredge can b e either a comp ositional (structural) or subtype (set membership) relation. As in the case of S S T s, we are th us able to in tegrate the tw o types of hierarch y (comp ositional and set) introduced abov e within a common even t t yp e. The formal recursiv e definition can b e given as: C E T :: ( { C E T } , { R } ) | S E T , (4) where: • R is a comp ositional or subset relation connecting n C E T s • ( | stands for O R ) This definition is mainly for formal purp oses. While it is p ossible to sp ecify a C E T explicitly b y defining the relationships b et ween its constituen t or subtypes, this is not alwa ys p ossible in practice since these relationships are not alwa ys known or, if they are, it w ould b e extremely cumbersome to sp ecify them in the represen tation ab ov e. Indeed, the goal of simulation ma y b e to discov er such relationships. In practice therefore, it is more feasible to sp ecify C E T s implicitly using aggregated state v ariables; for example, w e migh t sp ecify a C E T that includes all those structured ev ents where a c hange in systemic v ariable X (e.g. mean p opulation crime rate) exceeds a giv en threshold a . One could then discov er the execution structures after sim ulation by examining the sim ulations where X exceeds a . ABM framew ork for m ulti-level studies of complex so cial systems 27 T able 1 outlines the empirical equiv alents of the S S T /C E T constructs defined ab ov e and gives examples of empirical data to whic h they can b e mapp ed. 5 In ter- and m ulti-lev el v alidation of agen t- based mo dels with empirical data Ha ving defined ho w w e can ‘observ e’ the dynamic instan tiation prop erties and b eha viours in simulation, we can also use these to classify the set of systems generated by an agent-based mo del (just as w e can use input parameter configurations to classify systems). The rep ertoire of mo dels that we can study has therefore b een extended from h yp otheses ab out how agent-lev el rules generate systemic properties, to hypotheses ab out how agen t-level rules generate r elationships b etwe en systemic prop erties. 5.1 In ter-lev el mo dels and v alidation Graph-based representations such as structural equation mo dels and Ba y esian net works hav e b een used in the so cial sciences to describ e structures of re- lated phenomena (usually represen ted as v ariables) and the nature of the relationships (e.g. their strength, p ositivity). W e call these structur al mo d- els . Com bined with the S S T / C E T framew ork defined ab ov e, we hav e a means to represent structured, defined relationships b etw een phenomena at ABM framew ork for m ulti-level studies of complex so cial systems 28 differen t lev els in terms of the agent-b ase d mo del itself , and not only as ad-ho c system-lev el state v ariables. W e call these inter-level mo dels . T o give a concrete example, as illustrated in Figure 8, if v ariables x 1 , x 2 , and x 3 resp ectiv ely (i) o verall crime rate, (ii) clan marriage rate, and (iii) clan size, w e can ask whether the agen t-based mo del is able to generate an in ter- lev el mo del suc h that x 2 is p ositively asso ciated with x 1 , and x 1 increases x 3 (V alue ranges for x 1 , x 2 , and x 3 are also implicit sp ecifications for three differen t C E T s). Assuming that the agen t-based model w as dev elop ed and parametised in an empirically-driv en fashion, we w ould require m ultiple data sets with data corresp onding to x 1 , x 2 and x 3 to v alidate the inter-lev el mo del. If the asso ciations sp ecified by the mo del are found in these empirical data, the inter-lev el mo del is said to b e v alid, in the sense that it has not b een sho wn to b e wrong. 5 . Similarly , if we ha v e data corresp onding to v ariations in parameter v alues (e.g. differen t policies at the individual level, which could b e translated in to agen t prop ensities for action), w e can hypothesise ab out the effects of inter- v entions at the agent level on the structural or inter-lev el mo del. Or, if we ha ve v ery little information ab out what migh t b e going on at the individual lev el, w e can classify simulations into those which generate these inter-lev el relationships and those whic h do not (or do so with a far w eak er degree), and then conduct further analyses to determine what the ‘unsuccessful’ sim ula- tions hav e in common. This might in v olve sp ecifying and identifying further C E T s or, more simply , ev aluating S E T frequencies (and hence agent level ABM framew ork for m ulti-level studies of complex so cial systems 29 S T R execution frequencies). If, say , w e find that a giv en S E T is asso ci- ated with an inter-lev el mo del, it would b e w orth probing further on the effects of the particular S T R asso ciated with this S E T . In real w orld terms, this migh t, for example, corresp ond to iden tifying a particular la w as b eing asso ciated with a self-p erp etuating w eb of so cial problems. 5.2 Multi-lev el mo dels and parameter spaces Giv en that an agen t-based mo del aims to represent the essen tial individual- lev el mechanisms underlying systemic phenomena, a deep er understanding of these mechanisms can only b e attained through probing the mo del’s b e- ha viour under different conditions. In practice, this is done through system ti- cally v arying the mo del’s parameters, which (either individually or together) can b e used to represen t differen t real-world scenarios. A c haracterisation of the parameter space can therefore b e seen as a statement of how our mo delled mec hanisms interact under differen t conditions. The multi-lev el statistical framework has prov ed to be extremely promis- ing in the analysis of data in the so cial sciences. In multi-lev el mo delling (also known as hierarchical linear mo dels), effects can v ary dep ending on the lev el of analysis. F or example, a model relating tw o v ariables q and s , repre- sen ting say , the salary p er year of an individual and an individual’s level of education, and a parameter p , represen ting age, w e migh t find that differen t lev els (precisions) of p grouping exp ose differen t relationships or relationship strengths. If w e choose a precision of 1 year to group individuals (i.e. 1, 2, ABM framew ork for m ulti-level studies of complex so cial systems 30 3....), there may b e little difference b etw een groups, while a precision of 10 y ears (i.e. 1-10, 11-20, 21-30...) might yield a stronger relationship b etw een q and s for some groups than for others. This same framew ork can b e applied to the parametisation of agent-based mo dels. If, sa y , an agen t-based mo del has t w o parameters p 1 and p 2, we can prob e the mo del b y sim ulating with different p 1 × p 2 configurations, giving an n 1 × n 2 matrix of C E T s, with each matrix cell corresp onding to simulation under the particular p 1 × p 2 configuration ( n 1 is the set of v alues for p 1 we sim ulate with, and n 2 is the set of v alues for p 2). If some region of this matrix con tains C E T s differing greatly from the rest of the matrix (but similar to each other), we separate it from the remainder of the matrix using the p 1 and p 2 v alues. F or example, we could disco v er a m ulti-lev el mo del in which M 1 holds b etw een ranges p 1 = [ a 1 , a 2 ] and p 2 = [ b 1 , b 2 ]; M 2 holds b etw een ranges p 1 = [ a 1 , a 2 ] and p 2 = [ b 3 , b 4 ]; and M 3 holds b etw een ranges p 1 = [ a 3 , a 4 ] and p 2 = [ b 1 , b 4 ], where M 1 , M 2 and M 3 could b e any sp ecified relationship, from simple linear correlation to an in ter-level netw ork mo del. (In terms of C E T s we can also say that the C E T asso ciated with M 1 and the C E T associated with M 2 are b oth subtypes of a third C E T defined by the parameter range p 1 = [ a 1 , a 2 ].) Figure 7 illustrates this. As in the case of in ter-level mo dels, the multi-lev el mo del itself implicitly sp ecifies a C E T , as do its sub-mo dels. Regions in which parameters (either on their o wn or in com bination) are particularly sensitiv e are regions in whic h the resolution defining groups has ABM framew ork for m ulti-level studies of complex so cial systems 31 to b e higher to observe the differences. Complexity comes in when the levels are defined irregularly (i.e. the resolution for defining groups v aries; this can b e within or b etw een dimensions). T o v alidate these regions, w e need to also split the empirical data in to the appropriate groupings, p ossibly requiring relativ ely high resolution data for some ranges. In the abov e example, we would need t w o empirical datasets corresp ond- ing to the t w o interv al p 2 = [ b 1 , b 2 ] and p 2 = [ b 3 , b 4 ] within p 1 = [ a 1 , a 2 ]. These t wo datasets corresp ond to the tw o groups (‘lev els’): ( p 1 = [ a 1 , a 2 ] × p 2 = [ b 1 , b 2 ]) and ( p 1 = [ a 1 , a 2 ] × p 2 = [ b 3 , b 4]). A third dataset is required for the group ( p 1 = [ a 3 , a 4 ] × p 2 = [ b 1 , b 4 ]). If, in these data groupings, the relationships defined b y M 1 , M 2 and M 3 hold, the m ulti-lev el mo del gener- ated through sim ulations can b e said to hav e b een v alidated b y the empirical data. 6 This multi-lev el approach to describing the state space of an agen t-based mo del maps more naturally to data obtained from empirical studies than the equation-based descriptions of phase transitions t ypically used to char- acterise complex systems b y ph ysicists while still b eing formally related to this description. 6 Summary and conclusions In this article, w e ha v e in tro duced subsystem types ( S S T s) and complex ev ent t yp es ( C E T s), which allo w us to formally describe or ‘observe’ at any ABM framew ork for m ulti-level studies of complex so cial systems 32 lev el of abstraction the states and b eha viours generated by an agen t-based mo del. Therefore, we can characterise an agen t-based mo del as a function that generates a set of S S T s and C E T s with giv en probabilit y distributions. 7 S S T s and C E T s can be used as the building blo cks for defining sophisti- cated inter-lev el and multi-lev el mo dels (formally sp eaking, in ter-lev el mo dels and multi-lev el mo dels are also C E T s). Structural and inter-lev el mo dels al- lo w us to define a structure of statistically related C E T s and/or S S T s, and the types of statistical relationships that need to hold b etw een them. The m ulti-level mo delling framew ork allo ws us to define different classes of system for whic h differen t mo dels hold (mo dels might b e structural, inter-lev el, or simple linear models). This can also b e link ed to the sensitivit y of parameters and the c haracterisation of the mo del’s parameter phase space. F rom a more practical p ersp ective, the abilit y to sp ecify structured sta- tistical relationships b etw een phenomena at differen t abstraction levels in ABM terms allo ws us to formally define the isomorphism b etw een mo dels and empirical observ ations and data. Netw orks and hierarc hies of statis- tical asso ciations then give us more stringen t sets of criteria for empirically v alidating these t yp es of mo dels. Rather than simply requiring that an agen t- based mo del can generate phenomenon X for example, we can stipulate that it should b e able to generate asso ciations with particular strengths b etw een phenomena X , Y and Z in scenario A , and a different set of strengths in scenario B . By identifying emergen t structures of b eha viour, we are able to formally relate the agen t-based mo del to empirical observ ables. This repre- ABM framew ork for m ulti-level studies of complex so cial systems 33 sen ts a significant step to wards true integration of empirical and mo del-driv en researc h in the so cial sciences. ABM framew ork for m ulti-level studies of complex so cial systems 34 Notes 1 T wo simulation instances with the same sequence and structure of agent rule executions and resulting state changes are indistinguishable. 2 It is imp ortant to note that in many cases, the agen t-based mo del itself implies a finite set of systems e.g. a closed system with b o olean v alues deterministically go verning rule execution. 3 Note that structure here is meant in the most general sense here and do es not necessarily imply spatial structure 4 In the ab ov e example, we can express ( v ar 1 , v ar 2) → ( v ar 1 0 , v ar 2 0 ) in S S T terms as sst A → sst B , where sst A = ( { ( v ar 1 , r g 1) , ( v ar 2 , r g 2) } , { AN D } ) and sst B = ( { ( v ar 1 , r g 1 0 ) , ( v ar 2 , r g 2 0 ) } , { AN D } ) ( r g 1 and rg 1 0 represen t dif- feren t v alue ranges for v ar1; rg 2 and r g 2 0 represen t differen t v alue ranges for v ar 2) 5 The precise t yp e of asso ciation relationship e.g. correlation, mec hanistic causation, phenomenal causation, dep ends on the statistical constraints that need to b e satisfied; these would dep end on the goals of the mo delling pro ject. 6 Of course, when we wish to establish stricter, more sp ecific relationships b et w een mo dels and parameters (e.g. causal relationships), v alidation b e- comes more problematic, since it is then necessary not only to sho w the ABM framew ork for m ulti-level studies of complex so cial systems 35 same irregular regions show up in empirical data as in simulation-generated data, but also that they do so for the correct reasons. F or example, should w e sa y that p 1 must lie within [ a 1 , a 2 ] for M 1 and M 2 to hold for p 2 = [ b 1 , b 2 ] and p 2 = [ b 3 , b 4 ], or is it that in the v alue range [ a 1 , a 2 ], p 1 has no effe ct when p 2 lies b et ween b 1 and b 4 ? The difficulty of v alidating suc h relations is a general one ho w ever, and the c hallenge comes mainly from finding the appropriate ‘treated’ and untreated’ cases. This can b e particularly chal- lenging in the so cial sciences, since assumptions often hav e to b e made ab out the commonalities b etw een t wo cases since active treatment (the metho dol- ogy of the experimental sciences) is not usually appropriate (one could ev en argue that it is inconsisten t with the v ery p oin t of the so cial sciences). Data that would allo w us to distinguish, for example, necessary conditions from irrelev ant background conditions, are therefore extremely difficult to obtain. 7 Ho wev er, if an agent-based mo del contains real v alues or sto chasticit y , the computational representation of the mo del will only b e an appro ximation, and the set of computationally generated C E T s (sim ulations) generated ma y b e a biased sample from the true set of systems that could b e generated by the mo del. ABM framew ork for m ulti-level studies of complex so cial systems 36 References Axtell, R., Axelro d, R., Epstein, J. M., and Cohen, M. D. (1996), “Align- ing simulation mo dels: A case study and results,” Computational & Mathematic al Or ganization The ory , 1, 123–141–141. Barab´ asi, A.-L. (2002), Linke d: The New Scienc e of Networks , Basic Bo oks, 1st ed. Blok, H. J., Bergersen, B., and Revtex, T. U. (1999), “Sync hronous versus async hronous up dating in the "game of Life",” R ev. E , 59. Bonab eau, E. (2002), “Agent-based mo deling: Metho ds and techniques for sim ulating human systems,” Pr o c e e dings of the National A c ademy of Sci- enc es of the Unite d States of A meric a , 99, 7280–7287. Bonab eau, E. and Dessalles, J. L. (1997), “Detection and emergence,” Intel- le ctic a , 2, 85–94. Brenner, T. and W erk er, C. (2007), “A T axonom y of Inference in Simulation Mo dels,” Computational Ec onomics , 30, 227–244. Crutc hfield, J. P . (1994), “The calculi of emergence: Computation, Dynam- ics, and Induction,” Physic a D , 75, 11–54. Crutc hfield, J. P . and F eldman, D. P . (2003), “Regularities unseen, random- ness observ ed: Levels of entrop y conv ergence,” Chaos , 13, 25–54. ABM framew ork for m ulti-level studies of complex so cial systems 37 Darley , V. (1994), “Emergent phenomena and complexit y,” A rificial Life , 4, 411–416. Deguet, J., Demazeau, Y., and Magnin, L. (2006), “Elements ab out the Emergence Issue - A Survey of Emergence Definitions,” ComPlexU s , 3, 24–31. Dorogo vtsev, S. N. and Mendes, J. F. F. (2003), Evolution of Networks: F r om Biolo gic al Nets to the Internet and WWW (Physics) , Oxford Universit y Press, USA. Ellis, G. F. R. (2005), “Physics, complexity and causalit y,” Natur e , 435. F agiolo, G. (2003), “Exploitation, exploration and innov ation in a mo del of endogenous growth with locally in teracting agents,” Structur al Change and Ec onomic Dynamics , 14, 237–273. F eldman, D. P . and Crutc hfield, J. P . (2003), “Structural information in t wo- dimensional patterns: Entrop y conv ergence and excess entrop y,” Physic al R eview E , 67, 051104+. F o cus, E. (2010), “Agents of change,” The Ec onomist . Garg, A., Di Cara, A., Xenarios, I., Mendoza, L., and De Micheli, G. (2008), “Sync hronous versus asynchronous modeling of gene regulatory net works.” Bioinformatics (Oxfor d, England) , 24, 1917–1925. ABM framew ork for m ulti-level studies of complex so cial systems 38 Gelman, A. (2006), “Multilev el (Hierarc hical) Mo deling: What it can and cannot do,” T e chnometrics , 48, 432–436. Gelman, A. and Hill, J. (2006), Data analysis using r e gr ession and multi- level/hier ar chic al mo dels , Cambridge Universit y Press. Gilb ert, N. and T rioitzsc h, K. G. (2005), Simulation for the so cial scientist , Op en Universit y Press. Gunji, Y.-P . and Kamiura, M. (2004), “Observ ational heterarch y enhancing activ e coupling,” Physic a D: Nonline ar Phenomena , 198, 74–105. Hart well, L. H., Hopfield, J. J., Leibler, S., and Murra y , A. W. (1999), “F rom molecular to mo dular cell biology .” Natur e , 402, C47–C52. Henric kson, L. and McKelv ey , B. (2002), “F oundations of ”new” so cial science: institutional legitimacy from philosoph y , complexit y science, p ostmo dernism, and agen t-based mo deling.” Pr o c e e dings of the National A c ademy of Scienc es of the Unite d States of A meric a , 99 Suppl 3, 7288– 7295. Holland, J. (2000), Emer genc e - fr om chaos to or der , Oxford Universit y Press. Johnson, J. (2006), “Hyp ernetw orks for reconstructing the dynamics of m ul- tilev el sys tems,” in Pr o c e e dings of Eur op e an Confer enc e on Complex Sys- tems . ABM framew ork for m ulti-level studies of complex so cial systems 39 — (2007), Multidimensional Events in Multilevel Systems , Physica-V erlag HD, pp. 311–334. Kauffman, S. (1993), The Origins of Or der: Self-Or ganisation and Sele ction in Evolution , Oxford Univ ersit y Press. Kleijnen, J. (1995), “V erification and v alidation of sim ulation mo dels,” Eu- r op e an Journal of Op er ational R ese ar ch , 82, 145–162. Kleijnen, J. P . C. and Kleijnen, J. P . C. (2001), “Exp erimen tal Designs for Sensitivit y Analysis of Simulation Models,” in Pr o c e e dings of EUR OSIM 2001 . Krull, J. L. and MacKinnon, D. P . (2001), “Multilev el Mo deling of Individual and Group Lev el Mediated Effects,” Multivariate Behavior al R ese ar ch , 36, 249–277. Kubik, A. (2003), “T o ward a formalization of emergence,” A rtificial Life , 9, 41–66. Marks, R. (2007), “V alidating Simulation Mo dels: A General F ramew ork and F our Applied Examples,” Computational Ec onomics , 30, 265–290–290. McCauley , J. L. (2006), “Resp onse to W orrying T rends in Econoph ysics,” Physic a A , 2, 601–609. Moss, S. (2008), “Alternativ e approac hes to the empirical v alidation of agen t- based mo dels,” Journal of A rtificial So cieties and So cial Simulation . ABM framew ork for m ulti-level studies of complex so cial systems 40 P earl, J. (1998), “Graphs, Causality and ST ructural Equation Mo dels,” So- ciolo gic al metho ds and r ese ar ch , 27, 226–284. Prok op enk o, M., Boschetti, F., and Ry an, A. J. (2009), “An information- theoretic primer on complexity , self-organization, and emergence,” Com- plexity , 15, 11–28. R yan, A. (2007), “Emergence is coupled to scop e, not level,” Nonline ar Sci- enc es . Sasai, K. and Gunji, Y.-P . (2008), “Heterarch y in biological systems: A logic-based dynamical mo del of abstract biological netw ork deriv ed from time-state-scale re-en trant form,” Biosystems , 92, 182–188. Sa wyer, R. K. (2001), “Sim ulating emergence and down w ard causation in small groups,” in Pr o c e e dings of the Se c ond International W orkshop on Multi-A gent Base d Simulation , eds. Moss, S. and Da vidsson, P ., Berlin: Springer-V erlag, pp. 49–67. Seth, A. (2008), “Measuring emergence via nonlinear Granger causality,” in A rtificial Life XI: Pr o c e e dings of the Eleventh International Confer enc e on the Simulation and Synthesis of Living Systems , eds. Bullo ck, S., Noble, J., W atson, R., and Bedau, M. A., MIT Press, Cam bridge, MA, pp. 545–552. T ononi, G., Sp orns, O., and Edelman, G. M. (1994), “A measure for brain complexit y: Relating functional seggregation and integration in the ner- v ous system.” PNAS , 91, 5033–5037. ABM framew ork for m ulti-level studies of complex so cial systems 41 T roitzsch, K. (2004), “V alidating sim ulation mo dels,” in Pr o c e e dings of the 18th Eur op e an Simulation Multic onfer enc e , pp. 98–106. V arela, F. (1979), Principles of Biolo gic al A utonomy , North-Holland. W erker, C. and Brenner, T. (2004), “Empirical Calibration of Sim ulation Mo dels,” T ech. rep., Max Planck Institute of Economics, Evolutionary Economics Group. Windrum, P ., F agiolo, G., and Moneta, A. (2007), “Empirical V alidation of Agen t-based models,” Journal of A rtificial So cieties and So cial Simulation , 10. Y am, B. Y. (2003), Dynamics of Complex Systems , W estview Press Inc. ABM framew ork for m ulti-level studies of complex so cial systems 42 7 Figures and tables ABM framew ork for m ulti-level studies of complex so cial systems 43 Figure 1: Graphical represen tation of hypothesis that mechanisms and/or in teractions a , b and c at the micro-level give rise to phenomenon X at the systemic macro-lev el. ABM framew ork for m ulti-level studies of complex so cial systems 44 Figure 2: Graphical representation of hypothesis that under condition A , mec hanisms and/or interactions a , b and c at the micro-lev el giv e rise to phenomenon X at the systemic macro-level, but under condition B , a , b and c giv e rise to phenomenon Y . ABM framew ork for m ulti-level studies of complex so cial systems 45 Figure 3: Graphical represen tation of hypothesis that mechanisms and/or in teractions a , b and c at the micro-level need to b e related b y sp ecific as- so ciations, represented by i , j and k , to giv e rise to phenomenon X at the systemic macro-lev el. ABM framew ork for m ulti-level studies of complex so cial systems 46 Figure 4: Graphical representation of hypothesis that under condition A , mec hanisms and/or interactions a , b and c at the micro-level need to b e related by sp ecific asso ciations, represented by i 1, j 1 and k 1, to giv e rise to phenomenon X at the systemic macro-level, but under condition B , they need different relations i 2, j 2, and k 2. This is a multi-lev el model, where each of the sub-mo dels is distinguished only by the strengths of the relationships (and not the structure). ABM framew ork for m ulti-level studies of complex so cial systems 47 Figure 5: Graphical represen tation of hypothesis that (i) mec hanisms and/or in teractions a , b and c at the micro-level need to b e related b y sp ecific as- so ciations, represented by i , j and k , to giv e rise to phenomenon X at the systemic macro-level; (ii) mechanisms and/or interactions d , e and f at the micro-lev el need to b e related by sp ecific asso ciations, n , o , p , to giv e rise to phenomenon Y ; and (iii) Y is asso ciated with X by relation q . X and Y could also represen t phenomena at differen t abstraction lev els ABM framew ork for m ulti-level studies of complex so cial systems 48 ABM Si mu l a t i o n 3 Si mu l a t i o n 2 Si mu l a t i o n 1 Si mu l a t i o n n Sa t i sf i e s X Sa t i sf i e s Y Figure 6: An agen t-based mo del (abm) generates a set of p ossible system tra jectories, of which a sim ulation is an instantiation. The o ccurrence rate of sim ulations with a particular set of attributes (X and Y) reflects the proba- bilit y or frequency with whic h this type of system is exp ected to occur giv en the agen t-based mo del. A ttributes X and Y could include an y com bination of within-sim ulation observ ations and measures discussed ab ov e in Section 4, suc h as the the emergence of a particular global phenomenon or end state. ABM framew ork for m ulti-level studies of complex so cial systems 49 Figure 7: T w o categories of hierarch y . (a) Compositional hierarch y/ α - aggregation: P 2 , P 3 and P 4 are constituen ts of P 1 . W e can also say that P 1 has a greater scop e than its constituents. (b) Set membership hierarch y/ β - aggregation: P 6 , P 7 and P 8 fall in the set defined b y P 5 . W e can also say that P 5 has a lo w er resolution than its mem b ers P 6 , P 7 and P 8 . ABM framew ork for m ulti-level studies of complex so cial systems 50 x 1 x 2 x 3 + + CE T x1 + + CE T x2 CE T x1 CE T x1, x2, x3 Figure 8: Left: Example of an inter-lev el mo del where the no des in the graph, x 1 , x 2 and x 3 can represen t phenomena at differen t lev els. The edges b et w een the no des represent statistical asso ciations b etw een x 1 , x 2 and x 3 . These can b e heterogeneous in terms of their nature (correlation, mo dular, causal), direction, and strength. Right: F ormally , the in ter-lev el mo del is an implicit sp ecification for a C E T since the statistical asso ciations b etw een phenomena at different levels define the relative v alue ranges that m ust hold for x 1 , x 2 and x 3 (whic h in turn sp ecify further C E T s). ABM framew ork for m ulti-level studies of complex so cial systems 51 p1 a 1 a 2 a 3 a 4 p2 b 1 M 1 M 1 M 3 M 3 b 2 M 1 M 1 M 3 M 3 b 3 M 2 M 2 M 3 M 3 b 4 M 2 M 2 M 3 M 3 p1= [ a 1 , a 4 ] p2= [ b 1 , b 4 ] M 1 M 2 p1 = [ a 1 , a 2 ] p1 = [ a 3 , a 4 ] M 3 p2 = [ b 1 , b 2 ] p2 = [ b 3 , b 4 ] Figure 9: T op: Matrix represen ting different mo dels (system b eha viours) for different parameter ranges of p 1 and p 2. M 1 , M 2 and M 3 migh t b e radically differen t mo dels (e.g. M 1 migh t represent a simple linear relation while M 2 could b e an in ter-level netw ork relation, or they could simply b e differen t strengths of of the same mo del structure. Bottom: Multiple multi- lev el mo dels represen ted in a single hierarch y (‘heterarc hy’), where eac h node also represents a distinct C E T . Within the range p 1 = [ a 1 , a 2 , M 1 and M 2 can b e treated as submo dels defined b y tw o different p 2 v alue ranges: [ b 1 , b 2 ] and [ b 3 , b 4 ]. Within the range p 2 = [ b 1 , b 2 ], there are also tw o submo dels, M 1 and M 3. Hence, M 1 can b e multiple classified as a submo del of both p 1 = [ a 1 , a 2 and p 2 = [ b 1 , b 2 ]. All three mo dels, M 1, M 2, and M 3 can b e treated as submodels of the m ulti-level mo del defined b y the range p 1 = [ a 1 , a 4 ] and p 2 = [ b 1 , b 4 ]. ABM framew ork for m ulti-level studies of complex so cial systems 52 Empirical equiv alen t V alidation data S S T Observ ed situation in a system at a giv en p oin t in time Individual, collectiv e, p opulation measures and/or statistics e.g. an individual’s curren t emplo ymen t status, an organisation’s current rev enue, a coun try’s GDP at time t i S T R Hyp othesised micro-level (which can be individuals, organisations, coun tries dep ending on what the agen ts are mo delling) resp onses to environmen t. e.g. if individ- ual unable to pay bills and feels c heated, more lik ely to steal; if tax imp osed on activity A , firm less lik ely to do A . Ma y b e largely theory-based, so data not alw ays a v ailable. If a v ailable, ma y be from exp eri- men tal or case studies at micro- lev el e.g. So cial Psyc hology stud- ies inv estigating the resp onses of h uman sub jects, case studies on firms. S E T Micro-lev el b ehaviour in a sys- tem that arises as a direct con- sequence of the en tity’s resp onse to his/her/its en vironment e.g. stealing when unable to pay bills. Data from so- cial/b eha vioural/cognitiv e psyc hology studies and/or case studies (esp ecially when the en tity is an organisation or geographical region). C E T (includes S E T s) Observ ed b eha viour in a system. As well as micro-lev el b eha viour, this also includes collectiv e or sys- temic b ehaviours at other lev els e.g. increase in criminal activity in comm unity X. Data from exp erimental studies and/or case studies addressing micro-lev el behaviour; population statistics and changes in p opula- tion statistics o ver time. T able 1: T able outlining the empirical equiv alen ts and v alidation data for differen t constructs in the S S T /C E T framework.