Normative design using inductive learning

Under conside ratio n for public ation in Theory and Practice of Logic Pro grammi ng 1 Normative Design using Inductive Learning DOMENICO CORAPI, ALESSANDRA R USSO Department of Computin g Imperial Colle ge London 180 Queen’ s Gate , SW7 2AZ London, UK E-mail: { d.cor api,a.russo } @ic.ac.uk MARIN A DE V OS, JULIAN P ADGET Department of Computin g Univer sity of Bath, BA2 7A Y Bath, UK E-mail: { mdv ,jap } @cs.bath.ac .uk KEN SA TOH Principle s of Informat ics Researc h Division National Institut e of Informat ics Chiyoda -ku, 2-1-2, Hito tsubashi T ok yo 101-8430, Jap an E-mail: ksatoh@nii .ac.jp submitte d 1 J anuary 2003; r evised 1 Ja nuary 2003; accepted 1 Ja nuary 2003 Abstract In this paper we propose a use-case dri ven iterativ e design methodology for normativ e framewo rks, also called virtual institutions, which are used to go vern open systems. Our computational model rep- resents the normativ e frame work as a logic program under answer set semantics (ASP). By means of an inductiv e l ogic programming (ILP) approach, i mplemented using ASP , it is possible to syn- thesise ne w rules and revise existing ones. The learning mechanism is guided by the designer who describes the desired properties of the framewo rk through use cases, comprising (i) eve nt traces that capture possible scenarios, and (ii) a state that describes the desired outcome. The l earning process then proposes additional rules, or changes to current rules, to satisfy the constraints expressed in the use cases. Thus, the contribution of this paper is a process for the elaboration and revision of a nor- mativ e f rame work by means of a semi-automatic and iterative process driven from specifications of (un)desirable beha viour . The process inte grates a no ve l and general methodolo gy for theory rev ision based on ASP . KEYWORDS : normativ e fr ame works, inducti ve log ic programming, theory revision 1 Introduction Norms and regulations p lay an important r ole in the governance of hum an society . Social rules su ch a s la ws, conventions and c ontracts prescribe and regulate our beha viour . B y pro- viding the means to d escribe an d reason ab out no rms in a computa tional context, normative frameworks (also called institution s or virtu al organisations) m ay be ap plied to so ftware 2 D. Corapi, M. De V os, J. P adget, A. Russo and K. Satoh systems. Normati ve f rameworks allow for automated reasoning about the consequences of socially ac ceptable and unacceptab le behaviour b y m onitorin g the perm issions, empow- erment and oblig ations o f the pa rticipants and ge nerating violations when no rms are not followed. Just as legislators, and societies, find inconsistencies in their r ules (o r conventions), so too may designers of normati ve fram ew orks. The details of the specification makes it rela- ti vely easy to miss crucial op erations need ed to h elp or i nhibit in tended behaviour . T o make an a nalogy with software enginee ring, this characterises th e gap between requirements and implementatio n an d wha t we describe here can b e seen as an automated m echanism to support the validation of norma ti ve f rameworks, coup led wit h regression testing. The contribution of the work is twofold. Firstly , we show how inductiv e logic program- ming (I LP) can be used to fill g aps in the rules of an existing normative framework. T he designer nor mally develops a system with a ce rtain b ehaviour in mind. Th is in tended be- haviour can be captured in use cases which compr ise two comp onents: a d escription o f a scenario and the expecte d ou tcome when executing the scenario. Use cases ar e added to the prog ram to validate the existence of an a nswer set. Failure to solve the progr am in di- cates that the specification does not yield the intended behaviour . I n this case, the prog ram and the failing use case(s) are given to an inductive learnin g to ol, wh ich will then return suggestions for improving the norma ti ve sp ecification such that the use cases are satis fied. Secondly , we present a n ovel integrated methodo logy for th eory r evision that can be used to revise a logic p rogram un der the answer set semantics ( ASP) an d sup ports the d evel- opment process by associating answer sets (that can be used for debugging purpo ses) to propo sed r evisions. Due to the non-mono tonic nature of ASP , the design er can provide the essential parts of th e use case creating a template rath er th at a fu lly sp ecified descriptio n. The revision mecha nism is general and can be applied to other domains. W e de monstrate the methodo logy throu gh a case study showing the iterati ve revision process. The pap er is organised as follows. Section 2 pr esents some backgr ound material on the normative fr amew ork, while Section 3 introduc es the ILP setting used in our proposed approa ch. Sec tion 4 illustrates the methodo logy an d h ow th e revision task can be f ormu- lated into an ILP pr oblem. W e illustrate th e flexibility and expressiveness of our approach throug h specifications of a recipr ocal file shar ing normative system. Section 5 discusses the details of the revision mechanism and the learnin g system. Sectio n 6 relates o ur app roach to existing w ork. W e co nclude with a summary and remarks on future work. 2 Normative Frameworks The essential idea of normative fram ew orks is a (consistent) collection of ru les whose purpo se is to describ e a principle of right action b inding upon the members of a gr oup and serving to guide, contr ol, or r egulate pr oper an d acceptable behaviour [Merriam-W ebster dictionary] . T hese rules may be stated in terms of e vents, specifically the ev ents that matter for the functioning of the normative fra mew ork. Normative Design using Inductive Learning 3 N = hE , F , C , G , I i , where 1. F = W ∪ P ∪ O ∪ D 2. G : X × E → 2 E norm 3. C : X × E → 2 F × 2 F where C ( X, e ) = ( C ↑ ( φ, e ) , C ↓ ( φ, e )) where (i) C ↑ ( φ, e ) initi ates a fluent (ii) C ↓ ( φ, e ) terminat es a fluent 4. E = E ex ∪ E nor m with E nor m = E act ∪ E vio l 5. I 6. State Formul a: X = 2 F ∪¬F (a) p ∈ F ⇔ ifluent ( p ) . (1) e ∈ E ⇔ event ( e ) . (2) e ∈ E ex ⇔ evtype ( e , obs ) . (3) e ∈ E act ⇔ evtype ( e , act ) . (4) e ∈ E vio l ⇔ evtype ( e , viol ) . (5) C ↑ ( φ, e ) = P ⇔∀ p ∈ P · initiated ( p , T ) ← occur r e d ( e, I ) , E X ( φ, T ) . (6) C ↓ ( φ, e ) = P ⇔∀ p ∈ P · terminated ( p , T ) ← occur r e d ( e, I ) , E X ( φ, T ) . (7) G ( φ, e ) = E ⇔ g ∈ E , occurred ( g , T ) ← occu rred ( e , T ) , holdsat ( pow ( e ) , I ) ,E X ( φ, T ) . (8) p ∈ I ⇔ holdsat ( p , i00 ) . (9 ) (b) Fig. 1. (a) Form al specification of th e norm ativ e frame work and (b) translation o f norm a- ti ve framew ork s pecific rules into AnsP rol og 2.1 F ormal Model The formalizatio n of the ab ove may b e defined as conditio nal operations on a set of ter ms that represent the normativ e state. T o provid e the context for this pape r , we gi ve an outline of a formal e vent- based model for the specification of normative fr amew orks that captures all th e essential proper ties, namely empo werment , permission , obliga tion an d vio lation . W e adopt the f ormalisation f rom (Cliffe et al. 2006), summa rized in Figure 1(a), b ecause of its straightforward mapping to answer set programmin g. The essential elemen ts of th e no rmative framework are events ( E ), which brin g abou t changes in state, an d fluents ( F ), which char acterise the state at a given instant. The fu nc- tion of the fram ew ork i s to d efine the interplay between these con cepts over time, in ord er to cap ture the e volution of a particu lar in stitution throug h the interactio n of its particip ants. W e disting uish two kinds of events: normative e vents ( E nor m ), that are the events defined by th e framework, and exogenous events ( E ex ), some of wh ose occurrence may trigger normative e vents in a direct r eflection of “counts-as” (Jones and Sergot 1996), a nd others that are of no relev ance to this particu lar f ramework. Norm ati ve events are f urther parti- tioned into normative action s ( E act ) that denote changes in norm ativ e state and violation ev ents ( E viol ), that signal th e occurre nce of vio lations. V iolations may ar ise e ither from explicit gen eration, (i.e. from the occurren ce of a non -permitted event), or fr om th e no n- fulfilment of an oblig ation. W e also distinguish two k inds of fluents: normative fluents that denote n ormative pro perties of the state such as permissions ( P ), powers ( W ) and obli- gations ( O ), and d omain fl uents ( D ) that correspond to properties specific to a particular normative fr amework. A no rmative state is represen ted by the fluents that hold true in this state. Fluen ts that are no t present ar e consider ed to be false. Cond itions on a state ( X ) are expressed by a set of fluents that should be true or false. When th e crea tion event occ urs, the normative state is initialised with th e fluents specified in I . Changes in a norm ati ve state are achieved thr ough the d efinition of tw o relations: (i) the generation relation ( G ), which implements counts-as b y specifying how the occurren ce of one (exog enous or nor mativ e) event generates another (n ormative) event, subject to th e empowerment of the actor and the condition s on the state, and (ii) the conseq uence r elation 4 D. Corapi, M. De V os, J. P adget, A. Russo and K. Satoh ( C ), which specifies the initiation and termination of fluents, subject to the perfo rmance of some action in a state matching some condition. The sem antics of a n ormative fra mew ork is d efined over a sequenc e, called a trace , of exogenou s events. Startin g from the initial state, each exogenou s event is r esponsible for a state change, through initiation and termination of fluents. This is achieved by a three -step process: (i) th e transitive closure of G with resp ect to a given exogenous e vent determines all the g enerated (norm ativ e) e vents, (ii) to this all vio lations of non -perm itted e vents and non-f ulfilled obligations are added, giving the set of all events whose consequences deter- mine the n ew state, (iii) the application of C to this set of e vents ide ntifies all fluents that are initiated and terminated with respect to the curre nt state, so determining the next state. For e ach trace, we can theref ore compu te a sequence of states that constitutes the mo del of the nor mativ e framew ork for th at trac e. This process is realised as a com putation al model throug h answer set pro grammin g ( see Section 2.2) and it is this repr esentation that is used in the learning process d escribed in Section 4. A deta iled examp le of the for mal model of an institution can be found in (Clif fe et al. 2006). 2.2 Computational Model The formal m odel described above can be translated into an equiv alent comp utational model using answer set prog ramming (ASP) ( Gelfond and Lifschitz 1991) with AnsP r ol og as the impleme ntation language. AnsP r ol og is a knowledge representation lang uage that allows the progr ammer to descr ibe a problem an d th e req uiremen ts on th e solutions in an intuitive w ay , rather than the algo rithm to find the so lutions to th e proble m. For o ur map- ping, we followed the namin g con vention used in the event calculus (Ko walski and Sergot 1986) and action languages (Gelfond and Lifschitz 1998). The b asic compon ents of the language a re atoms, elements that can be assign ed a truth value. An atom c an be negated using negation as failur e . Literals are ato ms a or negated ato ms not a . W e say that not a is true if we cann ot find e vidence suppor t- ing the truth of a . Atom s and literals are used to create rules o f the g eneral for m: a ← b 1 , ..., b m , not c 1 , ..., not c n , where a , b i and c j are atoms. Intuitively , this m eans if a ll atoms b i ar e known/true and no atom c j is known/true, then a must be kno wn/true . W e ref er to a as the he ad an d b 1 , ..., b m , not c 1 , ..., not c n as the body of the rule. Rules with empty body are called facts . Rules with empty head are referred to as constraints , in- dicating th at no solution should be able to satisfy the body . A (n ormal) pr ogr am ( or the ory) is a conjunction o f r ules and is also denoted by a set of rules. The s emantics of AnsP rol og is defined in terms of answer sets , i.e. assignments of true and f alse to all atoms in the pro- gram tha t satisfy the rules in a m inimal and con sistent fashion. A p rogram may have zero or more answer sets, each correspo nding to a solution. The mapping of a norm ativ e fram ew ork con sists of three p arts: a base compon ent which is ind epende nt of the f ramework being modelled , the time compo nent and th e frame- work specific co mponen t . The independen t compon ent deals with inertia of the fluents, the gener ation of vio lation events of non -permitted actions an d of u nfulfilled obligatio ns. The time componen t defines th e pr edicates for time and is responsible fo r generatin g a sing le obser ved event at e very time instance. The mapp ing uses the following atoms: ifluen t ( p ) to identify fluents, ev type ( e , t ) to describe the typ e of an event, e vent ( e ) Normative Design using Inductive Learning 5 to deno te the events, insta nt ( i ) for time instances, final ( i ) for the last tim e instanc e, next ( i1 , i2 ) to estab lish time o rdering , occu rred ( e , i ) to in dicate that th e (normative) ev ent happ ened at time i , observed ( e , i ) that the (exogenous) event was observed at time i , hol dsat ( p , i ) to state th at the norm ativ e fluen t p h olds at i , an d finally ini tiate d ( p , i ) and termina ted ( p , i ) for flue nts that ar e initiated and ter minated at i . Note that exoge- nous events are always e mpowered, so th at ob served e vents are always occurred events, but that normative events are not, so their occurrence is conditional on their empowerment. Figure 1(b) provides the fram ew ork spe cific tran slation rules, in cluding the d efinition of all the fluents and ev ents as facts . W e translate expressions into AnsP rol og rule bodies as conjunc tions of literals using negation as failure for negated expressions. The translation o f the formal model is augmented with a trace pro gram, specifying the length of tra ces that th e designer is interested in and rules to ensure that, all b ut the fi- nal time instan ce, is associated with exactly one exog enous e vent. Specific occurrenc es of ev ents can be specified as facts (e.g. ob serve d ( event , in stance ) ). W e refer to a c om- plete tr ace when all exogen ous ev ents for a g iving time interval are spec ified. I f a tr ace is incomplete when the mo del needs to dete rmine the m issing exog enous events. Wh ile not discussed in this paper, both the no rmative framework and the learnin g tool can deal with both types of traces. When the m odel is supp lemented with the AnsP r olog specification of a com plete trace, we o btain a single answer set corr espondin g to the mod el matching the trace 1 . In this case th e complexity of compu ting the answer set is linear with resp ect to the nu mber of time instance be ing mod elled. This result can easily b e d eriv ed from th e structure o f the pr ogram. Of co urse, in the ab sence o f a c omplete trace, th e com plexity is NP-complete as the traces c omposed of all po ssible combina tions o f missing exogenou s ev ents are computed. See (Clif f e 2007) for further details and proofs. 3 Learning Inductive Logic Programming (IL P) ( Muggleto n 1 995) is a machine learn ing tech nique concern ed with the induction of logic theor ies that g eneralise (positiv e and negative) ex- amples with respec t to a prior backgrou nd knowledge. For exam ple, fr om the ob serva- tions (proper ties in this paper) P f ly = { f l y ( a ) , f l y ( b ) , not f l y ( c ) } and a backgroun d knowledge containing the two facts bird ( a ) and bird ( b ) , we can generalise the co ncept f l y ( X ) ← bird ( X ) . In non -trivial problem s it is cruc ial to define the space of possible solutions accurately . T arget theories are within a space defined by a language bias , that can be expressed using the notio n of mode declaration (Muggleton 1995). Definition 1 A mode d eclaration is either a head de claration , written modeh ( s ) , or a body declaration , written modeb ( s ) , wher e s is a schema . A schema is a gro und literal containing special terms called p lacemarkers . A placemarker is either ‘ + ty pe ’ , ‘ − ty pe ’ or ‘ # ty pe ’ wh ere type d enotes the type of the placem arker and th e thr ee symbo ls ‘ + ’ , ‘ − ’ and ‘ # ’ indicate that the placemarker is an input, an output and a constant respectiv ely . 1 The st ructure of th e progra m (the stratified base part a nd observ ed eve nts as facts), guarante es that the program has exa ctly one answer set. See (Clif fe 2007) for further details and proofs. 6 D. Corapi, M. De V os, J. P adget, A. Russo and K. Satoh In the previous example a possible language bias would be expressed by three mode declaration s in M f ly : modeh ( f l y (+ a nimal )) , modeb ( bird (+ a nimal )) and modeb ( peng uin (+ ani mal )) . A rule h ← b 1 , ..., b n is compatible w ith a set M of mode d eclarations iff (a) h is th e schema of a h ead dec laration in M and b i are the schemas of b ody declar ations in M wh ere ev ery input and o utput placema rkers are replaced by variables, and constant p lacemarkers are replac ed b y constants; (b) e very input variable in any atom b i is either an i nput v ariable in h or an o utput variable in som e b j , j < i ; and (c) all variables and con stants are of the correspo nding type (enforced by implicit conditions in the bo dy of the ru les). From a user perspective, mode declaration s establish how rules in the fin al hypotheses are structu red, defining literals that c an be u sed in the h ead and in the bod y of a well-form ed hypoth esis. s ( M ) is the set of all the rules compa tible with M . Definition 2 An ILP task is a tuple h P , B , M i where P is a set of con junctions of liter als, called pr op- erties , B is a n ormal progra m, called backgr ou nd theo ry , and M is a set of mode decla- rations. A theory H , called hypothesis , is a n ind uctive solution fo r the task h P , B , M i , if (i) H ⊆ s ( M ) , an d (ii) P is true in all the answer sets of B ∪ H . Our appro ach for incremental development of a normati ve system suppo rts the s ynthesis of ne w rules and re vision of existing one from gi ven use-cases. W e are therefo re i nterested in the task o f Th eory Revision ( TR). As d iscussed in (Corap i et al. 2009), no n-mon otonic inductive logic pr ogramm ing can b e used to revise an existing theor y . The key notion is that of minimal r evision . In general, a TR system is biased towards the comp utation of theories that are similar to a gi ven revisable theory . Our revision algor ithm uses a measure of minimality similar to that proposed in (W ogulis and Pazzani 1993), and defined in terms of number of r evision operations required to transform one theory into another . Definition 3 Let T ′ and T be norm al log ic prog rams. A re vision transform ation r is su ch that r ( T ) = T ′ , an d T ′ is o btained fro m T by deleting a ru le, adding a fact, adding a con dition to a rule in T or deleting a cond ition from a rule in T . T ′ is a revision o f T with d istance c ( T , T ′ ) = n iff T ′ = r n ( T ) a nd there is no m < n suc h that T ′ = r m ( T ) . For examp le, given the theo ry T f ly = { f l y ( X ) ← bi rd ( X ) } , T ′ f ly = { f l y ( X ) ← bir d ( X ) , not peng uin ( X ) } is a revision o f T with distance 1 . Note th at, although we refer to Definition 3, it is also p ossible to weight revisions differently or introduce different transform ations. Definition 4 A TR task is a tuple h P , B , T , M i where P is a set of co njunction s of literals, called p r op- erties , B is a normal program , called backgr ou nd theory , T ⊆ s ( M ) is a normal progr am, called r evisable theory , and M is a set of mode declaration s. The theory T ′ , called r evised theory , is a TR solution f or the task h P, B , T , M i with distance c ( T , T ′ ) , iff (i) T ′ ⊆ s ( M ) , (ii) P is true in all the answer sets of B ∪ T ′ , (iii) if a theory S exists that s atisfies cond itions (i) and (ii) then c ( T , S ) ≥ c ( T , T ′ ) , (i.e. minimal revision). Normative Design using Inductive Learning 7 Designer Normativ e framework AnsP rol og formalisation Use Cases Learning Suggested revisions Fig. 2. Iterative d esign dri ven by use cases. For example, let B f ly = { anima l ( X ) . bird ( X ) . pe ng u in ( c ) . } , T f ly , P f ly and M f ly as in the previous examp les. T ′ f ly is a TR solution for the task h P f ly , B f ly , T f ly , M f ly i with distance 1 . The main difference with the ILP task gi ven in De finition 2 is the av ailability of an initial revisable theory a nd the con sequent bias, as discussed in mor e detail in the following sections. 4 Revising Normative Rules 4.1 M ethodolog y Use cases represen t instances of executions that are known to the design er a nd that drive the elaboration of a normative system. If the curren t forma lisation of a normativ e system does not match the intended behaviour in the use cases then the forma lisation is not complete o r is incorrect, and an extension or re vision is required. Each use case u ∈ U is a tuple h T , O i where T , a trace , specifies a set of exogenou s ev ents ( obse rved ( e , t ) ) , and O is a set of hold sat an d oc curre d literals that represen t the expected ou tput of the u se case. Giv en a set U of use cases, T U and O U denote, re- spectiv ely , the set of all the traces and expected outputs in all the use cases in U . Th e time points of the dif ferent use cases relate to dif f erent instances of executions of the normative system to av oid the effect of events in o ne u se case affecting the fluents o f another use case. The use cases can, but do not have to, be complete traces (i.e. a n e vent for each time instance) and expected output can contain positi ve as well as negativ e literals. For a gi ven translation of a normative framew ork N , the designer must sp ecify what part of th e theory is subject to revision. The theory is split into two pa rts: a “r evisable” part, N T , an d a “fixed” pa rt, N B . By d efault the for mer includes r ules of the form (6), (7) and (8), g iv en in Figure 1( b), and the latter includes th e rest o f the rep resentation of the normative system and th e set T U of the traces in U . Given a set U of use cases, a TR task for a normative framework N is d e- fined as the tuple h O U , N B ∪ T U , N T , M i , wh ere M includ es by default a body declaration for a ny static relation declared in N B , and th e following mode dec- larations (where the sch ema is op portun ely f ormed by sub stituting arguments with input p lacemarkers): modeh ( occur r ed ( e ∗ , + i nstant )) , for each e ∈ E nor m ; modeh ( initiate d ( f ∗ , + i nstant )) and modeh ( termi nated ( f ∗ , + i nstant )) , for each f ∈ F ; modeb ( hol dsat ( f ∗ , + i nstant )) , for each f ∈ F ; modeb ( occur re d ( e ∗ , + i nstant )) , for each e ∈ E . The choice of the set of mo de declaration M is crucial and is ultimately the respon sibility 8 D. Corapi, M. De V os, J. P adget, A. Russo and K. Satoh of the d esigner . Many mode declarations ensur e higher coverage o f the specificatio n but increase the compu tation time. Con versely , fewer mode declarations improve perform ance but may result in par tial solutio ns. The choice may be driven, for example, by pre vious design cycles, or interest in more problematic parts of the s pecification. As shown in Figure 2 the design of a norm ati ve system is an iterative pro cess. The representatio n N in AnsP rol og of a system descr ibed by the design er using a norm ativ e languag e is te sted against a set o f use cases also p rovided by the designer . T his analysis step is perf ormed by ru nning an ASP solver over N , extended with the o bserved ev ents included in th e use cases, an d a con straint indicatin g that no answer set that d oes not satisfy O is accep table. Conceptually , if the solver is not able to find an answer set (i. e. returns unsatisfiable), then som e of the given use cases are no t satisfied in th e answer sets of N and a revision step is perfor med. Possible revisions are provided to the designer w ho ultimately chooses the most appropr iate one. 4.2 Ca se Study W e illustrate th e methodolo gy with a small but rich enough case study that d emonstrates the key prope rties and benefits o f our propo sed app roach. The fo llowing is a descrip tion of a reciprocal file sharing normative fr amew ork. The activ e parties—agents—of the scenario find themselv es i nitially in the situation of having o wnership o f se vera l (digital) objects—the block s—that form part of some larger composite (digital) entity—a file. An agent is required t o share a copy of a block they hold before they can downloa d a copy o f block they are missing . Initially each agent holds the only co py of a gi ven block and there is only one copy of each block in t he agent population. Some vip agents are able to do wnload blocks without any restriction. Agents that request a do wnload and have not shared a block after a previou s do wnload generate a vio lation for the do wnload action and a misuse violation for the agent. A mi suse terminates the empo werment of the agent to do wnload blocks. The designer de vises the following use case h T , O i : T =                  observ ed ( star t, i 00) . observ ed ( dow nload ( alice, bob, x 3) , i 01) . observ ed ( dow nload ( char lie, bob, x 3) , i 02) . observ ed ( dow nload ( bob, alice, x 1) , i 03) . observ ed ( dow nload ( char lie, alice, x 1) , i 04) . observ ed ( dow nload ( alice, ch ar lie, x 5) , i 05) . observ ed ( dow nload ( alice, bob, x 4) , i 06) . O =              not v iol ( my D ow nload ( al ice, x 3) , i 01) . not v iol ( my D ow nload ( c har lie, x 3) , i 02) . not v iol ( my D ow nload ( bo b, x 1) , i 03) . not v iol ( my D ow nload ( c har lie, x 1) , i 04) . not v iol ( my D ow nload ( al ice, x 5) , i 05) . viol ( my D ow nload ( alice, x 4) , i 06) . The u se case models a sequence of e vents that inc ludes a violation at the tim e p oint i 06 , wh ile the dow nl oad events at th e oth er time po ints do not gener ate violation s. In the trace, cha rl ie perform s a download a t time point i 0 4 withou t sharin g a b lock after the last download. This is no t expected to generate a violation since char l ie is d efined as v ip ( isV I P ( charl i e ) ∈ N ). The initial normative system includ es the domain componen t and type definitions giv en in Figure 1(b) and a specific componen t given by the following revisable theory N T : % r u l e 1 i n i t i a t e d ( h a s b l o c k ( X , B ) , I ) : − o c c u r r e d ( m y D o w n l o a d ( X , B ) , I ) . % r u l e 2 i n i t i a t e d ( p e r m ( m y D o w n l o a d ( X , B ) ) , I ) : − o c c u r r e d ( m y S h a r e ( X ) , I ) . % r u l e 3 Normative Design using Inductive Learning 9 t e r m i n a t e d ( po w ( e x t e n d e d f i l e s h a r i n g , m y D o w n l o a d ( X , B ) ) , I ) : − o c c u r r e d ( m i s u s e ( X ) , I ) . % r u l e 4 t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( X , B2 ) ) , I ) : − o c c u r r e d ( m y D o w n l o a d ( X , B ) , I ) . % r u l e 5 o c c u r r e d ( m y D o w n l o a d ( X , B ) , I ) : − o c c u r r e d ( d o w n l o a d ( Y , Y , B ) , I ) , h o l d s a t ( h a s b l o c k ( Y , B ) , I ) . % r u l e 6 o c c u r r e d ( m y S h a r e ( X ) , I ) : − o c c u r r e d ( d o w n l o a d ( Y , X , B ) , I ) , h o l d s a t ( h a s b l o c k ( X , B ) , I ) . Giv en the use case an d the above formalisation of the nor mativ e system , the first itera tion of our ap proach prop oses, through the revision process, the deletio n of a con dition in rule 5 and addition of a condition to rule 4 as sho wn below (leaving the other rules unaltered): % r u l e 4 − r e v i s e d t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( X , B2 ) ) , I ) : − n o t i s V I P ( X ) , o c c u r r e d ( m y D o w n l o a d ( X , B ) , I ) . % r u l e 5 − r e v i s e d o c c u r r e d ( m y D o w n l o a d ( X , B ) , I ) : − h o l d s a t ( h a s b l o c k ( Y , B ) , I ) . Howe ver, this is not y et the intend ed for malisation. As an add itional debuggin g facility the designe r can request the set of vio lations that are true in the answer sets th at cor- respond s to th e revision and no tice that un wanted vio lations are g enerated at each time point. This feedba ck can b e used to refine the use case provid ed. In f act the use case spec- ifies the single specific violations that must not oc cur b u t it does not request e xplicitly that no violations sho uld occur in th e first fi ve time poin ts ( e.g. viol(myDownload (alice,x3),i02), viol(myDownload(alice ,x4),i02) ). These violations ca n be observed in the answer set associ- ated with the re vision. The designe r can then improve the use case by modifying the set of expected outputs: O =      viol ( my D ow nload ( alice, x 4) , i 06) . not v iol ( my D ow nload ( A , B ) , T ) , T ! = i 06 . occur r ed ( misuse ( alice ) , i 06) . not oc cur r ed ( misuse ( X ) , T ) , T ! = i 06 . In the subsequen t iteratio n, the re vision process suggests changes that include those iden- tified in the previous iteratio n (i.e. addition of condition in rule 4 and deletion of condition in rule 5), and the addition o f a further cond ition in the body of rule 5. The combined effect of these ch anges fixes the or iginal error in th e specificatio n, by also ch anging the name of one of the variables. Furthe rmore, since the output O of the use case includes a desired misuse event, which is no t curren tly form alised in the system, the revision also suggests the new rule 7 given below . The final theory N ′ T includes the following rules (leaving untouch ed rules 1, 2, 3 and 6) 2 : % r u l e 4 − r e v i s e d t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( X , B2 ) ) , I ) : − n o t i s V I P ( X ) , o c c u r r e d ( m y D o w n l o a d ( X , B ) , I ) . % r u l e 5 − r e v i s e d o c c u r r e d ( m y D o w n l o a d ( X , B ) , I ) : − o c c u r r e d ( d o w n l o a d ( X , Y , B ) , I ) , h o l d s a t ( h a s b l o c k ( Y , B ) , I ) . % r u l e 7 − n e w o c c u r r e d ( m i s u s e ( X ) , I ) : − o c c u r r e d ( v i o l ( m y D o w n l o a d ( X , B ) ) , I ) . 2 The revi sion is generated in 23 seconds by I C L I N G O (Gebser et al. 2007) on a 2.8 GHz Intel Core 2 Duo iMac with 4 GB of RAM. 10 D. Corapi, M. De V os, J. P adget, A. Russo and K. Satoh In summary , after a few iteration s rule 4 is corrected by adding an exception not isVIP(X) , rule 5 is revised by correcting a typograph ical erro r in its condition (i.e. th e name of a vari- able was n ot the intende d o ne – occurred(download( Y , Y ,B),I) ), and finally , a new r ule is learnt that defines misuse cohe rently with respect to the provided use case. 5 Theory revision through ASP In this section we provide m ore details abou t the re vision pr ocess. W e first introdu ce all the computatio nal steps to d eriv e a revision with respect to a set of use cases. Then we d elve into the details of the learning system, describing the integrated ASP-based ILP approach. The revised nor mativ e system N B ∪ N ′ T is comp uted b y m eans of two program transfo r- mations and an abduc ti ve reasoning process executed in ASP , wh ich d erives prescriptions for re visions and new rules in the form of abdu cibles. The abductive solution has a one-to- one mapping to a revision of the initial theory . 5.1 Revision The ap proach describ ed in this section can be applied to other problems of T R. T o the best of our k nowledge, our m ethodolo gy is the only one cu rrently a vailable that is a ble to suppo rt revision of no n-mon otonic AnsP r olog theo ries that supports integrity co n- straints, aggregates and oth er ASP constructs, pr oviding revisions as an swer sets. Op er- ationally , the revision is performed u sing a similar tran sformation to the one described in (Corapi et al. 2009). Figure 3 details the revision steps for on e of the rules in the case study described above and Algorith m 1 illustrates the phases. W e present the concep tual steps and refer the reader to (Corapi et al. 2009) for further details. Input : N B fixed theory; N T ∈ s ( M ) revisable theory; P set properties; M mode declarations Output : N ′ T rev ised theory according to the giv en P ( N T , M ) = pre-proces sing ( N T , M ) ; H = ASP AL ( P, N B ∪ N T , M ) ; N ′ T = post-processing ( N T , H ) ; return N ′ T ; Algorithm 1 : Phases of the revision algor ithm. A pr e-pr oc essing pha se lifts the standard IL P pro cess of learn ing hypoth eses about ex- amples up to the (meta-) process of learning hypothe sis abou t the rules and their exception cases. For e very rule in N T , e very body literal c i j is replaced by the ato m try ( i, j, c i j ) , where i is th e ind ex o f the rule, j is the index of the b ody literal in th e ru le and the third argument is a reified ter m for the litera l c i j . not exception ( i , h i , v i ) is added to the body of the rule where i is the ind ex of the rule, h i is the reified term for the head of the rule and v i is an optional list of add itional variables appe aring in the bod y (see Figur e 3). The try p redicate is defin ed in such a way that when ev er del ( i, j ) is true, the m eta-cond ition try ( i, j, c i j ) is always true. Oth erwise tr y ( i, j, c i j ) is tr ue wh enever c i j is tr ue. Facts o f th e type del ( i, j ) can be learn t by the ILP system used within the revision. M specifies mod e declaration of rules that can be added togeth er with additional head declarations that are added to take into account the ne wly introduced del an d exception predicate s. Normative Design using Inductive Learning 11 1 – Pr e-pr ocessing (rules in N T ) t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( X , B2 ) ) , I ) : − t r y ( 4 , 1 , o c c u r r e d ( my D o w n l o a d ( X , B ) , I ) ) , n o t e x c e p t i o n ( t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( X , B2 ) ) , I ) , B ) . t r y ( 4 , 1 , o c c u r r e d ( my D o w n l o a d ( X , B ) , I ) ) : − n o t d e l ( 4 , 1 ) , o c c u r r e d ( m y D o w n l o a d ( X , B ) , I ) . t r y ( 4 , 1 , o c c u r r e d ( my D o w n l o a d ( X , B ) , I ) ) : − d e l ( 4 , 1 ) . 2 – Learning (rule in H) e x c e p t i o n ( t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( X , B2 ) ) , I ) , B ) : − i s V I P ( X ) . 3 – P ostpr ocessing (rule in N ′ T ) t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( X , B2 ) ) , I ) : − n o t i s V I P ( X ) , o c c u r r e d ( m y D o w n l o a d ( X , B ) , I ) . Fig. 3. Detailed revision transfo rmations for r ule 4 (Section 4.2) In the learning p hase , given th e p re-pro cessed th eory N T and the new mode decla rations M , the following ILP task i s executed h P, N B ∪ N T , M i , using A S PA L , the learn ing system described in Section 5.2. The outcome of the learning p hase H is used in a post-pr oce ssing phase wh ich gene rates a re v ised theory N ′ T semantically eq uiv alent to N T ∪ H . Informally , for each del ( i , j ) fact in H the correspo nding condition j in rule i in N T is deleted. For each exception ru le in H of the fo rm except ion ( i, h i , v i ) ← c 1 , ..., c n , the correspo nding rule i in N T is substituted with n new rules, one for each conditio n c h , 1 ≤ k ≤ n . Each of these rules k will have in the hea d the predica te h i and in th e bo dy all cond itions present in the original rule i in N T plus the additional condition not c ( k ) . An exception with em pty body results i n th e origin al rule i bein g deleted. An exception for which at least two co nditions share variables is kept as an ad ditional “exception con cept” in the revised theory . The pre-pro cessing and post-pr ocessing ph ases perf orm syntactic transformatio ns that are answer set preserving and do not in volve the answer set solver . 5.2 ASP AL The system u sed in this work, called A S PA L (ASP Abductive Learn ing), though used here to suppor t the revision of a no rmative system, can be applied more gene rally to no n- monoto nic ILP problems. I t is based on th e transfor mation from an I LP task to an ab ductive reasoning task, used in a recently proposed ILP system (Corapi et al. 2010). This system offers se veral advantages over other existing ILP appr oaches, makin g it par - ticularly suited for normative d esign. A S PA L is ab le to handle negation within the learning process, an d therefo re reason abou t default assumption s governing inertial fluents; to per- form non-o bservational an d multiple predicate learning, thus compu ting hypotheses about causal depen dencies between observed sequences of events and no rmative states; and to learn non- monoto nic h ypothe ses, which is also essential f or theo ry revision. Furthermore , the learnin g can be en abled by a simple tr ansformatio n of th e mode declaration s and does not require the co mputation of a bridge th eory ( Y amamoto et al. 2010). As discussed in (Corapi et al. 2010), none of the existing ILP systems p rovides the above m entioned fea- tures. Emb edding the learnin g p rocess with in ASP red uces the seman tic gap be tween the normative system and the learning proce ss and permits an easier control of the whole pro - cess. The notion of re vision d istance as in Definition 3 can be m anaged by the optimisation facilities pr ovided by mo dern ASP solvers (Geb ser et al. 2007). Optimisation statemen ts 12 D. Corapi, M. De V os, J. P adget, A. Russo and K. Satoh can be used to deri ve an swer sets that contain a minimal number of atoms of a certain typ e that ultimately relate to new rules or revisions, as e xplained in this section. As in (Corapi et al. 2010), an I LP task h P, B , M i is transform ed in to an abductiv e log ic progr amming pro blem (Kakas et al. 1992), thus en abling the use of AnsP r ol og . Let us introdu ce some prelimina ry n otation. Giv en a m ode declaration modeh ( s ) or modeb ( s ) , id is a unique identifier for t he mode declaration, s is the literal obtained from s b y replac- ing a ll p lacemarkers with different variables X 1 , ..., X n ; t y pe ( s , s ) denotes th e sequ ence of literals t 1 ( X 1 ) , ..., t n ( X n ) su ch that t i is th e type o f the placemarker replaced b y the variable X i ; con ( s , s ) = ( C 1 , ..., C c ) is th e constant list of v ariables in s that replace on ly constant placemarkers in s . inp ( s , s ) = ( I 1 , ..., I i ) and out ( s , s ) = ( O 1 , ..., O o ) are de- fined similarly for inp ut and output placem arkers. Since s is clea r fr om the c ontext, in the following we omit the second argument from ty pe ( s , s ) , con ( s , s ) , inp ( s , s ) and out ( s , s ) . Giv en a set of mode declarations M , a top theory ⊤ = t ( M ) is co nstructed as follows: • For each head declaration modeh ( s ) , with uniqu e identifier id , the following rule is in ⊤ s ← r ule ( R I d, ( id, con ( s ) , ()) , r ule id ( RI d ) , ty pe ( s ) , body ( R I d, 1 , inp ( s )) (10) • For each body declaration modeb ( s ) , with uniqu e identifier id t he follo wing clause is in ⊤ body ( R I d, L, I ) ← r ule ( R I d, L, ( id, con ( s ) , Link s )) , link ( inp ( s ) , I , Link ) , s , ty pe ( s ) , append ( I , out ( s ) , O ) , body ( R I d, L + 1 , O ) (11) • The following rule is in ⊤ together with the definitions for the l ink , r ul e id and appe nd predicates: body ( R I d, L, ) ← r ule ( RI d, L, las t ) rul e id ( rid ) is true wh enever 1 ≤ r id ≤ r n where rn is the m aximum numb er of ne w rules allowed. li nk (( a 1 , ..., a m ) , ( b 1 , ..., b n ) , ( o 1 , ..., o m )) is true if for each eleme nt in the first list a i , there exists an element in th e second list b j such that a i unifies with b j and o i = j . Giv en the top theory , we seek a set of r ul e atoms ∆ , such that P is true all m odels of B ∪ ⊤ ∪ ∆ . ∆ has a one-to -one mapp ing to a set of rules H = u (∆ , M ) . Intu iti vely , e ach abduced atom r epresents a litera l of the rule labelled by the first argument. The secon d argument co llects th e co nstant used in the literal and the thir d disambig uates the variable linking. Fig. 4 shows the learning steps for rule 4 of our example. For spac e lim itations we only state the main soundn ess and comp leteness theo rem (Corapi and Russo 2011) of the learning system. Normative Design using Inductive Learning 13 Inputs Mode declara tions M e x c e p t i o n ( t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( + a g e n t , + b l o c k ) ) , + i n s t a n t ) , + b l o c k ) . Pr operties P v i o l ( m y D o w n l o a d ( a l i c e , x 4 ) , i 0 6 ) . n o t v i o l ( m y D o w n l o a d ( A , B ) , T ) , T ! = i 0 6 . o c c u r r e d ( m i s u s e ( a l i c e ) , i 0 6 ) . n o t o c c u r r e d ( m i s u s e ( X ) , T ) , T ! = i 0 6 . Backgr ound theory B t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( X , B2 ) ) , I ) : − t r y ( 4 , 1 , o c c u r r e d ( my D o w n l o a d ( X , B ) , I ) ) , n o t e x c e p t i o n ( t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( X , B2 ) ) , I ) , B ) . t r y ( 4 , 1 , o c c u r r e d ( my D o w n l o a d ( X , B ) , I ) ) : − n o t d e l ( 4 , 1 ) , o c c u r r e d ( m y D o w n l o a d ( X , B ) , I ) . t r y ( 4 , 1 , o c c u r r e d ( my D o w n l o a d ( X , B ) , I ) ) : − d e l ( 4 , 1 ) . T op theory ⊤ e x c e p t i o n ( 4 , t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( A , B ) ) , T ) ) : − i n s t a n t ( T ) , b l o c k ( B ) , a g e n t ( A ) , r u l e i d ( RID ) , r u l e ( RI D , 0 , ( e 4 , ( ) , ( ) ) ) , b o d y ( R ID , 1 , ( A , B , T ) ) . b o d y ( RID , L e v e l , ( A , B , T ) ) : − a g e n t ( A ) , b l o c k ( B ) , i n s t a n t ( T ) , r u l e i d ( RID ) , l i n k ( L 1 , ( A , B , T ) , LR1 ) , r u l e ( RI D , L e v e l , ( i s v , ( ) , ( LR1 ) ) ) , i s V I P ( L 1 ) , b o d y ( L + 1 , R ID , ( A , B , T ) ) . b o d y ( RID , L , ) : − r u l e ( RI D , L , l a s t ) . Abducti ve solution ∆ r u l e ( 0 , 0 , ( e 4 , ( ) , ( ) ) ) , r u l e ( 0 , 1 , ( i s v , ( ) , ( 1 ) ) ) , r u l e ( 0 , 2 , l a s t ) Output Inducti ve so lution H e x c e p t i o n ( t e r m i n a t e d ( p e r m ( m y D o w n l o a d ( X , B2 ) ) , I ) , B ) : − i s V I P ( X ) . Fig. 4 . Learn ing steps for rule 4 ( Sec. 4 .2). W e show only the relev ant mode declaration s and rules. Theor em 1 Giv en an ILP task h P , B , M i , H is an inductive solution if and o nly if there is a ∆ such that H = u (∆ , M ) , ⊤ = t ( M ) and P is true in all the answer sets of B ∪ ⊤ ∪ ∆ . The ASP solver is used to compu te a set of solution s ∆ , that can be translated back into a set of in ductive solution. Sound ness and comp leteness fo r the revision procedu re rely on Theo rem 1 and on the un derlying ASP solver p roperties. T hese p roperties also e nsure that if a set of theories th at ma tches the requ irements exists within the languag e bias o f the lear ning, in the limit, if a comp lete set of all use cases (an extensional specification of the req uirements) is provided , th e r evision conv erges to the expected theory . This is of course an ideal case. In p ractice the system outputs mo re accurate solutions as m ore compreh ensive use case sets are provided. 6 Discussion and Related W ork The motivation beh ind this pap er is the problem of h ow to con verge upon a complete and cor rect norm ativ e system with respect to the in tended range of applicatio n , where in practice these p roperties may b e manifested by incorrect or u nexpected b ehaviour in use. Additionally , we observe, from pr actical experience with o ur par ticular framework, that it 14 D. Corapi, M. De V os, J. P adget, A. Russo and K. Satoh is often desirable to be able to develop and test incr ementally and regressively rather th an attempt verification once the system is (notionally) complete. The literature seems to fall bro adly into three categories: ( a) concrete langu age frameworks (OMASE (Gar c´ ıa-Ojeda et al. 2007), Op eretta (Okouy a and Dignu m 2008), InstAL (Cliffe et al. 200 6), MOISE (H ¨ ubner et al. 2007), Islander (Este va et al. 200 2), OCeAN ( Fornara et al. 2008) and the constraint appro ach of Garcia-Camin o et al. (Garc´ ıa-Camino et al. 2009)) f or the specification of n ormative systems, that are typ- ically suppor ted by some form of model-c hecking, and in some cases allow for change in the normative structu re; (b) logical fo rmalisms, such as ( Garion et al. 2009), that captu re consistency and completen ess v ia mo dalities and other formalisms like (Boella et al. 2009a), th at cap ture the concept of no rm chan ge, or (V asconcelo s et al. 200 7) and (Cardoso and Oli veira 2008); (c ) mechanisms that look out for (new) co n ven- tions and hand le th eir assimilation into th e norm ati ve framework over time and sub- ject to th e curren t normative state and the position of other agen ts (Artikis 2009; Christelis and Rov atsos 2009). Essentially , the objective of each o f the above is to real- ize a transformation o f the n ormative framework to accommod ate some form of short- coming. These shortcoming s can be identified in sev eral ways: (a ) by observing that a particular state is r arely achieved, which can indicate there is insufficient nor mativ e guid- ance for particip ants, or (b) a norm co nflict occurs, such that an a gent is un able to act consistently under the governin g norms (K ollingbau m et al. ), or (c) a particular v iolation occurs fr equently , which may ind icate tha t the v iolation co nflicts with an effecti ve co urse of action that agen ts prefer to take, the pen alty n otwithstandin g. All of these can be viewed as characterising emergent (Savarimuthu and Cranefield 20 09) appr oaches to the ev olution of nor mative framew orks, where som e mechan ism, either in the f ramework, or in the en- vironm ent, is u sed to re vise the no rms. I n the app roach taken her e, th e designer pre sents use cases that effectiv ely cap ture the b ehavioural requ irements fo r th e system, in o rder to ‘fix’ bad states. T his ha s an in teresting p arallel with the schem e p ut forward by Serrano and Saugar (Serrano and Saugar 2010), wher e they propo se the specification o f incom plete the- ories and their managemen t throug h incom plete normative states identified as “pen ding”. In (Boella et al. 2009b), whether the norms here are ‘strong ’ o r ‘weak’ —the first guideline— depend s on whether the purpose of the normativ e model is to dev elop the system sp ecifica- tion or add itionally to p rovide an explicit rep resentation fo r run- time reference. Likewise, in respec t o f the remaining guidelin es, it all depends on how the f ramework is actually used: we h av e cho sen, for th e purpo se of this presentation, to stage norm refinement so that it is an off-line (in the sense of prior to deploymen t) process, wh ile mu ch of the dis- cussion in (Boella et al. 2009b) addresses run-time issues. Wheth er the process we ha ve outlined h ere cou ld effectiv ely be a mean s for on-line mech anism design, is somethin g we have yet to explore. W ithin th e context of software engin eering, (Alrajeh et al. 2007) shows how exam ples of desira ble and u ndesirable behaviour of a software sy stem can be used by an ILP system, tog ether with an incom plete b ackgro und knowledge o f the en vi- sioned system and its en viro nment, to compute miss ing requiremen ts specification s. There are se veral elements in common with the s cheme prop osed here. From an ILP p erspective, we employ a system that can learn logic p rogram s with nega- tion (stratified or otherwise) and, unlike other existing no nmon otonic ILP systems (Saka ma 2001b) is sup ported by co mpleteness results, is integrated into ASP and can be tailored to p artic- Normative Design using Inductive Learning 15 ular design requ irements. Some prop erties an d r esults of ILP in the context of ASP a re shown in (Sak ama 2001a). The author also p ropo ses an alg orithm for learning that is sound but no t c omplete and, differently fro m th e approach prop osed here, e mploys a covering loop approach . 7 Conclusions and Future W ork The motiv ation for this work stems f rom a real ne ed for too l support in th e design of normative frameworks, because, altho ugh hig h-level, it is nevertheless har d for humans to identify error s in specifications, or indeed to p ropose the most appropriate co rrective actions. W e hav e described a meth odolog y for the re vision of normative frameworks and how to use tools with formal underp innings to sup port the proce ss. Sp ecifically , we are able to revise a form al mode l—represented as a logic p rogram —that captures the rules of a norm ati ve system. The r evision is achieved by means of induc ti ve logic p rogram ming, working with the same representatio n, in formed by u se cases that describe instances of expected beh aviour of the no rmative system . If ac tual b ehaviour does not coincide with expected, theo ry revision pr oposes new r ules, or m odifications of existing r ules, for the normative fr amew ork. Fu rthermor e, given correct traces, the learnin g p rocess guarantees conv ergence—the prop erty of “learning in the limit”. From this firm foundatio n, which properly connects a theory of normativ e systems with a practical representation , there are th ree directions that we aim to pursue: (i) definitio n of criteria for selecting solu tions from alternative sugg estions provided b y the learning ( we are currently in vestigating the use o f crucial literals (Sattar and Goebel 1991)) (ii) intro - duction of le vels of con fidence in the use cases and their use for selecting the “m ost likely” revision, in addition to the general criteria of m inimal re v ision: i.e . combine some domain- indepen dent heuristics with some do main-specific heu ristics suc h as le vel o f confiden ce in use cases ( iii) extension to interaction s b etween no rmative frameworks and a form of cooper ati ve r evision. Additio nally , there is the matter of scalability . The compu tation time increases with the nu mber of rules, time steps, e rrors in the theor y and in particu lar , mode declaration s an d lan guage bias for the lea rning. That is, it gr ows with the state sp ace of the normative framework and the “learning space”, i.e. is all possible theories we can con- struct g iv en our lang uage b ias. W e n eed to e xperiment fur ther to under stand better to which factors performance is sensiti ve and how to address these issues. References A L R A J E H , D . , R AY , O . , R U S S O , A . , A N D U C H I T E L , S . 2007. E xtracting requirements from Sce- narios using ILP. In L NAI . V ol. 4455 . Springer-V erlag, 63–77. A RT I K I S , A . 2009. Dynamic protocols for open agent systems. In AAMAS . 97–104. B O E L L A , G . , N O R I E G A , P., P I G O Z Z I , G . , A N D V E R H A G E N , H . , Eds. 2009 . Normative Mult-Agent Systems . Number 09121 in Dagstuhl Seminar Proceedings. Schloss Dagstuhl. B O E L L A , G . , P I G O Z Z I , G . , A N D V A N D E R T O R R E , L . 2009 a. Normative frame work for normativ e system change. S ee Sierra et al. (200 9), 169–176. B O E L L A , G . , P I G O Z Z I , G . , A N D V A N D E R T O R R E , L . 2009b . Normati ve Systems in Computer Science – T en Guidelines for Normativ e Multiagent Systems. See Boella et al. (2009). 16 D. Corapi, M. De V os, J. P adget, A. Russo and K. Satoh C A R D O S O , H . L . A N D O L I V E I R A , E . C . 2008. Norm defeasibility in an institutional normati ve frame work. In ECAI , M. Ghallab, C. D. Spyrop oulos, N. Fakotak is, and N. M. A vouris, Eds. Frontiers in Artificial Intelligence and Applications, vol. 178. IOS Press, 46 8–472. C H R I S T E L I S , G . A N D R OV A T S O S , M . 2009 . Automated norm synthesis in an agent-ba sed planning en vironment. S ee Sierra et al. (200 9), 161–168. C L I FF E , O . 2007. Specifyin g and analysing institutions in multi-agent systems using answer set programming. P h.D. thesis, Uni v ersity of Bath. C L I FF E , O . , D E V O S , M . , A N D P A D G E T , J . 2006. Answer set programming for representing and reasoning about virtual institutions. In CL IMA VII . LN AI, vol. 4371. Springer , 60–7 9. C O R A P I , D . , R AY , O . , R U S S O , A . , B A N D A R A , A . K . , A N D L U P U , E . C . 2009. Learning rules fro m user beha viour . In AIAI . 459–468. C O R A P I , D . A N D R U S S O , A . 2011. Aspal. proo f of soundne ss and completeness. T ech. R ep. DTR11-5, Department of Computing, Imperial College London . C O R A P I , D . , R U S S O , A . , A N D L U P U , E . 2010. Inducti ve logic programming as abductiv e search. In T ec. C omm. of the 26th ICLP , M. Hermenegildo and T . Schaub, Eds. LIPIcs, vol. 7. Dagstuhl, Germany , 54–63. E S T E V A , M . , D E L A C R U Z , D . , A N D S I E R R A , C . 2002 . Islander: an electronic institutions editor . In AAMAS . A CM, 1045–1 052. F O R N A R A , N . , V I G A N ` O , F., V E R D I C C H I O , M . , A N D C O L O M B E T T I , M . 2008. Artificial institu- tions: a model o f institutiona l reality for op en multiagen t system s. Artif. In tell. Law 16, 1, 89–1 05. G A R C ´ I A - C A M I N O , A . , R O D R ´ I G U E Z - A G U I L A R , J . A . , S I E R R A , C . , A N D V A S C O N C E L O S , W. W. 2009. Constraint rule-based programming of norms for elec tronic institutions. Autonomous Agen ts and Multi-Agent Systems 18, 1, 186–2 17. G A R C ´ I A - O J E D A , J . C . , D E L O A C H , S . A . , R O B B Y , O Y E N A N , W. H . , A N D V A L E N Z U E L A , J. 2007. O-mase: A customizable approach to dev eloping multiagent dev elopment processes. In AOSE , M. Luck and L. Padgham, Eds. Lecture Notes in Compu ter Science, vol. 4951. Springer , 1–15. G A R I O N , C . , R O U S S E L , S . , A N D C H O LV Y , L . 2009. A modal logic for reasoning on consistency and completeness of regulations. See Boella et al. (2009). G E B S E R , M . , K AU F M A N N , B . , N E U M A N N , A . , A N D S C H AU B , T. 2007. clasp: A conflict-driv en answer set solver . In In LP NMR’07 . Springer , 260–265. G E L F O N D , M . A N D L I F S C H I T Z , V . 1991 . Classical negation in logic programs and disjunctiv e databases. New Generation Computing 9, 3-4, 365– 386. G E L F O N D , M . A N D L I F S C H I T Z , V . 1998. Action languages. E lectr on. T rans. Artif. Intell. 2 , 193– 210. H ¨ U B N E R , J . F., S I C H M A N , J . S . , A N D B O I S S I E R , O . 2007. Dev eloping organised multiagent sys- tems using the moise. IJA OSE 1, 3/4, 370–395. J O N E S , A . J . A N D S E R G O T , M . 1996 . A F ormal Characterisation of Institutionalised Power . ACM Computing Surve ys 28, 4es, 121. Read 28/11/2004. K A K A S , A . C . , K O WAL S K I , R . A . , A N D T O N I , F. 1992. Abductiv e logic programming. J. Log . Comput. 2, 6, 719–770. K O L L I N G B AU M , M . , N O R M A N , T., P R E E C E , A . , A N D S L E E M A N , D . Norm conflicts an d inconsis- tencies in virtual organisation s. In Proceed ings of COIN 2006 . 245–258. K O WA L S K I , R . A N D S E R G O T , M . 1986. A logic-based calculus of ev ents. New Gen. Comput. 4, 1, 67–95. M U G G L E T O N , S . 1995. In v erse entailment and progol. New Gen. Comp. 13, 3&4, 245–286 . O K O U Y A , D . A N D D I G N U M , V . 2008. Operetta: a prototype tool for the design, analys is and dev el- opment of multi-agent organ izations. In AAMAS (Demos) . IF AAMAS, 1677– 1678. S A K A M A , C . 2001a. Learning by answer sets. In In AAAI Spring Symposium: Answer set p r ogram- ming . Normative Design using Inductive Learning 17 S A K A M A , C . 2001b . Nonmonotonic inductiv e logic programming. In LPNMR . 62. S A T TA R , A . A N D G O E B E L , R . 19 91. Using crucial literals to se lect better t heories. Comput. Intell. 7 , 11–22. S A V A R I M U T H U , B . T. R . A N D C R A N E FI E L D , S . 2009. A categorization of simulation works on norms. S ee Boella et al. (2009). S E R R A N O , J . - M . A N D S A U G A R , S . 20 10. Dealing with incomplete normativ e states. In Pr oceedings of COIN 2009 . LNCS, vol. 6069 . Springer . in press. S I E R R A , C . , C A S T E L F R A N C H I , C . , D E C K E R , K . S . , A N D S I C H M A N , J . S . , Eds. 2009. A AMAS 2009, Budapest, Hungary , May 10-15, 2009, V olume 1 . IF AAMAS. V A S C O N C E L O S , W., K O L L I N G B AU M , M . , A N D N O R M A N , T . 2007. Resolving conflict and in- consistenc y in norm-regulated virtual org anizations. In AAMAS , E. H. Durfee, M. Y okoo, M. N. Huhns, and O. Shehory , Eds. IF AAMAS, 91. W O G U L I S , J . A N D P A Z Z A N I , M . J . 1993. A methodology for ev aluating theory r e vision systems: Results with audrey ii. In IJCAI . 1128–1 134. Y A M A M O T O , Y . , I N O U E , K . , A N D I W A N U M A , K . 2010. From in verse entailment to in verse sub- sumption. In ILP .