Hybrid Reasoning and the Future of Iconic Representations

Hyb rid Reaso ning an d the Future of Icon ic Rep resentatio ns Cather ine RECA N ATI LIPN – CNRS U MR 70 30 , Uni versi té Par is 13 Insti tut Ga lil ée, A v. J-B . C lém ent, 93 43 0 V ille taneuse , Fran ce cather ine .recan at i@lipn .un iv-p aris13 . fr Abstrac t . We giv e a b rief o verv iew o f the main cha racte ristics of diag ramm atic re asoning , analy ze a case o f huma n reasoning in a mastermin d gam e, an d explain why h yb rid repres entatio n system s (HRS) a re p arti cula rly attractive and prom ising fo r AGI and Comp uter Scien ce in general. Key wor ds . Diagra mma tic represe ntation . Iconic represen tation, Ana logical repre sen tation. Hy brid rep resentatio n sys tems. Cog nitive mo deling an d re asoning . Introd uct ion Logic al li ngu is tic repr e sen tations have a h igh pow er o f abs tract ion an d many p eop le think th at they can mode l our re asonin g ab iliti es, part ly because our know led ge expr esses in l ingu isti c t erms , and al so be caus e o ur f or mal tools ar e bui lt on alphanu m erica l repr esen tat ions. N ev erth eless , inferen tial sy stems so lely b a sed on textual r epres en tatio ns ar e v ery in eff ici ent. Moreov er, thes e sy stems r ais e d iff icul ties at the r ep resen tation al l evel , be caus e they r equ ire a co mpl ete s pecif ication of th e concre te and abs trac t p rop er ties of the mod eled ob ject s. Thi s is wh y co mpu ter s ci ent ists, u sed to think in term s of da ta st ru cture s, h ave e ar ly def en ded the us e o f d iagra mm atic represen tations , for i nstanc e i n prob lem solving , on t he b asis of t he f act tha t the se represen tations w ere b etter adap ted to sp ecif ic d oma ins (s ee [ 1] f or an histor ic al su rv ey and cr itiqu es of log icis t AI ). Althou gh com mo nly used in Sc ien ce, for instanc e in Ma them atic s or Phy sics, diagram ma tic r epres ent ation s h av e lon g suff ered fro m th eir r eputa tion as mer e tools in the s ear ch for so lu tions. A t the b eginn in g of the 90 's, Barw ise and Et che mend y (B &E) have stron gly den ou nc ed this g enera l p re judic e ag ains t di agrams ([2 ], [3 ], [ 4]) . To co pe with co mp lex s itu ation s, they defen ded a g enera l th eor y of valid inf er enc es tha t is indepen den t of the mod e of rep re sen tation , and th ese wo rk s lead on the f irst demon str ation that di agram ma tic sys te ms can b e so un d and co mplete [5] . As far as hu man r easonin g is co ncern ed , there ar e many examp les using non linguis tic for m of rep res ent ation, and, to qu ot e B&E , “hum an langu ag es ar e inf inite ly richer an d mo re su bt le than th e for mal lang uages for whic h we hav e anyth ing like a compl ete accou nt of i nferen ce. [...]. As the c omp ut er giv e s u s ever r icher too ls f or represen ti ng inform at ion, we must beg in to study the logi cal aspe cts o f reason ing that uses no nl ingu istic fo rms of repre sen ta tion” [2] . Follow ing in B& E fo ot st eps, o ur g ener al p roj ect is to d ef end the in ter est o f hyb ri d represen tation sy stem s (H RS) – i.e. sys tem s l inking t ogether sev era l kind s of represen tations . W e c lai med in [6] an d [7 ], th at on ly H RS co u ld y ield to the b ui lding o f mod els of reaso ning , b oth comp ut ation ally effi cien t and co gnit ively plau sib le. In this paper, w e will f irst re ca ll the mo s t in terest in g ch aracter istic s of diagram ma tic inferen tial sy stems , and add so me comm ents a b out an exa mpl e o f hu man hyb rid reasoning i n a m as term ind gam e. In the nex t s ect ion, w e w ill g ive so me argu ment s fo r the system at ic stu dy ( and use) of H RS in A GI an d co gn ition mo de ling , and so m e hin ts for the ir us efuln ess in pr og ram spe cif ication and se man tics. 1. S ome cha racte rist ics of dia gra mmat ic inf erent ia l syst ems In [2] , B&E emp h asized tha t th e m ain pro per ties o f diagra m mati c sy stem s deriv e fro m the ex is tenc e o f a sy n tact ical h o mo mor phism b etw een icons and repr esen ted o bj ects . In many cas es, this h omo m orp hism y ields to a very strong p ro p erty c a lled closu re und er constra ints . In clos ed un der constra ints sy stems , th e conseq u ences of in itia ls f a cts are includ ed de f acto in t he r ep res entat ion and d o not requ ire extra compu tatio n. Th is makes these sys te ms ver y efficien t. A s we h av e und erl ined in [6 ] and [7] , this al so sho ws a d eep dual ity b etwe en t wo mode s of re asoning . Lingu istic (o r tradi t ional l og ica l) reason in g req uires : (1) the repres ent ation of initial p rop ert ies of ob jects ; (2) an explic it r epres ent ation o f a bstra ct pro pert ies (or relat ions amon g ob ject s); and (3 ) a compu ta tion al mech anis m link ing the two so urces of infor mation ( to est abl ish the v alidi ty o f a n on -exp licit conseq uenc e). Th us, b y constru ction , su ch syst ems requ ire c alcul ation s. For ins tanc e, if y ou kno w that Ann is on the left o f Gas ton on a bench , and that G a ston is on the left of Isab el, yo u n eed to add that th e re lat ion “be o n the left o f” is tran s itive to pr ov e th at Ann is on the left o f Isabel . To th e op po site, di agr amma tic r easo ning u sual ly d oes n ot r equ ire th e exp lic it represen tation of such abstr ac t p ro pert ies, b ecause these p ro pert ies ar e taken autom atic ally into a ccou nt b y syn tact ic cons tra ints on the r ep resen tation i ts elf. In o ur examp le, an icon ic r epr esen tat ion o f th e firs t fa ct w ill loo k l ike th e ( left) jux tapos ition of two sy mbo ls (say , A fo r An n and G f or G aston , as in : A G); an d the s econ d fa ct wi ll yield to the jux tapos it ion of a th ird sy mbo l (say, I fo r Isab el), as in : A G I. Thus, yo u will just “see ” on the result ing repr es entat ion tha t A is o n the left o f I, withou t any compu tation. Sin ce many cons equen c es autom atic ally appe ar on represen tations , d iag ram ma tic sy s tems pro vide an ea sy tre atmen t o f conju n ction s and are comp u tationa l ly v ery eff ici ent. Unf or tuna tely, they hav e diff icult ies w ith disjunc t ive c ases i . Al terna tives may requ ire the u s e o f sev e r al diag ra ms, wh ich must then be travers ed on e after the other, as in the ling u istic cas e 1 . Note also tha t in man y diagram ma tic sys tems, ea ch repre sen ta tion corre sp on ds to a g enuin e s itu ation , and that contradi c tion is i mpo ssib le to represen t ( wh ich c an b e go od o r b ad d epend ing on what you need to repr es ent) . Many r esear cher s have tri ed (in the nin eties) to an alyz e d iag ramm a tic inf er ent ia l system s pr op ertie s and cl osur e under cons tra ints in p ar ticul ar. Fo r St enn ing and Oberland er ( S &O) [9 ], di agr amma tic repre sen ta tions s ee m ma inly to diff er fr om 1 The d ifficulty with disju nctio n re inforc es th e thesis tha t cognitive rep rese ntations are ma inly diag ramm atical, bec ause hu man perfo rma nces are b etter in co njunctive than in disjun ctive ca ses [8 ]. linguis tic o nes b y a mo re l im ited p ow er o f ab s tract ion, bu t gr eater co mpu tationa l efficien cy. Th ey clai med tha t there ar e thr ee class es o f r e p resent ation al sy s tems: th e MARS (M inima l Ab str action R epresen ta tion al Sys tem s), the LA RS (Li mi ted Abstra ctio n R epres en tatio nal Systems) and the UA RS ( Un limited A bs trac tion Repre sen tationa l S yst ems) . Th ey argue tha t thi s h ierar chy of rep resentat ional sys tems is an a logo us to that o f l ang uages isola ted by Chom sky , and tha t most d iag ram ma tic represen tation sy s tems are LA RS. A MARS is a system in w hi ch a repr esen tat ion corresp on ds to a un ique mo d el of the wo rld und er the cons idered int erpr eta tion. Fo r instan ce in a mas ter mind ga me , a ro w of l etters stand ing fo r a r ow o f colored p awns , as [ B B Y Y R] , w ill b e a min imal abs tract ion rep resen tatio n of a p ossible solu tion. How ever, y ou c an easily au gm ent the number of mode ls captured in a MA RS b y introd ucing n ew symb o ls that a llow ab str acting on r ep resen t ation s. Fo r ins tanc e, in the masterm ind examp le, you can hav e a “ - ” sy mbo l stand ing for an undet ermin ed color , as in [ B B - Y R] . Su ch sy stems c an q uantify m assiv el y o n p oss ible mo de ls , but canno t spe cify arb itrar ily co mpl ex dep enden ces bet we en the s pec ified dimens io ns. Th is is wh y S &O cal led the m L A RS. Th ey cla imed that on ly lin gu istic symb ols , add ed to a represen tation, cou ld allow th e descr ipt ion of arbitrar ily fine depend ences betwe en dimens ion s. They d efined a LA RS as “a sys t em that ke eps its repres ent ation s s imp le, and keeps ass ert ions out o f it s key s” and c laim ed th at mo s t diagra mm at ic inf erent ial system s are LA RS . S&O id ent ify the res tri cted cap acity of d iagra mm at ic sys tem s with a pr op erty called “ spec ifi city” , w h ich r equire s inf or mation o f a certa in k ind to b e expl icit in all interpr etab le r epres ent ation . In [ 10] , P err y an d Mack en ( P &M) h ave op po sed to this strong notion o f spec ifici ty (i.e. the man datory spe cific atio n of values o f pro perties other than th e o ne y ou try to repr esen t) the not ion of “det ermin ed ch aracter ” d ue to Berke ley. B erk eley's n ot ion of a d etermin ed ch aract er is that i t is n ot p ossibl e to represen t an objec t as having a cer tain p rop erty, withou t repr esen ting at th e sam e time a spec ified v alue fo r this p rop erty . Thus , I cann ot r epre sen t a tr iangl e o n a figur e, wi tho ut ending with a part icu lar tr iangl e. A s w ell, it is n ot p ossib le to rep resent a colored o bj ec t on a dr awing witho ut spe cifying i ts color , but I can perfec tly say, « this o bj ect has a n interes ting color », w itho ut spe cifying w h ich o ne ii . For P& M , clo sed u nd er con s traints system s hav e, in add ition to this d eterm ined ch arac ter , a pro perty called “ loc aliz ation ” (alread y iden tif ied by Lark in and S imon in [12] ). Loca lizat io n is more impo rtan t than spec ificity to ch aract erize di agram ma tic repr esen ta tions . N everth eless, ther e ar e two pro perties of loca liza tion. The on e iden tif ied by P& M is a pu rely log ica l pr op erty al so called u niqu e token cons tra int . It is the pr op erty of u sing on ly one tok en o f a s ymb o l to represen t an o b je ct. Th is pr op erty d isapp ears g enera lly wh en you u se a typ ed sys tem iii . Fina lly, P & M d isting u ish f iv e kin ds o f r epres ent ation g o ing f rom text to im ages : grap hic t exts , charts, diagr am s, m ap s and pictures. Th eir catego rizat ion uses two additio na l pro pert ies, i con ici ty and a constr aint and s y stema tic ho mom orp his m (requ ired to hand le closur e under con str ain ts). As far as g eom etr ic or spati al aspe cts are co n cerned , M ack en, Perr y, and Hass emphas iz ed the i mp ortan ce o f icon ici ty in [13 ]. I conicity allows repre sen ta tions w ith richly g rou nd ed me an ing – that is mean in g w ho se relat ion to form is n ot arb itrary . An iconic s ign m ay have a r ead ily infer ab le mean ing ( RI M), an eas ily rem emb ered mean ing ( ERM) , o r an int ernal ly mo dif iab le m ean ing (I MM). Road s ign s p ro vide num erou s ex amp les o f ERM, RIM and IM M (fo r instan ce, signpo sting bend s). Th er e also are m any examp les of sy mbo ls h aving a RI M in mu s ica l scor es (as f or ins tan ce, crescen do situated un der the stave). Howev er , ico nici ty is o nly partially analyzed unt il now , and IMM is stil l p uzz ling. W e think that it co uld be some times link ed to the syn tactic ho mo m orp hism , be caus e o ur per son al conclu sion is th at the main dist inc tion betwe en l ing uis tic ( or sym bol ic) r epresen tat ion sy stems an d analogi cal repr esen ta tion system s (as d iagra mm atic sys te ms) mu s t be ch ar acter ized in terms of th e p ow er of the meta-l angu age r equ ired to p rov ide the s em antics o f the sys t em. In th e an alog ica l c ase, the me talang uag e ne eds to r efer en ce sy ntac tica l pro pert ies of the ob jec t langu age , wh ile in the sy m boli c cas e, th is is no t ob ligato ry iv . 2. Hy brid hu man rea son ing in mas t erm ind The p rec eding sec tion re calls that iconi c rep res ent ation s ca n be first class cit izen, i.e. valid syntact ical ob je cts in inferen tial sys tems . It also u nd erlines w ha t ico nic represen tations are go od fo r and wha t th ey are not. A t f irst sigh t, a limi ted p ow er o f abstra ction and th e reque st of a un iq ue syn tac tical ho momo rph ism are res trict ive , and situat ions to w hich pur ely d iagra mma tic re aso ning app lies seem li mited 2 . Nev er theless , grap hical and textu al r epr esen tat ion s ys tems bein g co mplemen tary (a t rep r esen tation al and algo r ithm ic l eve ls), the sho rtco mings of both sy st ems can d is app ear in H RS. Therefo re , the pr eced ing revi ew acts as a cri tique of cur ren t appr oa ches to r eason ing , which tend to emph asize o nly o ne mod e, d iagr amm at ic o r l ing uistic , and are se t u p in opp osition to th e oth er mode ( e.g. th e men tal log ic vs. men tal mod els d eb ate). Let us now look at an ex ampl e of hy brid human reason ing in a m astermind g am e. Master m ind 3 is w ell su ited to the s tudy of hu m an r easoning , b ecau se it constr ains the p layer to perfo rm lo gi cal r eason ing . F urth erm ore, t he g eom etry o f the g rid encou rages th e p layer s to u se diagra mm at ic repre sent ation s. For most p layer s , reasonin g is fr agm en ted and oppor tunis tic, and c ons is ts in partial deduc tions u sing severa l types of repr esen tat ions. I n [ 14 ], we h ighl ighted this hyb rid ch ar acter : mos t o f dedu ction s are g raph ical, w hile th e mod el un der constr u ction par ti ally ex presse s verb ally. In fact, t he use o f grap hical represen ta tions mitig ates limi tations in th e cogn itive c apa cities o f th e pl ayer, an ch oring reaso n ing on inexp ensive v isu al capaci ties , and r eliev ing thus v erba l me mor y. In return , visua l c apacit ies b eing thems elves r es tric ted, the sh ape of the d iagr ams and th e o rd ering of h yp othese s ar e biased ( this b ecaus e, even wh en th ey exp ress v erb ally, h yp ot heses ar e al so gr ou nd ed on the g rid) . For inst ance , the left-to- righ t or der (o f pins and p a wns) an d th e e as e o f visu al trans lations, inf luen ce the choice o f hy po thes es to b e cons ider ed f irst . N everth eless , some play ers use thes e b iases to dev elop their own strateg y of resolution in an intel ligent w ay. We have insu ffi cien t room her e to rep ort all of our observ ation s, but w e can sho rtly comm ent a ga me of one p layer (g rid on Figu re 1) . Th e grid ensur es the memor iz ing of p receding r esu lts, bu t, as w e wi ll s ee, it is als o a geo me trica l sup po rt for org anizing p ro of and b ack track ing. O ur player separ ates h er gam e in two p has es: first determ in ing th e color s, and then deter min ing the pl aces. In bo th phas es, she u ses 2 Co ntrary to w hat ma y see m initially, g rap hical rep rese ntatio ns are n ot only he lpful in mo delin g situations where a (con cre te) spatial h omomo rph ism applies. 3 The gam e consists in disc overing a hidden row o f five co lored pawns. On e play er (the leader) h ides a con figu ration of p awns. The sec ond player can the n disp ose on a g rid a tentative con figu ration o f pawns, and the lead er replies by posting pins (o n the rig ht) indic ating if and how pawns co rresp ond to the s olution on e’s. A white pin mean s a g ood position a nd color fo r on e pawn, and a bla ck on e a misp laced color. The rows remain visible d uring the g ame , and the play er ha s to find o ut the so lution w ith a limite d nu mbe r of rows . represen tations tha t can b e q ua lified as men tal mo d els b eca us e th ey ar e v ery s imi lar to those o f Jo hn son -Laird [15] . Th e int eres ting f act her e is th a t the se mo d els (w h ich a lso corresp on d to LA RS of S&O) are o rder ed b o th b y incre asin g order of spe cif icity , and by d ecreasin g or der o f p rob ability. Th is m akes ba cktr acki ng easier, s ince the mo d el consider ed next is d etermin ed, and guarant ee s a qu ick co nverg ence to th e solu tion, since thes e mod els ar e in de cre asing o rder of pro bab ility. Figure 1. Gam e of an exp erien ced player The p layer b egins on ro w 1 by her favo ri te attemp t (a 2 /2/1 distrib ut ion), whi c h pos sible repl ies reve aled be ing stat istica lly more inf orm at ive than tho s e o f other co lors distr ibutions (su ch as 3 /2, 4 /1, 5 , 1 /1/3 o r 1/1/1 /2, etc.). G ive n the pins on the r ight s ide, she con s iders f irst th e int erpret ation d isp layed o n Fig ure 2 , i.e. that on e b lue is placed correc tly, o n e y ellow m isp lac ed, and tha t th er e is n o red . (S he might tak e in hi s h and a blue and a y ellow p awn to h elp m emo riz ing, an d no te men ta lly tha t th e thr ee co lors are exhau sted) . Figure 2. A first inte rpreta tion sch em a We n ote th is men tal mod e l by [1B] [1 Y] (an d “no r ed”) – u sing squ are b rackets f or the n ot ion of exh austio n intro du ced by Joh nso n-La ird . (N ote h ow ev er that the mod el behind the sch ema o f F igure 2 is mo re sp ec ific, sinc e i t inc l ud es some informa t ion o n places , b ut in this f irst phase o f the g ame, th e pl ayer do es no t pay mu ch attention to them) . Th en, sh e plays th e se con d row , try ing new pla ces fo r blue (anticipa tion on futur e r easo ning ab ou t b lue p la ces) , and in trodu cing a n ew c o lor: o rang e . By luck , b oth oran ge and blu e are missing co lors, an d the in terp re tation of the s econ d row is ob viou s . Blue being exc luded, she sw itche s to a n ew mod el based o n a n ew interp retat ion of the first r ow: [1 Y ] 1R . Then, she p lays the th ird r ow b oth to try new pla ces fo r red , and to try a new co lo r . Getting fo ur p ins as a resul t, sh e con cludes eas ily th at the col ors o f the so lution must be yellow , red and gr een. Given th at there i s o nly o ne yello w, she con sid ers first [1 Y] [2R] [2G ] ( wh ich se ems mor e pro bab le th an [1 Y] [3R] [1G ]). She then beg ins reasonin g o n places and su pp oses tha t on the f irst r ow , i t is the firs t left yel low th at is correc t ( we w ill n ot e th is model by [ – – Y – –] , k now ing that the emp ty p lac es m us t be fill ed by the mi ssing paw ns w i thin [1Y , 2 R, 2G]). Figure 3 . A diagra mma tic reaso nin g 6. R R G Y G o o o o o 5. G R R Y G o o o   4. R G R Y G o o o   3. R R R G G o o o  2. O O B B B 1. B B Y Y R o  1. B B Y Y R o  3. R R R G G o o o  2. 1. B B Y Y R o  With th e di agram ma tic r eason ing illus trated in F igur e 3 ( st art fol lowing the arro w s fro m the first ro w), she infers th at on row 3, a red is m isplaced , and thus, two greens well pl aced ([ – – Y G G] ). Th e so lut ion sho uld be [ R R Y G G] , bu t this conf lic ts w ith the fo ur pins o f ro w 3, which shou ld then be all w h ite. Th u s, sh e has to ba cktr ack and reconsid er the po s ition of the y ellow pa wn on th e f irst r ow ( [ – – – Y – ]) . A graph ic r eason ing very simi lar to th e p re ceding on e reve a ls th at in this ca se, the left green is m isp laced and the righ t on e cor re ct ( i. e. [– – – Y G]). She th en tr ies a fou rth plausib le row , bu t is this t ime un lucky . N everth eless , colors are conf irmed an d she k no ws b y exper ience tha t, ge tting 3 wh ite and 2 b la ck pins means tha t tw o pawns have jus t to be exchan g ed to give the so lut ion. The tw o pawn s to sw itch are to b e fo un d in th e f irst thr ee p awn s [ R G R – –], thus the gr een mus t b e exch anged w ith on e o f th e two reds. S he tries [ G R R Y G] on ro w 5, but is unlu cky agai n. H ow ev er, ther e is no w only one so lut ion f or th e sw itch , and sh e w ins on the l ast ro w . An int erest ing f a ct abou t th is g ame i s the use of graph ic al infer en ces as those depic ted o n Figu r e 3 . Th ere are o ther sor ts of g raph ical infer ences us ed b y exp erienced players . F or instan ce, by fo cusing on the commo n parts o f severa l ro ws, infer ence s can be draw from th e requ es ted map ping s b etw een the s et of com mon pins and the set of common p awns. A ll o f thes e inf eren ces ar e in a way “local ” wi thin the g lob al reasonin g, an d th ey u se cr eat ive g raph ic s chem as m ap ped on the fly onto the g rid. At a high er l eve l, th e s tra tegy us ed by this p l ayer co nsi sts in a systema tic ord er ing of the pos sibiliti es open ed by a g iv en r ow. This k ind of stra t eg y is of ten us ed. A t the beginn ing, th e rea son ing is ro ot ed o n the firs t ro w, and th e mo st pro babl e mod el i s consider ed f irs t, her e with a lef t-to-r ight b ias in c ase o f equality. For instanc e, in the preced ing gam e, the order ing of the sever al mode ls com p atible with th e first row (withou t con s idering p laces) is the fo llowin g: [B][Y ] no red < [B] R no y ellow < [Y] R no b lue. The pl ayers compar e co mp eti tive mod e ls and their re lativ e p ro babilit ies dire ctly from the nu mb er of pins and paw ns . Th is is w hy s ome p layer s have a t end ency to pref er continuo us co lor arrang em en ts to separ at e ones, b ecau se qu an tities (or mass) are th en more sali ent, an d th e co mpar ison analog ical ly perfo rmed easier . In many cases, the player bui lds a mo de l in easy s tag es by cov ering a la ttice w here mod els fit into e ach other o n a br anch ( by b eing mo re specifi c). The na ture o f con sidered mo de ls is not alway s as sys te mat ic as in o ur examp le , and m ay vary amo n g p layer s an d /or s itua tion s. Neverth el ess , an impor tant fact i s th at t hes e mo de ls lay o u t on the g rid i n a visua l manner. Figu r e 5 g ives exa mpl es of sever al mod els (fit ting tog ether gr ap hic ally). Figure 5. Some g rap hical sc hem as of inte rpreta tion How ever, conc ern ing ou r player, the global reason in g path is oriented by two direction s (b o th gr ou nd ed on the grid ): ( 1) a left- to-r ight orient ation o f the p ossib le mod els w ithin a row, and (2 ) the n atural ver tical o rder ing o f the rows. This syst emat ic ord ering helps re me mber ing w hich mod el h as to be con sid er next in case of back tra ck. This g loba l stra t egy appli es as well in th e se con d ph ase of the game . Her e fo r instance , the order ing on the f irst row is: [– – Y – –] < [ – – – Y –] < [– – – – R] B B Y Y R o  B B Y Y R o  B B Y Y R o  The firs t mode l [– – Y – – ] was qu ickly elimina ted, and [– – – Y –] evolved pro gr essively in a more spe cif ic solu tion . Ano ther int eres ting fa ct about this ex amp le is tha t diagra mma tic repr esen tat ion s prev ent here from incoher en ce, ins tead of introd ucing err ors (as many peop le c la imed they merely do ). Here this is d ue to th e use of limited abs t raction d iagra ms in wh ich contradi c tion is i mpo ssib le to repr esen t. Further mor e, p artially bec ause of the spec ificity p rop erty m ention ed in th e f irst se ct ion, LA RS ap pear to b e g oo d cand ida tes for o rd ering mo dels b y inclusion. Mode ls may a lso be or derl y amon g oth er d imensions , by u sing p rob abi lities or o ther spe cific attr ibute s. Fro m this poin t of view, o ur ex amp le can b e s een a s a pro totyp e fo r a fam i ly o f pro gr ams, wh ere in for m ation aris es increm en tally ( here on each n ew row ) and which are mo r e o r l ess d eterm ining ( or ap pro xi ma ting) the “solu tio n ” – tho ug ht o f as a m atrix of values. In su ch case s, th e art icu lat ion of loca l ( po ssibly g r aph ical) sub sys te ms w ithin the lat tic e and the g enera l l eve l contro lling f low ( in ch ar ge of backtrack ing) is s imp le, becaus e the rol e of each mod ul e is well def in ite. Each new infor m ation m ay bring spec ific constra int s b etw een s pe cific va lues , and expr esses par tia lly in som e sub syste m, but the g ener al p ro gram canno t be pr ep are to a l l o f them. Th en , oth er loca l h euri st ics or strateg ies wil l help and give a co re to th e gen er al re ason ing. For in stanc e, this so rt of a rchi tectur e could n atur al ly appl y to natur al langu ag e pro cessin g, b e caus e t ext app ear s seq u ent ially, b oth a t di sco u rse l eve l an d at sen tenc e level. Su ppo se w e h ave to p roc ess so me text ( i.e. already o rg anized in w ord s , b ut a similar arch itecture wo uld app ly to s pe ech). Each wo rd arr ives w ith n ew inf orm ation abou t the “mean ing ” of a s ent ence . Such mean ing co uld loc ally be a matrix o f severa l sorts of at tribu tes ( wh ich m igh t be va lues or a c tions) dep end ing on w hat the softwar e is sup posed to do . With curren t se man tics theor ies , it could be made o f linguis tic f eatur es fro m severa l d omains ( mor ph olog y, sy nt ax, sem ant ics, e tc.). I n eac h d omain, th ere are spec ific co ns tra ints th at c an be h andl e b y mo du lar and mo re o r les s indepen den t sub system s (f or instan ce , in even ts s em antics , y ou migh t ha ve specif ic r epr esen tat ions for time, s p ace, c ausa lity, et c.) . Thus, the g en era l p ro gram may used sta tist ics, pro per strateg ies ( as try to d iscov er synt ac tical fea tur es fir st) or a lef t-to-r ight b ias as in our masterm ind examp le, to f ind a way throu gh the severa l po ssibi lities o f f illing u p the manda tor y f eatur es – witho ut pr esu pp osing tha t so m e of the se f eatur es (synta cti c on es in p articu lar) h av e to b e co mp let ely d et ermin ed f irs t. P articu lar fe a tures m ay also b e l et und eterm ined, to k eep the na tur al lack o f pre ci sion in langu ag e. 3. Per spe ctiv es for Hyb rid Re pr esentat ion Syst em s in A GI HRS may lead to am azing r esults co nc ern ing eff iciency . A p aradox is that a given demon str ation may b e lim ited by a min im al cost in any sy mb olic sys te m, and still be less co stly in a hy br id sy stem includ ing and b inding the two so rts of rep resentat ion s (iconi c o r symb o lic on es). N ot e th at th er e is n oth ing sop his tic h ere , b ecaus e in a hy br id system there is no ne ed o f a g loba l lang uage to b ind its su bsy stems 4 (remind Göd el’s pro of ). Fur thermor e, th e articul ation of s ever al sub syste ms in a co mp lex represen tationa l sys tem, bases so m etim es s imply on th e f act that they d eno te the s am e objec ts in th e wo rld, and th erefo re coh eren ce be tw een two sub sys tem s h as no t 4 This is w hy B&E gav e a theoretical justification of th e two main alg orithms imp leme nted in H ype rproof [16] b ase d on pure ly math ematical gro unds, witho ut usin g an y interm edia te l ang ua ge. necessar il y to b e h andl e. In the do ma in o f r easo ning , the objection that si tua tions in which a un ique homo mor ph ism applies are r are is as wel l n ot too s erio us, b ecau se y ou can u se sev era l ho momo rph is ms. The s i tuation i s jus t that the sub sys te ms d eno te differen t prop erties o f mo d els or o bj ects, and w h at expres se s in on e su bsy s tem d o not expr ess n ecess ari ly in the other. Nev erth el ess, som e info rm ation can b e transf er f rom one syst em to ano th er (on th e bas is o f safe corr espon den ces), endo wing the g lobal system with su per ior inferen tial an d comp u tation al cap aci ties. And ther e is no sp eci al need of an inter med ia te lan gu age. Contrary to wh at may se em ini tial ly, g raph ica l rep res entat ions are not only helpfu l in mode l ing si tuation s where a spa tial ho momo rp his m ap pl ies. Their increa sed us e in scienc e is also d ue to the ir ob viou s ab il ity to con vey abs trac t mean ing s. V ia spa ce, th ey bring new possib il ities of s tructur ing an d abs tract ing (comp ared to sequ en ces of l etters alone) . F rom this poin t of v iew , H RS are d efin itive ly on top o f tradi tion al UA RS in the hierar chy o f S&O. Bes ides th eir app lica tion to r eason ing , th eir system at ic s tudy sh ou ld impro ve form aliza tion in many do main s, which are relev ant for AGI , as cog nit ive scienc e, natur al l angu age se man tics and ling ui stics . The re are man y d omain s in seman tics w her e icon ic rep re sen tations se em b et ter s ui ted t han logic al fo rma lisms . In linguis tics , th e numero us sche mas , fou nd in wo rk s on t im e and a spec t (fo r ins tan ce [17 ], [1 8], [1 9] ), are an indi cation o f the p laus ibili ty of th is thes is. We b elieve th at concern ing thes e dom ains , it is du e to th e na ture of o ur co g nitive app ar atus ( see n ex t sub section) . We a lso b e lieve that the ad d ition of iconic featu r es in theor e tical languag es or tools cou ld br ing ma jor advanc es in o th er fi elds o f Comp u ter S c ienc e, less concern ed by world repr esenta tions , as for in s tance , in th e d oma in of sem ant ics of pro gr amming lang uag es, or in so f twar e d es ign in g ener al. By w ay o f conc lus ion , w e add two sub sect ions to r einfor ce th ese claims. The f o llow i ng are s ure ly contro vers ia l pro po sals. (They ar e a lso r ather indep en dent and so me mig ht be va lid, o thers n ot.) 3.1. Outl ine o f a m od el of th e h uma n min d (r e lation bet ween tho u gh t and langu ag e) To furth er u nd erst anding o f the h uman m ind , its h igh er co gn iti v e c apac i ties, and more spec ifica lly the natur e of the rel ation b etw een l ang uage and tho ug ht, the go al is to develop a mode l of la ng uag e und erstan ding an d use that atta ins observ a tion al adequ acy , i. e. that is able to pas s the Turin g test. To a chi ev e this go al, w e mu st a im high er, b y trying to r each ex p lana tory adeq ua cy, that is , to d evelop a mo de l o f how the system can r eason ab ly acqu ir e the “kn ow led ge” ( i. e ., sys tem s o f kn ow ledg e /belief , etc.) th at enabl es i t to a tta in ob serva tio nal adeq u acy. The o nly w ay a m ind c an acqu ire the r ich v ar iety of k no wled ge hu mans do a cqu ire is to start w ith a s tron g inn ate b asis . The on ly way to bu ild a sy stem wi th a stron g innate b asis i s to org ani ze th is b asi s into modu les that are w ell adap ted to repre sent ing the aspe ct s of the wo rld they repr es ent. This is be caus e of the w ay the wo rld is (i t is rich and v ar ied, and the ba si c con cep tu al ap para tus n eed ed to rep res ent time and tempora l r el ation s, f or ins tan ce, mu s t u se d iff eren t r es ou rces o beyin g diff erent constra ints than tha t ne eded to repr esen t sp atia l r elat ions, or interp erson al r el ations and other m ind s, or c aus al in teraction s, e tc) . Th ere ar e pr ob ably also g ener al comp u ta tional constra ints (p ro blems of tra ctabi lity an d ex pre ssiv e ad eq uacy) , and th e ne ed f or revis ion with in rel evant con s traints ( as w e ll a s many o ther fac tors) , whi ch will determ in e the em ergenc e of a s et of mod ul es. The min d’s ri ch set o f inna te mo dul es, its “kno wledg e” abo u t the w or ld ( inclu ding itself) i s thu s in the for m o f r epresent ation al capacit ies. Wh ile it c an b e heu ristic ally usefu l to form ulate k no wl edg e/bel iefs abo ut ti me, fo r instan ce, a s a se t of ax ioms ( i. e., declara tiv ely) it is mor e plaus ibl e to co nsid er tha t the m ind e mbo dies th is kn ow ledg e as a c apac ity f or repr esen tation (fo r ins tance, for repr esen ting temp oral ent itie s and relat ions a mon g the m). Th e k no w ledge is then emb ed ded as con stra ints on wha t can b e represen ted, and i t w ill be usefu l to appr oach the p rob lem of sp ecify ing kn ow ledg e in a certain do m ain, as the pr ob lem of sp ecify ing a ‘gr am mar’ of p ossibl e r epres ent ation s in that d oma in (e. g. p ossib le rep res ent ation s o f temp or al r ela tions amon g situ ation s — preceden ce , over lapping , inclus ion). Beside s this rich se t o f do main-spe c ific mod ules, th e m ind n eeds to b e equ ippe d with a s et o f pro cedur es fo r deve loping and enhan cing the i nn ate b as is. Whi le som e o f these ar e no d ou bt d oma in-sp ecifi c, o thers must b e dom ain-ind ep enden t. W e hyp othesiz e that th e human mind st arts life w it h an inn ate b asis for doma in- spec ific kno wledg e that is mo r e analog ica l or diagramm a tic in natur e, a nd th at one of the impor tant ways it dev elop s is in th e enr ich m ent o f the inn ate r epresent a tiona l c apac ities with mo re symb ol ic rep res entat iona l cap aci ties 5 . A mind that h as th e ab ility to choos e ho w it w ill r epr esen t a parti cular p rob lem i t needs to solve , choo s ing fro m a r ep erto ire o f r epre sen ta tion al capa ci ties that in clud e more ana logic al and mo re symb olic no tations i s more flex ibl e, hen c e mo re “ inte llig ent” (mor e ap t to so lve it s p ro blems , hen ce to sur v ive). W e po stu late tha t h um ans h ave this kind of mind . To h andle th is ab ility to cho os e b etwe e n s evera l repr esen tationa l capaci ties , and to ke ep its rep erto ire re lat ively unchan ge d (after a cert a in level of develop m ent), a m ind needs also to h ave g en eri c and glo b al cogn itive p rocedu re s to constru ct repr esen tations on th e fly. Follow ing th e gen eral f ram ewo rk of cogn itive app roach es to languag e, w e b eliev e that l ingu isti c for ms are ( p art ial and un de term ined) instru ction s fo r co ns truct ing interco nn ected doma ins wi th in tern al struc ture. A s c la imed in [20 ] by G . Fau conn ier, this con s truc tion takes p lace a t a cognitiv e l eve l C. Th is l evel i s d istin ct f rom lan gu age struc ture. Construc tion s at l eve l C are n ot “meaning s ”, ne ither r epresen tations assoc iated wi th an y part icu lar se t o f linguis tic exp ress ions. T hey are no t rep resen tation s of the wo r ld, o r o f mo del s o f w or ld, or w ha tsoev er of this so rt. H ow ev er, thes e constru ction s r el ate langu age to r eal w orld , an d th ey p r ov ide var ious r eal-wo rld inferenc es . Th ey also are n ov el and diff eren t fo r each ca se o f lang uage use, an d men tal spac es and conn ections bui ld u p as d iscou rs e un fo lds. Th e pri mary go al of (and primary eviden ce f or) th e ap pro ach in te rm s of intercon ne cted do mains is sc ien tific genera liza tion. The f irs t dev elo pmen ts of Faucon ni er “men tal sp ace s” theo ry focu s o n processes of transf er fro m a source (or bas e) to a targe t. The c apac ity of org anisms to carr y ou t such pro jections lies a t the h eart of cogn ition in its m any for ms. Th e ana lyse s g iven by Faucon ni er are n umer ous and b ased on a r ich array o f lingu istic d ata ( coun terf a ctuals ; time, ten se, and moo d ; o pac i ty; m e tapho r; f ic tive mo tion; g ramma tic al constru c tion s ; and qu antif ica tion ov er co gn itive do ma ins) . Fur ther d ev elo pments of the th eory study another v ery inter esting opera t ion, con cep tua l b len ding [2 1] , w hich also dep end s central ly on s truc tur e pro ject ion and dyn ami c simu la tion . Like stand ard an alogi cal mapping , b lending al igns two p artia l s truc tures , b ut in a d dition, b lend ing pro jects 5 Similar hy po these s relative to the arch itecture of mind and compa tible with data (in psyc ho logy of reaso nin g), conclude to the existe nce of a meta-repre sentationa l reflective leve l (i.e. h an dling meta - repre sen tation, as tho ught ab out thou gh t, and the like) whe re slow log ical inferenc es are drawn consciously . selectiv ely to f orm a third s tru c ture. ( Crea tiv ity in S cien ce is of ten b ased o n concep tua l blending ). All these wor ks in Cogni tive Seman tics g iv e us g uide lin es and examp les to inves tigate in de tai ls ho w sym bol ic an d i coni c repr es entat ions m igh t re late in an intel ligent comp lex sy s tem. 3.2. Addit iona l rema r ks from a Comp u ter S cient ist po in t o f view The p rob lem of the building ( and, at first, of the d escrip tion ) of comp lex prog ram archite ctur es on compu ters is the concern o f softw are en gi neering . We think that the num erou s diff icul ties arising at t his level are du e to t he defi cien cies of ou r pro gr amming languag es, in particu lar bec ause th ey d o no t incor por ate a mo r e sop histic ated level of des crip tion of the impor t fe atur es fro m o ther modu les. Our cl aim is th at they d o n ot describ e the ir ow n archit ectur e (and th er efor e cann ot incor po rate impor t featu res at this lev el of des cription ). W e wi ll n o t deve lop thi s claim here , althou gh diagra ms are ob viou s ly h e lpful for th e des crip tion of archi tec tures. W e wi ll only add a f ew rem ark s o n iconicity (o r o n the no n ar bitrar y s hape o f a symb ol), to sho w that this dimen s ion cou ld be h elpfu l at v ariou s lev els of se mant ic d escrip tion . A first r em ark is tha t a n on ar bi trary shap e ch aract er may ap p ear in sma ll tou ch es, at the lev el of an isola ted symb o l, wi thou t ev en being inc lu d ed in a true ico nic sys t em (with pro per an alog ic al pro per ti es). Fo r instanc e, a s imp le d ifferen ce in the char acter fon t, as th e add ition o f bo ld f ace, could mo dify traditio na l sy mbo lic r epresen ta tions in a creativ e way. Yo u can k eep th e old me aning fo r the new expr essio n (f or ins tance a value 0 and a value 0 b oth r efering to z ero as usu al) and nev ertheless h av e a sup plemen tary me aning , r el ative to ano ther di mens ion in the mo de led w or ld, o r in th e calcul ation pr oces s i tself. You can fo r instanc e disting u ish betwe en a true ( and fin al) value , fr om on e that cou ld still ch ange, o r be s et by defau lt b y the sy stem . Or, when added to more abstr ac t sy mb ols , a s tho s e de scr ibing the rewriting rules of a log ic system , i t cou ld in trod uces s econ d o rder rewri ting r ules , al lowing to ign or e intermed iate terms (for ins tanc e, if not in b oldf ace). Thus, trad itiona l e limina tio n r ules could app ly in a more eff icien t way (i .e. be twe en dis tan t e lement s), jus t b y m eans of an additio na l graph ica l fea ture, def ined an d used at the lev el of t he m eta-l ang uage itse lf. Our seco nd rem ark is tha t, in th e con tex t of a compu ter , the genera l sch em a fo r the implem ent ation o f th e h omo mor ph ism betw een syn ta ctic rep resent ation s and se man ti c represen tations d o no t stand on a simple line, as ph iloso ph er s an d logicians cons id er. It will trans late into a pro gr am that will cal culat e, from int ernal repres ent ation s (com ing fro m o ur syntactic repres ent ation s, by an o per at ion of “int er naliza tio n”), o th er in terna l represen tations – wh i ch w e hav e t o “ex tern aliz e” i f we want to g et th em e xp licit . Therefo re , there ar e many oth er mean s to estab lis h corre sp on dences, or explo it ing particu lar di agram ma t ic f ea tures , betw een all of the se r ep resent a tions. I n par ticu lar, some iconic r elat ions m ay b ind the sy n tax of the p ro gram mi ng languag e u sed to th at o f the interna l rep r esen tation s used , yield ing to int erna lly mo d ifi able mean ing (I MM) . In reflect ive interp re ters (cf . l isp), an ob je ct inter pre ts as p rog ram or d at a, dep endin g on its co ntex t o f use. In such frame wo rk, the tradition al d a ta/pr o gr am d istinc tion van ished (as in machin e l ang uages) . W ith refl ec tive fe a tures, eva lu ation can be susp ended or delay ed (som e f un ctiona l languag es implem ents lazy eva lu ation ). Som e pr og ram ming languag es m ay a lso h av e oth er spe cif iciti es, as for ins tanc e, a p attern-m atching oper ation as in P LAS MA ( an ac tor lan guag e o f the eigh ties). Ther efor e, on a comp u ter, very co mplex rela tions betw een th e repr esen ted w orld and the rep res ent ing wor ld ar e virtua lly p ossib le . Ano ther r emark rela tive to th e use of b old (or oth er su ch f eatur es) is that it c a n obv ious ly be use to hand le so me not ion o f f ocus. Fo cus th eories h ave n ot y et b ee n succ essful ly design , bu t it i s a lack in o ur theor etical tools. T here are m any fie lds wher e some not ion of fo cus wou ld be o f great h elp (in perc ept ion th eory , in dis cou rse th eo ry, etc.). On e re ason of this f ailur e might b e pre cisely tha t th e theor ies of f ocus r equ ire referenc es to the un der lying compu tat iona l mech an ism (a s reflect ive pro pert ies of the pro gr amming langu ag e) v . If we take s eriou sly the assu mpt ion o f endn ote IV, i.e., that the me ta-lang uage requ ired to p rov ide the seman ti cs o f a sy stem ha s to r eflec t ( in some way) the pos sibiliti es of conf igur at ion s of term s in the repr es en tation al lang uage , th en we hav e to inv estiga te the fol lowing q uest ions: w ha t syn tax do w e need to easi ly p ro vide the seman tics o f H RS? W ou ld it b e enou gh to add simp le reflec tive and loca l gr aph ic al featur e ( as tho s e o f some of o ur p ro gramming lang uag es) to a trad ition al fu nction al and symb olic langu ag e, or sho uld this synt ax be tr ickier? Con clusion Wor ks d one so far o n d iagra mm atic r eason ing pr ovid e fra gmen ts of eviden ce about how p eop le use icon ic repr es entat ions, and id entify s ome of the pr ob lems raised b y the pro ject of A GI. Y et , th ere is s till m uch to do to un derst a nd the vari ety of f orm s in which inf orm ation c an stor ed and m an ipul ated in in telli gent con trol sys tems. We believe tha t we cou ld m ake imp ort ant p ro gress in stu dyi n g in deta ils th e r ela tion betwe en icon ic an d sy mbo lic f eatur es in h yb rid r epresen ta tion sy stems, as wel l as in paying att ention to th em in the th eoret ical too ls and sy mbo lic langu ag es that w e us e. Endnot es i Ho wev er, c ontrary to wh at m an y au thors have s aid, it is no t d ifficult to repre sen t d isjunc tive cas es on diag rams, and we will s ee so me ex emplars in the n ext se ction (see Figure 5). It is also p ossib le to have iconic symb ols o f se cond o rder in p ure ly diagram matic systems. C.S. Peirce first s ug ges ted to rep resent disjunction s in the form of a line co nn ectin g two ico nic sy mbo ls. But in a forma l system , the introdu ction of suc h sy mbo ls requ ires the definition of trans forma tion ru les on diagrams. ii Th e analog ical/digital distin ction a lso relies on a no tion o f specificity for Dretske [11 ]. For h im, every signa l tran smitting in formation n ece ssarily c arries this info rmatio n unde r two a spects: an a nalogica l fo rm and a dig ital form. T he a nalog ical f orm alway s c on tains an a dd itional s pecificity relative to the i nformation prop erly conveyed by th e digital form . iii Th e omn ipres enc e of rep resentation of the sam e typ e designa ting the sa me objec t is thu s obs erve d in hum an lan guag e, where refe rences to an ob ject can b e spre ad out everywh ere in a docume nt, so th at inform ation is no t « loca lized » (q uo ted from [1 3]). For P&M, this ad ditiona l c hara cter is the on e req uired to give diag rammatic systems t he closure under constraints pro pe rty, when c om bine d wit h i con icity and a con straint an d sy stema tic homomo rphism. iv Let u s take the ex am ple of An n, Ga ston , an d Is abe l, who are re prese nted a s « orde red » in the diag ramm atic c ase. A minim al d ifferen ce, b ut a n e ssential o ne, b etwee n th e two typ es o f re pres entations is the follo wing: (I) left-of (a, g) & left-o f (g , i) and (II) ordere d ([ a, g, i]) (or jus t [a , g, i] ) There is an a dd itional syntac tical c omplex ity for (II) w hich p reven ts its meanin g, c on trary to that o f (I), from being describe d as a function o f one a rgument o f its pre dica te's meanin g. In deed , y ou ca n ea sily assig n a mea ning to th e sem antic eq ua tion: [ [ left-of ( a  , g) ] ] = [ [ left-o f ] ] ( [ [ a ] ] , [ [ g ] ] ), while y ou ca nnot write any thin g else but: [ [ ordered ([ a, g, i] ) ] ] = [ [ ord ered ] ] ( [ [ [ a, g , i] ] ] ), w hich implies g iving me aning, at th e me ta- lang uage lev el , to a c onfigura tion o f term s (th e list figurin g be tween simple squ are b rackets). Th erefo re, the sema ntic de scriptive meta-la nguage must offer possib ilities o f synta ctica l struc turing o f data similar to the one s figuring i n the representa tion language , because it will sometime s be necessary t o assign th em a mean ing . This is not to say tha t all sy ntac tical n uan ce s o f the repres enta ti ona l syste m m ust re flected in the interpre tation sys tem, b ecause no t all ico nic rep resentation fea tures are interp reted in a d iagra mmatic repre sen tation (think to the u se of m arked feature s in ma them atical fig ure to derive geom etrical proofs). Neve rthele ss, it shows tha t seman tic com po sitiona lity relies on sy nta ctic conside ration s. v Note that in the co nte xt of gra ph ical interfac es, sev eral no tion o f focus a re requ ired a t a very low level (in the graphic se rver itself), in ord er to link the k eyboard (and /or events on the po inter o f th e mouse) to a particu lar w indow. Th e develop ment of graphic al in terfac es (and networks) ha s in troduced conside rable cha nges in the p rev ious progra mmin g fra mework. (1) There a re o the r so urces o f in put than letters (at least, mou se inp uts), an d oth er sorts o f outp ut (graphic s, soun d). (2) The inpu t/output da ta are of d istinct natu re, bu t they may b e link tog eth er in the sy stem (as the mo use and the scre en). (3) The sha ring o f inpu t/output dev ices by se vera l program s add s some ad ditiona l complex ity to the em ergin g fram ework. Refer en ces [1] A . Slom an , Mu sing s on the role of log ical a nd non lo gical represen tation s in in telligence, Diagr amma tic Rea son ing : co gnitive a nd computa tional pe rspective . J. Gl asg ow, N . Nari, Nara ya nan , B. Chand rase kara n, (eds .), MIT Press , AAA I Cambrid ge , MA and Lond on , 199 5, 7-32. [2] J . Barwise and J. Etch emend y, Visu al In formation and Va lid Rea soning , in Visu alization in Mathema tics , Zimmerma n, W., ed ., Mathem atical Asso ciatio n of Ame rica, Wash ingto n DC, 199 0. [3] J . Barwise and J. Etch eme nd y, Hyp erproo f . CSL I Pub lication s, Stanford , 199 4. [4] J . Barwise and J. Etch em endy, Hetero geneo us Log ic. J. Glas go w a nd alii (e ds.), MIT P ress, A AAI Camb ridge , MA and Lon do n, 19 95, 211-234 . [5] S-J. Shin, The log ical statu s of Diag rams . Cam bridg e Un iversity Press , 1994 . [6] C. Reca na ti, Raison ne r avec de s diag ramm es: perspec tives co gnitives et comp utation ne lles, In tellectica 40, 20 05 , 9-42. http://hal.arc hive s-ou ve rtes.fr/hal-0 008500 4/fr. [7] C. Recanati, Cha racte ristics of d iag ramm atic reason ing, Proceeding s o f Eur oCo gSc i07 , The s econd Europ ean c ognitive science co nferenc e, May 23 -27 2 007, Law renc e Erlb aum Ass ociates, Delp hi, Greec e, 200 7, 510-5 15. http://hal.arc hive s-ou ve rtes.fr/hal-0 015332 8/fr. [8] J . Bruner, J. G oo dn ow, and A. Austin, A stu dy o f thinking , Wiley, New York , 195 6. [9] K . Stenn ing an d I. Ob erland er, A Co gn itive Theory o f Graph ical and Lin guistic Rea son ing : Log ic and Implem en tation, Cog nitive S cien ce , 19 (1), 19 95. [10] J. Perry and E. Macke n, Interfa cing Situatio ns, in Logic, Lan gua ge and Computatio n , J. Seligm ann and D. Westerstah l (eds.) , Stan ford Un ive rsity Press, 199 6. [11] F. Dretske, Kno wledg e an d the flow of informa tion , Black well, Oxfo rd, 19 81, 13 7. [12] J. Larkin and H. Si mon , Why a Diag ram Is (Some times) Worth Ten T hous and Words. Cognitive Scien ce , 11, (19 87). [13] E. Mack en , J. P erry, an d C. Hass, Richly Groun din g Symb ols in ASL. CSLI Repo rt no. 93 -18 0, 1993 . [14] C. Rec an ati, Diag ramm es pour résoud re le p roblème d’Einstein e t ce lui d’un joueur de Masterm ind, Rapp ort LIPN, dec 20 04 , Unive rsité Paris13. http ://h al.arch ives-o uv ertes.fr/h al-00 085056 /fr/. [15] P.N. John so n-Laird , Mental Mod els: to ward s a co gnitive sc ienc e of lan gu age, infer ence, a nd con sciousn ess , Camb ridg e University Press , Cambridg e, 19 83 . [16] J. Barwise an d J. Etchem en dy, Hyp erpro of . CSLI Publica tions, Stanfo rd,19 94 . [17] H. Reiche nb ach , Elements of sy mbo lic Logic , Macmil lan , New York , 194 7. [18] C. Smith, The para mete r of Aspect . Studies in Lin guistics and Philoso ph y 43 , Kluwer Acad emic Publishe rs, 199 1. [19] N. Horn stein, As Time Go es By - Tense and Unive rsal Gra mma r . MIT Press, Cambrid ge MA, 19 93 . [20] G. Fauco nn ier, Mappin gs in tho ug ht and lan guag e . Camb ridge University Press, 19 97 . [21] G. Fauc on nier, Co ncep tual blend ing and analogy . The analog ical min d . D. G entn er, et alii (e ds). MIT Press, Cambrid ge MA and Lon do n, 20 01 .