Unbeatable Imitation

Unbea t able Imit a tion ∗ P eter Duersc h † J¨ org Oec hssler ‡ Burkhar d C. Sc hipp er § F ebruary 18, 20 1 0 Abstract W e sho w that for man y classes of symmetric t w o-pla y e r games, the simple de- cision rule “imitate- the-b est” can hardly b e b eaten b y an y other decision rule. W e pro vide necessary and sufficien t conditions for imita tion to b e unbeatable and sho w that it can only be beaten by muc h in games that are of the ro c k-scissors-pap er v ariet y . Th us, in many int eresting examples, like 2x2 games, Courn ot duop oly , price comp etition, rent seeking, p ublic go o ds games, common p o ol resource games, minim um effort coordin ation games, arms race, searc h, bargaining, etc ., imitation cannot b e beaten b y m uch eve n b y a v ery clev er opp onent. Keyw ords: Imitate-the-best, learning, symm etric games, relativ e pay offs, zero- sum ga mes, ro ck-paper-scissors, fin ite p opulation ESS, p oten tial games, quasisub- mo dular games, quasisup ermo dular g ames, quasiconca ve games, aggrega tiv e games. JEL-Classifications: C72, C 73, D43. ∗ W e thank Ca rlos Al´ os-F err er, Chen Bo, Drew F udenberg , Alexander Matr os, and John Stac hurski for interesting discussions. Seminar audiences at Australian Na tional Univ er sity , Melbourne Universit y , Monash Univ er sity , UC Da vis, UC San Diego, the Univ e r sity of Heidelberg, the Univ ersity of Queens- land, and at the International Conference on Game Theory in Ston y Bro ok, 2009, con tributed helpful comments. † Department of Econo mics, Universit y of Heidelb erg ‡ Department of Econo mics, Universit y of Heidelb erg, Email: oechssler@uni-hd.de § Department of Econo mics, Universit y of California , Davis, Ema il: bcs chipper@ucdavis.edu “Who ever wants to set a g o o d e xample must add a gr ain of fo olish ness to his virtue: then others c an imitate and yet at the sam e time surp ass the one they imitate - w h i ch h uma n b eings love to do . ” F rie drich Nietzsche 1 In tro du ction Psyc hologists and behav ioral economis ts stress the role of s imple heuris tics or rules for h uman decision making under limited computatio na l capabilities (see G igerenzer and Selten, 20 02). While suc h heuristics lead to suc cessful decis ions in some particular tasks, they may b e sub optimal in others. It is plausible that decision make rs may cease to adopt heuristics that do worse than others in relev ant situations. If v arious heuristics are pitted against eac h other in a con test, then in the long run the heuristic with the highest pa y off should surviv e. One of the heuristics in the contes t ma y b e a rat io nal, omniscien t, and forward lo oking decision rule. Ev en if suc h a rational rule is not curren tly among the contestan ts, there can alw ays be a “m utat io n”, i.e., an in v ention of a new rule, that ente rs the p o ol of rules. A heuristic that do es v ery ba dly against suc h a rational rule will not b e around for long as it will not b elong to the top p erformers. Being sub ject to exploitatio n b y the rational opp onen t in strategic situations w ould b e an ev olutionary liability . Consequen tly , w e lik e to raise the following question: Is there a simple adaptiv e heuristic that can not b e b eaten eve n b y a rational, omniscien t and forw a r d lo oking maximizer in lar g e classes of economically relev an t situations? The idea for this pap er emerged from a prior observ ation in experimen tal data. In Duersc h, Kolb, Oec hssler, and Sc hipp er (2 010), sub jects play ed a gainst computers that w ere programmed according to v arious learning algorithms in an Cournot duop oly . On a v erage, hum an sub jects easily w on ag ainst all of their computer o pp onen ts with one exception: the computer follow ing the rule “imitat e- t he-b est”, the rule that simply pre- scrib es t o mimic the action of the most success ful play er in the previous round. This suggested to us that imitation ma y b e hard to b eat ev en b y forw ard–lo oking play ers. In this pap er, w e pro v e that this holds more generally . The decision heuristic “imita t e- the-b est” is v ery hard to b eat b y any other decision rule in larg e class es of symmetric t wo-pla ye r games that are highly relev an t for economics a nd include games such as all symmetric 2x2 games, Courno t duop oly , Bertrand duop oly , ren t seeking, public g o o ds games, common p o ol resource ga mes, minim um effort co o rdination games, synergistic relationship, arms r a ces, Diamond’s searc h, Nash demand bargaining, etc. 1 W e s hall cons ider tw o notio ns of b eing “ un b eatable”. W e call imitation “essen tially un b eatable” if during the infinitely repeated game against the some opp o nen t, the opp o- nen t cannot obtain, in total, ov er an infinite n umber of p erio ds, a pa y o ff difference that is more than the maximal pa yoff difference for the one–perio d ga me. As a w eaker not io n w e consider the conce pt of b eing “ no t sub ject to a money pump”. W e s a y imitation is not sub ject to a money pump if there is a b ound on the sum of pa yoff differences an opp onen t can achiev e in the infinitely rep eated game. It w ould be in tractable to explicitly consider how all p ossible opp onen ts – who ma y ha v e any arbitrary decision rule – w ould pla y against an imita t o r. That is wh y we consider the toughest p ossible opp onent ag a inst t he imitator in order to obtain an upper b ound for ho w muc h an imitat o r can b e b eaten b y any p ossible opp onent. This toughest p ossible opp onen t clearly is a dynamic relativ e pa y o ff maximizer who maximizes the sum of all future differences b et wee n her pa y o ff a nd the imitator’s pay off. The relative pa y off maximizer is assume d to b e infinitely patien t and forward loo king and nev er to mak e a mistak e. More imp ortan tly , the dyn amic pa y off maximiz er is assumed t o kno w that she is ma t c hed ag ainst an imitator. That is, she kno ws exactly what her opp onent, the imitator, will do at all times, including the imitator’s starting v alue. F inally , the dynamic pay off maximizer is enabled t o commit to an y strat egy including an y closed- lo op strategy . Although these a ssumptions are certainly extreme, they mak e sure that if imitation cannot b e b eat en by this maximizer, then it cannot b e b eaten b y an y decision rule including dynamic absolute pa y off maximization or an y decision rule that is more m yopic or less omniscien t. Our results are as follows. W e presen t ne c essary and sufficient c onditions for imitation b eing sub ject to a money pump. The paradigma t ic example for a money pump is playing rep eatedly the g ame ro c k–pap er-scissors, in whic h, obvious ly , an imitator can b e exploited without b ounds b y the maximizer. The main result of this pap er is that imitation is sub j ect to a money pump if and only if the r elative p ayoff game in question con tains a generalized ro ck–paper-scissors game as a submatrix. Since the existence o f a r o c k–pap er-scissors submatrix may b e cumbersome to c hec k in some instances, w e also provide a n umber of sufficien t conditions for imitation not to b e sub ject to a money pump that ar e based on more familiar concepts like quasicon- ca vity , generalized ordinal p oten tials, or quasisubmo dularity/q uasisup ermo dularity and aggregation of actions. W e also provide a n um b er of s ufficien t conditions for imitation to b e essen tially un- b eatable lik e exact p oten tials, increasing/decreasing differences, or additiv e separability . 2 One suc h condition is that the game is a symmetric 2 x2 game. T o g ain some in tuition for this, consider the g ame of “c hic ken ” presen ted in the f ollo wing pa y o ff matrix. sw erv e straigh t sw erv e straigh t  3 , 3 1 , 4 4 , 1 0 , 0  Supp ose that initially t he imitato r starts o ut with pla ying “sw erv e”. What should a forw ard lo o king opp onen t do? If she decides to pla y “straight”, she will earn more tha n the imitator to da y but will b e copied b y the imitator to morro w. F rom then on, the imitator will sta y with “ straigh t” foreve r. If she decides to pla y “sw erve” to da y , then she will earn the same as the imitator a nd the imitator will s ta y with “sw erv e” as long as the oppo nent stay s with “sw erve”. Supp ose the opp o nent is a dynamic relativ e pa y off maximizer. In that case, the dynamic relativ e pay off maximizer can b eat the imitator at most by the maximal one-perio d pa y off differen tial of 3. No w supp ose the opp onen t maximizes the sum of her ab s olute pa yoffs. The b est an absolute pay off maximizer c an do is to play sw erve forev er. 1 In this case the imitato r cannot b e b eaten at all a s he receiv es the same pay off as his opp onen t. In either case, imitation comes v ery close to the top–p erfor ming heuristics and there is no need to abandon suc h an heuristic. Imitate-the-b est ha s b een prev iously studie d theoretically and exp erimen tally mostly in Cournot oligop olies. V ega-R edondo (1997) shows that in sym metric Cournot oligop oly with imitators, the long run outcome con v erges to the comp etitiv e output if small mis- tak es are a llo w ed. Huc k, Normann, and Oechs sler (1999), Offerman, Potters, and Son- nemans (2002), and Ap esteguia et al. (20 07, 2010) pro vide some exp erimen tal evidenc e. V ega- Redondo’s result has b een g eneralized to agg r egativ e quasisubmo dular games b y Sc hipp er (2003) and Al´ os-F errer and Ania (2005). F or Cournot oligo p oly with imitators and m yopic b est reply pla y ers, Schipper (20 09) sho ws that the imita t o rs’ long run av erage pa y offs are strictly higher than the b est reply play ers’ av erage pa y offs. The a rticle is organized as follows. In the next section, w e presen t the mo del a nd pro vide fo rmal definitions for b eing un b eatable. Our main result, whic h pro vides a nec- essary and sufficien t condition for a money pump, is con tained in Section 3. Suffic ien t conditions for imitation to b e essen tially un b eatable are giv en in Section 4. Section 5 pro vides sufficien t conditions for imitation not being sub ject to a money pump. W e finish with Section 6, where w e summarize and discuss the results. 1 Pa yoffs ar e ev alua ted acc o rding to the ov er – taking criterio n (see b elow). 3 2 Mo d el W e consider a s ymmetric 2 t wo–pla y er game ( X, π ), in whic h b oth pla yers are endo w ed with the same (finite or infinite) set of pure actions X and the same b ounded pa y off function π : X × X − → R , where π ( x, y ) denotes the pa y off to the play er c ho o sing the first argumen t. W e will frequen tly mak e use of the fo llo wing definition. Definition 1 (Relativ e pa y off gam e) Given a symme tric two-player ga m e ( X , π ) , the r elative p ayoff game is ( X , ∆) , whe r e the r elative p ayoff function ∆ : X × X − → R is define d by ∆( x, y ) = π ( x, y ) − π ( y , x ) . Note that, by construction, ev ery relative pay off game is a symmetric zero- sum game since ∆( x, y ) = − ∆( y , x ). W e introduce tw o t yp es of pla y ers. The imitator follo ws the simple rule “imitate- the-b est”. T o b e precise, the imitator adopts the opp onent’s action if and only if in the previous round the opp o nen t’s pa y off w as strictly higher than that o f the imitator. F ormally , the action of the imitator y t in p erio d t giv en the action of the other play er from the previous p erio d x t − 1 is y t =  x t − 1 if ∆( x t − 1 , y t − 1 ) > 0 y t − 1 else (1) for some initial action y 0 ∈ X . The sec ond type w e consider is a dynamic r elative p a yoff m aximizer . The dynamic relativ e pay off maximizer, from no w on call her the maximizer , maximizes the sum of a ll future pa y o ff differen tials b et wee n her and the imitat o r, D ( T ) := T X t =0 ∆( x t , y t ) , (2) where y t is kno wn to b e given b y (1). Since this sum may b ecome infinite for T → ∞ , we a ssume that the maximizer ev aluat es her strat egies according to the overtaking-criterion (see e.g. Osb orne and Rubinstein, 1994 , p. 1 39). Accordingly , a sequence of relativ e pa yoffs { ∆( x t , y t ) } ∞ t =0 is strictly preferred to a sequence { ∆( x ′ t , y ′ t ) } ∞ t =0 if lim T inf P T t =0 (∆( x t , y t ) − ∆( x ′ t , y ′ t )) > 0. 3 2 See for instance W eibull (19 95, Definition 1.1 0 ). 3 General r esults on the existence of optimal over-taking s tr ategies are develop ed in Leizarowitz (19 96) and the literature cited therein. Here we ca n side-step the issue of existence beca use our pro ofs will b e 4 If w e used time-av eraging instead of the ov er-taking criterion, then the maximizer w ould b e indifferen t b et wee n a sequence of zero relativ e pay o ff s and a sequence with a finite num b er o f strictly p o sitive relative pay o ffs and zero thereafter. Y et, our aim is to pit t he imitator aga inst a maximizer who cares even ab out a finite num b er of pa y off adv an t a ges. If we used time discoun ting instead the o v er- t a king criterion, then the maximizer ma y prefer a sequence with a large pa y o ff at the b eginning ov er a sequence with an endless cycle of small but p ositiv e relativ e pay o ffs. Ho w eve r, w e b eliev e that money pumps – ev en if small – are a feature of irrationalit y tha t a r ational opp onen t w ould tak e adv an tage of. It is imp ortant to realize just how extreme our assum ptions regarding the maximizer are. The maximizer is infinitely patien t and forw ard lo oking and nev er mak es a mistak e. More imp o rtan tly , s he is assumed to know exactly what her opp o nent, the imitat o r, will do at all times, including the imita tor’s starting v a lue. Although these assumptions are certainly extreme, they mak e sure tha t if imitation cannot b e b eaten b y this max- imizer, then it can no t b eaten b y an y decision rule including dynamic absolute pay off maximization or any decision rule that is more m y opic o r less omniscien t. Definition 2 (No money pump) We say that imitation is not sub ject to a money pump if ther e exists a b ound M ∈ R + such that for any initial action of the imitator y 0 ∈ X , lim T →∞ sup T X t =0 ∆( x t , y t ) ≤ M , (3) wher e y t is given by (1). That is, imitation is not sub ject to a money pump if it can b e b eaten only b y a finite amoun t alt ho ugh the game b et w een the imita tor and the maximizer runs fo r an infinite n umber of p erio ds. In some cases w e can show that imita tion can in fact not b e b eaten b y more than the pay off differen tial from a single p erio d. Definition 3 (Essen tially un b eatable) We say that imitation is esse n tially unbeat- able if it c an b e b e aten in total by at most the maximal one-p erio d p ayoff d i ff er ential, i.e., if M in ine quality (3) is at most ˆ ∆ := max x,y ∆( x, y ) . constructive in the s ense that (a) we constr uct strateg ies - no matter whether optimal o r not - that b eat the imitator o r (b) we show that no strategy can beat the imitator. 5 As in previous studies of imitation (see e.g. Al´ os-F errer a nd Ania, 20 05; Sc hipp er, 2003; V ega- Redondo, 1997), the concept of a finite p opulation ev olutionary stable strat- egy (Sc haff er, 198 8, 1989) pla ys a prominen t ro le in our analysis. Definition 4 (fESS) An action x ∗ ∈ X i s a finite p opulat io n evolutionary stable strat- egy (fESS) of the game ( X, π ) if π ( x ∗ , x ) ≥ π ( x, x ∗ ) for al l x ∈ X. (4) In terms of the relativ e pay off game, inequalit y (4) is equiv alent to ∆( x ∗ , x ) ≥ 0 for all x ∈ X . Already Schaffer (1988, 1989) o bserv ed that the fESS of the game ( X , π ) and the symmetric pure Nash equilibria of the relativ e pay off game ( X, ∆) coincide. 3 A Nece s sary and Sufficien t Cond i tion for a Money Pump The game ro c k–pap er-scissors is the paradigma t ic ex ample for ho w an imita t o r can b e exploited without b ounds b y the maximizer. In our terminology , imitation is sub j ect to a money pump. Example 1 (Ro c k-P ap er-Scissors) Consider the wel l known r o c k - p ap er-scissors gam e. 4 R P S R P S   0 − 1 1 1 0 − 1 − 1 1 0   Cle arly, if the imitator starts for in s tanc e w i th R, then the dynamic al ly optimal str ate gy of the maximizer is the cycle P-S-R... In this w a y, the maximizer wins in every p erio d and the imitator lo s es in every p erio d. Over time, the p ayoff differ enc e wil l gr ow wi thout b ound in favor of the ma ximizer. W e can generalize Example 1 b y noting that the crucial feature of the example is that the maximizer can find for each action of the imitator an action whic h yields her a strictly p ositive relativ e pa yoff. 4 In the follo wing, we will repr esent symmetric pay off matrices b y the matrix of the r ow pla yer’s pay offs only . 6 Definition 5 (Generalized Ro c k-P ap er-Scissors Ma trix) A symmetric zer o-sum game ( X , π ) is c al le d a gener alize d r o ck-p a p er-scissors m atrix if fo r e ach c olumn ther e e x ists a r ow with a st rictly p ositive p ayoff to player 1. It should b e fairly obv ious that if a zero–sum game con tains somewhere a submatrix that is a generalized ro c k-pap er-scissors matrix, then this is sufficien t fo r a money pump as the maximizer can mak e sure that the pro cess cy cles forev er in this submatrix. What is probably less obvious is that the existence of suc h a submatrix is also neces sary for a money pump. Definition 6 (Generalized Ro c k-P ap er-Scissors Gam e) A symmetric zer o-sum gam e ( X , π ) is c al le d a g e ner alize d r o ck-p ap e r- s c issors gam e if it c ontains a submatrix ( ¯ X , ¯ π ) with ¯ X ⊆ X and ¯ π ( x, y ) = π ( x, y ) for al l x, y ∈ ¯ X , and ( ¯ X , ¯ π ) is a gener alize d r o ck- p ap er-scissors matrix. This leads us t o our main result. Theorem 1 Imi tation is subje ct t o a mon ey pump in the finite symmetric game ( X , π ) if and only if its r elative p ayoff game ( X , ∆) is a gener alize d r o ck-p ap er-scissors game. Recall that a ccording to Definition 2, imitation is sub ject to a money pump if there exists some initial condition y 0 ∈ X suc h that inequalit y (3) is violated. The pro of of the theorem follow s directly from the following three lemmata. W e use the follo wing preliminary observ ation rep eatedly in the analysis. Lemma 1 Consi d er a symmetric gam e ( X, π ) with its r elative p ayoff game ( X, ∆) . The maximizer wil l never cho ose an ac tion x t such that ∆( x t , y t ) < 0 . Proo f. Suppose to the con tra ry that the maximizer c ho oses an action x t suc h tha t ∆( x t , y t ) < 0. Then in p erio d t + 1, t he imitator will not imitate her p erio d t action. But then, she could improv e her relativ e pay o ff in t by setting the same action in t as the imitato r without influencing the actions of the imitato r in p erio d t + 1 or an y o t her future p erio d.  Giv en a symmetric t w o- play er game ( X , π ) and its relative pa y o ff game ( X , ∆), a pat h in the action space X × X is a sequence of action profiles ( x 0 , y 0 ) , ( x 1 , y 1 ) , ... . A path is 7 constan t if ( x t , y t ) = ( x t +1 , y t +1 ) for all t = 0 , 1 , ... . Otherwise, the path is called non– constan t. A non–constant finite path ( x 0 , y 0 ) , ..., ( x n , y n ) is a cycle if ( x 0 , y 0 ) = ( x n , y n ). A cycle is an imitation cycle if for all ( x t , y t ) and ( x t +1 , y t +1 ) on the path of the cycle ∆( x t , y t ) > 0 and y t +1 = x t . Along an imitation cycle, one play er alw ays obtains a strictly p ositiv e relativ e pay off and the other pla ye r mim ics the action of the first play er in the previous round. Th us, a n imitation cycle nev er contains an action profile on the diago na l of the pa y o ff matrix. Lemma 2 F or an y finite symmetric ga me ( X , π ) , imitation is subje ct to a money pump if and only if ther e exists an imitation cycle. Proo f. Consider a finite symmetric game ( X, π ) and its relativ e pa y o ff ga me ( X, ∆). W e sho w first that if imitation is sub j ect to a money pump, then there is a imita t ion cycle. Since the game is finite, there can not b e infinitely man y strictly p o sitiv e relative pa y off impro veme n ts unless there is a cycle. T o sho w that suc h a cycle implies an imitation cycle, supp ose b y contradiction that there exists a p erio d t suc h that ∆( x t , y t ) ≤ 0. W.l.o.g. assume that ∆( x t +1 , y t +1 ) > 0. This is w.l.o.g. b ecause w e assumed a money pump. By equation (1) the imitator will no t imitate in t + 1 the previous p erio d’s actio n of t he maximizer, i.e., y t +1 = y t . But then the maximizer could strictly improv e the sum of her relative pa y offs alr eady in t b y setting x t = x t +1 . Th us, there m ust b e a cyc le with ∆( x t , y t ) > 0 for a ll t . The decision rule of the imitator then r equires that y t +1 = x t for all t , whic h pro v es that suc h a cycle is an imitation cycle. The con v erse is trivial.  Lemma 3 Consi d er a finite symmetric game ( X , π ) with its r elative p ayoff game ( X, ∆) . ( X , ∆) is a gener alize d r o ck-p ap e r-scissors ga me if and only if ther e exists an imitation cycle. Proo f. “ ⇐ ”: If there exists an imitation cy cle in ( X, ∆), let ¯ X b e the orbit o f the cycle, i.e., all actions of X that are play ed a long the imitation cycle. F or eac h a ctio n (i.e., column) y ∈ ¯ X , t here exists an a ctio n (i.e., row ) x ∈ ¯ X suc h that ∆( x, y ) > 0 . Hence, ( ¯ X , ¯ ∆), whe re ¯ ∆ is defined by ¯ ∆( x, y ) = ∆( x, y ) for all x, y ∈ ¯ X , is a generalized ro c k-pap er-scissors submatrix. Thus , ( X, ∆) is a generalized ro c k-pa p er-scissors game. “ ⇒ ”: If the relative pay o ff game ( X, ∆) is a generalized ro c k-pap er-scissors game, then it con tains a generalized ro c k-pap er-scissors submatrix ( ¯ X , ¯ ∆). That is, for each column of the matrix game ( ¯ X , ¯ ∆) there exists a r ow with a strictly p ositiv e relative 8 pa y off to pla y er 1. Let the initial action o f the imitator y b e con tained in ¯ X . If the maximizer selects suc h a row x ∈ ¯ X f o r whic h she earns a strict p o sitiv e relativ e pa yoff, i.e., ∆( x, y ) > 0, then she will b e imitated by the imita t o r in the next p erio d. Y et, at the next round, w hen the imitator pla ys x , the maximizer has anot her action x ′ ∈ ¯ X with a s trictly p ositiv e relativ e pay o ff, i.e., ∆( x ′ , x ) > 0. Thus the imitator will imitate he r in the follo wing perio d. More generally , for eac h a ction y ∈ ¯ X of the imita t or, the re is another action x ∈ ¯ X , x 6 = y of the maximizer that earns the la tter a strictly p ositive relativ e pay off. Since ¯ X is finite, suc h a sequenc e of actions m ust contain a cycle. More- o v er, w e just argued that ∆ ( x t , y t ) > 0 and y t +1 = x t for all t . Th us, it is an imitat ion cycle.  Theorem 1 is used to obtain an in teresting necessary condition for imitation b eing not sub ject to a money pump. Prop osition 1 L et ( X , π ) b e a finite symmetric game with its r elative p ayoff game ( X , ∆) . If ( X , ∆) has no pur e sadd le p o i n t, then imitation is subje ct to a money pump. Proo f. By Theorem 1 in D uersc h, Oec hssler, and Sc hipp er (2010), ( X , ∆) has no symmetric pure saddle p oint if a nd only if it is a generalized ro c k-pa p er-scissors matrix. Th us, if ( X, ∆) has no symmetric pure saddle p oint, then it is a g eneralized ro c k-pap er- scissors game. Hence, b y Theorem 1 imitation is sub ject to a money pump.  Corollary 1 I f the finite symmetric g a me ( X , π ) h a s no fESS, then imitation is subje ct to a money p ump . In other w o r ds, the existence o f a fESS is a necessary condition for imitatio n not b eing sub j ect to a money pump. The reason for the existen ce of a f ESS not b eing sufficien t is that there could b e a generalized rock-paper- scissors s ubmatrix o f the game (“disjoint” from the fESS profile) that gives rise to an imitation cycle. If the initial action of the imitator lies within the actio n set corresp onding to this submatrix, then imitation is sub j ect to a money pump. Since the relativ e pa yoff game of a symmetric z ero-sum game is a generalized ro ck- pap er-scissors game if and only if the underlying symme tric zero- sum game is a general- ized ro ck-paper- scissors game, w e obtain fro m Theorem 1 the following corollary . Corollary 2 I mitation is subje ct to a money pump in the finite symmetric zer o-sum game ( X, π ) if and only if ( X, π ) is a gener alize d r o ck-p a p er-scissors g ame. 9 4 Sufficien t Conditi ons for Essentially Un b e atable 4.1 Symmetric 2x2 games In this section, w e extend the “chic k en” example o f the intro duction to all symmetric 2x2 g a mes. Not e that the relative pay off game of an y symmetric 2 x2 ga me cannot b e a generalized ro c k–pap er–scissors matrix since latter mus t b y a symmetric zero–sum game. If o ne of the ro w pla y er’s off-diagonal relativ e pay offs is a > 0, then the ot her m ust b e − a violating the definition of generalized ro ck -pap er-scissors matrix. Th us Theorem 1 implies that for an y symme tric 2x2 game imitation is not sub j ect to a money pump. W e can strengthen the r esult to imitation b eing essen tially un b eat a ble. Prop osition 2 I n any symm etric 2x2 game, im itation is e s s ential ly unb e a table . Proo f. Let X = { x, x ′ } . Consider a p erio d t in whic h the maximizer achie v es a strictly p ositiv e relativ e pay o ff, ∆( x, x ′ ) > 0. (If no suc h p erio d t in whic h the maximizer ac hieve s a strictly p ositiv e relativ e pay off exists, then trivially imitation is essen tia lly un b eatable.) By definition, ∆( x, x ′ ) ≤ ˆ ∆. Since ∆( x, x ′ ) > 0, the imitat or imitates x in p erio d t + 1. F or there to b e another p erio d in whic h t he maximizer ac hieve s a strictly p ositiv e relativ e pa yoff, it m ust hold t ha t ∆( x ′ , x ) > 0, whic h b ecause o f symmetry yields a con tra diction as ∆( x ′ , x ) = − ∆( x, x ′ ). Th us there can b e at most one p erio d in whic h the maximizer achie v es a strictly p ositiv e relativ e pa yoff.  Note that “Matc hing p ennies” is not a coun ter-example since it is not symmetric. 4.2 Additiv ely Separable Relativ e P a y off F unctions Next, w e consider relative pay o ff functions that are additiv ely s eparable in the play ers’ actions. In this class of sym metric games, imitation is also es sen tially unbeatable. While additiv e separabilit y ma y app ear to b e restrictiv e, we will show b elow that there is a fairly large nu m b er o f imp ortant examples that fall in to this class. Definition 7 (Additiv e Separab le) W e say that a r elative p ayoff function ∆ is add i - tively sep ar able if ∆( x, y ) = f ( x ) + g ( y ) fo r some functions f , g : X − → R . Prop osition 3 L et ( X , π ) b e a symm e tric game with its r elative p ayoff game ( X , ∆) . If X is c omp ac t and the r e lative p ayoff function ∆ i s upp er semic ontinuous and additively sep ar able, then im itation is e s sential ly unb e atable. 10 Proo f. Since ∆ is separable, w e ha v e that for all x ′′ , x ′ , x ∈ X , ∆( x ′′ , x ) − ∆( x ′ , x ) = ∆( x ′′ , x ′ ) − ∆( x ′ , x ′ ) , whic h is equiv alent to ∆( x ′′ , x ) = ∆( x ′′ , x ′ ) + ∆( x ′ , x ) (5) b ecause ∆( x ′ , x ′ ) = 0 since the relativ e pay off game is a symme tric zero–sum game. By induction, equation (5) implies that one la rge step is just a s profitable as any n umber o f steps. Supp ose three steps w ere optima l f o r the maximizer. By equation (5 ) the maximizer is no w orse off b y merging t w o of the three steps t o one larg er step. Applying equation (5 ) a gain yields the claim. Th us, if a fESS exist, then the maximizer cannot do b etter than jumping directly t o a fESS x ∗ since for all x, y ∈ X , ∆( x ∗ , y ) = ∆( x ∗ , x ) + ∆( x, y ) ≥ ∆( x, y ) , where the equalit y follo ws from equation (5) and t he inequalit y from the definition of fESS. If the inequalit y is strict, then once the maximizer has c hosen x ∗ , the imitator will follo w and remain there for ev er. Otherwise, if the ineq ualit y holds w ith equalit y then the maximizer can not improv e further his relativ e pay off. Finally , in Duersc h, Oec hssler, and Sc hipp er (2010, Corollar y 6), w e show that if X is compact and ∆ upp er semicon tinuous and additiv ely separable, then a fESS of ( X , π ) indeed exists.  The f o llo wing corollar y follows dir ectly fro m Prop osition 3 and ma y b e useful in applications. Corollary 3 Co n sider a game ( X , π ) with a c omp act action set X and a p ayoff function that c an b e written as π ( x, y ) = f ( x ) + g ( y ) + a ( x, y ) for some c ontinuous functions f , g : X − → R and a symmetric function a : X × X − → R ( i .e., a ( x, y ) = a ( y , x ) for al l x, y ∈ X ). Then imitation is essential ly unb e atable. Prop erties suc h as increasing or decreasing differences are often useful for pro ving the existence of pure equilibria and con vergence of learning pro cesses. Definition 8 L et X b e a total ly or de r e d set. A (r elative) p ayoff function ∆ has de cr e as- ing (r esp. incr e asing) diff er enc es on X × X if for al l x ′′ , x ′ , y ′′ , y ′ ∈ X with x ′′ > x ′ and 11 y ′′ > y ′ , ∆( x ′′ , y ′′ ) − ∆( x ′ , y ′′ ) ≤ ( ≥ )∆( x ′′ , y ′ ) − ∆( x ′ , y ′ ) . (6) ∆ is a val uation if i t has b oth de cr e asing and incr e asing differ enc es. Our or ig inal inten t w as to study the consequences of ∆( x, y ) ha ving either increasing or decreasing difference s. Ho w ev er, in D uersc h, Oec hssler, and Sc hipp er (2010, Lemm a 1) w e show that for all symmetric t wo-pla y er zero-sum games, increasing differences is equiv alen t to decreasing differences. By T opkis (19 98, Theorem 2.6.4.), a function is additiv ely separable on a totally ordered set X if and only if it is a v aluation. Hence, w e ha v e the follow ing corollary to Prop osition 3. Corollary 4 L et ( X , π ) b e a finite symmetric gam e with its r elative p a yoff game ( X, ∆) . If X is a total ly or der e d set and ∆ has inc r e asing or de cr e asing differ enc es or is a valu- ation, then imitation is ess e n tial ly unb e atable. Brˆ anzei, Mallozzi, and Tijs (2003, Theorem 1) sho w that a zero-sum ga me is an exact p oten tial game if and only if it is additively separable. Thus , Prop osition 3 implies that imitation is es sen tially un b eatable in exact p oten t ia l games. The follo wing notio n is due to Monderer and Sha pley ( 1 996). Definition 9 (Exact pot en t ial games) The symmetric gam e ( X , π ) is an exact p o- tential game if ther e exists an exact p otential function P : X × X − → R such that for al l y ∈ X and al l x, x ′ ∈ X , π ( x, y ) − π ( x ′ , y ) = P ( x, y ) − P ( x ′ , y ) , π ( x, y ) − π ( x ′ , y ) = P ( y , x ) − P ( y , x ′ ) . Corollary 5 L et ( X , π ) b e a finite symmetric gam e with its r elative p a yoff game ( X, ∆) . If ( X, ∆) is an exact p otential game, then imitation is essential ly unb e atable . All of the follo wing example s follow from Prop osition 3 or Corollary 3. They demon- strate that the assumption of a dditiv ely separable relativ e pa yoffs is not as res trictiv e as ma y b e though t at first g lance. Example 2 (Cournot Duop oly with Linear Demand) Consider a Cournot duop oly given by the symmetric p ayoff function by π ( x, y ) = x ( b − x − y ) − c ( x ) with b > 0 . Sinc e π ( x, y ) c an b e written as π ( x, y ) = bx − bx 2 − c ( x ) − xy , Cor o l lary 3 applies, and imi tation is essential ly unb e atable. 12 The fo llo wing example with strategic complemen tarities shows that the result is not restricted to strategic substitutes. Example 3 (Bert rand Duop oly with Pro duct Differen tiation) Consider a differ- entiate d duop oly with c onstant mar ginal c osts, in whi c h firms 1 and 2 set pric es x and y , r esp e ctively. F i rm 1’s p r ofi t function is giv en by π ( x, y ) = ( x − c )( a + by − 1 2 x ) , for a > 0 , b ∈ [0 , 1 / 2) . Sinc e π ( x, y ) c an b e written as π ( x, y ) = ax − ac + 1 2 cx − 1 2 x 2 − bcy + bxy , Cor ol lary 3 applies, and imitation is es s e ntial ly unb e atable . Example 4 (Public Go o ds) Consi d er the class of symmetric public g o o d game s de- fine d by π ( x, y ) = g ( x, y ) − c ( x ) w her e g ( x, y ) is some symmetric monotone incr e asing b enefit function and c ( x ) is an incr e asing c ost function. Usual ly, i t is assume d t hat g is an incr e a sing function of the sum of pr o v isions, that is the sum x + y . V arious assump- tions on g have b e en studie d in the liter atur e such as incr e asing or de cr e as ing r eturns. In any c ase, Cor ol lary 3 applies, and im itation is es s ential ly unb e a table . Example 5 (Common P o ol Resources) Con sider a c ommon p o ol r eso ur c e ga me with two a p pr opriators. Each appr opriator has an endowment e > 0 that she c an invest in an outside activity with mar ginal p ayoff c > 0 or into the c o m mon p o ol r esour c e. x ∈ X ⊆ [0 , e ] denotes the maximizer’s investment into the c om m on p o ol r eso ur c e (like wise y denotes the imitator’s investment). The r eturn fr om investment into the c ommon p o ol r e- sour c e is x x + y ( a ( x + y ) − b ( x + y ) 2 ) , with a, b > 0 . So the symmetric p ayoff function is given by π ( x, y ) = c ( e − x ) + x x + y ( a ( x + y ) − b ( x + y ) 2 ) if x, y > 0 and ce otherwise (se e Walker, Gar dner, an d Ostr om, 1990). Sinc e ∆( x, y ) = ( c ( e − x ) + ax − bx 2 ) − ( c ( e − y ) + ay − by 2 ) , Pr op osition 3 implies that im itation is es sential ly unb e a table. Example 6 (Minim um Effort Co ordination) Consider the class of minimum effort games given by the s ymmetric p ayoff function π ( x, y ) = min { x, y } − c ( x ) for some c ost function c (se e B ryant, 1983 and V an Huyck, B attalio, and Be i l , 1 9 9 0). Cor ol lary 3 implies that imitation is es s e ntial ly unb e atable . Example 7 (Synergistic Relationship) Consider a syner gistic r elations h ip amon g two individuals. If b oth devote mor e effort t o the r elations hip, then they a r e b oth b etter off, but for any given effort of the opp onent, the r eturn of the player’s effort first incr e ases and then de c r e ases. The symm e tric p ayoff function is given by π ( x, y ) = x ( c + y − x ) with c > 0 and x, y ∈ X ⊂ R + with X c omp act (se e O sb orne, 2004, p. 39). Cor ol lary 3 implies that imitation is es s e ntial ly unb e atable . 13 Example 8 (Arms Race) Co nsider two c ountries engage d in an arms r ac e (se e e.g. Milgr om and R ob erts, 199 0, p. 1272). Each player cho oses a level of arms in a c omp act total ly or der e d set X . The symmetric p a yoff function is given by π ( x, y ) = h ( x − y ) − c ( x ) wher e h is a c onc av e function of the differ en c e b etwe en b o th pla yers’ level of arms, x − y , satisfying h ( x − y ) = − h ( y − x ) . By Pr op osition 3 imitation is esse ntial ly unb e atable . Example 9 (Diamond’s Searc h) C onsider two pla yers who exert effort se ar ch ing for a tr ading p a rtner. Any tr ader’s pr ob ability of fi n ding another p articular tr ader is pr o- p ortional to h i s own effort and the effort by the other. The p ayoff function is given by π ( x, y ) = α xy − c ( x ) for α > 0 and c in c r e asing (se e Milgr om and R ob erts, 1990, p. 1270). The r elative p ayoff gam e of this two-player game is additively sep a r a b le. By Pr op osition 3 imitation is es sential ly unb e a table. A natural question is whether additive separability of relative pay offs is also necessary for imitation to b e essen tially unbeatable. The follo wing counter-example sho ws that this is not the case. Example 10 (Co ordination gam e with ou tside option) Cons i d er the fol low ing c o- or dination game with an outside op tion ( C ) for b oth players of not p articip a ting (left matrix). π = A B C A B C   4 − 1 0 2 3 0 0 0 0   ∆ = A B C A B C   0 − 3 0 3 0 0 0 0 0   Note that the r elative p ayoff game ∆ (right matrix) do es not have c onstant differ enc es. E.g., ∆( A, B ) − ( B , B ) = − 3 6 = ∆( A, C ) − ∆( B , C ) = 0 . Thus, by T o pkis (1998 , The or em 2.6.4.), it is not additively sep ar abl e . Y et, imitation is essential ly unb e atable. If the imitator’s initial action is A , the maxi m izer c an e arn at m ost a r elative p ayoff differ ential of 3 after which the imitator adjusts and b oth e arn zer o fr om ther e on. F or other initial actions of the imi tator, the maxi mal p ayoff diffe r en c e is at most 0 . 5 Sufficien t Conditi ons for No Money Pump 5.1 Relativ e P a y off Games with P oten tials P otential functions are often useful for obtaining results on conv ergence o f learning al- gorithms to equilibrium, existence of pure equilibrium, and equilibrium selection. In 14 the previous section, w e ha v e sho wn that if the relativ e pa y off g ame is an exact po t en tial game, then imitation is essen tially un b eatable. It is na t ur a l to explore the implicatio ns of more general notions of p oten tials. Besides exact po ten tial games (see Definition 9), the follo wing notions of p oten tial games we re intro duced b y Monderer and Shapley (1996). Definition 10 (Poten tial games) The symmetric gam e ( X , π ) is (W) a weig h te d p o tential game if ther e exists a weighte d p otential func tion P : X × X − → R and a weight w ∈ R + such that for al l y ∈ X and al l x, x ′ ∈ X , π ( x, y ) − π ( x ′ , y ) = w ( P ( x, y ) − P ( x ′ , y )) , π ( x, y ) − π ( x ′ , y ) = w ( P ( y , x ) − P ( y , x ′ )) . (O) an or dinal p o tential game if ther e exists an or dinal p o tential function P : X × X − → R such that for al l y ∈ X and al l x, x ′ ∈ X , π ( x, y ) − π ( x ′ , y ) > 0 if and only if P ( x, y ) − P ( x ′ , y ) > 0 , π ( x, y ) − π ( x ′ , y ) > 0 if and only if P ( y , x ) − P ( y , x ′ ) > 0 . (G) a gener alize d or dinal p otential game if ther e exists a gene r a l i z e d or dinal p otential function P : X × X − → R such that for al l y ∈ X and al l x, x ′ ∈ X , π ( x, y ) − π ( x ′ , y ) > 0 implies P ( x, y ) − P ( x ′ , y ) > 0 , π ( x, y ) − π ( x ′ , y ) > 0 implies P ( y , x ) − P ( y , x ′ ) > 0 . Note that e v ery exact p oten tial game is a w eigh ted p ot ential g a me, ev ery w eigh ted p oten tial game is an ordinal p oten tial game, and ev ery ordinal p otential game is a gen- eralized ordinal p otential game. Monderer and Shapley ( 1996, Lemma 2.5 and the first paragraph o n p. 12 9 ) sho w that any finite strat egic game admitting a generalized o r - dinal p otential p ossesses a pure Nash equilibrium. Thus , if ( X, π ) is a finite symmetric game with relativ e play off ga me ( X, ∆) and t he latter is an exact, w eigh ted, ordinal or generalized ordinal p o ten tia l ga me, then ( X , π ) p ossesses a fESS. A sequen tial path in the action space X × X is a sequence ( x 0 , y 0 ) , ( x 1 , y 1 ) , ... of profiles ( x t , y t ) ∈ X × X suc h that for all t = 0 , 1 , ... , the a ctio n pro files ( x t , y t ) and ( x t +1 , y t +1 ) differ in ex actly one pla yer’s action. A sequen tial path is a strict impro v emen t path if for eac h t = 0 , 1 , ... , the play er who switc hes her action at t strictly impro ves her pay o ff. A finite sequen tial path ( x 0 , y 0 ) , ..., ( x m , y m ) is a strict impr oveme n t cycle if it is a strict impro veme n t path and ( x 0 , y 0 ) = ( x m , y m ). 15 Lemma 4 If ( X, ∆) d o es not c on tain a strict im pr ovement cycle , then it do es not c o n tain an imitation cycle. 5 Proo f. W e prov e the con t rap ositiv e. I.e., if ( X , ∆) con ta ins a n imitation cycle, then it contains a strict improv emen t cycle. Let ( x 0 , y 0 ) , ..., ( x m , y m ) b e an imitation cycle. F rom this imitation cycle, w e construct a strict impro v emen t cycle as follows: F or t = 0 , ..., m − 1, we add the elemen t ( x t , y t +1 ) a s succes sor to ( x t , y t ) a nd predecess or to ( x t +1 , y t +1 ). That is, instead of sim ultaneous a djustmen ts of actions at each round as in an imitation cycle, w e let pla yers adjust actions sequen tially b y taking turns. The imitator adjusts fr o m ( x t , y t ) to ( x t , y t +1 ) and the maximize r from ( x t , y t +1 ) to ( x t +1 , y t +1 ) for t = 0 , ..., m − 1. This construction yields a sequen tial path. W e now sho w that it is a strict improv emen t cycle. First, for the imitator, whenev er she adjusts in t = 0 , ..., m − 1, w e claim ∆( y t , x t ) < ∆( y t +1 , x t ) = 0. Note that by symmetric zero-sum, ∆( y t , x t ) = − ∆( x t , y t ) < 0 b ecause ( x t , y t ) is an elemen t of an imitation cycle, i.e., ∆( x t , y t ) > 0. ∆( y t +1 , x t ) = 0 b ecause the imitator mimics the action of the maximizer, y t +1 = x t . Thus ∆( y t +1 , x t ) = ∆( x t , x t ) = 0 by symmetric zero-sum. Second, for the maximizer, whenev er she adjusts in t = 1 , ..., m , ∆( x t , y t ) > ∆( x t − 1 , y t ) = 0 b ecause ( x t , y t ) is an elemen t of an imitation cycle, so ∆( x t , y t ) > 0 . Moreo v er, the imitator mimics the a ction of the maximizer, i.e., y t = x t − 1 , and th us ∆( x t − 1 , y t ) = ∆( x t − 1 , x t − 1 ) = 0. Hence ( x 0 , y 0 ) , ( x 0 , y 1 ) , ( x 1 , y 1 ) , ..., ( x m − 1 , y m ) , ( x m , y m ) is indeed a strict impro v emen t cycle.  The con v erse is not true as the follo wing coun ter-example sho ws. Example 11 Conside r the fol low i n g r elative p ayoff game. 6 ∆ = a b c a b c   0 0 − 1 0 0 1 1 − 1 0   5 Ania (2008, Prop os ition 3) presents a similar result according to which if all play e rs are imitators and imita tion is payoff improving, then fESS implies Nash equilibrium action. T his is different from Lemma 4 as we consider one maximizer and one imitator a nd fo cus on the re la tionship b et ween relative pay off games that p oss ess a gener alized o r dinal p otential and imitation cycles. 6 This example a ppea rs also in Ania (2008 , Exa mple 2 ), where it is used to demonstrate that the class of games where imitation is pa yoff improving (when a ll play ers are imitators) is not a subclass of generalized ordinal potential ga mes. 16 Cle arly, this game is not a gener alize d r o ck-p ap er-scissors game. Thus, b y L e mma 2 it do es no t p oss ess an imitation cycle. However, we c an c onstruct a strict impr ovemen t cycle ( b, a ) , ( c, a ) , ( c, c ) , ( b, c ) an d ( b , a ) . Prop osition 4 L et ( X , π ) b e a finite symmetric game with its r elative p ayoff game ( X , ∆) . If ( X , ∆) is a gener alize d or dinal p otential gam e , then imitation is no t subje ct to a money p ump . Proo f. Monderer and Shapley (1996, Lemm a 2.5) sho w that a finite strategic game has no strict impro vem en t cycle (what they call the finite impro v emen t prop erty) if and only if it is a generalized ordinal p otential game. Since this r esult holds fo r a n y finite strategic game, it holds also for an y finite symmetric zero- sum game ( X , ∆). Lemma 4 sho ws that if ( X, ∆) do es not con tain a strict improv emen t cycle, then it do es not con tain an imitation cycle. Th us Lemma 2 implies that imitation is not sub ject to a money pump.  If the conv erse we re true, then the class of generalized ordinal p oten t ia l relativ e pay off games and relative pay o ff g ames that are not generalized ro c k-pap er-scissors games w o uld coincide. Y et, the con ve rse is not t rue. This follo ws again fro m Example 11. It is not a generalized ro c k-pap er-scissors game but due to the existenc e of a strict impro ve men t cycle it do es not p ossess a generalized ordinal pot ential b y Monderer and Shapley (1996, Lemma 2.5). F or a n example of a game whose relative pay o ff game is a generalized or dinal p oten tial game see again the co ordination ga me with an outside opt io n presen ted in Example 10. A generalized ordinal p oten tial function is g iv en b y G = A B C A B C   − 2 − 1 − 2 − 1 0 0 − 2 0 0   It is straigh tfor w ard to c hec k that exact, w eigh ted, and or dina l po t ential games are generalized ordinal p o ten tia l ga mes. Corollary 6 L et ( X , π ) b e a finite symmetric gam e with its r elative p a yoff game ( X, ∆) . If ( X, ∆) is an exact p otential g a me, a weighte d p otential game, or an or din a l p otential game, then imitation i s n ot subje ct to a money pump. 17 5.2 Quasiconca v e Relativ e P a y off Games Here w e sh o w that imitation is essen tia lly un b eatable if the relativ e pa y off ga me is “qua- siconca v e”. The f ollo wing definition naturally extends the definition of quasi-conca v e pa y off function on con v ex real- v a lued spaces to t he case of finite, symmetric games. It is the notion of single-p eak edness. Definition 11 (Quasiconca v e) A finite s ymm etric game ( X, π ) with symmetric m × m p ayoff matrix π = ( π xy ) is quasi c onc ave (or sin gle-p e ake d) if for e ach y ∈ X , ther e exis ts a k y such that π 1 y ≤ π 2 y ≤ . . . ≤ π k y y ≥ π k y +1 y ≥ . . . ≥ π my . That is, w e say a symmetric game is quasiconcav e if eac h column has a single p eak. In our companion pa p er, Duersc h, Oec hssler, and Sc hipp er ( 2 010, Corollary 4), w e sho w that if X is finite and ∆ is quasiconca v e, then a fESS of ( X, π ) exists. It is clear from the definition of quasiconca vit y that if ∆ is quasiconcav e then there exists a total order on the a ctio n space. With some abuse of notation, we denote this order also by ≤ . If ∆ is quasiconca v e, we sa y that x is b etwe en some x ′ and x ′′ if x ′ ≤ x ≤ x ′′ or x ′′ ≤ x ≤ x ′ . Lemma 5 L et ( X , ∆) b e the r e lative p a yoff ga m e o f the symmetric game ( X, π ) . Supp ose ∆ is quasic onc ave. 1. If x is b etwe en some y and som e fES S x ∗ , then ∆( x, y ) = − ∆( y , x ) ≥ 0 . 2. If x ∗ and x ∗∗ ar e fESS, then so ar e al l x b etwe en x ∗ and x ∗∗ . Proo f. (1 ) Let x b e b et we en y and x ∗ . By definition of the fESS, ∆( x ∗ , y ) ≥ 0 . By symmetry of pay offs, ∆( y , y ) = 0 . P art (1) o f the lemma fo llo ws then b y quasiconca vity . (2) Let x ∗ and x ∗∗ b oth b e fESS. Th us, ∆( x ∗ , x ∗∗ ) ≥ 0 a nd ∆( x ∗∗ , x ∗ ) ≥ 0. Since ∆( x ∗ , x ∗∗ ) = − ∆( x ∗∗ , x ∗ ), w e mus t hav e ∆( x ∗ , x ∗∗ ) = 0. By quasiconcav it y , ∆( x ′ , x ∗∗ ) ≥ 0 a nd ∆( x ′ , x ∗ ) ≥ 0 for a ll x ′ b et w een x ∗ and x ∗∗ . By part (1) of the lemma, ∆( x ′ , x ) ≥ 0, for all x ∈ X . Hence, all x ′ b et w een x ∗ and x ∗∗ are fESS as well.  Prop osition 5 L et ( X , π ) b e a finite symmetric game with its r elative p ayoff game ( X , ∆) . If ∆ is quasic onc ave, then imitation is not subje ct to a money pump. 18 Proo f. W e w ill sho w that the imitator’s pla y m ust reac h the set of fESS in finitely man y steps, whic h implies that imitatio n is not sub ject to a money pump. W e ne ed the follow ing notation. Let E denote the (finite) set of fESS of ( X , π ). By Lemma 5 (2), if x ∗ and x ∗∗ are t wo fESS with x ∗ ≤ x ∗∗ , then for any x with x ∗ ≤ x ≤ x ∗∗ also x is a f ESS, where the total order ≤ is induced b y quasiconca vit y of ∆. D enote x ∗ := min E , x ∗∗ := max E the smallest and largest fESS, resp ectiv ely , where again the max and min are ta ken with respect t o the tot al o rder induced b y quasiconca vit y of ∆. W e denote b y < the strict part of ≤ , i.e., x < x ′ if and only if x ≤ x ′ and not x ′ ≤ x . F or an y v alue y ∈ X, y > x ∗∗ , w e define the f o llo wing lo wer b ound (whic h need no t alw ays exist), l ( y ) := max { x ∈ X : ∆( x, y ) < 0 , x < x ∗ } . If y is larg er than the largest fESS, then l ( y ) is the largest action low er than the low est fESS at whic h relativ e pay o ffs are strictly negativ e. Lik ewise, for y ∈ X , y < x ∗ , w e define the following upp er b ound (which need not alw a ys exist), u ( y ) := min { x ∈ X : ∆( x, y ) < 0 , x > x ∗∗ } . Without loss of generality , let y 0 > x ∗∗ b e the starting v alue o f the imitato r. (The case of y 0 < x ∗ follo ws analogously .). Let us consider all p ossible c hoices of the maximizer. By L emma 1, the maximizer will nev er choose an action x suc h that ∆( x, y 0 ) < 0. If the maximizer c ho o ses an y x suc h tha t ∆( x, y 0 ) = 0, then the maximizer will not b e imitated and the situation in t = 1 will b e iden tical to t = 0. Th us, from now on we can restrict atten t io n to x ∈ X suc h that ∆( x, y 0 ) > 0. W e claim that ∆( x, y 0 ) > 0 can o ccur only if x < y 0 and l ( y 0 ) < x , where the second requiremen t is empty should l ( y 0 ) not exist. T o see that w e can exclude x ≤ l ( y 0 ) note that ∆( l ( y 0 ) , y 0 ) < 0 b y definition. By quasiconca vit y , ∆( x, y 0 ) < 0 for all x ≤ l ( y 0 ). T o see that w e can exclude x ≥ y 0 note that ∆( y 0 , y 0 ) = 0. By quasiconca vit y , ∆( x, y 0 ) ≤ 0 , for all x > y 0 . This pro ves the claim. When the maximizer c ho oses an y x suc h that l ( y 0 ) < x < y 0 , the imitator imitates x and c ho oses y 1 = x in the next p erio d. Consider the case y 1 < x ∗ . W e claim that u ( y 1 ) ≤ y 0 . T o see this note that ∆( y 1 , y 0 ) > 0 ( o therwise the imitator would not hav e imitated) and hence ∆( y 0 , y 1 ) < 0. By quasiconca vit y and the definition of fESS w e hav e ∆( x ′ , y 1 ) < 0 for all x ′ ≥ y 0 . Hence, u ( y 1 ) ≤ y 0 . Next, consider the case that y 1 > x ∗∗ . In that case simply restart the pro cedure with the new starting v alue y 1 . F ina lly , consider the case where the maximizer selects an elemen t in E . Then no further relativ e pa y off impro veme n ts are p ossible. 19 Th us, in this first step w e ha v e strictly narrow ed do wn the range of the p ossible c hoice y 1 of the imitator in p erio d t = 1 to l ( y 0 ) < y 1 < y 0 . Since X is finite, when w e repeat this step, the imitator m ust reac h the set of fESS in a finite n um b er of steps. Once the imitator has reac hed a fESS, he has reac hed a statio nary state since then ∆( x, x ∗ ) ≤ 0 for all x . The imitator will nev er lea v e x ∗ and the maximizer w ill nev er again obtain a p ositiv e relativ e pa yoff. Since there are only finitely man y r ounds in whic h ∆( x t , y t ) > 0, imitation is not sub ject to a money pump.  The f ollo wing corollary ma y b e useful f o r applications. Let X ⊂ R m b e a finite subset of a finite dimensional Euclidean space. A function f : X − → R is c o nvex (r esp. c onc ave) if for any x, x ′ ∈ X and for an y λ ∈ [0 , 1] suc h tha t λx + (1 − λ ) x ′ ∈ X , f ( λx + (1 − λ ) x ′ ) ≤ ( ≥ ) λf ( x ) + (1 − λ ) f ( x ′ ). Corollary 7 L et ( R m , π ) b e a symm etric two-player game for whic h π ( · , · ) is c on c ave in its first ar gument and c onvex in its se c ond ar gumen t. I f the players’ actions ar e r estricte d to a finite subset X of the fin i te dimensional Euclidian sp ac e R m , then imitation is not subje ct to a money pump . Bargaining is an economically relev ant situation in volving t wo pla y ers. Our results imply that imitation is not sub ject to a money pump in bargaining as mo deled in the Nash Demand game. Example 12 (Nash Deman d Game) Consid er the fol lowing version of the Nash De- mand game (se e Nash, 1953). Two players s i m ultane ously dem and an amount in R + . If the sum is within a fe asible set, i.e . , x + y ≤ s for s > 0 , then player 1 r e c eives the p ayoff π ( x, y ) = x . Otherwise π ( x, y ) = 0 (analo gously for pla yer 2). The r elative p ay- off function is quasic onc av e . If the players’ demands ar e r estricte d to a finite set, then Pr op osition 5 implies that im itation is n o t subje c t to a money pump. Example 13 Conside r a symmetric two-player gam e with the p ayoff function given by π ( x, y ) = x y with x, y ∈ X ⊂ [1 , 2] with X b ein g fi n ite. T h is game’s r el a tive p a yoff f unc tion is quasic onc ave. Thus our r esult i m plies that imitation is not subje ct to a money pump. Mor e over, the example demons tr ates that not every quasic onc ave r elative p ayoff function is additively sep ar able. Finally , w e lik e to remark that Example 11 is an instance of a quasiconca v e relativ e pa y off game but due to the strict improv emen t cycle it do es not p osses a generalized 20 ordinal p oten tia l. Moreov er, in Duersc h, Oechs sler, and Sc hipp er (2010, Example 1) w e sho w that there are relativ e pa y o ff games that are neither generalized ro c k-pap er-scissors games nor quasiconca v e. 5.3 Aggregativ e Games Man y games relev a n t to economics p ossess a natural aggr ega te of all play ers actions. F or instance, in Cournot games the total mark et quan tit y or the price is an aggregate. But also o ther games lik e ren t-seeking games, common po ol resource ga mes, public go o d games etc. can be view ed as games with a n ag gregate. The a ggregation prop erty has b een useful f o r the study of imitation and fESS in the literat ur e (see Sc hipp er, 2003, a nd Al´ os-F errer and Ania, 2005). In this section, w e will derive results for aggregat iv e games whose absolute pay off functions satisfy some second-order prop erties. 7 W e say that ( X , Π) is an aggr e gative game if it satisfies the following prop erties. (i) X is a totally ordered set o f a ctions and Z is a tota lly ordered set. (ii) There exists an aggregator a : X × X − → Z that is – monot one increasing in its a r gumen ts, i.e. if ( x ′′ , y ′′ ) > ( x ′ , y ′ ), then a ( x ′′ , y ′′ ) > a ( x ′ , y ′ ), and – symmetric, i.e., a ( x, y ) = a ( y , x ) for all x, y ∈ X . (iii) π is extendable to Π : X × Z − → R with Π( x, a ( x, y )) = π ( x, y ) for all x, y ∈ X . W e say that an aggregative game ( X , Π) is quasisubmo dular (r esp. quasisup ermo dular) if Π is quasisubmo dular (r e s p . quasisup ermo d ular) in ( x, y ) on X × Z , i.e., for all z ′′ > z ′ , x ′′ > x ′ , Π( x ′′ , z ′′ ) − Π( x ′ , z ′′ ) ≥ 0 ⇒ ( ⇐ ) Π( x ′′ , z ′ ) − Π( x ′ , z ′ ) ≥ 0 , (7) Π( x ′′ , z ′′ ) − Π( x ′ , z ′′ ) > 0 ⇒ ( ⇐ ) Π( x ′′ , z ′ ) − Π( x ′ , z ′ ) > 0 . (8) 7 A t a fir st glance, the aggr egation prop er t y may b e les s comp e lling in the co nt ext of tw o -play er games. How ever, the results we obtain in this section allows us to cov er imp orta n t ex amples that ar e not cov ered by any of our other r esults. 21 Quasisup ermo dularit y ( resp. quasisubmo dularit y) is sometimes also called the (dual) single crossing prop ert y (e.g. Milgrom and Shannon, 199 4 ). 8 W e sa y that an aggregativ e game ( X , Π) is submo dular (r esp . sup ermo dular) if Π has decreasing (resp. increasing) differences in ( x, z ) on X × Z . I.e., for all z ′′ > z ′ , x ′′ > x ′ , Π( x ′′ , z ′′ ) − Π( x ′ , z ′′ ) ≤ ( ≥ )Π( x ′′ , z ′ ) − Π( x ′ , z ′ ) . (9) It is straigh t-forward to c hec k that if an aggregativ e game ( X, Π) is submo dular (resp. sup ermo dular), then it is quasisubmo dular (resp. quasisup ermo dular). The conv erse is false. A finite aggregativ e game is quasic onc av e (or single-p eaked) if for an y x, x ′ , x ′′ ∈ X with x < x ′ < x ′′ and z ∈ Z , Π( x ′ , z ) ≥ min { Π( x, z ) , Π( x ′′ , z ) } . A finite aggregative game is quasic on v ex if for any x, x ′ , x ′′ ∈ X with x < x ′ < x ′′ and z ∈ Z , Π( x ′ , z ) ≤ max { Π( x, z ) , Π( x ′′ , z ) } . It is strictly quasic onvex if the inequality holds strictly . An action x ∗ ∈ X is a fESS of the aggregative game ( X, Π) if Π( x ∗ , a ( x ∗ , x )) ≥ Π( x, a ( x ∗ , x )) for all x ∈ X. The follo wing lemma is the k ey insight f o r our r esult on quasiconca ve quasisubmo dular aggregative g ames. Lemma 6 Supp ose ( X , Π) is a quasic on c ave quasisubmo dular aggr e gative game. If x is b etwe en some x ′ and a fESS x ∗ , then Π( x, a ( x, x ′ )) ≥ Π( x ′ , a ( x, x ′ )) . 8 It is impo rtant to rea liz e that q ua sisubmo dularity in ( x, z ) where z is the aggr egate of al l players’ actions is differen t from quasisubmo dula r ity in ( x, y ) where y is the aggr egate of al l op p onents’ actions . F or insta nce, Schipper (200 9, Lemma 1) shows that quasisubmo dula r ity in ( x, z ) where z is the agg regate of all players’ actions is sa tisfied in a Cournot oligop oly if the inv e r se demand function is decr easing. No assumptions on co sts are re quired. It is known fro m Amir (199 6, Theorem 2.1 ) that further assumptions on costs are require d if the Cournot oligop oly s ho uld b e quasisubmo dular in ( x, y ) where y is the aggreg ate of all o ppo nent s’ actions. 22 Proo f. Suppo se that x ′ ≤ x ≤ x ∗ . The case x ′ ≥ x ≥ x ∗ can b e dealt with a na lo gously . By the definition of a fESS Π( x ∗ , a ( x ∗ , x ′ )) − Π( x ′ , a ( x ∗ , x ′ )) ≥ 0 . By quasiconca vity , Π( x, a ( x ∗ , x ′ )) − Π( x ′ , a ( x ∗ , x ′ )) ≥ 0 . The result follows then b y quasisub mo dularity , Π( x, a ( x, x ′ )) − Π( x ′ , a ( x, x ′ )) ≥ 0 , since a ( x ∗ , x ′ ) ≤ a ( x, x ′ ).  Prop osition 6 I f ( X , Π) is a fini te quasic onc a v e quasi s ubm o dular aggr e gative game for which a fESS e x ists, then im itation is no t subje c t to a mone y pump. Proo f. W e will sho w that from an y initial action, an y relativ e pay o ff improv ing sequence of actions reac hes a fESS in a finite n umber of steps. Once reac hed, there a re no further impro v ement p ossibilities for the maximizer b y definition of the fESS. Note that since the game is quasiconca v e, if x ∗ and x ∗∗ are fESS, then so is a ny x ∈ X with x ∗ < x < x ∗∗ or x ∗∗ < x < x ∗ . W e write E for the set of fESS. Step 1 : Let y 0 ∈ X b e the starting action of the imitator. Assume that y 0 < x ∗ = min E (the pro of f or y 0 > x ∗∗ = max E works analogously). W e claim that when the imitat o r switc hes to a new action y 1 6 = y 0 , w e m ust ha ve that y 1 > y 0 . Supp o se by contradiction that y 1 < y 0 . By Lemma 1, the imitators w ould only ch o ose y 1 if in the previous p erio d the maximizer c hose x = y 1 and receiv ed a strictly higher pa yoff than the imitator, ∆( y 1 , y 0 ) = Π ( y 1 , a ( y 1 , y 0 )) − Π ( y 0 , a ( y 1 , y 0 )) > 0 . (10) But this contradicts Lemma 6 as y 1 < y 0 < x ∗ . Th us, y 1 > y 0 . • If y 1 ∈ E , w e are done. • If y 0 < y 1 < x ∗ , then take y 1 as the new start ing a ction and rep eat Step 1. • Else, go to Step 2 . 23 Step 2 : W e hav e that y 1 > x ∗∗ . W e claim that when the imitators switc hes to a new action y 2 6 = y 1 , we m ust ha v e tha t y 2 < y 1 . Supp ose b y contradiction that y 2 > y 1 . By Lemma 1, the imitators would only c ho o se y 2 if in the previous perio d the maxim izer c hose x = y 2 and receiv ed a higher pa y off, ∆( y 2 , y 1 ) > 0. But this contradicts Lemma 6 as y 2 > y 1 > x ∗∗ . Th us y 2 < y 1 . • If y 2 ∈ E , w e are done. • If y 0 < y 2 < x ∗ , then take y 2 as the new start ing a ction and rep eat Step 1. • If x ∗∗ < y 2 < y 1 , then take y 2 as the new starting a ction and rep eat Step 2. W e c laim that y 2 ≤ y 0 can b e ruled out. Since X is finite, the algo rithm then stops after finite p erio ds. Thus the pro of of the prop osition is complete o nce we verify this last claim. Supp ose to the contrary that y 2 ≤ y 0 . By Lemma 1, the imita tors w ould only c ho ose y 2 if in the previous p erio d the maximizer chose x = y 2 and receiv ed a strictly higher pa y off than the imitator, ∆( y 2 , y 1 ) = Π ( y 2 , a ( y 2 , y 1 )) − Π ( y 1 , a ( y 2 , y 1 )) > 0 . By quasiconca vity , w e ha ve Π ( y 0 , a ( y 2 , y 1 )) − Π ( y 1 , a ( y 2 , y 1 )) ≥ 0 . Since a ( y 0 , y 1 ) ≥ a ( y 2 , y 1 ), w e ha v e b y quasisubmo dularity Π ( y 0 , a ( y 0 , y 1 )) − Π ( y 1 , a ( y 0 , y 1 )) ≥ 0 . But this contradicts inequalit y (10) and prov es the claim.  The f ollo wing examples presen t applications of the previous result. The first exam- ple extends the linear Cournot olig o p oly of Example 2 to general symmetric Cournot oligop oly . Example 14 (Cournot Duop oly) L et the s ymm etric p ayoff function b e π ( x, y ) = xp ( x + y ) − c ( x ) and assume that π ( x, y ) is quasic o nc ave in x . Schipp er (2009, L em m a 1) shows that a symmetric Cournot duop oly with an arbitr ary de cr e asing inverse dem and function p and arbitr ary incr e asing c ost function c is an aggr e gative quasisubmo d ular game. Thus, Pr op osition 6 implies that imitation is not subje c t to a money pump in Cournot duop oly. 24 Example 15 (Rent Seeking) Two c ontestants c omp ete for a r ent v > 0 by bidding x, y ∈ X ⊆ R + . A player’s pr ob abi l i ty of wi n ning is pr op ortional to h e r bid, x x + y and z er o if b oth players bid zer o. T he c o st of bidding e quals the bid. The symmetric p ayoff func tion is given b y π ( x, y ) = x x + y v − x (se e T ul l o ck, 1980, and Hehenkam p, L eininger, and Possajennikov, 2004). This game is an ag gr e gative quasisubmo dular game (se e Schipp er, 2003, Example 6, and A l´ os-F err er and A nia , 2 0 05, Example 2) and π ( x, y ) is c onc ave in x . Thus Pr op osition 6 implies that imitation i s n ot subje ct to a money pump. F or quasicon v ex quasisupermo dular aggregative games w e can prov e an analogo us result. W e first observ e that in a strictly quasicon ve x quasisubmo dular game a fESS m ust b e a “corner” solution if it exists. It follow s that there can b e at most tw o fESS. Lemma 7 L et ( X , Π) b e a finite strictly quasic on v ex quasisup ermo dular aggr e gative gam e. If x ∗ is a f ES S, then x ∗ = max X or x ∗ = min X . Proo f. Let x ∗ b e a fESS and supp o se to the con trary that there exis t x ′ , x ′′ ∈ X suc h that x ′ < x ∗ < x ′′ . W e distinguish four cases: Case 1: If Π( x ′′ , a ( x ∗ , x ′′ )) ≥ Π( x ′ , a ( x ∗ , x ′′ )) , then b y strict quasicon v exity Π( x ∗ , a ( x ∗ , x ′′ )) < Π( x ′′ , a ( x ∗ , x ′′ )) , a con tra diction to x ∗ b eing a fESS. Case 2: The case Π( x ′ , a ( x ∗ , x ′ )) ≥ Π( x ′′ , a ( x ∗ , x ′ )) is analogous to Case 1. Case 3: If Π( x ′ , a ( x ∗ , x ′′ )) ≥ Π( x ′′ , a ( x ∗ , x ′′ )) , then b y strict quasicon v exity Π( x ∗ , a ( x ∗ , x ′′ )) < Π( x ′ , a ( x ∗ , x ′′ )) . By quasisupermo dularit y , Π( x ∗ , a ( x ∗ , x ′ )) < Π( x ′ , a ( x ∗ , x ′ )) . a con tra diction to x ∗ b eing a fESS. Case 4: The case Π( x ′′ , a ( x ∗ , x ′ )) ≥ Π( x ′ , a ( x ∗ , x ′ )) is analogous to Case 3. Th us, if x ∗ is a fESS, then x ∗ = max X or x ∗ = min X .  25 Prop osition 7 I f ( X , Π) is a finite strictly q uas i c o nvex quasisup erm o dular aggr e gative game for which a fESS ex i s ts, then imitation is not subje ct to a money pump. Proo f. W e s ho w that a pro cess of steps that strictly increase the s um of the maxi- mizer’s relativ e pa y o ffs mus t lead t o a fESS. Note that only nontrivial steps, in whic h the maximizer do es not repeat her action, can impro v e the sum of her r elat ive pay offs. Consider a seq uence of non t r ivial steps x 1 , x 2 , x 3 the maximizer ma y take. Supp ose that x 2 < x 1 (the case x 2 > x 1 is dealt with analogously). By Lemma 1 it m ust hold that Π( x 2 , a ( x 2 , x 1 )) ≥ Π( x 1 , a ( x 2 , x 1 )) . (11) T o show that the pro cess mov es to one of the corners, we need to sho w that either x 3 > x 1 or x 3 < x 2 . Supp ose to the contrary tha t x 2 < x 3 ≤ x 1 . 9 By Lemma 1 it m ust hold that Π( x 3 , a ( x 3 , x 2 )) ≥ Π( x 2 , a ( x 3 , x 2 )) . Th us, b y quasisup ermo dularit y Π( x 3 , a ( x 1 , x 2 )) ≥ Π( x 2 , a ( x 1 , x 2 )) . (12) F rom inequalit y (11) follows b y strict quasicon v exity that Π( x 2 , a ( x 2 , x 1 )) ≥ Π( x 3 , a ( x 2 , x 1 )) , (13) with eq ualit y only for x 3 = x 1 . Th us for x 3 < x 1 , inqualit y (13) yields a con tradiction to inequalit y (12). F or x 3 = x 1 the sequence of steps has not impro v ed the maximizer’s sum of relative pa y o ffs since b oth pla y ers obtained the same pa yoff throughout. Th us w e ha ve sho wn that with ev ery non trivial step, the maximizer gets closer to a corner. Since there are only finitely man y actions, if the sequence of actions is non- constan t, then a corner m ust b e reac hed in finitely man y steps. If the corner is a fESS, then no further c hanges of actions o ccur. Ot herwise, the other corner ma y be reac hed in one additional step. This m ust b e a fESS b y Lemma 7 since a fESS is assumed to exist. Once it is reache d, no furt her c ha ng es of actions o ccur.  6 Summary and Discuss i on In T able 1 w e summarize our results. 10 The only class of symmetric games in whic h imitation can really b e beaten is the c lass of games whose relative pay off function is a 9 The case of x 2 6 = x 3 is already excluded by the re q uirement of non-triv ia l steps. 10 More results on the clas ses of ga mes and their relationships are con tained in o ur companion pap er, Duersch, Oechssler, and Sc hipp er (2010). 26 generalized ro ck–paper–scissors game. While this is a g eneric class of symmetric games, so is its complemen t. More imp ortantly , man y economically relev ant games are con tained in this complemen t . Th us it is f a ir to say that imitation se ems very hard to b eat in large classes of economically relev an t and g eneric games. T able 1 : Summary of Results Class Result Reference Examples Symmetric 2x2 g ames essentially un b eatable Prop. 2 Chick en, Prisoner s’ Dilemma, Stag Hun t Additiv ely separable relative essentially un b eatable Prop. 3 Linear Co urnot duopoly pay off function Heterogeneous Bertrand duo po ly or Public goo ds Relative pay o ff functions essentially un b eatable Cor. 4 Common po ol resour c e s with increasing o r decreasing Minim um effort co ordination differences Synergistic relationship or Arms race Relative pay o ff games with essentially un b eatable Cor. 5 Diamond’s search exact potential Relative pay o ff games with no money pump Prop. 4 Example 10 generalized ordinal potential Quasiconcave relative no money pump Prop. 5 Nash dema nd game pay off games Example 11 Example 13 Quasiconcave quasisub- no money pump Prop. 6 Cournot g ames mo dular aggrega tive g ames Rent seeking Quasiconvex quasisup er- no money pump Prop. 7 mo dular aggrega tive g ames No g eneralized no money pump Thm. 1 all of the above Ro ck-P ap er-Scisso rs games Ho wev er, one ne eds to b e aw are of the limitations of our analysis, primar ily the re- striction to tw o–play er games. While a full treatmen t of the n –play er case is b ey ond 27 the scop e of the current pap er, w e provide here an example that show s ho w imita- tion can b e b eaten in a standard Cournot game when there are t hr ee pla y ers. Let the in vers e demand f unction b e p ( Q ) = 100 − Q and the cost function b e c ( q i ) = 10 q i . No w consider the case of t w o maximizers and one imitator. W rit ing a v ector of quan- tities as ( q I , q M , q M ), it is easy to c hec k that the following sequenc e of action pro files (0 , 22 . 5 , 22 . 5) , (22 . 5 , 0 , 68) , (0 , 22 . 5 , 22 . 5) , (22 . 5 , 68 , 0 ) , (0 , 22 . 5 , 22 . 5) ... is an imitation cy- cle. The t w o maximizers tak e turns in inducing the imitato r to reduce his quan t it y to zero by increasing quan tity so m uc h that price is below marginal cost. Since the other maximizer has zero losses, she is imitated in the next p erio d, whic h yields half o f the monop oly profit for bo t h maximizers. Clearly , this requires co ordination among the t w o maximizers but this can b e ac hiev ed in an infinitely rep eated game by the use of a trigger strategy . Th us, imitatio n is sub ject to a money pump. R ecall, how ev er, that w e pitted imitation aga inst truly sophisticated oppo nen ts. Whether imitation can b e b eaten also b y less sophisticated (e.g. h uman) opp onen ts remains to b e seen in future exp erimen ts. F urthermore, o ur analysis w a s based on the assumption that an imitator stic ks to his action in case o f a tie in pa yoffs. T o see wh at go es wrong with an alternativ e tie-braking rule consider a homogenous Bertrand duop oly with constan t marginal costs. Supp ose the imitator star t s with a price equal to marginal cost. If the maximizer chooses a price strictly ab ov e marginal cost, her profit is also zero. If nev ertheless, the maximizer w ere imitated, she could start the money pump b y undercutting the imitator until they reac h aga in price equal to marginal c ost and then start the cycle a g ain. This example sho ws that o ur results dep end crucially on the details of the imitation heuristics. It w ould b e in teresting to exactly characterize the class of simple de cision heuristics that are essen tially un b eatable in large classes of economically relev an t games. W e leav e this for future w ork. References [1] Al´ os-F errer, C. and A.B. Ania (2005). The ev olutionary stabilit y of p erfectly com- p etitiv e b eha vior, Ec onomic The ory 26 , 497-516 . [2] Amir, R. (1996). Cournot olig o p oly and the theory of sup ermo dular games, Games and Ec on omic Beh avior 15 , 132 -148. [3] Ania, A. (2008). Ev olutionary stability and Nash equilibrium in finite p opulations, with an application to price comp etitio n, Journal of Ec onomic B e havior and O r g a - 28 nization 65 , 472-48 8. [4] Ap esteguia, J., Huck , S., and J. Oec hssler (200 7). Imitation - Theory and experi- men tal evidence , Journal of Ec on o m ic T he ory 136 , 217-235 . [5] Ap esteguia, J., Huc k, S., Oec hssler, J., and W eidenholzer, S. (2 010). Imitatio n and the ev olutio n of W alrasian b eha vior: Theoretically f ragile but behaviorally robust, Journal o f Ec o nomic The ory , forthcoming. [6] Brˆ anzei, R., Mallozzi, L., and S. Tijs (2003 ). Sup ermo dular ga mes and p o ten tial games, Journal of Mathematic a l Ec onomics 39 , 3 9 -49. [7] Bry an t, J. (1983). A simple r ational exp ectations Keynes-t yp e co ordinatio n mo del, Quarterly Journal of Ec ono mics 98 , 525 - 528. [8] Duersc h, P ., Kolb, A., Oec hssler, J., and B.C. Sc hipp er (2 0 10). Rage aga inst the mac hines: How sub j ects play aga inst learning algorithms, Ec o n omic The ory , fo r th- coming. [9] Duersc h, P ., Oec hssler, J., and B.C. Sc hipp er ( 2 010). Pure saddle p o ints and sym- metric relat ive pa y o ff games, mimeo, Univ ersit y o f Heidelberg and the Univ ersit y of California, Dav is. [10] Gigerenzer, G. and R . Selten (20 0 2). Bounde d r ational i ty. T he a daptive to olb ox , Cam bridge, MA: The MIT Press. [11] Hehenk a mp, B., Leininger, W., and A. P ossa jennik o v (2 004). Ev o lutio nary equilib- rium in T ullo c k con tests: Spite and o verdis sipation, Eur op e an Journal of Politic al Ec onomy 20 , 1045-1057 . [12] Huc k, S., Normann, H.-T., a nd J. Oec hssler (1999). Learning in Cournot olig op oly - An experiment, Ec onomic Journal 109 , C80-C95. [13] Leizarow itz, A. (1 996). Ov ertaking and almost-sure optimalit y for infinite horizon Mark ov decision pro cesses, Mathematics of Op er ations R ese ar ch 21 , 15 8-181. [14] Milgrom, P . and J. Rob erts (1990 ). Rationalizabilit y , learning, and equilibrium in games with strategic complemen ta rities, Econometrica 58 , 125 5 -1277. [15] Milgrom, P . and C. Shannon (1994). Monotone comparativ e statics , Econometrica 62 , 157-180. 29 [16] Monderer, D. and L.S. Shapley (199 6). P oten tial games, Games and Ec onom ic Be- havior 14 , 12 4-143. [17] Nash, J. (1953 ). Tw o–p erson co op erat iv e ga mes, Ec onometric a 21 , 128 -140. [18] Offerman, T., P otters, J., and J. Sonnemans (2002). Im itation and b elief learning in an oligop o ly experimen t, R eview of Ec onomic S tudies 69 , 9 73-997. [19] Osb orne, M. (2004). A n intr o duction to game the ory , Oxford Univ ersit y Press. [20] Osb orne, M. and A. Rubinstein (1994 ). A c ourse in game the ory , MIT Press. [21] Sc haffer, M.E. (1989 ). Are profit-maximizers the b est surviv ors?, Journal o f Ec o- nomic B e havior and O r g a nization 12 , 29- 45. [22] Sc haffer, M.E. (1988). Ev olutionary stable strategies for a finite p opulation and a v a r ia ble con test size, Journal of The or etic al B iolo gy 132 , 469-478. [23] Sc hipp er, B.C. (2009 ). Imitators and o ptimizers in Cournot oligop oly , Journal of Ec onomic D ynamics and C ontr ol 33 , 1981 -1990. [24] Sc hipp er, B.C. (2003). Submo dularity and the ev olution of W alrasian b eha vior, In- ternational Journal of Game Th e ory 32 , 471 - 477. [25] T opkis, D. M. (1 998). S up e rmo dularity and c om p lementarity , Princeton, New Jersey: Princeton Univ ersit y Press. [26] T ullo c k, G. (198 0). Efficien t ren t seeking, in: Buchanan, T ollison, T ullo c k (eds.), T owar ds a t he ory of the r ent se ekin g so ciety , T exas A & M Univ ersit y Press, 3-15. [27] V an Huyc k, J., Battalio, R., and R. Beil (19 90). T acit co ordinatio n games, strategic uncertain ty and co ordinat io n failure, Americ an Ec onomic R evi e w 80 , 234-248. [28] V ega-Redondo, F. (1997). The ev olutio n of W alrasian b eha vior, Ec onometric a 65 , 375-384 . [29] W alk er, J.M., G ardner, R., and E. Ostrom ( 1 990). R ent dissipation in a limited- access Common-P o ol resource: Experimen tal evidence, Journal of En vironmen tal Economics and Managemen t 19 , 203-211. [30] W eibull, J. (1995 ). Ev olutionary game theory , MIT Press. 30