On the Effects of Idiotypic Interactions for Recommendation Communities in Artificial Immune Systems

On the Effe cts of Idiotypic Intera ctions f or Recomme ndation Communities i n Artificial I mmune Syste ms Proceedings of the 1st Internat Conference on ARtificial Immune Sy stems (ICARIS-2002 ), pp 154-1 60, Canterbury, UK, 2002. Steve Cayz er Hewlett-Packard Laborator ies Filton Road Bristol BS12 6QZ Steve_Cayzer @hp.com Uwe Aickelin School of Computer Science University of Notti ngham NG8 1BB UK uxa@cs.nott.ac .uk Abstract It has previously been shown that a recommender based o n immune system idio typic principles can outperfor m one based on correlation alone. This paper r eports the results of work in p rogress, wher e we undert ake so me investigatio ns i nto the n ature of thi s beneficial effect. T he initial finding s are t hat the i mmune system recomme nder tends to p roduce different neighbourhoo ds, and that the superior performance of this reco mmender is due par tly to the di fferent neighbourhoods, and par tly t o the way t hat the idiotypic ef fect i s used to w eight each neighbour’ s recommenda tions. 1 INTRODUCTION The idiotypic e ffect bui lds o n t he pre mise that antib odies can ma tch ot her antibo dies a s well as antige ns. It was first proposed by Jerne [6] a nd formalised i nto a m odel by Farmer et al [3]. T he theory is currently debated by immunologists, with no clear consens us yet o n its effects in the humoral i mmune system [5] . In a previous pape r [1], we have shown that the incorpora tion of idiotypic effects can be beneficial for Artificial Immune System based r ecommender systems. However, in that paper we did not explor e the mechanis ms of t hat be neficial effect. Such a n e xploration would seem worth while, particularly if this results in identifying the u nderlying ca uses o f t he i mprovements o f the ‘character istics’ o f a co mmunity (ei ther b y cha nging its membership, or b y evaluating the relative merit of each member). Such a n effect will be generall y useful in a range of applicatio ns, o f w hich reco mmender systems provide just o ne example. In ad dition, a deeper understanding of the idiot ypic effect m ay p rove useful to the designers o f other Arti ficial Im mune System applications. In t his paper, we prese nt the results o f work undertaken to better und erstand the idiot ypic effect. I n order to se t the context, the ne xt section provides a d efinition of the idiotypic effect and the fo llowing one a brief review o f Artificial Immune System based recomme nders. W e then present and di scuss the results of our anal ysis to date. 2 IDIOTYPIC EFFECTS The idiotypic networ k hypot hesis was first prop osed b y Jerne [6]. It builds on the rec ognition that antibodies can match ot her antibod ies a s well as antigens. Hence, an antibody may be matched b y other antibod ies, which in turn may be matched by yet other ant ibodi es. This activation can continue to spread t hrough the po pulation. The idiotypic network has been formalised b y a number of theoretical immunologists in [7]. This theory could help explain how t he memory o f past infections i s maintained. Furthermore, it co uld result in the suppression of si milar antibod ies t hus encouragi ng diversit y in the antibody poo l. The follo wing is a for mal eq uation for the idiot ypic effect adapted from Equation 3 fro m Farmer [3]: ) 1 ( 2 1 1 1 1 i n j j i ji N j N j j i ij j i ji i x k y x m x x m k x x m c rate death recognised antigens recognised am I recognised antibodies c dt dx −       + − =         −               +         −         = ∑ ∑ ∑ = = − Where: N is the number of a ntibodies n is the number o f antigens. x i (or x i ) is the concentration o f antibody i ( or j ) y i is the concentration o f antigen j c is a rate co nstant k 1 is a suppressive ef fect and k 2 is the death rate m ji is the matchi ng function betwee n antibody i and antibody (or antigen) j As can b e seen from the ab ove eq uation, the nature of a n idiotypic interaction can be either positive or negative. Moreover, if the matching functio n is symmetric, then the balance between “I am recognised” and “Antibodies recognised” (para meters c an d k 1 in the equation) wholly determines whether the idio typic ef fect is positi ve or negative, and we can simplify the equatio n. W e can simplify the eq uation stil l further if w e only al low one antigen i n the Artificial Immune S ystem. The simplified equation looks l ike this: ) 2 ( 3 1 2 1 i j i n j ij i i i x k x x m n k y x m k dt dx − − = ∑ = Where: k 1 is stimulation, k 2 suppression and k 3 death rate m i is the corre lation betwee n antibody i and the ( sole) antigen x i (or x i ) is the concentration o f antibody i ( or j ) y is the concentr ation of the (so le) antigen m ij is the correlation b etween antibod ies i and j n is the number o f antibodies. 3 RECOMMENDER SYST EM At t his poi nt, it is worth revie wing how this model can be applied to recommender s ystems. Ful l details can b e found in [1], but a brief over view follo ws. Recommende r systems are those that use collaborative filtering techniques to pr oduce predictions and recommendations [4 ]. So for example a m ovie recommender s ystem w ould, give n a film, provide a predictio n for that film (i.e. a n es timated rat ing for you). It m ight also provide a list o f r ecommended films ( i.e. films whic h it esti mates that you would pre fer over others). It d oes this b y comparing use rs together (b ased on their votes for mo vies), and prepari ng so me ‘neighbourhood ’ of like-minded users from which it can produce p redictions and recom mendations. The main lo op o f the r ecommender algorithm i s sho wn in Figure 1 and is the core o f our Artificial Im mune S ystem. The aim o f this algori thm is t o increa se the conce ntrations of those a ntibodies (datab ase users) t hat ar e similar to the antigen ( target user) and yet dif ferent from eac h o ther. The p rocess is thus s ubject to the suppressio n of si milar antibodies following J erne’s idiotypic ideas mentioned above. T hus, over tim e the Art ificial Im mune Syste m contains hig h concentra tions of a d iverse set of users who have similar fil m preferences t o the target user. The algorith m is ter minated e ither whe n there are no more users to try, or when the Artificial Immune System is stabilis ed , i.e. it is full, and has not c hanged in consistency f or m ore t han ten iteratio ns. The concentrations and co rrelations of the users i n the final neighbourhoo d, i.e. final immune system iteratio n, are then used t o calculate a weighted sum o f the ratin gs of movies. Initialise Artificial I mmune System Encode user for whom to make predictions as antigen Ag WHILE (Artificial Immune System not stabilised) & (More da ta available) DO Add next user as an a ntibody Ab Calculate matching sc ore between Ab and Ag Calculate matching scores between Ab and other antibodies WHILE (Artificial Immune System at full size) & (Artificia l Immune Syst em not stable) DO Iterate Ar tificial Immu ne System OD OD Figure 1 : Main loop of the Artifi cial Imm une Sy stem’s a l gorithm for recomm endation . Our previo us work [1] co mpared t w o pred ictors, one based o n a Simple Pear son test and one on o ur Artificial Immune System. In each case, a test user is take n from a database, and then pr edictions and r ecommendatio ns are made f or that user. Both pre dictors work by finding a neighbourhoo d and using that neighb ourhood to produce predictions and recommendati ons. Predict ion quality is as sessed by mea suring the mea n absolute erro r (de tails in [ 1]). Reco mmendation quality i s assessed by co mparing the ranked reco mmendations with the user’s ranked ratings fo r the recommended films. Kendall’s Tau can now be ap plied . T his measure reflects the level o f concordance in the lists, an d pr oceeds b y counting the number o f disco rdant pair s. To do this we order the films b y actual v ot e and appl y t he following formulae to the reco mmended films: ( ) ( ) ( )    > = = − − = ∑ ∑ = + = otherwise r r if r r D r r D N n n N j i j i n i n i j j i D D 0 1 , , 1 4 1 1 1 τ (3) Where: n is the overlap size r i is the ac tual rank of film i as recommended by the neighbourhoo d. Note that i here re fers to the reco mmended rank of the film, not the film ID. N D is the number of discord ant pairs, or, equivalentl y, the expected cost of a bub ble sort to reconcile the two lists. D is se t to one if the r ankings are discordant . 2a) E ffe ct o f S tim u latio n o n N e igh b ou r ho o d siz e 0 10 20 30 40 50 60 70 80 90 100 0 0. 2 0.4 0.6 0.8 1 Sti m u l a ti on Ra te Ne i g h b o u r h o o d S i z e (2b) E ffec t o f s timu la tion o n n u m b er of us er s lo o k e d at 0 5000 10000 15000 0 0. 2 0. 4 0. 6 0.8 1 S tim ul a tio n Ra te N u m be r o f u s e r s l o o k e d a t Figure 2: Effect of stim ulation rate on neighbourhood s ize and reviewers l ooked at. For t he Si mple P earson case, the nei ghbourhood is composed of the ‘top N’ correlated users, where correlation is mea sured by the Simple P earson stati stical measure. In the Artifici al Immune System case, the neighbourhoo d is cre ated by b uilding an i mmune syste m with the test user as the antigen, t he neighbo urs as antibodies, and the Simple P earson measure as a matching function. (I n fact, i n our exp eri ments, this measure was weighted b y the a fraction prop ortional to the number of films b oth user s had seen, in o rde r to penalise co rrelations made on t he basis o f onl y a fe w films). The b ehaviour of the neig hbourhood is then governed b y equati on 2, with poorly p erforming ant ibodies being deleted fro m the neighbourhoo d. Not e that we have treat ed the idiotypic effect as supp ressive. 4 ANALYSIS OF EFFECTS Although b oth the Artificial Immune System a nd Simple Pearson recommender algorithms are based on P earson correlations, t hey act differentl y for a number of reaso ns: • The cho ice of neighbours is differe nt. I n the S imple Pearson, the 100 highest co rrelated users (or all users that sho w a ny co rrelation, if this is less than 1 00) are chosen to for m a neighbour hood. In the Artifici al Immune System, this general rule is follo wed, except that stimulatio n adds t hreshold and idi otypic effect adds diversit y. • Even given the same neighbo urs, the weighting is different. I n the S imple Pe arson, t he neighbour weight i s simply the co rrelatio n bet ween t hat neighbour and the test u ser. In the Ar tificial I mmune System, this correl ation is multiplied b y t hat antibody’s (nei ghbour’ s) concentration, which in turn is deter mined by r unning t he Artificia l I mmune System algorithm o ver the nei ghbourhoo d. To deal with the first point, the stimulation rate provides some fixed threshold for the correlation of any antibo dy with the antigen. Even i n the absence of any i diot ypic interactions, an anti body’s correlation (weighted by the stimulation rate) must o utweigh the death rate; other wise, it will not survive in the Artific ial Immune S ystem. So, at low stimulatio n r ates it may p rove difficult to fill the Artificial I mmune Syste m completel y. Conversel y, at very high s timulation rates it may not be n ecessar y to examine a ll the supplied users in order to fill an Artificial Immune System. This e ffect was noted in our previous p aper [1] and ca n be seen in F igure 2. S uch a thresholding ef fect has bee n shown to b e beneficial by Gokhale [4] i n maintaini ng the quality o f a neighbourhoo d by filteri ng o ut poorly correlated users (t he Si mple P earson w ill co nsider all reviewers who ha ve at least o ne vote in common with the test user). Thus, t he idio typic e ffect should be viewed in the context of providing further re finement to a neig hbourhood that i s already kno wn t o be in some sense ‘good ’. Since the effect (i n o ur m odel) i s al ways negative, its i m pact may be to i mprove diversity by re moving ‘subo ptimal’ users from the Artific ial Immune Syste m. Conversely, it mi ght be that the idio typic effect is eff ective because, given a neighbourhoo d, i t changes t he weight of each neighbour (or concentration o f each antibody) in t hat neighbourhoo d. This is the second poin t highlighted above. In order to test out these hypotheses, we took a sample result, b ased on 1 00 predictions for detailed anal ys is. T he 3 settings for eac h al gorith m w ere as detailed in [1] except that de fault votes were not used. Thus, if a neighbour ha s not seen a fil m then tha t neighbour i s ignored when making a pred iction for that film. The Artificial Im mune Syste m pa ra met ers w ere set to ‘good’ values (as observed in t he previou s paper) : thus stimulation rate was set to 0.3 and suppression r ate to 0 .2. As reporte d previously, the pred iction perfor mance (mean absolute err or) was not significantl y d ifferent b etween the two algorith ms, but recomme ndation (Ke ndall’s T au) was significantly better for the Artificial I mmune Syste m recommender (a s before, a Wilcoxon matched pairs signed rank te st was used to as sess significance). Comparison of neighbourhood s for A IS and SP predictors 0 20 40 60 80 100 120 AIS SP Pred ictor type N ei gh bo u rh oo d s ize Unique Common Figure 3: Comparison of Artifici al Immune System and Simple Pearson neighbourhoods. The total siz e of each bar represents the total size of the ne ighbou rhoods produced by eac h predictor (averaged over 100 predictions; bar sho ws standard deviatio n). The lower part of e ach bar shows the average nu m ber of common neighbours (i.e. appearing in both ne i ghbourhoods). The remainder of the bar is com posed of unique n eighbo urs – that is, neighbours who appeared in one neighb ourhood but not the other . The first t hing to observe i s that t he nei ghbourhood s produced by each algorithm are different. As implied from the abo ve, Si mple Pears on tended to p roduce large neighbourhoo ds (avera ge 95.4 as op posed to 73.8 using the Artificial I mmune System) and Figure 3 shows t hat the co mposition of these neighbourhood s is different. In particular, i t d oes not s eem that t he Arti ficial I mmune System neighbourhoods are merely subsets o f t he Simple P earson neighbourhoo ds. In fact, t he vast majority of nei ghbours are ‘unique’ – that is, chosen by one algorithm bu t not the othe r Is it the neighbourhoods that make the difference to prediction and r ecommendation p erformance? Figure 4 shows Artificial I mmune Sys tem a nd Simple P earson performance on both neighbo urhoods. For this experiment, we reco rded the neighbour hoods found by both t he Artificial Im mune System and Si m ple P earson algorithms. We t hen reran the pr edictions, with e verything t he same except that this time we forced the Artificial I mmune System and Si mple Pea rson algor ithms to use o ur ‘fixed’ nei ghbourhood s. W e can see that for pred iction, changing the neighbourhood (o r indeed algorithm) did not seem to make any sig nificant di fference (Ta ble 1 has the d etails o f the stat istical test s). Ho wever, for recommendation, although the means ar e very si milar (Fig 4), the Artificial I mmune System neighbour hood usually produced better r ecommendations than the Simple Pea rson neig hbourhoo d (T able 1b). In fact, the neighbourhoo d effect seems t o dominate, s ince given the Artificial Immune Sys te m neighbourhood , the Simple Pearso n algorithm app ears to do significantly better than t he Artificial I mmune System algorith m for recommendation. There is o ne exception to this tre nd, where the Artificial Immun e S ystem al gorithm does not do significantly better for either neighbourhood . In addition, t he Artificial Immu ne System algorit hm does better on the Simple Pe arson neighbo urhood than the Simple P earson algorithm d oes, indicating t hat the neighbour wei ghtings, as wel l as t he neighbo urs themselves, als o contrib ute to the reco mmendation quality. We ran these experiments using default votes (neighbours who had no t voted on a film were a ssumed to give t he film a slightl y ne gative ratin g) and o btained similar results. It is worth pointing out at this stage t hat these results should not be t aken to be ex haustive, merel y i ndicative. Indeed, we would not want to draw an y fir m conclusions based o n o nly 10 0 predictions. T his poi nt will b e re turned to in the d iscussion. Neverthele ss, the results obtained so far see med to indicate that it was worth investigati ng the contribution of neig hbourhood composition to reco mmendatio n performance. Fig 4a Fig 4b Effect of neigh bo ur h oo d on pr ed iction pe rfor m anc e 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 SPp r ed ic torS Pn ei gh b ourh ood A ISpr edict or SP ne ighbo u rho o d SPp redic to r AIS ne ighbo u rhoo d AIS pr editorA ISne ig h bo urho od Me a n a b so lu te er ro r Effect of neigh bou r ho od on r ec omme nd ation pe r for man ce 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 SPp r edic t o rSPn e ighbourhood AISpr e d i ct o r S Pneig h b o urh o o d SPpredi c torAISneig h b o urh o o d AIS pre d i c to r A I S n e i ghb o u r hood Reco mmendation perf ormance (Kenda ll 's Tau) Figure 4: Effect of neighbourhood com positio n for Artificial Imm une System and Simpl e Pearson algorithm s. See text for detail s on fixing the neighbourhoods. Fi g 4a shows p rediction performance (measure d as mean absol ute error averaged over 100 p redictions) fo r each algorithm and each neighbourhood. Fig 4b shows recommendation p erfor m ance deviation. ( measur ed as Kendall ’s Tau averaged over 100 p redictions) for each al gorithm and each neighbour hood. Bars sho w standard deviation . Tab le 1: Analysis o f differen ces be tween neighbourhood s and algorithms for both pred iction (1 a) and reco mmendation (1b). In each case, the Wilcoxon significa nce te st was ap plie d to the results o btained from each pai r of regimes. Regi mes that are si gnificantl y be tter ar e shown in b old (there were no significant d ifferences found for prediction). [AIS = Artificial Im mune System; SP = Simple Pe arson] Table 1a 1 st Predictor 1 st neighbourh ood 2 nd Predictor 2 nd neighbourh ood Median 1 Median 2 Num ber of (unequal) predictio ns compared 1 st regim e better (sum of ranks) 2 nd regim e better (sum of ranks) Significanc e (upper bound) SP SP AIS SP 0.682 0. 697 97 2212 2541 0.5551 SP SP SP AIS 0.682 0.658 9 7 2163 2590 0.4434 SP SP AIS AI S 0.682 0.652 97 2176 2577 0.4717 AIS SP SP AIS 0.697 0. 658 97 2256 2497 0.6659 AIS SP AIS AIS 0.697 0.652 97 2258 2495 0.6711 SP AI S AIS AIS 0.658 0. 652 84 1706 1864 0.7263 Table 1b 1 st Predictor 1 st neighbourh ood 2 nd Predictor 2 nd neighbourh ood Median 1 Median 2 Num ber of (unequal) predictio ns compared 1 st regim e better (sum of ranks) 2 nd regim e better (sum of ranks) Significanc e (upper bound) SP SP AIS SP 0.525 0.557 8 3 801 2685 1.917e-05 SP SP SP AIS 0.525 0.549 8 3 7 07.50 277 8.50 2.617e-06 SP SP AIS AIS 0.525 0.542 8 5 930 2725 8.483e-05 AIS SP SP AIS 0.557 0.549 8 2 1218.50 2184.50 0.02571 AIS SP AIS AIS 0.557 0.542 80 1426 1814 0.3534 SP AIS AIS AIS 0.549 0.542 78 214 9 932 0.002459 We looked a t a variety of neighbo urhood par ameters ( we might term these communit y c haracteristics) across Simple Pearson and Artificial I mmune System neighbourhoo ds. Four characte ristics are of par ticular interest, and each will be discussed i n turn. Firstly, i t might seem reasonable to assu me t hat performance improves with the num ber of neighbours in a neighbourhoo d. However, clearl y t here is a cost in collecting neighbour s (of appr opriate qualit y) together , and thus it will be useful i f we ca n provide good quality recommendations from smalle r neighbourhood s. Another c haracteristic is the overlap size, which govern s the nu mber o f re commendations we can as sess ( An overlap i s a test u ser vote t hat is also contained i n the union of all neighb ours’ votes). Thirdly, we looked at correlation bet ween each nei ghbour and the test user. A high correlation shows t hat neighbours are cluste red ‘tightly’ a round the test us er, whi ch w e might i magine would provide for better recommendatio ns. Fourthly, the idiotypic e ffect i s expected to re duce t he inter-neighbo ur correlations. An o bvious i ntuition m ight b e t hat such a reduction causes a n increase i n recommendatio n quality. Tab le 2 sho ws the d ifference in t hese communit y characteristics acro ss S imple Pearson and Artificial Immune S ystem neig hbourhoods. It can be seen that t he Artificial Immune System does produce neighbourhoo ds that are meas urably differ ent in character to the Simple Pearson neighbourhood s. In summar y, the Artificia l Immune System neighbourho ods are sm aller, have less overlap, are generally less co rrelated with the test u ser and have lower i nter-neighbo ur cor relations. In or der to test out which (i f any) o f these chara cteristics is crucial, we plotted recommendation perfor mance against eac h for the Artificial Immune System a lgorithm. The results see m to show that none of the se characteristic s on th eir own in fluences the p erfor mance in a clear way. Figure 5 shows s catter plots generated for each characteristic agai nst reco mmendatio n q uality. T rend lines (base d o n a power law) have bee n add ed to emphasise an y underlying dat a trends. The first p lot suggests t hat neighb ourhood size is not essential in or der to ob tain high quality r eco mmendations. The second p lot, how ever, does suggest t hat small overlap sizes might be be neficial for pro ducing good recommendations ( regressio n analysis has not bee n performed so at this stage this is merely a suggestion). This in some sense i s intuitive, as it might be easier to produce higher qualit y reco mmendations i f there are less of them. However, a b alance need s to be struck here; once the overlap size gets too lo w, t he nei ghbourhoo d may no longer prove useful to the user . The third plo t shows that, pe rhaps s urprisingly, high correlation bet w een neighbour s and the test user ma y not be essential f or high quality recommendations. Finally, the fo urth plot would seem to indicate that reduced inter- neighbour cor relation is no t importa nt in reco mmendation accuracy, or at least if it is re sponsible, it is part of a wider effect. Table 2 : Ana lysis of difference in n eighbourhood characteristics between Simpl e Pearson and Artificial Immune Syst em a lgorithms. F our characteristi cs ar e shown. In each case, the Wilcoxon significanc e test was applied to the n eighbour hoods obtained from the algorithms. In all four cases, the val ue for the Sim ple Pearson was sig nificantly higher; this i s indicated by bold typ e. 1 st Predictor 2 nd Predicto r Neighbo urhood characteristi c tested Mean 1 Mean 2 Number of (unequal) neighbourhoods compared 1 st neighourhood has higher value (sum of ranks) 2 nd neighourhood has higher value (sum of ranks) Significance (upper bound) Simple Pearson Artificial Imm une System Neighbours 95.40 73.75 97 4602 151 1. 196e-15 Simple Pearson Artificial Imm une System Overlap 47.46 46.39 26 3 34.50 16.50 5.686e-05 Simple Pearson Artificial Imm une System Correlatio n 0.12 0.10 79 2566 594 1. 465e-06 Simple Pearson Artificial Imm une System Neighbour correlati on 0.15 0.04 83 3477 9 3.572e-15 Fig 5a Fig 5b Effect of neighb ou rho od s ize o n re comme n dation accu r acy 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 10 20 30 40 5 0 60 70 80 90 100 Neighbou rh ood Size Recommendat ion Accuracy ( Kendal l's Tau ) Effect of over lap siz e on r ec omm enda tion ac cur ac y 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 50 100 150 200 250 300 Ov erl ap Size Recommendat ion Accuracy ( Kendall 's Tau ) Fig 5c F ig 5d Effect of cor r elation with test use r on r eco mmen dation acc ur ac y 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.05 0.1 0.15 0 .2 0.25 Adju sted Correlation with t est u ser ( median) Recommendation Accuracy (Kendal l's Tau ) Effect of inter-ne ighbo ur co rr ela tion on r eco mme nd ation ac cur a cy 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 Adju sted I n ter n eighbou r correl ation ( median) Recommendation Accuracy (Kendal l's Tau) Figure 5. Effect of var ious n eighbour hood measures on Artificial Immune System reco mm endation per form ance. In ea ch gr aph, the measure is shown on th e x-axi s. The recomm endation perform ance (where ava il able) f or each of 100 Artificial I mm une System pred ictio ns is plotted a gainst this neighbourhood m easure. Trend li nes are added to i ndicate the underly ing data trend (if any ). 5 DISCUSSION AND CONCLUSIONS As mentio ned p reviously, it is not c laimed that these results a re conc lusive. Indeed, muc h more data is r equired before any firm co nclu sions can be drawn. In this respect, this p aper is ver y much a work in p rogress. Nevert heless, the results to da te certainly a re indicative, and challenge certain assu mptions. It is ho ped that the p resentation o f these results will stimulate di scussion and interest i n the nature of the idio typic effect. It does not seem likely that the idiotypic effect can be captured by one par ticular measure ment. Never theless, it is likely to be so me combination of factors. For example, we have s hown that bot h t he neighbourhood choice and the w eighti ng o f ne ighbours w ithin that neighbourhood can influence t he r ecommend ation performance. Pinning do w n the ef fect further has p roved to be pro blematic. Our first intuitio n – tha t spread ing out neighbours b y reducing inter-neighbour corr elation im pro ves recommendation – appears to be at best inco mplete and at worst i ncorrect. The mecha nisms unde rlying the effect are clearl y subtler than this. There a re of course o ther community c haracteristics tha t we co uld explore. S ome (for example, number o f recommendations, overlaps per neighbour, abso lute correlation scores) have bee n e xamined and shown to be equally inconcl usive. Some ( for example, nu mber of neighbours voting o n each f ilm) re main po tential future subjects for invest igation. Other t ests (e.g. set ting each neighb our’s conce ntration to a ra ndom number for immune system pred ictions, to see whether acc urate concentrat ions are reall y nece ssary) might shed further li ght on the relative importance o f each measure. But it is our intuition that such stud ies might not really get a t t he nature o f t he effect, a nd t hat larger scale or more sophisticated tests will b e needed, coupled with perhaps analytical work, to get at the heart of this intriguing pheno menon. There are wider implicatio ns f or such work. The database used for this study [2] is ba sed o n real p eoples’ pro files. Thus, any head way made i nto i mproving neighbo urhoods by the id iotypic effect can have rea l benef it for other recommender s – and indeed an y community based application. References [1] Cayzer S, Aickelin U , A Reco mmender Syste m based on the I mmune Networ k, Proce edings o f the 2002 Congress on Evolutionar y Computation, 2002. [2] Compaq Systems Research Centre. EachMovie collabor ative filter ing data set, http://www.re search.compaq.c om/SRC/eac hmovie/. [3] Far mer J D, Packard NH and P erelson AS, T he immune system, ada ptation, and m achine l earning Physica, vol. 22, pp. 187 -204, 1986. [4] Gokhale A, Impro vements to Collabor ative Filte ring Algorithms 1999 . Worc ester Poly tec hnic Institute. http://www.cs. wpi.edu/~clayp ool/ms /cf-i mprove/. [5] Gold sby R, Kindt T , Osborne B , Kuby Immunolog y, Fourth Edition, W H Freeman, 2000. [6] Jerne NK, T owards a net work theor y o f t he immune system Annals o f I mmunology, vol. 125, no. C, pp. 373-38 9, 1973. [7] Pe relson AS and Weisb uch G, I mmunology for physicists Re views of Mo dern Ph ysics, vol. 69, pp. 1219 -1267, 1997.