Cooperative searching for stochastic targets

Co op erativ e searc hing for sto c hastic targets V adas Gin tautas, 1, 2 , ∗ Aric Hagberg , 2 and Lu ´ ıs M. A. Bettencour t 2, 3 1 Center for Nonline ar Studies, L os A lamos National L ab or atory, L os A lamos NM 87545, USA 2 Applie d Mathematics and Plasma Physics, The or etic al Division, L os Alamos National L ab or atory, L os Alamos NM 87545, USA † 3 Santa F e Institute, 1399 Hyde Park R o ad, Santa F e NM 87501, USA (Dated: Novem ber 8, 201 8) Spatial search p roblems aboun d in the real worl d, from locating hidden nuclear or chemical sources to finding skiers after an a v alanc he. W e exemplify th e forma lism and solution for spatial searc hes inv olving t w o age nts that ma y or m ay not choose to share information during a searc h. F or certain classes of tasks, sharing information betw een multiple searc hers mak es co op erativ e searc hing adv anta geous. In some examples, agen ts are able to realize synergy by aggregating information and moving based on lo cal judgments ab out maximal informatio n gathering ex p ectations. W e also explore one- and tw o-dimensional simplified situations analyt ically and numerical ly to p ro vide a framew ork for analyzing more complex problems. These general considerations provide a guide for designing optimal algorithms for real-w orld searc h problems. I. INTRO DUCTION In the real w orld, there are man y spatial search prob- lems that inv olve multiple agents or searchers. Co mm u- nication betw een such agents or b e t ween agents and a centralized command cen ter may be s ensitiv e, costly , or difficult for v arious reasons. In this work, we explore spatial search problems in this con text and examine the classes of sea rch problems for whic h communication and co ordination b et ween m ultiple agents will enable qua n- titative adv antages ov er independent information gath- ering. These pro ble ms are very gener al and ca n b e for - malized and solved in terms o f informa tion theory . The solutions are essential to the dev elopment of quantita- tive decisio n supp ort to ols under uncertain ty (e.g. in medicine [9 ]) and fo r automated m ulti-agent searches for sto c hastic infor mation [1], such as those in volving dis- tributed senso r net works [19]. A searc h ma y be thought o f a s a series of steps by which an agent or a gen ts reduce the uncertaint y of the lo cation of a targ e t to zer o . T his lo cation may b e in physical s pa ce or in a “ s pace of p ossibilities,” i.e., a se t of a lternative scenarios . F o r example, if you need to find y our k eys in the morning b efore g oing to work and there ar e five ro oms in your ho use, initia lly you k now your keys m ust be in one o f these ro oms. After thor oughly searching one r oo m and not finding the keys, your uncer tain ty is reduced; the keys must now b e in one of the four remaining ro oms, and so on. If you find the keys in the se cond ro om, then the uncertaint y of their lo cation is immediately zero b ecause you know with complete certaint y that they are in your hand. Now imagine you ha ve a friend help you search. The t wo of you would be searching twice a s fast since , as - ∗ Electronic address: v gintaut as@chat ham.edu † Ph ysics Departmen t, Chatham Univ ersity , Pittsburgh P A 1523 2, USA suming y ou and y our friend share informatio n, a pair of s e archers ca n eliminate ro oms a t a rate t wice as fa s t as that of a single searcher. In more co mplex sea rch problems, the tar get might emit some k ind of compli- cated sig nal that makes it p ossible for multiple co or di- nated se a rchers to dra matically increas e efficiency (imag - ine there ar e o ne thous and ro oms to search but the keys are attached to some kind of radioactive homing be a con or can emit a so und). In these situations, understanding the nature of the c lues and shar ing infor ma tion can b e extraor dinarily b eneficial. A sea rc h whereby agents move ba sed expr essly on in- formation cues r ather than following gra dien ts is known as an infotaxis search. In a 200 7 pap er, V erga ssola et al. explored infotax is in the context of a moth following a pheromone trail thro ugh air to find a mate. The moth was p erforming a spatia l sea rch in turbulent air currents that carried the trail h [18]. Compared to more conv en- tional metho ds s uch as c hemotaxis (following a chemical gradient [7, 14]), infotaxis gives a n adv antage in situ- ations when the signals from the target of the search are uncerta in. This migh t b e the case if signals from the target are sto chastic, difficult to measure, or highly v arying in time. In the cas e of the moth, the signal was sparse and widely dis p ersed by the turbulent air cur rent s. When the moth follo wed the gra dien t of the strength of the trail dir ectly chemotaxis it was forced to take a very circuitous ro ute due to the turbulen t disp ersal of the trail. How ever, when the moth employ ed an info- taxis algorithm to mov e to p ositions where it mig h t gain the mos t information a bout the sour c e o f the tr ail, its per formance improv ed significantly . Since we will con- sider only sear ch pro blems s uc h as these, in the following we will refer to the target of a n infotaxis sea rch as the “source.” The formalism of infota xis balances the com- peting goals of ex ploiting the curr e nt information av ail- able and exploring to gain more infor mation, a familiar compromise from other unsuper vised lea rning metho ds, such as reinfor cemen t learning [17]. While effectiv e path planning a lgorithms may b e based on the o ptimization UNCLASSIFIED (LA-UR 09-0767 6) of so me ob jective function, thes e often rely on exploiting some features o f the known environment [12] r ather than a so lid foundation based on information theor y . Info- taxis provides a framework for studying search problems in gener al and is therefo re broadly applicable. In this article, w e explo re conditions in which co op er- ation b et ween multiple infotaxis agents is adv antageous. W e fo cus o n ex a mples in which ag en ts are able to r e- alize synergetic co op eration by a ggregating infor mation and moving ba sed on a lo cal infota x is algo rithm. Syn- er gy , and its opp osite, r e dundancy , ar e infor mation the- oretic quantities that a r e defined in terms o f relative probabilities o f the sto c hastic v ariables inv olved [4]. In a recent pap er, w e show ed that spatiotemp oral corre la- tions are nece ssary for synergy [11]. When sy nergy is exploited e ffectiv ely it can le a d to an exp onen tial re- duction in the search effort, in terms of time, ener gy , or n um b er of steps [8, 10, 16]. Here w e use a simple one-dimensional search example and a mor e rea listic tw o- dimensional ge neralization to show how correlatio ns lead to sy ne r gy . These simple exa mples provide a fr amew ork for a nalyzing more complex problems. Since, in gener al, the computational c ost is greater for sea rchers to com- m unicate and p erform co ordinated movemen ts instead of moving based on indep endent decisio ns, w e will describ e situations in whic h co or dina tion is w orthwhile. II. INFO RMA T ION THEOR Y APPR OA CH TO STOCHASTIC SEAR CH Effective and robust search metho ds for lo cating sto c hastic sour c es balance the comp eting str ategies of exploratio n and exploitatio n [17]. Giv en a current esti- mated pr obabilit y distribution for the lo cation o f a so urce a searcher might either exploit the da ta a lready c o llected by moving towards the lo cation that maximizes this like- liho od or shar pen the distribution (reduce uncertaint y further) by moving to gather mo r e div erse data. The in- fotaxis search balances these tw o s trategies by optimiz- ing the exp ected information gain over the p ossible next search mov es. In the following w e review some basic con- cepts from information theor y a nd formalize the infotax is algorithm in terms of these quan tities. A. Information, synergy , and redundancy T o determine whether sear c hers can be effectively co- ordinated we define define synerg y and r edundancy as information theoretica l qua n tities [6] and use them a s a measure of co ordination. Sy ne r gy is found when meas ur- ing tw o or more v ariables to gether with resp ect to another (e.g. the so urce’s s ignal) results in a greater infor ma- tion gain than the sum o f that from each v ariable sep a- r ately [4, 5]. In sear c h problems , sy ne r gy is adv antageous bec ause then the c o or dination of tw o or more sea rc hers is more efficient than the same searchers w orking indep en- dent ly . In this section we will intro duce thes e concepts in general terms befor e applying them to a sp ecific search problem. Consider the sto chastic v ariables X i , i = 1 . . . n . Each v ariable X i , r epresentin g a searcher or so urce lo cation, can take on sp ecific states, denoted by the cor respo nding low ercase letter x i . F or a single v a riable X i the Shannon ent ro p y (henceforth “ e n tropy”) is S ( X i ) = − X x i P ( x i ) log 2 P ( x i ) , where P ( x i ) is the probability that the v ariable X i takes on the v alue x i [6]. The sum is ov er all o f the po ssible states x i ; since P ( x i ) < 1 always,the entrop y is a lw ays po sitiv e. The ent ro py is a meas ur e of uncertain ty abo ut the state of X i , therefore entrop y can only decr ease or remain unc hanged as mor e v ariables are measured. The conditional en tropy of a v ariable X 1 given a second v a ri- able X 2 is S ( X 1 | X 2 ) = − X x 1 ,x 2 P ( x 1 , x 2 ) log 2 P ( x 1 , x 2 ) P ( x 2 ) ≤ S ( X 1 ) . This e x pression contains a s um over the join t probabil- it y distribution of t wo v ariables. Since measuring a sec- ond v ariable can only decr ease entrop y (or leav e it un- changed), the conditional e ntropy is bounded ab ov e by the en tropy of the first v ariable. The mutual informa tio n betw een t wo v ariables, which pla ys an impo rtant role in search s trategy , is defined as the change in en tropy when a v ariable is mea sured: I ( X 1 , X 2 ) = S ( X 1 ) − S ( X 1 | X 2 ) ≥ 0 . This is also the difference betw een the en tropy o f one v ariable and its entrop y co nditioned o n the measur emen t of a second v ariable. Mutual information is a lways p osi- tive. These definitions can b e directly extended to m ul- tiple v ariables. Just a s entrop y may b e conditioned on an a dditional measur emen t, m utual information may b e conditioned on the knowledge of other v ariables. These quantities may be used to generate new information the- oretic co nstructs that we will us e in sp ecific sear c h pro b- lems. F o r three v ariables [15] the quantit y R ( X 1 , X 2 , X 3 ) ≡ I ( X 1 , X 2 ) − I ( { X 1 , X 2 }| X 3 ) (1) measures the degree of “ o verlap” in the information con- tained in v ariables X 1 and X 2 with resp ect to X 3 . The sign of this quantit y is meaningful. If R ( X 1 , X 2 , X 3 ) > 0, there is ov erlap and X 1 and X 2 are sa id to b e redundant with r espect to X 3 . If R ( X 1 , X 2 , X 3 ) < 0, mor e infor- mation is av ailable whe n these v a riables ar e considered together than when consider ed separa tely . In this case X 1 and X 2 are said to be s ynergetic with resp ect to X 3 . If R ( X 1 , X 2 , X 3 ) = 0, X 1 and X 2 are indep enden t. 2 UNCLASSIFIED (LA-UR 09-0767 6) B. Bay esian i nference and spati al i nfotaxis W e first formulate the general spa tial sto chastic search problem for N searc hers s e eking to find a sto chastic source lo cated in a finite, D -dimensional space. This is a generaliza tion of the single searcher formalism pre- sented in Ref. [18]. A t any time step, the searchers s i , i = 1 , 2 , . . . , N , are lo cated at po s ition r i and ob- serve some num be r o f particles h i from the source. The searchers do not g e t information ab out the tra jecto r ies or sp eed of the particles; they only get information if a par- ticle was observed or no t. Therefor e simple geometrica l metho ds such as triangulatio n a r e not p ossible. Consider a r andom v ariable R 0 , which can assume a nu mber of sp ecific v alues, denoted by r 0 . The v alues of r 0 refer to p ositions in spa ce tha t ma y contain the sto c has- tic source. Only one v alue of r 0 corres p onds to the (yet unknown) lo cation of the source s 0 . The sear c hers com- pute and share a probability distribution P ( t ) ( r 0 ) for the source lo cation at eac h time index t . Initially the proba- bilit y for the source P (0) ( r 0 ) is assumed to b e to b e uni- form. After each mea suremen t { h i , r i } , the sea rc hers up- date their estimated probability distribution of so urce p o- sitions via Bay esian inference [2] and decide what move to make (p ossibly remaining at the same p osition). The goal of Bay esian inference is to improv e an estimated proba- bilit y distribution P ( X ), where X is a r a ndom v a riable that can ass ume a set o f v a lues denoted b y x . Assuming that Y is another ra ndom v ariable (that can ass ume a set of v alues denoted b y y ) a nd that X and Y are not in- depe ndent (that is, I ( X , Y ) 6 = 0), k no wledge of the state of Y can be used to improv e P ( X ). After a measure- men t reveals Y = y , the pr obabilit y o f this measurement given the current e s timated is co mputed. The probabil- it y of this measurement is P ( Y = y | X ). Ba yesian in- ference ma k es it p ossible to a ssimilate this informatio n int o the current estimate of P ( X ) via a Bay esian up date step: P ( X ) = P ( X | Y = y ) = P ( Y = y | X ) P ( X ) / A , where A is a normaliza tion factor . This step includes an explicit statement of equiv alence beca use eac h new mea- surement is included implicitly in P ( X ). Therefore the measurement Y = y impr o ves the estimate of P ( X ). The searchers will us e this Bay esian inference framework to improv e their es timate o f the probability distr ibutio n of source lo cations P (0) ( r 0 ) after eac h measure ment h i , r i . T o decide where to move nex t, the searchers follow an infotaxis algor ithm for multiple sea r c hers. First the conditional pro babilit y P ( t +1) ( r 0 |{ h i , r i } ) ≡ 1 A P ( t ) ( r 0 ) P ( { h i , r i }| r 0 ) , (2) is calcula ted, where A is a norma liz a tion o ver all p ossi- ble source lo cations r 0 as required by B ayesian inference. This is then assimilated via Ba yesian update, P ( t +1) ( r 0 ) ≡ P ( t +1) ( r 0 |{ h i , r i } ) . (3) If the searchers do not find the s ource a t their prese n t lo cations they choos e the nex t lo c al move using an in- fotaxis step to maximize the exp ected infor mation gain. The exp ected infor mation gain is computed in the fol- lowing w ay . The entropy of the distributio n P ( t ) ( r 0 ) at time t is defined as S ( t ) ( R 0 ) ≡ − X r 0 P ( t ) ( r 0 ) log 2 P ( t ) ( r 0 ) . (4) F or a sp ecific measurement { h i , r i } the entrop y b efor e the Bay esian update is S ( t ) { h i ,r i } ( R 0 ) ≡ − X r 0 P ( t ) ( r 0 |{ h i , r i } ) lo g 2 P ( t ) ( r 0 |{ h i , r i } ) . (5) W e define the difference b etw een the entrop y at time t and the entrop y at time t + 1 after a measur emen t { h i , r i } to b e ∆ S ( t +1) { h i ,r i } ≡ S ( t +1) { h i ,r i } ( R 0 ) − S ( t ) ( R 0 ) . (6) F or a uniform prior, P (0) ( r 0 ) = 1 / M for M pos s ible lo- cations of the source in the discrete spa ce, the entrop y is max im um, S (0) ( R 0 ) = log 2 M . F o r each po ssible joint mov e { r i } , the change in exp ected entropy ∆ S is co m- puted and the mo ve with the minimu m (most negative) ∆ S is executed. The exp ected informatio n gain is found by computing the entrop y change for all of the p o ssible joint sea rc her mov es ∆ S = −  X i P ( t ) ( R 0 = r i )  S ( t ) ( R 0 ) +  1 − X i P ( t ) ( R 0 = r i )  × X h 1 ,h 2 ∆ S ( t +1) { h i ,r i }  X r 0 P ( t ) ( r 0 ) P ( t +1) ( { h i , r i }| r 0 )  . (7) The first ter m in Eq . (7) corres ponds to one o f the searchers finding the source in the next time step (the final entrop y will b e S = 0 so ∆ S = − S ). The second term considers the reduction in entrop y for all p ossible measurements at the prop osed lo c a tion, weigh ted by the probability of each o f those meas uremen ts. The pr oba- bilit y of the sear c hers obtaining the measure ment { h i } at the lo cation { r i } is given b y the trace o f the probabilit y P ( t +1) ( { h i , r i }| r 0 ) ov er all p ossible source lo cations. A t each step the searchers mov e join tly to increase the exp ected information gain as measured by the change in ent ro p y of the probability distribution. Although this algorithm is general in the following we conside r only the case of tw o s earchers ( N = 2) a nd b oth o ne- and t wo- dimensional spatia l domains. II I. SEAR CHING F OR CORRELA T ED SIGNALS IN ONE DIMENSION Sources that emit uncorrelated signals provide no op- po rtunit y for co ordination because the sea r c hers are 3 UNCLASSIFIED (LA-UR 09-0767 6) never sy nergetic [11]. W e instead consider s ignals with spatial, tempor al, or other correla tions. The simplest nontrivial example is searching in a one-dimensional do - main for a source tha t emits t wo particles simultaneously in o pp osite directions. Two searchers should b e able to exploit the cor relations in the signa l; if bo th sea rchers simult aneo usly o bserve particles, they can immediately conclude that the s ource is lo cated b etw een them. There- fore we exp ect synerg y to b e p o ssible for some spatial arrang emen ts of the s o urce and sear c hers. First consider a finite one-dimensional domain with a source s 0 and tw o searchers s 1 and s 2 at the corres p ond- ing po sitions { r 0 , r 1 , r 2 } ∈ [0 , 1]. The source is assumed emit t w o particles simultaneously and in opp osite direc - tions. That is, one par ticle is emitted to the left a nd one to the rig h t of the s ource. The tw o s earchers s 1 and s 2 are ident ical with a fixed cro ss section such that 0 < a < 1 is the probability of a sea rc her capturing one of the parti- cles emitted from the source . At each step in the s e arch the num ber o f pa r ticle “hits” meas ured by searchers s 1 and s 2 are denoted by h 1 ∈ { 0 , 1 } and h 2 ∈ { 0 , 1 } , re- sp ectiv ely . T o calculate R ( r 0 , h 1 , h 2 ) it is first necessary to com- pute the probabilities of h i for each searcher g iv en the po sition of the source r 0 . W e note that since there is no distance dependence in the ca pture probability a , it is sufficient to consider three separate cases dep ending on the relativ e positions, or ordering, of the s ource a nd the searchers, as shown in Fig. 1. F or exa mple, if s 1 is to the left of the s o urce a nd s 2 is to the right of the source, the order is s 1 s 0 s 2 . Note this is eq uiv alent to the case s 2 s 0 s 1 since the s earchers a re iden tical. s 0 s 1 s 2 r 0 r 1 r 2 0 1 s 1 s 0 s 2 r 1 r 0 r 2 0 1 s 1 s 2 s 0 r 1 r 2 r 0 0 1 FIG. 1: The three uniqu e cases for relative p ositio ns of the source and t wo iden tical searc hers on a one-dimensional do- main. Since the probab ly of detection in this example do es not dep end on distance, we need only consider the spatial ar- rangemen t of the source and searchers. The cases are labeled s i s j s k according to the relative spatial ordering for the source s 0 and searc hers s 1 , s 2 . Since the searc hers are identical, s 1 and s 2 are interc hangeable and there are only three uniqu e cases. If a searcher obser v es a par ticle, it is assumed to b e absorb ed so that the other se archer will not b e able to observe it. F or example if s 1 is b etw een s 2 and the source (case s 0 s 1 s 2 or s 2 s 1 s 0 ) then the proba bilit y that s 2 ob- serves a particle depends on whether s 1 observed it. If s 1 observed it, then the pro babilit y that s 2 observes it m ust b e 0. If s 1 did not observe it then the probability that s 2 will obs e r v e it is a P ( h 2 = 1 | h 1 = 1 , r 0 ) = 0 , (8) P ( h 2 = 1 | h 1 = 0 , r 0 ) = a. (9) W e ca n use the probability relation P ( h 2 , h 1 | r 0 ) = P ( h 2 | h 1 , r 0 ) P ( h 1 | r 0 ) (10) to compute the t wo-searcher conditional pr obabilities P ( h 2 = 1 , h 1 = 1 | r 0 ) = 0 , (11) P ( h 2 = 1 , h 1 = 0 | r 0 ) = a (1 − a ) . (12) The other pro babilit y distributions ar e computed using similar reaso ning. The r e s ults are summarized in T a ble I. Cases h 1 , h 2 P ( h 1 | r 0 ) P ( h 2 | r 0 ) P ( { h 1 , h 2 }| r 0 ) 1 , 1 a a (1 − a ) 0 s 0 s 1 s 2 1 , 0 a 1 − a (1 − a ) a s 2 s 1 s 0 0 , 1 1 − a a (1 − a ) a (1 − a ) 0 , 0 1 − a 1 − a ( 1 − a ) (1 − a ) 2 1 , 1 a a a 2 s 1 s 0 s 2 1 , 0 a 1 − a a (1 − a ) s 2 s 0 s 1 0 , 1 1 − a a a (1 − a ) 0 , 0 1 − a 1 − a (1 − a ) 2 1 , 1 a (1 − a ) a 0 s 0 s 2 s 1 1 , 0 a (1 − a ) 1 − a a (1 − a ) s 1 s 2 s 0 0 , 1 1 − a (1 − a ) a a 0 , 0 1 − a (1 − a ) 1 − a (1 − a ) 2 T ABLE I: Conditional probabilities for the six p ossible ar- rangemen ts of th e source s 0 and searc hers s 1 , s 2 sho wn in Fig. 1. S ince the searc hers are iden tical, s 1 and s 2 are inter- changea ble and th ere are only three unique cases. In cases s 0 s 1 s 2 and s 2 s 1 s 0 , searcher one is b et ween the source and searc her tw o. A particle emitted by th e source in the direc- tion of the searc hers wil l reach searc her one fi rst. If searcher one detects th e particle, searcher tw o will not be able to d e- tect it. S earc her tw o will only hav e a chance of detecting the particle if the particle passes through searc her one unde- tected. Similarly , in cases s 0 s 2 s 1 and s 1 s 2 s 0 , the source is b et wee n the searchers and the searc hers d o not interfere with eac h other. F urthermore, since the source emits tw o p articles sim ultaneously in opposite directions, in only these cases do b oth searchers h a ve a c hance of each d etecting a particle. Using T able I, we can ana ly tically compute the infor - mation theoretic quantities we need to de ter mine s ynergy and redundancy . These are R ( h 1 , h 2 , r 0 ) = I ( h 1 , h 2 ) − I ( h 1 , h 2 | r 0 ) , (13) I ( h 1 , h 2 ) ≡ X h 1 ,h 2 P ( h 1 , h 2 ) log 2 P ( h 1 , h 2 ) P ( h 1 ) P ( h 2 ) , (14) and I ( h 1 , h 2 | r 0 ) ≡ P h 1 ,h 2 R 1 0 dr 0 P ( h 1 , h 2 , r 0 ) (15) × log 2 P ( h 1 , h 2 | r 0 ) P ( h 1 | r 0 ) P ( h 2 | r 0 ) ; 4 UNCLASSIFIED (LA-UR 09-0767 6) where the probabilit y distributions are calculated as fol- lows: P ( h 1 , h 2 , r 0 ) = P ( h 1 , h 2 | r 0 ) P ( r 0 ) , (16) P ( h 1 , h 2 ) = Z 1 0 dr 0 P ( h 1 , h 2 | r 0 ) P ( r 0 ) , (17) P ( h 1 ) = Z 1 0 dr 0 P ( h 1 | r 0 ) P ( r 0 ) , (18 ) P ( h 2 ) = Z 1 0 dr 0 P ( h 2 | r 0 ) P ( r 0 ) . (19 ) Initially we consider a uniform pr obabilit y distribution (prior) of source lo cations: P ( r 0 ) = 1. F or small capture probability , a , we can expand Eq. (13) as a T aylor series in a to g et an analytica l so- lution for the cr itical v alues a c where R | a = a c = 0 . F or r 1 > r 2 , the critical v alues are given b y a c = s 3( r 2 − r 1 ) log ( r 1 − r 2 ) r 3 1 − r 3 2 − 3 r 2 1 + 2 r 1 + r 2 . (20) F or r 1 < r 2 , a c is given by E q. (2 0) under the trans- formations r 1 → r 2 and r 2 → r 1 . These r elations give the approximate b oundary b et ween the regions of syn- ergy and redundancy . In the limit r 1 → r 2 , R → 0. F or these for m ulas we used a uniform s ource distr ibution P ( r 0 ) = 1 but it is p ossible to rep eat these ca lculations with a differ e n t P ( r 0 ). F or la rger v alues of a , when the expans ion is no longer v alid the co ndition R = 0 [as in Eq. (13)] can b e solved nu merica lly . Figure 2 shows R ( h 1 , h 2 , r 0 ) as a function of the captur e probability a and sear c her s 1 lo cation r 1 with searcher s 2 fixed at r 2 = 2 / 3. This fig ure also shows R ( h 1 , h 2 , r 0 ) for a Gauss ian proba bilit y of the sourc e lo- cation P ( r 0 ) = B exp( − ( r 0 − 1 / 3) 2 ), where B is a nor mal- ization factor . In both ca ses, R > 0 (indicating redun- dancy) when the searchers are clos e together and R < 0 (indicating synergy) when they are further a part. F or the Gaussian distribution so ur ce lo cation synerg y is s trongest when s 1 is clo se to the s ource b ecause the mutual infor- mation b et ween r 0 and h 1 is p eaked there as well. Figure 2(a) shows how the capture probability a influ- ences the p ossible s trategies of the sea rchers. If a is close to 1 , then the sea rchers only realize synergy if they a re far a part, whic h max imize s the c hance of the source be- ing b et ween them. But if a is is clos e to 0 then nea r ly any arrang emen t of sea rchers is syner getic but only w eakly . Figure 2(b) demonstra tes the significa nce of the prior P ( r 0 ) in the c a lculation of R ( h 1 , h 2 , r 0 ). The v ariance of P ( r 0 ) is equal to 1, the size o f the domain, making the Gaussian distribution quite broad. Nonetheless, the fact that the prior is weakly p eaked at some point in the domain dramatically reduces the ar ea o f the r edundan t ( R > 0 ) region and a llo ws for muc h g reater synergy be- t ween the sear c hers. Although sea rch pro ble ms in the real world are r arely one-dimensiona l, this exa mple illus- trates the basic calculations for determining synerg y . 0 0 . 2 0 . 4 0 . 6 0 . 8 1 a ( a ) R e dund a n t ( R > 0 ) S yn e r g e ti c ( R < 0 ) 0 0 . 2 0 . 4 0 . 6 0 . 8 1 r 1 0 0 . 2 0 . 4 0 . 6 0 . 8 a ( b ) S yn e r g e ti c ( R < 0 ) r 2 r 2 FIG. 2: Syn ergetic and redundant regions for t w o sea rchers in the one-dimensional correlated searc h problem. The v alue of R ( h 1 , h 2 , r 0 ) is shown for different lo cations r 1 of s 1 with s 2 held fi xed at r 2 = 2 / 3 and the capture p robabilit y v arying from a = 0 (no capture) to a = 1 (complete capture). D ark er regions represen t higher synergy (larger n egativ e v alues of R ) and th e white areas represent redund an cy . The critical con- tour line R = 0 separates the syn ergetic and redundant re- gions. (a) When the source is equally lik ely to be anywhere, P ( r 0 ) = 1, there is only w eak synergy . (b) Then the proba- bilit y of the source is a Gaussian distribution centered at 1 / 3, P ( r 0 ) ∝ exp( − ( r 0 − 1 / 3) 2 ), the synergy is highest n ear the source at 1 / 3 since the mutual information is highest there as w ell. The dashed line in (a) show s the critical va lues a c com- puted by th e approximation in Eq. (20); this approxima tion is go od for a c . 0 . 3. IV. SEARCHING FOR CORRELA TED SIGNALS IN TW O DIMENSIONS The one-dimensional example pr o vides a tracta ble starting p oint for generaliza tion to tw o-dimensiona l prob- lems. The simplest gener alization is the case of tw o mobile searchers and one stationary source in a tw o- dimensional finite do main. Imagine a source in whic h chemical or n uclear reactions are o ccurring and the pro d- ucts of the reactions lea ve the source with relative angu- lar correlatio ns. Our t w o-dimensio nal idealized pro blem consists o f a so ur ce that emits 2 particles simultaneously at each time step in oppos ite dire c tions a long some emis- sion axis. At ea c h time step, a new emission axis angle is chosen uniformly at r andom from (0 , 2 π ]. The particles mov e a long straig h t-line tra jectories but the sea rc hers are not able to meas ur e the velo cities of any detected pa r ti- cles, so g eometric metho ds such as triang ulation are not po ssible. 5 UNCLASSIFIED (LA-UR 09-0767 6) FIG. 3: Diagram for the t wo-dimensional example. The source p osition r 0 is fixed in the cen ter and th e searchers are d isks of radius d . At eac h time step the source emits tw o particles in in opp osite directions with a rand om angle. Eac h p ossible emission axis passing through the source correspond s to one of six cases for source and searc her confi gurations as giv en in T ables. I and I I. In the re a l world, sea rc hers could b e, for exa mple, au- tonomous mobile senso rs capable of detecting r adiation or o ther rea ction pro ducts fr o m the ab ov e exa mple. Most mo dern-day autonomous rob ots do not mov e muc h faster than a walking pace [13], la rgely due to the difficulties of navigating uncer tain ter rain sa fely . Thus mov ement is r elatively costly (backtrac king will take a significant amount of time) and the sear c hers s hould make decisions to refine their tra jectorie s often, using an y new a v ail- able information. F or this pr oblem, this mea ns that the searchers can only make s mall discre te movemen ts b e- t ween measurements. T o repr esen t simplified mobile autono mous robo ts, we cast the sea rchers as iden tical disks of r adius d that mov e on a reg ula r Car tesian gr id. Unlike the one-dimensiona l case these searchers hav e spatial extent; there are tw o parts to the calculation of particle detection. First, if a particle trav els alo ng a str aight line tra jectory that passes through a searcher, the capture pro babilit y is 0 < a < 1 . As in the one-dimensiona l ex a mple, if a particle is ob- served b y a searcher, it is a bsorb ed so it ca nnot b e ob- served by the other sea rcher. Second, we must consider the probability that the particle’s tra jecto ry will pass through the sea rc her. This is a function of the searcher radius d and the distance to the so urce. The v ariables r 0 , r 1 , and r 2 for the p ositions of the sour c e and sea rch ers each hav e t wo comp onent s, e.g.( r 0 ,x , r 0 ,y ), since they r ep- resent positio ns on a tw o-dimensional grid. As in the one-dimensional example, there are differ- ent cases for the pr o babilities which dep end on the rel- ative p osition of the t w o sea rchers to the s ource. F or all po ssible straig h t line emission axe s passing thr ough the source, some lines ma y pass through no searchers (case s 0 ), only sear c her s 1 (case s 0 s 1 ), or only sea r c her s 2 (case s 0 s 2 ). If a line passes throug h b oth searchers, the so urce is b etw een the searchers (case s 1 s 0 s 2 as in the one-dimensional example), or one of the other sear c hers is in front of the other (cases s 0 s 1 s 2 and s 0 s 2 s 1 as in the one-dimensional example). These cases are illustr ated in Fig. 3 . F or any source lo cation, there will b e a rang e of a ngles ∆ θ c for each case c . The probabilities for the cases that do not appear in the one-dimensional example (see T able I) are detailed in T a ble II. Quantit ies such as P ( h 1 , h 2 | r 0 ) are a sup erpo sition of the v alues for the different cases, w eighted by the prop ortion of angles cor- resp onding to ea ch case P ( h 1 , h 2 | r 0 ) = X c ∈{ cases } ∆ θ c 2 π P c ( h 1 , h 2 | r 0 ) , (21) and similar ly for other quantities such as P ( h 1 | r 0 ), etc. While it is in principle po ssible to find the ∆ θ c analyti- cally using geometry , in pra ctice it is m uch mor e efficient to do this n umerically . Case h 1 , h 2 P ( h 1 | r 0 ) P ( h 2 | r 0 ) P ( { h 1 , h 2 }| r 0 ) 1 , 1 0 0 0 s 0 1 , 0 0 1 0 0 , 1 1 0 0 0 , 0 1 1 1 1 , 1 a 0 0 s 0 s 1 1 , 0 a 1 a 0 , 1 1 − a 0 0 0 , 0 1 − a 1 1 − a 1 , 1 0 a 0 s 0 s 2 1 , 0 0 1 − a 0 0 , 1 1 a a 0 , 0 1 1 − a 1 − a T ABLE I I: Co nditional probab ilities for cases uniqu e to the tw o dimensional example in Fig. 3. The cases s 1 s 0 s 2 , s 0 s 1 s 2 , and s 0 s 2 s 1 (not shown), are identical to those of the one di- mensional problem given in T able I. As sho wn in Figure 3, the case s 0 corresponds to th e range of angles for which it is imp ossible fo r either searc her to detect a particle. Case s 0 s 1 corresponds to the range of angles for whic h only searcher one has a c hance of detecting a particle, and case s 0 s 1 corre- sp on d s to t h e range of angles for which only searcher tw o has a c hance of detectin g a particle. Note that the differen t c ases in the one- dimens io nal example were taken in to ac coun t implicitly dur ing inte- gration as in E q. (19). In the tw o-dimensional exa mple they are taken into account when the effective probabil- it y distributions are computed as a superp osition of the probabilities of the individual cases. This is b ecause ther e is mo r e than one case for each so urce lo cation rela tiv e to the sea r c hers. W e illus tr ate the tw o-dimensional sea rc her pro blem with a setup of sea rc her and source lo cations that illus- trate the sy ner getic and redundant p ositions. W e use the Gaussian prior for the initial guess P ( r 0 ) = A exp − || r 0 − s || 2 σ 2 , (22) 6 UNCLASSIFIED (LA-UR 09-0767 6) where A is the nor malization and σ deter mines the ov er- all shap e o f the distribution. The v ector s = (2 , 2) is the most probable po sition of the source. The quan- tit y R ( h 1 , h 2 , r 0 ) [see E q. (1)] determines whether the searchers ar e po sitioned synergetica lly relative to the source. When the sear chers are per forming an infotaxis search, r ealizing s ynergetic r elative p ositions will in prin- ciple lea d to the fastest reduction in uncertaint y . In Fig. 4 we plot R as a function of the p osition of one searcher r 1 with the po sition o f the se cond s e a rcher r 2 held fixed. W e find for this exa mple that o nly s ynergy ( R < 0) is possible. The light ar e as in Fig. 4 co r resp ond to weak synergy and the dark area s to strong e r synergy . As in the one-dimensional exa mple synergy is stro ngest for large a and when s earcher s 1 is near the sour c e and not b ehind searcher s 2 . A s ma ll c r oss section [ a = 0 . 25 in Fig. 4(a)] allows only for w eak synergy , wherea s a larger cross section [ a = 0 . 75 in Fig. 4(b)] gives muc h s tronger synergy for certain relative p ositions. The strong e st sy n- ergy comes fro m a large cro s s section pair ed with opti- mal p ositioning. The maximum s y nergy is r ealized when searcher s 1 is clo se to the pe ak of P and esp ecially when the p eak o f P is b etw een the se archers, Note that un- fav orable positions (such as r 1 = (5 , 2), be hind sear c her s 2 ) provide minimal synergy regardles s o f the v alue of a . F or real-world problems, larger cros s sections yield more information and therefor e s tr onger syner gy is p os- sible. Howev er for some applicatio ns the cross section of a senso r o n a searcher ma y b e limited by pr actical con- siderations such as weigh t or p o wer consumption. This example shows that synergy is still p ossible for small a . F urthermore, this e xample emphasizes the importa nc e of the pro babilit y es timate of so urce lo cations P ( r 0 ). The infotaxis algorithm is designed such that the searchers will explore, gathering new information, if their ar range- men t is not sufficient ly syne r getic to warrant exploita- tion. This allows the sear c hers to succeed even with no starting informatio n, but they may not realize stro ng syn- ergy until the estimate of P ( r 0 ) is sufficien tly refined. V. CONCLUSION In this work we studied sear ch alg orithms for au- tonomous a g en ts lo o king for the spatial lo catio n of a sto c hastic sourc e . In spatia l search problems, since the exploitation of syner gy r e q uires spa tia l or tempora l co r- relations, we c o nsidered pr oblems in which a so urce emits t wo par ticle s sim ultaneously in opp osite direc tio ns. This is a simplification o f physical problems in whic h there is a r eaction and the pr oducts travel in directions that have angular correlatio ns . W e show ed that both syner gy and redundancy are p ossible for o ne-dimensional sea r c h pro b- lems but not fo r t wo-dimensional searches, where only synergy is po ssible. Since even unfav ora ble arr angements of sear c hers a re syner getic, in tw o-dimensio nal sea r c h problems like these coo rdination is a lw ays a dv a n tageous. 0 1 2 3 4 y ( a ) r 2 S yn e r g e ti c ( R < 0 ) 0 1 2 3 4 5 6 x 0 1 2 3 y ( b ) r 2 S yn e r g e ti c ( R < 0 ) FIG. 4: Synergy for t wo searc hers for tw o-dimensional corre- lated signals. The v alue of R ( h 1 , h 2 , r 0 ) is shown as a function of th e p osition r 1 of searcher s 1 for tw o different v alues of the capture probability a : (a) a = 0 . 25; (b) a = 0 . 75. The p o- sition of searcher s 2 is fixed r 2 = (2 , 4). The most probable source lo cation (th e p eak of t h e Gaussian distribution for the initial source location) is at r 0 = (2 , 2). Darker blue corre- sp on d s to stronger sy nergy ( R < 0). Synergy is strongest for large a and when sea rcher s 1 is near the source and n ot b ehind searcher s 2 relativ e to the source. Simple exa mples such as these that can be studied an- alytically provide insight into real- w orld problems. In real-world problems, there will necessa r ily b e additional consideratio ns. It may not always b e p ossible to write a closed- fo rm equation for the nature o f the corr elations in the signal from the s ource. It may b e neces sary to di- rectly measure any correla tions and us e this to estimate capture probabilities. F urthermore, v arious probabilities may no t b e stationary in time or the signals from the source may get prog r essively w eaker. F or example, sig - nals from a radio tra nsmitter may decrea se in streng th ov er time as its batteries ar e slowly ex ha usted. An a ddi- tional cons ideration is that in the real-world, communica- tion b etw een a gen ts may only be pos sible at certa in times and may not b e instantaneous as in our simple examples. These general consideratio ns are cr ucial for the exploita- tion of multi-agen t info ta xis in terms o f the design of optimal collective algor ithms in particular applications . The next steps for ma king this a ppr oach applicable to a broader class of problems, including those not limited to spatial sear ches[3], are to gener alize the results to more than tw o sear c hers and to explore how synergy may b e bes t leveraged to giv e increases in search sp eed and effi- ciency . 7 UNCLASSIFIED (LA-UR 09-0767 6) Ackno wledgments This work was supp orted in pa rt by a DCI IC Post- do ctoral F ellowship and the Department of Ener g y at Los Alamos National Lab oratory under co n tract DE-AC52- 06NA2539 6 throug h the Lab orator y Direc ted Resea rch and Developmen t Progra m. This is published under re- lease LA-UR 09 -07676 . [1] A. K. Agogino and K. T umer. Analyzing and visualizing multia gent rewards in dynamic and sto c hastic domains. Aut onomous A gent s and Multi-A gent Systems , 17(2):320, 2008. [2] J. M. Bernardo and A. F. M. Smith. Bayesian The e ge- sory . Wiley , New Y ork, 1994. [3] L. M. A. Bettencourt . The rules of information aggre- gation and emergence of collectiv e intel ligent b eha vior. T opics in Co gnitive Scienc e , 1:598, 2009. [4] L. M. A. Bettencourt, V. Gintautas, and M. I. Ham. Iden- tification of functional in formation subgraphs in complex netw orks. Phys. R ev. L ett. , 100:238 701, 2008. [5] L. M. A. Bettencourt, G. J. Stephen s, M. I. Ham, and G. W. Gross. F unctional structu re of cortical neuronal netw orks grow n in vitro. Phys. R ev. E , 75:021915, 2007. [6] T. M. Co ver and J. A. Thomas. El ements of I nformation The ory . Wiley , New Y ork, 1991. [7] M. Eisen bac h and J. W. L en geler. Chemotaxis . Imp erial College Press, London, 2004. [8] S. Fine, R. Gilad-Bachrac h, and E. S hamir. Query by committee, linear separation and rand om w alks. The or. Comput. Sci. , 284(1):25, 2002. [9] J. F o x, D. Glasspo ol, and J. Bury . Qu an titative and qualitative approaches to reas oning un d er un certainty in medical decision making. I n Artificial Intel ligenc e i n Me dicine: L e ctur e Notes i n Computer Scienc e , volume 2101, pages 272–282 . Springer, Berlin/Heidelb erg, Ger- many , 2001. [10] Y. F reund, E. Shamir, and N. Tishb y . Selectiv e sampling using th e query by committee algorithm. In Machine L e ar ning , page 133, 1997 . [11] V. Gintautas, A . Hagberg, and L. M. A. Bettencourt. When is so cial computation b etter than th e sum of its parts? In H. Liu, J. J. Salerno, and M. J. Y oung, ed itors, So cial Computing, Behavior Mo deling, and Pr e diction , page 93, 2009. [12] G. Hollinger, S. Singh, J. Djugash, and A. Kehagias. Ef- ficient m ulti-rob ot search for a moving target. The I n- ternational Journal of R ob otics R es e ar ch , 28(2):201–219, 2009. [13] R. Playter, M. Buehler, and M. Raib ert. Bigdog. In G. R. Gerhart, C. M. S hoemaker, and D. W. Gage, editors, Pr o c . SPIE: Unm anne d Systems T e chno lo gy VIII , volume 6230, 2006. [14] R. A. Russell, A. Bab-Hadiashar, R . L. S hepherd, and G. G. W all ace. A compari son of reactive rob ot c hemo- taxis algorithms. R ob ot ics and Auton omous Systems , 45(2):83, 2003. [15] E. S c hneidman, W. Bialek, and M. J. Berry I I. Synergy , redundancy , and indep endence in p opulation co des. J. Neur osci. , 23:1 1539, 2003. [16] H. S. Seung, M. Opp er, an d H. Sompolinsky . Query by committee. In C OL T ’92: Pr o c e e dings of the fifth annual workshop on C omputat ional le arning the or y , page 287, 1992. [17] R. S. Sulton and A. G. Barto. R einfor c ement le arning: an intr o duction . MIT Press, Cam brigde MA, 1998. [18] M. V ergasso la, E. V illermaux, and B. I. Shraiman. “In- fotaxis” as a strategy for searching without gradients. Natur e , 445: 406, 20 07. [19] W. Zhang, Z. Deng, G. W ang, L. Wittenburg, and Z. Xing. Distributed prob lem solving in sensor net- w orks. In AAMAS ’02: Pr o c e e dings of the first interna- tional j oint c onfer enc e on A utonomous agents and mul - tiagent systems , pages 988–989, New Y ork, NY, U SA, 2002. AC M. 8