Unspecified distribution in single disorder problem

Unsp ecified distribu tion in single diso rder problem W o jciec h Sarnowski a , ∗ Krzysztof Sza jowski a a Wr o c law University of T e ch nolo gy, Institute of Mathematics and Computer Scienc e, Wyb rze ˙ ze W yspia´ nskie go 27, 50-370 Wr o c law, Poland Abstract W e register a sto c hastic sequence affected by one disorder. Monitoring of the se- quence is made in the circumstances when not fu ll information ab out distributions b efore and after the c h ange is a v ailable. T he initial problem of disorder detection is transformed to optimal stopping of ob s erv ed sequence. F ormula for optimal decision functions is derive d. Keyw ords. Disorder problem, sequen tial detection, optimal stopping, Mark ov pro- cess, change p oin t. 1 In tro duction The pap er is fo cused on sequen tial detection using Ba yesia n approac h. Disorder problem in this framework wa s formulate d b y A.N. Kolmogoro v at the end of 50’s of previous cen tur y and solv ed by [Shiry aev(19 61)]. The next turning p oin t is pap er b y [P eskir an d Sh iry aev(20 02)] w here authors p ro vide complete solution of basic prob- lem. F rom this time man y publications p ro vide new solutions and generaliz ations in the area of sequentia l detectio n. Some of them are articles by [Karatzas(20 03) ] and [Ba yraktar et al.(2005) Ba yraktar, Da y anik, and Karatzas]. F or discrete time case there are some detailed analysis in the pap ers by [Bo jdec ki and Hosza(1984) ], ⋆ 1 Nov em b er 2018 ∗ Corresp ond ing author Email addr esses: W ojciech.S arnowski @pwr.wroc.pl (W o jciec h Sarn owski), Krzyszto f.Szajows ki@pwr.wroc.pl (Krzysztof Sza jo wski). URLs: http:// www.im.pw r.wroc.pl/~sarnowski (W o jciec h Sarnowski), http://n eyman.im. pwr.wroc.pl/~szajow (Krzysztof Sza jo wski). 1 [Moustakides(199 8) ], [Y akir(1994 ) ], [Y oshida(1983) ], [Sza jo wski(1996) ] and the pap ers cited there. Suc h mo del of data ap p ears in many practical p r oblems of the qu alit y control (see Bro dsky and Darkho vsky [Bro d sky and Darkhovsky(1 993)], Sh ewhart [Shewhart(1931 )] and in the collection of the pap ers [Basseville and Ben v en iste(198 6) ]), traffic anoma- lies in net works (in pap ers b y Dub e and Mazumdar [Dub e and Mazumdar(2001)], T artak o vsky et al. [T artak o vsky et al.(2006)T artak o vsky , Rozo vskii, Bla ˇ zek, an d Kim]), epidemiology mo dels (see Baron [Baron(2004)]). I n managemen t of man u facture it happ ens that the plan ts which pro d uce some details c hanges their paramet ers. It mak es that th e details c hange their qualit y . Th e aim is to r ecognize the moments of these changes as so on as p ossible. This pap er fo cuses atten tion on mo dels un d er assumption of uncertaint y ab out dis- tribution b efore or after th e c hange. The example of su c h mo d els can b e found in researc h by [Dub e and Mazumdar(2001)] with application to detection of traffic anomalies in netw orks or in pap er by [Sarnowski and S za jowski(20 08) ]. T h e solu- tion of a single disord er mo del with u nsp ecified distrib ution of observed sequence is present ed. Section 2 sp ecifies the details of inv estigated mo del. The transformation of the optimization job to the optimal stopping problem for the sp ecific sto c hastic pro cess is co nsidered in Section 3. A c onstruction o f the optimal estimator of th e disorder moment is given in Section 4. T echnical parts of inv estigations are mo v ed to App endix. 2 Description of the model 2.1 Basic notations F or f u rther considerations it will b e conv enien t to in tro duce the follo wing notation whic h will mak e our formulas more compact and clear x k ,n = ( x k , x k +1 , ... , x n − 1 , x n ) , k ≤ n, L i,j m ( x k ,n ) = n − m Y r = k +1 f 0 ,i x r − 1 ( x r ) n Y r = n − m +1 f 1 ,j x r − 1 ( x r ) , A k ,n = A k × A k +1 × . . . × A n , where: Q m 2 r = m 1 u r = 1 for m 1 > m 2 and u r ∈ ℜ , A i ∈ B , k ≤ i ≤ n . It will b e con ve nien t to write β = ( β 1 , β 2 ) and denote b y α = ( α 11 , . . . , α 1 l 1 , . . . , α l 0 1 , . . . , α l 0 l 1 ) 2 an y matrix l 0 × l 1 :           α 11 α 12 · · · α 1 l 1 α 21 α 22 · · · α 2 l 1 . . . . . . . . . . . . α l 0 1 α l 0 2 · · · α l 0 l 1           In consequen ce ve ctors π , b , p represent: π = ( π 11 , . . . , π 1 l 2 , . . . , π l 0 1 , . . . , π l 0 l 1 ) b = ( b 11 , . . . , b 1 l 2 , . . . , b l 0 1 , . . . , b l 0 l 1 ) p = ( p 11 , . . . , p 1 l 1 , . . . , p l 0 1 , . . . , p l 0 l 1 ) W e need also n otation for v ector of densities f 0 ,i x ( y ). Let b f 0 x ( y ), where x, y ∈ E stands b ehind: b f 0 x ( y ) = ( f 0 , 1 x ( y ) , . . . , f 0 , 1 x ( y ) | {z } l 1 times , . . . , f 0 ,l 0 x ( y ) , . . . , f 0 ,l 0 x ( y ) | {z } l 1 times ) . Moreo ve r let us int ro duce op eration ” ◦ ”. F or vecto rs α and β w e pu t: α ◦ β = ( α 11 β 11 , . . . , α 1 l 1 β 1 l 1 , . . . , α l 0 1 β l 0 1 , . . . , α l 0 l 1 β l 0 l 1 ) . 2.2 Change p oint pr oble m Let ( X n ) n ∈ N b e sequence of observ able ran d om v ariables defin ed on (Ω , F , P ) with v alue in ( E , B ), E ⊂ ℜ . Sequence ( X n ) generates filtration F n = σ ( X 0 , X 1 , ..., X n ). On the same sp ace th er e are also d efined v ariables θ , β 1 and β 2 . θ tak es v alues in { 1 , 2 , 3 , . . . } . V ariables β 1 , β 2 are v alued in I k = { 1 , 2 , . . . , l k } , where l k ∈ N , k = 0 , 1. Let us assume the f ollo wing parametrization: P ( β 1 = i, β 2 = j ) = b ij P ( θ = n | β 1 = i, β 2 = j ) =      π ij , if n = 1, (1 − π ij ) p n − 2 ij q ij , if n > 1, 3 where i ∈ I 0 , j ∈ I 1 , P i ∈ I 0 ,j ∈ I 1 b ij = 1, b ij ≥ 0, π ij ∈ [0 , 1], p ij = 1 − q ij ∈ (0 , 1). W e ha v e ∞ X k =1 X i ∈ I 0 X j ∈ I 1 P ( θ = k , β 1 = i, β 2 = j ) = 1 The c hange of the conditional densities in rand om moment θ is in v estigate in this mo del. Th e transfer b et ween distrib ution is describ ed b y conditional p robabilities b ij = P ( β 2 = j | β 1 = i ). F or completeness it will b e assumed that the state of β 1 is stable b efore θ and the sa me as at the moment 0. The marginal distrib ution of θ has a form P ( θ = k ) = X i,j P ( θ = k , β 1 = i, β 2 = j ) =      P i,j π ij · b ij if k = 1, P i,j (1 − π ij ) p k − 2 ij q ij b ij if k > 1. The observed sequence has a form X n = X 0 ,i n · I { θ>n , β 1 = i } + X 1 ,j n · I { θ ≤ n , β 2 = j ,X 1 ,j θ − 1 = X 0 ,i θ − 1 } , (1) where ( X r,i n , G r,i n , P r,i x ), r = 0 , 1, are Mark o v pro cesses a nd σ -fields: G r,i n = σ ( X r,i 0 , X r,i 1 , . . . , X r,i n ), with i ∈ I 0 , j ∈ I 1 , r = 0 , 1 and n ∈ { 0 , 1 , 2 , . . . } . V ariables θ , β 1 and β 2 are n ot measurable w.r.t F n . On th e space ( E , B ) there are σ -additiv e measur es µ ( · ) and measures µ • , • x absolutely con tinuous w ith resp ect to µ . I t is assumed that the measures P k ,i x ( · ), i = 1 , 2 , . . . , l k , k = 0 , 1, h a ve follo wing representat ion: P k ,i x ( { ω : X k ,i 1 ∈ B } ) = P ( X k ,i 1 ∈ B | X k ,i 0 = x ) = Z B f k ,i x ( y ) µ ( dy ) = Z B µ k ,i x ( dy ) = µ k ,i x ( B ) . for an y B ∈ B . The conditional densities f k , 1 x ( · ) , . . . , f k ,l k x ( · ) are differen t and su p- p orts of all measures µ · , · x are there same for giv en x ∈ E . It is the mod el of the follo w- ing random ph enomenon. A t the b eginning w e register pro cess { X 0 ,i n , n ∈ N } , where i ∈ I 0 is unk n o w n . A t random moment θ initial pro cess is switc h ed on { X 1 ,j n , n ∈ N } where j ∈ I 1 is unkn o wn . It can b e in terpreted as disord er of { X n , n ∈ N } causing c h ange in distribution of { X n } n ∈ N . W e monitor the p ro cess and w e wish to detect the change as close θ as p ossible. Ho w ev er our knowledge ab out densities b efore and after the c hange moment θ is limited generally to the information ab out sets of p os- 4 sible conditional densitie s only: { f 0 ,i x ( y ) , i ∈ I 0 } and { f 1 ,j x ( y ) , j ∈ I 1 } resp ectiv ely . W e also kno w pr obabilities of distribu tion pairs b ij and p arameters π ij . F or i ∈ I 0 , j ∈ I 1 let us introd uce fun ctions Ψ i,j , e Ψ i,j , Λ i,j , e Λ i,j defined on the pro du ct N × ( × l +2 i =1 E ) × [0 , 1] with v alues in ℜ : Ψ i,j ( l, x 0 ,l +1 , α ) = (1 − α ) " q ij l X k =0 p l − k ij L i,j k +1 ( x 0 ,l +1 ) + p l +1 ij L i,j 0 ( x 0 ,l +1 ) # + αL i,j l +1 ( x 0 ,l +1 ) (2) e Ψ i,j ( l, x 0 ,l +1 , α ) = (1 − α ) " q ij l X k =1 p l − k ij L i,j k ( x 0 ,l +1 ) + p l ij L i,j 0 ( x 0 ,l +1 ) # + αL i,j l +1 ( x 0 ,l +1 ) , (3) Λ i,j ( l, x 0 ,l +1 , α ) = Ψ i,j ( l, x 0 ,l +1 , α ) − (1 − α ) p l +1 ij L i,j 0 ( x 0 ,l +1 ) , e Λ i,j ( l, x 0 ,l +1 , α ) = e Ψ i,j ( l, x 0 ,l +1 , α ) − (1 − α ) p l ij L i,j 0 ( x 0 ,l +1 ) . Next let us define on N × ( × k +2 i =1 E ) × ( × l 1 l 2 i =1 [0 , 1]) × ( × l 1 l 2 i =1 [0 , 1]) function S, e S : S ( k , x 0 ,k +1 , γ , δ ) = X i,j γ ij Ψ i,j ( k , x 0 ,k +1 , δ ij ) , (4) e S ( k , x 0 ,k +1 , γ , δ ) = X i,j γ ij f Ψ i,j ( k , x 0 ,k +1 , δ ij ) . (5) F or an y D n = { ω : X i ∈ B i , i = 1 , 2 , . . . , n } , where B i ∈ B and an y x ∈ E define: P x ( D n ) = P ( D n | X 0 = x ) = Z × n i =1 B i e S ( n − 1 , x 0 ,n , b, π ) µ ( d x 1 ,n ) F or the pro cess (1 ) the set of estimators for the dis order momen t θ is S X – the set of stopping times with resp ect to {F n } n ∈ N ∪{ 0 } . The construction of the optimal estimator is to find a stopping time τ ∗ ∈ S X suc h tha t f or an y x ∈ E P x ( | θ − τ ∗ | ≤ d ) = sup τ ∈ S X P x ( | θ − τ | ≤ d ) , (6) where d ∈ { 0 , 1 , 2 , ... } is fixed lev el of detection precision. 3 Existence of solution In this section w e are going t o sho w that there exists solution o f the problem (6). Let us define: 5 Z n = P ( | θ − n | ≤ d | F n ) , n = 1 , 2 , . . . , V n = ess sup { τ ∈ S X , τ ≥ n } P ( | θ − n | ≤ d | F n ) , n = 0 , 1 , 2 , . . . τ 0 = inf { n : Z n = V n } (7) Notice that, if Z ∞ = 0, then Z τ = P ( | θ − τ | ≤ d | F τ )for τ ∈ S X . Because F n ⊆ F τ (when n ≤ τ ), w e obtain V n = ess sup τ ≥ n P ( | θ − τ | ≤ d | F n ) = ess sup τ ≥ n E ( E ( I {| θ − τ |≤ d } |F τ ) | F n ) = ess sup τ ≥ n E (Z τ | F n ) The f o llo wing lemma states that solution exists. Lemma 1 S topping time τ 0 given by (7) is a solution of the pr obl e m (6). PR OOF. Applying Theorem 1 fro m [Bo jdec ki(197 9)] it is enough to sho w that lim n →∞ Z n = 0. F or all n, k , where n ≥ k w e ha v e: Z n = E ( I {| θ − n |≤ d } | F n ) ≤ E (sup j ≥ k I {| θ − j |≤ d } | F n ) Basing on L evy’s the orem w e get lim sup n →∞ Z n ≤ E (sup j ≥ k I {| θ − j |≤ d } | F ∞ ) where F ∞ = σ ( S ∞ n =1 F n ). W e hav e: lim sup j ≥ k, k →∞ I {| θ − j |≤ d } = 0 a.s. Basing on do minated conv ergence theorem w e get we state that lim k →∞ E (sup j ≥ k I {| θ − j |≤ d } | F ∞ ) = 0 a.s. what ends the pro of. It turns out that we need at least d observ ations to detect disorder in optimal w ay: Lemma 2 L e t τ b e stopp i n g rule in the pr oblem (6). Then rule ˜ τ = max( τ , d + 1) is at le ast as go o d as τ (in the sense of ( 6)). PR OOF. F o r τ ≥ d + 1 the rules are the same. Let us consider case when τ < d + 1. Then ˜ τ = d + 1 and: 6 P ( | θ − τ | ≤ d ) = P ( τ − d ≤ θ ≤ τ + d ) = P (1 ≤ θ ≤ τ + d ) ≤ P (1 ≤ θ ≤ 2 d + 1 ) = P ( ˜ τ − d ≤ θ ≤ ˜ τ + d ) = P ( | θ − ˜ τ | ≤ d ) . 4 Construction of the d isorder momen t estimator 4.1 F unction and pr o c e sses Let us fix parameters π , b a nd set initial state of X n : P ( X 0 = x ) = 1. W e denote ϕ = ( π , b, x ) and w e will write P ϕ ( • ) to emphas is that the pro babilit y of the even ts defined by the pro cess a r e dep enden t on this a priori set parameters. Let us define t he following crucial p osterior pro cesses : Π i,j n = P ϕ ( θ ≤ n | β = ( i, j ) , F n ) = P ϕ ( θ ≤ n | ˜ F ij n ) (8) B i,j n = P ϕ ( β = ( i, j ) |F n ) (9) where n ∈ N , i ∈ I 0 , j ∈ I 1 , ˜ F i,j n = σ ( F n , I { β =( i,j ) } ). Pro cess Π i,j n is designed for up dating information a b out disorder distribution. B i,j n in turn refreshes information ab out distributions of v aria bles β 1 , β 2 . No t ice t ha t Π i,j n , B i,j n starts from f ollo wing states: Π i,j 0 = 0, B i,j 0 = b ij Dynamics of Π i,j n and B i,j n are c hara cterized by form ulas (A.10), (A.11). The ab ov e not a tions hold also fo r (8), (9): Π n =  Π 1 , 1 n , . . . , Π 1 ,l 2 n , . . . , Π l 1 , 1 n , . . . , Π l 1 l 2 n  , B n =  B 1 , 1 n , . . . , B 1 ,l 2 n , . . . , B l 1 , 1 n , . . . , B l 1 l 2 n  . A t the end of section let us define auxiliary functions Π · , · ( · , · , · ), Γ · , · ( · , · , · , · ). F or x, y ∈ E , α , γ ij , δ ij ∈ [0 , 1], i ∈ I 0 , j ∈ I 1 put: Π i,j ( k , x 0 ,n , α ) = Λ i,j ( k , x 0 ,n , α ) Ψ i,j ( k , x 0 ,n , α ) (10) Γ i,j ( k , x 0 ,n , γ , δ ) = γ ij Ψ i,j ( k , x 0 ,n , δ ij ) S ( k , x 0 ,n , γ , δ ) . (11) Let D n = { ω : X 0 ,n ∈ B 0 ,n } , X 0 = x and B i ∈ B . W e hav e 7 P ϕ ( θ > n, β = ( i, j ) , D n ) = Z { ω : β =( i,j ) ,D n } I { θ>n } d P ϕ (12) = Z B 0 ,n (1 − π ij ) p n − 1 ij L ij 0 ( x 0 ,n ) S i,j n ( x 0 ,n ) b ij S i,j n ( x 0 ,n ) S n ( x 0 ,n ) S n ( x 0 ,n ) µ ( dx 1 ,n ) = Z D n (1 − Π i,j n ) B i,j n d P ϕ , where S i,j n ( x 0 ,n ) = π ij L i,j n ( x 0 ,n ) + (1 − π ij ) p n − 1 ij L i,j 0 ( x 0 ,n ) (13) + (1 − π ij ) n X s =2 p s − 2 ij q ij L i,j n − s +1 ( x 0 ,n ) = Ψ i,j ( n − 1 , x 0 ,n , π ij ) and S n ( x 0 ,n ) = P i,j b ij S i,j n ( x 0 ,n ) = S ( n − 1 , x 0 ,n , ¯ b, ¯ π ). 5 Solution According to Shirya y ev’s metho dology (see [Shiry aev(1978) ] ) w e are go ing to find s olution red ucing initia l problem (6) to the case of stopping Random Mark ov F unction with sp ecial pay off function. This will b e done using p osterior pro cesses (8)-(9). Lemma 3 F or n ≥ d + 1 P ϕ ( | θ − n | ≤ d ) =                E ϕ h h ( X n − 1 − d,n , Π n , B n ) i , if n > d + 1 , E ϕ h e h ( Π d +1 , B d +1 ) i , if n = d + 1 . (14) wher e h ( x 1 ,d +2 , γ , δ ) = X i,j   1 − p d ij + q ij d +1 X k =1 L i,j k ( x 1 ,d +2 ) p k ij L i,j 0 ( x 1 ,d +2 )   (1 − γ ij ) δ ij , (15) e h ( γ , δ ) = X i,j  1 − p d ij (1 − γ ij )  δ ij , (16) x 1 , ..., x d +2 ∈ E , γ ij , δ ij ∈ [0 , 1] , i ∈ I 0 , j ∈ I 1 . 8 PR OOF. Let us rewrite initial criterion as exp ectation: P ϕ ( | θ − n | ≤ d ) = E ϕ [ P ϕ ( | θ − n | ≤ d | F n )] . (17) Let us analyze conditional proba bilit y under expectatio n in equation (17) using total proba bilit y formula P ϕ ( | θ − n | ≤ d | F n ) = P ϕ ( θ ≤ n + d | F n ) − P ϕ ( θ ≤ n − d − 1 | F n ) (18) = X i,j Π ij n + d B i,j n − X i,j Π ij n − d − 1 B i,j n , b ecause P ϕ ( θ ≤ n + d |F n ) = E ϕ ( I θ ≤ n + d |F n ) = X i,j E ϕ ( I { θ ≤ n + d } I { β =( i,j ) } |F n ) = X i,j E ϕ ( E ϕ ( I { θ ≤ n + d } I { β =( i,j ) } | ˜ F i,j n ) |F n ) = X i,j E ϕ ( I { β =( i,j ) } E ϕ ( I { θ ≤ n + d } | ˜ F i,j n ) |F n ) = X i,j E ϕ ( I { θ ≤ n + d } | ˜ F i,j n ) E ϕ ( I { β =( i,j ) } |F n ) . The la st equalit y is a consequence of the ve ry sp ecial form of the extended σ -field ˜ F i,j n . The random v aria ble measurable with res p ect to ˜ F i,j n is also F n measurable. Putting n = d + 1 in Lemma 5 w e get P ϕ ( θ ≤ n − d − 1 | F n , β = ( i, j )) = 0, for i ∈ I 0 , j ∈ I 1 . Hence P ϕ ( | θ − n | ≤ d | F n ) = X i,j P ϕ ( θ ≤ n + d | ˜ F n ) P ϕ ( β = ( i, j ) | F n ) . Lemma 4 implies that P ϕ ( | θ − n | ≤ d ) = E ϕ h e h ( Π d +1 , B d +1 ) i . Now let n > d + 1. Basing o n Lemma 4 probabilit y P ϕ ( θ ≤ n + d | ˜ F n ) is giv en b y (A.3). F rom Lemma 5 w e kno w t ha t P ϕ ( θ ≤ n − d − 1 | ˜ F n ) is expressed by equation ( A.7 ) . F orm ula (A.7) rev eals connection b et we en pa y off function (1 4) and posterior pro cess at instan ts n and n − d − 1, i.e Π i,j n , Π i,j n − d − 1 for i ∈ I 0 , j ∈ I 1 . Dep endence on Π i,j n − d − 1 can b e rule out by expressing Π i,j n − d − 1 in terms o f Π i,j n . By Lemma 6 and (A.9 ) w e g et 9 Π i,j n − d − 1 = " ( q ij − Π i,j n ) d X k =0 p d − k ij L i,j k +1 ( X n − d − 1 ,n ) − Π i,j n p d +1 ij L i,j 0 ( X n − d − 1 ,n ) # (19) × " (1 − Π i,j n )  q ij d X k =0 p d − k ij L i,j k +1 ( X n − d − 1 ,n ) − L i,j d +1 ( X n − d − 1 ,n )  − Π i,j n p d +1 ij L i,j 0 ( X n − d − 1 ,n ) # − 1 The result (19) a nd for mula (A.7) lead us to: P ϕ ( θ ≤ n − d − 1 | F n , β = ( i, j )) (20) = p d +1 ij L i,j 0 ( X n − d − 1 ,n )Π i,j n − q ij P d k =0 p d − k ij L i,j k +1 ( X n − d − 1 ,n )(1 − Π i,j n ) p d +1 ij L i,j 0 ( X n − d − 1 ,n ) . Applying equations (A.3) and (20) in for mula (18) w e get the thesis. Notice that for n ≥ d + 1 function h under exp ectation in (14) dep ends on pro cess η n = ( X n − d − 1 ,n , Π n , B n ). It turns out that { η n } is Mark o v R a ndom F unction (see Lemma 8 in App endix A). W e do not care about { η n } for n < d + 1. It is a consequence of discussion in Lemma 2 whic h leads to the conclusion that under the considered pay off function (criterion) it is not o ptimal to stop b efore instan t d + 1. The decision mak er can start his decision based on at least d + 1 observ ations X 1 , . . . , X d +1 . Lemmata 3 and 8 imply that initial problem can b e reduced to the optimal stopping of Mark o v Random F unction ( η n , F n , P ϕ y ) ∞ n =1 , where y = ( x n − d − 1 ,n , γ , δ ) ∈ Ξ = E d +2 × [0 , 1] l 1 l 2 × [0 , 1] l 1 l 2 with pay off describ ed b y (15) . Ho w ev er, the new problem is no longer homogeneous one as it is emphasized by the definition of y . It is a conse quence of the fact that the pro cess { η n } for n < d + 1 has formally differen t structure than for n ≥ d + 1. Th us, the pa y offs for instances n ≤ d + 1 are differen t. Lemma 8 give s a justification to w ork with the ho- mogeneous part of the pro cess in construction the optimal estimator of the disorder momen t. T o solv e the maximization problem (14), for any Borel function u : Ξ − → ℜ let us define o p erators: 10 T u ( x 1 ,d +2 , γ , δ ) = E ϕ ( x 1 ,d +2 , γ ,δ ) h u ( X n − d,n +1 , Π n +1 , B n +1 ) i = E ϕ h u ( X n − d,n +1 , Π n +1 , B n +1 ) | ( X n − d − 1 ,n , Π n , B n ) = ( x 1 ,d +2 , γ , δ ) i , Q u ( x 1 ,d +2 , γ , δ ) = max { u ( x 1 ,d +2 , γ , δ ) , T u ( x 1 ,d +2 , γ , δ ) } . Op erators T and Q act on function h and they determine the shap e of optimal stopping rule τ ⋆ . Recursiv e fo r mulas are giv en b y Lemma 9, whic h is presen ted in App endix A. Lemm a 9 c hara cterizes structure of sequence of functions s k ( x 1 ,d +2 , γ , δ ), where x ∈ E d +2 , γ , δ ∈ [0 , 1] l 1 l 2 , which is use d in the the orem stated b elow. Theorem 1 T he solution of pr o b lem (6) is t he fol lowin g stopping rule: τ ∗ =            inf  n ≥ d + 2 : ( X n − 1 − d,n , Π n , B n ) ∈ D ⋆  , if e h (Π d +1 , B d +1 ) < s ∗ ( X 1 ,d +2 , Π d +2 , B d +2 ) , d + 1 , if e h (Π d +1 , B d +1 ) ≥ s ∗ ( X 1 ,d +2 , Π d +2 , B d +2 ) , (21) wher e the stoppi n g ar e a D ⋆ : D ⋆ =    ( x 1 ,d +2 , γ , δ ) ∈ Ξ : h ( x 1 ,d +2 , γ , δ ) ≥ s ∗ ( x 1 ,d +2 , γ , δ )    , and s ∗ ( x 1 ,d +2 , γ , δ ) = lim k − →∞ s k ( x 1 ,d +2 , γ , δ ) . PR OOF. F irst let us consider subproblem of finding t he optimal rule e τ ⋆ ∈ F X d +2 : E ϕ h h ( X e τ ⋆ − d − 1 , e τ ⋆ , Π e τ ⋆ , B e τ ⋆ ) i = sup τ ∈ F X d +2 E ϕ h h ( X τ − d − 1 ,τ , Π τ , B τ ) i . (22) Then, basing on Lemmata 1, 2 and according to o pt ima l stopping theory (see [Shiry aev(1978 )]) it is kno wn that τ 0 defined by (7) can b e express ed as τ 0 = inf { n ≥ d + 2 : h ( X n − 1 − d,n , Π n , B n ) ≥ h ∗ ( X n − 1 − d,n , Π n , B n ) } , where h ∗ ( x 1 ,d +2 , γ , δ ) = lim k − →∞ Q k h ( x 1 ,d +2 , γ , δ ). The limit exists acc ording to the Leb esgue’s theorem and structure of functions h and s k . Lemma 9 implies that : 11 τ 0 = inf n n ≥ d + 2 : h ( X n − 1 − d,n , Π n , B n ) ≥ max n h ( X n − 1 − d,n , Π n , B n ) , s ∗ ( X n − 1 − d,n , Π n , B n ) oo = inf n n ≥ d + 2 : h ( X n − 1 − d,n , Π n , B n ) ≥ s ∗ ( X n − 1 − d,n , Π n , B n ) o . According to optimalit y principle rule e τ ∗ solv es maximization problem of (14) if only at n = d + 1 the pay off e h will b e smaller than exp ected pay off in success iv e perio ds (for n > d + 1). Th us, another w ords: τ ⋆ = e τ ∗ , if e h ( X 0 ,d +1 , Π d +1 , B d +1 ) < s ∗ ( X 1 ,d +2 , Π d +2 , B d +2 ). (23) In opp osite case τ ⋆ = d + 1. This ends the pro of of form ula (21). 5.1 A cknow le dgements W e ha ve b enefited from the remarks of a non ymous r eferee of submitted presen- tation to the Program Committee of IWSM 2009. He has pro vided numerous corrections to the man uscript. A Lemmata In appendix w e presen t useful form ulae and lemmata whic h help to obta in solution o f problem (6). Remark 1 F or n ≥ l ≥ 0 , k > 0 , i ∈ I 0 , j ∈ I 1 , on the s e t { ω : X 0 ,l ∈ A 0 ,l , A 0 = { x } , A i ∈ F i , i ≤ l } the fol lowin g e quations hold: P ϕ ( θ = n + k | X 0 ,l ∈ A 0 ,l , β = ( i, j ) , θ > n ) =                p k − 1 ij q ij , if n, k > 0 , π ij , if n = 0 , k = 1 , (1 − π ij ) p k − 2 ij q ij , if n = 0 , k > 1 , Remark 2 (1) The simple c ons e quenc e of the formula (A.1) we get 12 P ϕ ( θ > n + k | X 0 ,l ∈ A 0 ,l , β = ( i, j ) , θ > n ) =        p k ij , if n, k > 0 , (1 − π ij ) p k − 1 ij , if n = 0 , k > 0 . (A.1) (2) F ormula (A.1) for k = 1 is given by: P ϕ ( θ 6 = n + 1 | X 0 ,l ∈ A 0 ,l , β = ( i, j ) , θ > n ) = P ϕ ( θ > n + 1 | X 0 ,l ∈ A 0 ,l , β = ( i, j ) , θ > n ) =        p ij , if n > 0 , (1 − π ij ) , if n = 0 . (A.2) Lemma 4 F or n > 0 , k ≥ 0 , i ∈ I 0 , j ∈ I 1 the fol lowing e q uation is satisfie d: P ϕ ( θ ≤ n + k | ˜ F n ) = 1 − p k ij (1 − Π i,j n ) . (A.3) PR OOF. W e are going to sho w equalit y on the se t { ω : X 0 ,n ∈ B 0 ,n , B 0 = { x }} P ϕ ( θ > n + k , X 0 ,n ∈ B 0 ,n , β = ( i, j )) = Z { ω : X 0 ,n ∈ B 0 ,n ,β =( i,j ) } I { ω : θ> n + k } d P ϕ (A.4) = Z { ω : X 0 ,n ∈ B 0 ,n ,β =( i,j ) } E ϕ ( I { ω : θ> n + k } | ˜ F n ) d P ϕ = Z { ω : X 0 ,n ∈ B 0 ,n } E ϕ ( I { ω : β =( i,j ) } E ϕ ( I { ω : θ> n + k } | ˜ F n ) |F n ) d P ϕ . By direct computation calculatio n we get P ϕ ( θ > n + k , X 0 ,n ∈ B 0 ,n , β = ( i, j )) (A.5) = Z B 0 ,n ∞ X s = n + k +1 (1 − π ij ) q ij b ij p s − 2 ij L ij 0 ( x 0 ,n ) µ ( dx 0 ,n ) = p k ij Z B 0 ,n (1 − π ij ) b ij p n − 1 ij L ij 0 ( x 0 ,n ) S i,j n ( x 0 ,n ) S i,j n ( x 0 ,n ) S n ( x 0 ,n ) S n ( x 0 ,n ) µ ( dx 0 ,n ) = p k ij Z { ω : X 0 ,n ∈ B 0 ,n } I { ω : θ> n } I { ω : β =( i,j ) } d P ϕ = p k ij Z { ω : X 0 ,n ∈ B 0 ,n } (1 − Π i,j n ) B i,j n d P ϕ = p k ij Z { ω : X 0 ,n ∈ B 0 ,n } P ϕ ( β = ( i, j ) | F n ) P ϕ ( θ > n | ˜ F n ) d P ϕ (A.6) 13 Henceforth w e ha ve E ϕ ( I { ω : β =( i,j ) } E ϕ ( I { ω : θ> n + k } | ˜ F n ) |F n ) = p k ij P ϕ ( β = ( i, j ) |F n ) P ϕ ( θ > n | ˜ F n ) . Comparison of (A.4) and (A.5) implies A.3 and this ends t he pro of of lemma. Lemma 5 F or n > k ≥ 0 , i ∈ I 0 , j ∈ I 1 it is true that P ϕ ( θ ≤ n − k − 1 | ˜ F n ) = 1 − (1 − Π i,j n ) 1 + q ij k +1 X s =1 L i,j s ( X n − s +1 ,n ) p s ij L i,j 0 ( X n − s +1 ,n ! . (A.7) PR OOF. If n = k + 1 then P ϕ ( θ ≤ n − k − 1 | ˜ F n ) = P ϕ ( θ ≤ 0 | ˜ F k +1 ) = 0 . Because of the fact that θ > 0 a.s.: Π i,j n − k − 1 = Π i,j 0 = P ϕ ( θ ≤ 0 | ˜ F 0 ) = 0 . (A.8) Hence for m ula (A.7) ho lds. The case where n > k + 1 w e ha v e P ϕ ( θ > n − k − 1 | ˜ F n ) = P ϕ ( θ > n | ˜ F n ) + k +1 X s =1 P ϕ ( θ = n − s | ˜ F n ) . On the set D n = { ω : β = ( i, j ) , X 0 ,n ∈ B 0 ,n , B 0 = { x }} w e hav e P ϕ ( θ = n − s, D n ) = Z D n I { θ = n − s } d P ϕ = Z D n P ϕ ( θ = n − s | ˜ F n ) d P ϕ = Z × n r =1 B r (1 − π ij ) p n − s − 2 ij q ij b ij L i,j s +1 ( x n − s,n ) dµ ( x 0 ,n ) = Z × n r =1 B r q ij L i,j s +1 ( x n − s,n ) p s +1 ij L i,j 0 ( x n − s,n ) (1 − π ij ) p n − 1 ij b ij L i,j 0 ( x 0 ,n ) S i,j n ( x 0 ,n ) S i,j n ( x 0 ,n ) S n ( x 0 ,n ) S n ( x 0 ,n ) dµ ( x 0 ,n ) = p k ij Z { ω : X 0 ,n ∈ B 0 ,n } q ij L i,j s +1 ( X n − s,n ) p s +1 ij L i,j 0 ( X n − s,n ) (1 − Π ij n ) B i,j n d P ϕ . Therefore P ϕ ( θ = n − s | ˜ F n ) = q ij L i,j s +1 ( X n − s,n ) p s +1 ij L i,j 0 ( X n − s,n ) (1 − Π i,j n ) 14 and P ϕ ( θ > n − k − 1 | ˜ F n ) =   1 + q ij k X s =0 L i,j s +1 ( X n − s,n ) p s +1 ij L i,j 0 ( X n − s,n )   (1 − Π i,j n ) . Lemma 6 F or n > l ≥ 0 , i ∈ I 0 , j ∈ I 1 fol lowing e quation hold s: Π i,j n =                  Π i,j ( l , X n − l − 1 ,n , Π i,j n − l − 1 ) , if n > l + 1 , e Λ ( l,X n − l − 1 ,n ,π ij ) e Ψ i,j ( l,X 0 ,l +1 ,π ij ) , if n = l + 1 . (A.9) Remark 3 In p articular, taking l = 0 , we get e quation char acterizing ”one- step” dynamic s of the pr o c ess Π i,j n : Π i,j n =          f 1 ,j X n − 1 ( X n )( q ij + p ij Π i,j n − 1 ) f 1 ,j X n − 1 ( X n )( q ij + p ij Π i,j n − 1 )+ f 0 ,i X n − 1 ( X n ) p ij (1 − Π i,j n − 1 ) , if n > 1 , f 1 ,j X 0 ( X 1 ) π ij f 1 ,j X 0 ( X 1 ) π ij + f 0 ,i X 0 ( X 1 )(1 − π ij ) , if n = 1 , (A.10) with initial c on d ition Π i,j 0 = 0 . Remark 4 F or l > 0 r e cursive structur e define d in e quation (A.9) r e q uir es ve ctor of initial states Π i,j 0 , Π i,j 1 , . . . , Π i,j l . State Π i,j 0 is given ab ove. T o obtain r emaining states Π i,j 1 , . . . , Π i,j l it is e n ough to a p ply formula (A.10). PR OOF. ( o f Lemma 6) Condition Π ij 0 = 0 has b een sho wn in lemma 5 (equation (A.8) ) . W e ha ve following recursiv e relation; 15 S i,j n − s − 1 ( x 0 ,n − s − 1 ) Ψ i,j ( x n − s − 1 ,n , Π i,j ( n − s, x n − s − 1 ,n , π ij )) = S i,j n − s − 1 ( x 0 ,n − s − 1 )Π i,j n − s − 1 L l +1 ( x n − s − 1 ,n ) + S n − s − 1 ( x 0 ,n − s − 1 )(1 − Π i,j n − s − 1 ) × " q ij s X k =0 p s − k ij L k +1 ( x n − s − 1 ,n ) + p s +1 ij L 0 ( x n − s − 1 ,n ) # = π ij L i,j n − s ( x 0 ,n − s − 1 ) + (1 − π ij ) q ij n − s − 1 X k =1 p k − 1 ij L n − s − k ( x 0 ,n − s − 1 ) ! L s +1 ( x n − s − 1 ,n ) +(1 − π ij ) p n − s − 1 ij L 0 ( x 0 ,n − s − 1 ) l X k =0 p s − k ij q ij L k +1 ( x n − s − 1 ,n ) + p s +1 ij L 0 ( x n − s − 1 ,n ) ! = π ij L i,j n ( x 0 ,n ) + (1 − π ij h n − s − 1 X k =1 p k − 1 ij q ij L n − k +1 ( X 0 ,n ) + s X k =0 p n − k − 1 ij q ij L k +1 ( x 0 ,n ) + p n ij L 0 ( X 0 ,n ) i = π ij L i,j n ( x 0 ,n ) + (1 − π ij ) h n − s − 1 X k =1 p k − 1 ij q ij L n − k +1 ( X 0 ,n ) + n X k = n − s p k − 1 ij q ij L n − k +1 ( X 0 ,n ) + p n ij L 0 ( X 0 ,n ) i = π ij L i,j n ( x 0 ,n ) + (1 − π ij ) " n X k =1 p k − 1 ij q ij L n − k +1 ( X 0 ,n ) + p n ij L 0 ( X 0 ,n ) # = S i,j n ( x 0 ,n ) . No w, on the set D n = { ω : X 0 ,n ∈ B 0 ,n } , X 0 = x and B i ∈ B w e hav e by (12): P ( θ > n, β = ( i, j ) , D n ) = Z { β =( i,j ) ,D n } I { θ>n } d P ϕ = Z { β =( i,j ) ,D n } P ϕ ( θ > n | ˜ F n ) d P ϕ = Z B 0 ,n p s − 1 ij L ij 0 ( x n − s − 1 ,n ) Ψ i,j ( n − s, x n − s − 1 ,n , Π i,j ( n − s, x n − s − 1 ,n , π ij )) × (1 − π ij ) p n − s − 2 ij L i,j 0 ( x 0 ,n − s − 1 ) S i,j n − s − 1 ( x 0 ,n − s − 1 ) b ij S i,j n − s − 1 ( x 0 ,n − s − 1 ) S n ( x 0 ,n ) S n ( x 0 ,n ) µ ( dx 1 ,n ) = Z D n p s − 1 ij L ij 0 ( X n − s − 1 ,n ) Ψ i,j n − s − 1 ( X n − s − 1 ,n , Π i,j n − s − 1 ) (1 − Π i,j n − s − 1 ) B i,j n d P ϕ . This f ollo ws P ϕ ( θ > n | ˜ F n ) = p s − 1 ij L ij 0 ( X n − s − 1 ,n ) Ψ i,j n − s − 1 ( X n − s − 1 ,n , Π i,j n − s − 1 ) (1 − Π i,j n − s − 1 ) . In th e case where n = s + 1 the pro of is similar. 16 Lemma 7 F or n > 0 , i ∈ I 0 , j ∈ I 1 we have B i,j n =                  Γ i,j (0 , X n − 1 ,n , B n − 1 , Π n − 1 ) , if n > 1 , b i,j n − 1 e Ψ i,j (0 ,X 0 , 1 ,π ij ) e S (0 ,X 0 , 1 , b,π ) , if n = 1 . (A.11) with c ondition B i,j 0 = b ij . PR OOF. F irst, let us ve rify the initial condition: B i,j 0 = P ϕ ( β = ( i, j ) | F 0 ) = P ϕ ( β = ( i, j )) = b ij . Let n > 1. Let us consider f orm ula (A.11) on the set D n = { ω : X 0 ,n ∈ B 0 ,n ; B 0 = { x } , B i ∈ B for 1 ≤ i ≤ n } : P ϕ ( β = ( i, j ) , X 0 ,n ∈ B 0 ,n ) = Z D n I { β =( ij ) } d P ϕ = Z D n E ϕ ( I { β =( ij ) } |F n ) d P ϕ = Z B 1 ,n b ij S i,j n ( x 0 ,n ) S n ( x 0 ,n ) S n ( x 0 ,n ) µ ( dx 1 ,n ) = Z D n b ij S i,j n ( X 0 ,n ) S n ( X 0 ,n ) d P ϕ . (A.12) T a king into a ccoun t the form ulae (13), ( 2), (4) and (11) w e ha v e gotten (A.11) for n > 1. The case n = 1 is a consequence of (A.12 ) a nd (11) with (3) and (5). Lemma 8 L e t η n = ( X n − d − 1 ,n , Π n , B n ) , wher e n ≥ d + 1 . System ( η n , F n , P ϕ y ) is Markov R and om F unction. PR OOF. It is enough to sho w that η n +1 is a function of η n and v a r ia ble X n +1 as w ell a s that conditiona l distribution of X n +1 giv en F n dep ends only on η n (see [Shiryaev (1978) ]). F or x 1 , ..., x d +2 , y ∈ E , γ ij , δ ij ∈ [0 , 1], i ∈ I 0 , j ∈ I 1 let us consider the follow ing function 17 ϕ ( x 1 ,d +2 , γ , δ , y ) =  x 2 ,d +2 , y , Π 1 , 1 (0 , x d +2 , y , δ 11 ) , . . . , Π 1 ,l 2 (0 , x d +2 , y , δ 1 l 2 ) , . . . , Π l 1 , 1 (0 , x d +2 , y , δ l 1 1 ) , . . . , Π l 1 ,l 2 (0 , x d +2 , y , δ l 1 l 2 ) , Γ 1 , 1 (0 , x d +2 , y , γ , δ ) , . . . , Γ 1 ,l 2 (0 , x d +2 , y , γ , δ ) , . . . , Γ l 1 , 1 (0 , x d +2 , y , γ , δ ) , . . . , Γ l 1 ,l 2 (0 , x d +2 , y , γ , δ )  . W e will sho w that η n +1 = ϕ ( η n , X n +1 ). Using formulas (A.10 ) and (A.11) w e express Π i,j n +1 as a function of Π i,j n and B i,j n +1 as a function B i,j n . Then: ϕ ( η n , X n +1 ) = ϕ ( X n − d − 1 ,n , Π n , B n , X n +1 ) =  X n − d,n , X n +1 , Π 1 , 1 (0 , X n,n +1 , Π 1 , 1 n ) , . . . , Π 1 ,l 2 (0 , X n,n +1 , Π 1 ,l 2 n ) , . . . , Π l 1 , 1 (0 , X n,n +1 , Π l 1 , 1 n ) , . . . , Π l 1 ,l 2 (0 , X n,n +1 , Π l 1 ,l 2 n ) , Γ 1 , 1 (0 , X n,n +1 , B n , Π n ) , . . . , Γ 1 ,l 2 (0 , X n,n +1 , B n , Π n ) , . . . , Γ l 1 , 1 (0 , X n,n +1 , B n , Π n ) , . . . , Γ l 1 ,l 2 (0 , X n,n +1 , B n , Π n )  . = ( X n − d,n +1 , Π n +1 , B n +1 ) = η n +1 . Let us consider no w t he conditional expectation u ( X n +1 ) under the condition of σ -field F n , for Borel function u : E − → ℜ . Applying equation (A.3) ( k = 1) w e get: E ϕ ( u ( X n +1 ) | F n ) = X i,j E ϕ ( u ( X n +1 ) I { β =( i,j ) } | F n ) (A.13) = X i,j E ϕ ( u ( X n +1 ) I { θ ≤ n +1 } I { β =( i,j ) } | F n ) + X i,j E ϕ ( u ( X n +1 ) I { θ>n +1 } I { β =( i,j ) } | F n ) = X i,j B i,j n  E ϕ  p ij Z E u ( y )(1 − Π i,j (0 , y , Π i,j n )) f 0 ,i X n ( y ) µ ( dy ) | F n  + E ϕ  Z E u ( y )( q ij + p ij Π i,j (0 , y , Π i,j n )) f 1 ,j X n ( X n +1 ) µ ( dy ) | F n  W e se e that conditional distribution of X n +1 giv en F n dep ends only o n comp onen t of η n what ends the pro of. Lemma 9 L e t 18 s k ( x 1 ,d +2 , γ , δ ) =      T Q k h ( x 1 ,d +2 , γ , δ ) , if k ≥ 1 , T h ( x 1 ,d +2 , γ ) , if k = 0 . (A.14) Then, for function h ( x 1 ,d +2 , γ , δ ) given by (15) and k ≥ 1 , fol lowing e quali ties hold: Q k h ( x 1 ,d +2 , γ , δ ) = m ax    X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( x 1 ,d +2 ) p m ij L i,j 0 ( x 1 ,d +2 ) ! (1 − γ ij ) δ ij , s k − 1 ( x 1 ,d +2 , γ , δ )    , s k ( x 1 ,d +2 , γ , δ ) = Z E max ( X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( x 1 ,d +3 ) p m ij L i,j 0 ( x 1 ,d +3 ) ! f 0 ,i x d +2 ( x d +3 ) p ij (1 − γ ij ) δ ij , s k − 1 ( x 2 ,d +3 , γ , p ◦ b f 0 x d +2 ( x d +3 ) ◦ δ ) ) µ ( dx d +3 ) , wher e: s 0 ( x 1 ,d +2 , γ , δ ) = X i,j   1 − p d ij + q ij d +1 X m =1 L i,j m − 1 ( x 2 ,d +2 ) p m ij L i,j 0 ( x 2 ,d +2 )   p ij (1 − γ ij ) δ ij . (A.15) Mor e over for k ≥ 0 and ve c tor η n +1 = ( X n − d,n +1 , Π n +1 , B n +1 ) , function s k has the pr op e rty: s k ( X n − d,n +1 , Π n +1 , B n +1 ) = s k ( X n − d,n +1 , Π n , p ◦ b f 0 X n ( X n +1 ) ◦ B n ) S (0 , X n,n +1 , B n , Π n ) . (A.16) PR OOF. No t ice that lemmas 6, 7, form ulas (A.10) i (A.11) enable us to rewrite function h ( X n − d,n +1 , Π n +1 , B n +1 ) in the follo wing w ay: 19 h ( X n − d,n +1 , Π n +1 , B n +1 ) (A.17) = X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( X n − d,n +1 ) p m ij L i,j 0 ( X n − d,n +1 ) ! (1 − Π i,j n +1 ) B i,j n +1 = X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( X n − d,n +1 ) p m ij L i,j 0 ( X n − d,n +1 ) ! (1 − Π i,j (0 , X n,n +1 , Π i,j n )) × Γ i,j (0 , X n,n +1 , B n , Π n ) = X i,j (1 − p d ij ) p ij (1 − Π i,j n ) B i,j n S (0 , X n,n +1 , B n , Π n ) f 0 ,i X n ( X n +1 ) + q ij d +1 X m =1 L i,j m − 1 ( X n − d,n ) p m ij L i,j 0 ( X n − d,n ) p ij (1 − Π i,j n ) B i,j n S (0 , X n,n +1 , B n , Π n ) f 1 ,j X n ( X n +1 ) ! . Using definition of op erator T , equation ( A.1 7 ), for k = 0 and ( X n − 1 − d,n , Π n , B n ) = ( x 1 ,d +2 , γ , δ ) w e get s 0 ( x 1 ,d +2 , γ , δ ) = E ϕ ( h ( X n − d,n , X n +1 , Π n +1 , B n +1 ) |F n ) (A.18) = Z E h ( X n − d,n , y , Π n +1 , B n +1 ) S (0 , X n , y , B n , Π n ) µ ( dy ) = X i,j Z E (1 − p d ij ) p ij (1 − Π i,j n ) B i,j n f 0 ,i X n ( y ) µ ( dy ) + X i,j Z E q ij d +1 X m =1 L i,j m − 1 ( X n − d,n ) p m ij L i,j 0 ( X n − d,n ) p ij (1 − Π i,j n ) B i,j n f 1 ,j X n ( y ) µ ( dy ) = X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m − 1 ( X n − d,n ) p m ij L i,j 0 ( X n − d,n ) ! p ij (1 − Π i,j n ) B i,j n = X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m − 1 ( x 2 ,d +2 ) p m ij L i,j 0 ( x 2 ,d +2 ) ! p ij (1 − γ ij ) δ ij . Hence, applying equations (A.10) and (A.11) o ne more time we end with s 0 ( X n − d,n +1 , Π n +1 , B n +1 ) = X i,j   1 − p d ij + q ij d +1 X m =1 L i,j m − 1 ( X n − d +1 ,n +1 ) p m ij L i,j 0 ( X n − d +1 ,n +1 )   p ij (1 − Π i,j n ) p ij f 0 ,i X n ( X n +1 ) B i,j n S (0 , X n,n +1 , B n , Π n ) = s 0 ( X n − d,n +1 , Π n , p ◦ b f 0 X n ( X n +1 ) ◦ B n ) S (0 , X n,n +1 , B n , Π n ) . If k = 1, then b y definition of Q : 20 Q h ( x 1 ,d +2 , γ , δ ) = max n X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( x 1 ,d +2 ) p m ij L i,j 0 ( x 1 ,d +2 ) ! (1 − γ ij ) δ ij , (A.19) s 0 ( x 1 ,d +2 , γ , δ ) o . No w, for ( X n − 1 − d,n , Π n , B n ) = ( x 1 ,d +2 , γ , δ ), taking in to accoun t link b etw een Π i,j n − 1 and Π i,j n as w ell as b et w een B i,j n − 1 and B i,j n giv en b y (A.10) and (A.11), w e get with the suppor t of (A.13): s 1 ( x 1 ,d +2 , γ , δ ) = E ϕ h max { h ( X n − d,n , X n +1 , Π n +1 , B n +1 ) , (A.20) s 0 ( X n − d,n , X n +1 , Π n +1 , B n +1 ) } | F n i = Z E max    X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( X n − d,n , y ) p m ij L i,j 0 ( X n − d,n , y ) ! f 0 ,i X n ( y ) p ij (1 − Π i,j n ) B i,j n S (0 , X n , y , B n , Π n ) , s 0 ( X n − d,n , y , Π n , p ◦ b f 0 X n ( y ) ◦ B n ) S (0 , X n , y , B n , Π n ) ) S (0 , X n , y , B n , Π n ) µ ( dy ) = Z E max    X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( x 2 ,d +2 , y ) p m ij L i,j 0 ( x 2 ,d +2 , y ) ! f 0 ,i x d +2 ( y ) p ij (1 − γ ij ) δ ij , = s 0 ( x 2 ,d +2 , y , γ , p ◦ b f 0 x d +2 ( y ) ◦ δ ) o µ ( dy ) . Basing on (A.20) with the help of (A.10) and (A.11) let us v erify formula (A.16): s 1 ( X n − d,n +1 , Π n +1 , B n +1 ) = Z E max n X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( X n − d +1 ,n +1 , y ) p m ij L i,j 0 ( X n − d +1 ,n +1 , y ) ! f 0 ,i X n +1 ( y ) p ij (1 − Π i,j n +1 ) B i,j n +1 , s 0 ( X n +1 − d,n +1 , y , Π n +1 , p ◦ b f 0 X n +1 ( y ) ◦ B n +1 ) o µ ( dy ) = Z E max    X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( X n − d +1 ,n +1 , y ) p m ij L i,j 0 ( X n − d +1 ,n +1 , y ) ! f 0 ,i X n +1 ( y ) p ij (1 − Π i,j n ) B i,j n f 0 ,i X n ( X n +1 ) p ij S (0 , X n,n +1 , B n , Π n ) , X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( X n − d +2 ,n +1 , y ) p m ij L i,j 0 ( X n − d +2 ,n +1 , y ) ! p ij (1 − Π i,j n ) p ij f 0 ,i X n ( X n +1 ) p ij f 0 ,i X n +1 ( y ) B i,j n S (0 , X n,n +1 , B n , Π n )    µ ( dy ) = s 1 ( X n − d,n +1 , Π n , p ◦ b f 0 X n ( X n +1 ) ◦ B n ) S (0 , X n,n +1 , B n , Π n ) . Supp ose that lemma 9 holds for some k > 1. W e will sho w that equations c hara cterizing Q k +1 h and s k +1 are true a nd that condition (A.16) for s k +1 is satisfied. It follows fr o m definition of op erator Q k +1 that: 21 Q k +1 h ( x 1 ,d +2 , γ , δ ) (A.21) = max  X i,j   1 − p d ij + q ij d +1 X m =1 L i,j m ( x 1 ,d +2 ) p m ij L i,j 0 ( x 1 ,d +2 )   (1 − γ ij ) δ ij , s k ( x 1 ,d +2 , γ , δ )  . Giv en ( X n − 1 − d,n , Π n , B n ) = ( x 1 ,d +2 , γ , δ ) and basing on inductiv e assumption w e hav e also: s k +1 ( x 1 ,d +2 , γ , δ ) = E ϕ h max n h ( X n − d,n , X n +1 , Π n +1 , B n +1 ) , (A.22) s k ( X n − d,n , X n +1 , Π n +1 , B n +1 ) o | F n i = Z E max    X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( X n − d,n , y ) p m ij L i,j 0 ( X n − d,n , y ) ! f 0 ,i X n ( y ) p ij (1 − Π i,j n ) B i,j n S (0 , X n , y , B n , Π n ) , s k ( X n − d,n , y , Π n , p ◦ b f 0 X n ( y ) ◦ B n ) S (0 , X n , y , B n , Π n ) ) S (0 , X n , y , B n , Π n ) µ ( dy ) = Z E max    X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( X n − d,n , y ) p m ij L i,j 0 ( X n − d,n , y ) ! f 0 ,i X n ( y ) p ij (1 − Π i,j n ) B i,j n , s k ( X n − d,n , y , Π n , p ◦ b f 0 X n ( y ) ◦ B n ) o µ ( dy ) = Z E max    X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( x 2 ,d +2 , y ) p m ij L i,j 0 ( x 2 ,d +2 , y ) ! f 0 ,i x d +2 ( y ) p ij (1 − γ ij ) δ ij , s k ( x 2 ,d +2 , y , γ , p ◦ b f 0 x d +2 ( y ) ◦ δ ) o µ ( dy ) . Finally , using(A.22) w e obtain: s k +1 ( X n − d,n +1 , Π n +1 , B n +1 ) = Z E max    X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( X n − d +1 ,n +1 , y ) p m ij L i,j 0 ( X n − d +1 ,n +1 , y ) ! f 0 ,i X n +1 ( y ) p ij (1 − Π i,j n +1 ) B i,j n +1 , s k ( X n +1 − d,n +1 , y , Π n +1 , p ◦ b f 0 X n +1 ( y ) ◦ B n +1 ) o µ ( dy ) = Z E max    X i,j 1 − p d ij + q ij d +1 X m =1 L i,j m ( X n − d +1 ,n +1 , y ) p m ij L i,j 0 ( X n − d +1 ,n +1 , y ) ! f 0 ,i X n +1 ( y ) p ij (1 − Π i,j n ) B i,j n f 0 ,i X n ( X n +1 ) p ij S (0 , X n,n +1 , B n , Π n ) , s k ( X n +1 − d,n +1 , y , Π n , p ◦ b f 0 X n +1 ( y ) ◦ p ◦ b f 0 X n ( X n +1 ) ◦ B n ) S (0 , X n,n +1 , B n , Π n ) ) µ ( dy ) = s k +1 ( X n − d,n +1 , Π n , p ◦ b f 0 X n ( X n +1 ) ◦ B n ) S (0 , X n,n +1 , B n , Π n ) , 22 References [Baron(200 4)] M. Baron (2004) Early detect ion of epidemics as a sequ ential c hange- p oint problem. In V. Antono v, C. Hub er, M. Nikulin, and V. Po lisc ho ok, editors, L ongevity, aging and de gr adation mo dels in r eliability, public he alth, me dicine and biolo gy, LAD 2004. Sele c te d p ap ers fr om the first F r ench- R ussian c onfer enc e, St. Petersbur g, Russia, Ju ne 7–9, 2004 , v olume 2 o f IMS L e ctur e Notes-M ono gr aph Series , pages 31–43, St. P etersburg, Russia, 2004. St. P etersburg State Po litec hnical Univ ersit y . [Basseville and Ben v eniste(1986)] M. Basseville and A. Ben veniste, editors (1986) Dete ction of abrupt changes in signals and dyna mic al systems , vol ume 77 of L e ctur e Notes in Contr ol and Information Scie nc es , page 373. Springer-V erlag, Berlin. [Ba yraktar et al.(2005) Ba yraktar, Da y anik, and Karatzas] E. Bayrakta r, S. Da y anik, and I. Karatzas (2005) The standard P oisson disorder p roblem revisited. Sto chastic Pr o c esses Appl. , 115(9) :1437– 1450. doi: 10.101 6/j.spa.200 5.04.011. [Bo jd ec ki(1979)] T. Bo jdecki (1979) Prob ab ility maximiz ing approac h to optimal stopping and its application to a disord er problem. Sto c hastics , 3:61–71 . [Bo jd ec ki an d Hosza(1984) ] T. Bo jdec ki and J. Hosza (1984) On a generali zed disorder problem. Sto c hastic P r o c esses A ppl. , 18:349–359 . [Bro dsky an d Darkhovsky(19 93)] B.E. Brodsky and B.S. Darkhovsky ( 1993) Nonp ar ametric Metho ds i n Change-Point Pr oblems . Mathemat ics and its Applications (Dordrec ht). 243. Dord r ec ht: Klu w er Academic Publishers. 224 p., Dordrec ht, 1993. [Dub e and Mazumdar(2001)] P . Dub e and R. Mazumdar (2001) A F r amework for Quic k est Detection of T raffic Anomalies in Net wo rks. T ec hnical rep ort, Electrical and Computer Engineering, Purdu e Univ ersity , No vem b er 2001. citeseer.ist.psu.edu/5065 51.h tml . [Karatzas(20 03)] Ioannis Karatzas (2003) A n ote on Ba yesian detection of c hange- p oint s with an exp ected miss criterion. Stat. De cis. , 21(1):3–1 3. d oi: 10.152 4/ stnd.21.1.3.20 317. [Moustakides(199 8)] George V. Moustakides (1998) Quic k est detection of ab r upt c h anges for a class of r an d om p r o cesses. IEEE T r ans. Inf. The ory , 44(5):1965 – 1968. 23 [P eskir an d Sh iry aev(20 02)] G. Peskir and A.N. Shiryaev (2002) Solving the P oisson disorder problem. In Klaus Sandmann and Philipp J. Sch u on bu c her , editors, A dvanc es in financ e and sto c hastics. Essays in hono ur of Die ter Sondermann. , pages 295–312. Sp r inger, Berlin. [Sarno wski and Sza jo w ski(2008) ] W. Sarno wski and K . S za jowski (2008) On-line detection of a part of a sequen ce w ith unsp ecified distribution. Stat. P r ob ab. L ett. , 78(15):2511 –2516. doi: 10.1016 /j.spl.2008.0 2.040. [Shewhart(1931)] W.A. Shewhart (19 31) Ec onomic c ontr ol of qual ity of manufactur e d pr o ducts . D. V an Nostrand, Y ew Y ork, 1931. [Shirya ev(1961 )] A.N. Shiry aev (19 61) The d etectio n o f spontaneous effects. Sov. Math, Dokl. , 2: 740–7 43, 1961. translation from Dokl. Ak ad. Nauk SS SR 138, 799-8 01 (1961). [Shirya ev(1978 )] A.N. Shirya ev (1978) O ptimal Stopping Rules . Springer-V erlag, New Y ork, Heidelb erg, Berlin. [Sza jo wski(1996)] Kr zysztof Sza jo w ski (19 96) A t w o-disorder detect ion problem. Appl. Math. , 24(2):231 –241. [T artak o vsky et al.(2006 )T artak o vsky , Rozo vskii, Bla ˇ zek, and K im ] A. G. T artak o vsky , B. L. Rozo vsk ii, R . B. Bla ˇ zek, and H. Kim (2006) Detectio n of intrusions in inform ation systems by sequen tial c hange-p oin t metho ds. Stat. Metho dol. , 3(3):25 2–293 . ISSN 1572-31 27. [Y akir(1994)] Benjamin Y akir (1994) Optimal detection of a c hange in distribution when the observ ations f orm a Marko v c hain with a finite state space. In D. Siegm und E. Carlstein, H.-G. M ¨ u ller, editor, Change-p oint Pr oblems. Pap ers fr om the AM S-IMS-SIAM Summer R ese ar ch Confer enc e held at Mt. Holyoke Col le g e, South Had ley, M A, USA, July 11–16, 1992 , volume 23 of IMS L e ctur e Notes-Mono gr aph Series , pages 346–358 , In stitute of Mat hematical Statistics, Ha yward, California. [Y osh ida(1983 )] M. Y oshida (1983) Probabilit y m aximizing app r oac h for a qu ic kest detection problem with complicated Mark o v c hain. J. Inform . Optimization Sci. , 4:127– 145. 24