Asymptotically Optimal Estimator of the Parameter of Semi-Linear Autoregression

Theory of Sto c hastic Pro cesses V ol.12 (28), no.3-4, 2006, pp.*-* DMYTR O IV ANENK O ASYMPT OTICALL Y OPTIMAL ESTIMA T OR OF THE P ARAMETER OF SEMI-LINEA R A UTOREGRESSION 1 The d ifference equations ξ k = af ( ξ k − 1 ) + ε k , where ( ε k ) is a square in tegrable difference martingale, and the differen tial equation d ξ = − af ( ξ )d t + d η , wh ere η is a square in tegrable martingale , are con- sidered. A family of estimators dep ending, b esides the s ample size n (or the obser v ation p erio d, if time is con tin uous) on some rand om Lipsc hitz functions is constructed. Asymptotic optimalit y of this estimators is in v estigated. 1. Intr oduction Discr ete time W e consider the difference equation ξ k = af ( ξ k − 1 ) + ǫ k , k ∈ N , (1) where ξ 0 is a prescribed random v ariable, f is a prescrib ed nonrandom function, a is an unkno wn scalar par ameter and ( ǫ k ) is a square integrable difference martingale with resp ect to some flow (F k , k ∈ Z + ) of σ -algebras suc h that the random v ariable ξ 0 is F 0 -measurable. In the detailed form, the a ssumption ab o ut ( ǫ k ) means that for an y k ǫ k is F k -measurable, E ǫ 2 k < ∞ (2) and E( ǫ k | F k − 1 ) = 0 . (3) The w ord ”semi-linear” in the title means that the r ig h t-hand sides of (1) dep end linearly on a but not on ξ . 1 2000 Mathematics Subje ct Classific ations . P rimary 62F1 2. Secondary 60F05. Key wor ds and phr ases . Marting ale, e s timator, optimiza tio n, con vergence. 1 2 DMYTR O IV ANENKO W e use the notation: l . i . p . – limit in probabilit y; d → – the w eak con v er- gence of finite-dimensional distributions of random functions, in par t icular con v ergence in distribution of random v ariables. Let for eac h k ∈ Z + h k = h k ( ω , x ) b e an F k − 1 ⊗ B- measurable function (that is t he sequence ( h k ) b e predictable) suc h that E [( | ξ k +1 | + | af ( ξ k +1 ) | ) | h k ( ξ k ) | ] + E | h k ( ξ k ) | < ∞ . Then from (1) – (3) w e ha v e E ( ξ k +1 − af ( ξ k )) h k ( ξ k ) = 0, whence a = (E ξ k +1 h k ( ξ k )) (E f ( ξ k ) h k ( ξ k )) − 1 pro vided (E f ( ξ k ) h k ( ξ k )) 6 = 0. This prompts the estimator ˇ a n = n − 1 X k =0 ξ k +1 h k ( ξ k ) ! n − 1 X k =0 f ( ξ k ) h k ( ξ k ) ! − 1 , (4) coinciding with the LSE if h k ( x ) = f ( x ) for all k . Continuous time W e consider the differen tial equation d ξ ( t ) = − af ( ξ ( t ))d t + d η ( t ) , t ∈ R , (5) where η ( t ) is a lo cal square in t egr a ble martinga le w.r.t. a flo w (F( t )) suc h that the r andom v ariable ξ (0) is F ( 0 )-measurable. Let h ( t, x ) b e a predictable random function suc h that for all t ∈ R + E [( | ξ ( t ) | + | af ( ξ ( t )) | ) | h ( t, ξ ( t )) | ] + E | h ( t, ξ ( t )) | < ∞ . Let us m ultiply (5) on h ( t, ξ ( t )) and in tegrate from 0 to T . The same rationale as in the discrete case yields the estimator ˇ a T = − Z T 0 h ( t, ξ ( t ))d ξ ! Z T 0 f ( ξ ( t )) h ( t, ξ ( t ))d t ! − 1 , (6) coinciding with the LSE if h ( t, x ) = f ( x ). Asymptotic normalit y of √ n  ˇ A n − A  , where ˇ A n is the LSE of a matrix parameter A , was pro ved in [1] under the assumptions o f ergodicity and stationarit y of ( ξ n ). Conv ergence in distribution of this nor ma lized deviation w as prov ed in [2] with the use of sto c hastic calculus. Ergodicity and ev en stationarit y of ( ǫ k ) w as not assumed in [2], so the limiting distribution could b e o ther than normal. The goal of the article is to match a sequence ( h k ) (if time is discrete) or a function h ( t, · ) (if time is contin uous) so that to minimize t he v alue of some random functional V n whic h, as w e shall see in Section 3, is asymptotical close in distribution to some n umeral characteristic o f the estimator (in case the latter is asymptotically normal this c hara cteristic coincides with the v ariance). ASYMPTOTICALL Y OPTIMAL ESTIMA TO R 3 2. The main re sul ts Discr ete time Denote σ 2 k = E[ ǫ 2 k | F k − 1 ], µ k = h k ( ξ k ). Let Lip ( C ) denote the class of functions satisfying the Lipsc hitz condition with some constan t C and equal to zero at the origin, Lip = S C > 0 Lip( C ), and let H ( C ) denote the class of all predictable random functions on Z + × R (dis crete time) or R + × R (con tin uous time) whose realizations h k ( · ) (resp ectiv ely h ( t, · )) b elong, as functions of x , t o Lip( C ), H = S C > 0 H ( C ) . Predictability means P ⊗ B- measurabilit y in ( ω , t, x ) (the σ -alg ebra P is defined in [4, p. 28], [6, p. 13]). W e a re seeking for ( e h k ) ∈ H minimizing the functional V n ( h 0 , . . . , h n − 1 ) = 1 n P n − 1 k =0 σ 2 k +1 µ 2 k  1 n P n − 1 k =0 f ( ξ k ) µ k  2 . (7) Theorem 1. L et V n ( e h 0 , . . . , e h n − 1 ) = min h 0 ,...,h n − 1 ∈ H V n ( h 0 , . . . , h n − 1 ) . (8) Then σ 2 k +1 e µ k n − 1 X i =0 f ( ξ i ) e µ i = f ( ξ k ) n − 1 X i =0 σ 2 i +1 e µ 2 i , k = 0 , n − 1 . (9) Pr o of. T o obtain the necessary conditions for extrem um of the functional V n (9) w e will v ary [3] just o ne of functions h k , k = 0 , n − 1, leaving the other functions without changes . Th us regarding V n ( h 0 , . . . , h n − 1 ) as a f unctional dep ending on only one function V n ( h 0 , . . . , h n − 1 ) = e V n ( h k ). Let’s c ho ose some scalar func tion g ∈ H and denote g λ ( x ) = e h k ( x ) + λ ( g ( x ) − e h k ( x )), v ( λ ) = e V n ( g λ ). Ob viously , g λ ∈ H so the minim um of v ( λ ) is attained at zero and therefore v ′ (0) = 0 . (10) The express ion for the left-hand side is v ′ (0) = 2 n ( g ( ξ k ) − µ k )  σ 2 k +1 e µ k ( P n − 1 i =0 f ( ξ i ) e µ i − f ( ξ k ) P n − 1 i =0 σ 2 i +1 e µ 2 i   P n − 1 i =0 f ( ξ i ) e µ i  3 . Hence in view of (10) w e obtain the i th equation of system (9). It remains to apply this argumen t to eac h function h k , k = 0 , n − 1. Remark. The Lipsc hitz condition w as no t used in the pro of. It will b e required in Section 3 . Corollary 1. Let f ∈ Lip( C ) and there exist a constant q > 0 suc h t ha t σ 2 k ≥ q f or all k . Then h i ( x ) = f ( x ) /σ 2 i +1 , i = 0 , n − 1, is a solution to the problem (8). 4 DMYTR O IV ANENKO Continuous time Let m denote the quadratic c ha racteristic o f η . W e shall matc h e h = e h ( ω , t, x ) f rom H ( C ) ( C is indep enden t of t ) so that to minimize the v a lue of the functional V T ( h ) = 1 T R T 0 h ( t, ξ ( t )) 2 d m ( t )  1 T R T 0 f ( ξ ( t )) h ( t, ξ ( t ))d t  2 . (11) Theorem 2. L et V T ( e h ) = min h ∈ H V T ( h ) . (12) Then for al l g ∈ H R T 0 e h ( t, ξ ( t )) g ( t, ξ ( t ))d m ( t ) R T 0 f ( ξ ( t )) e h ( t, ξ ( t ))d t = R T 0 f ( ξ ( t )) g ( t, ξ ( t ))d t R T 0 e h ( t, ξ ( t )) 2 d m ( t ) . (13) Pr o of. Let’s c ho ose some scalar function g ∈ H and denote g λ ( t, x ) = e h ( t, x ) + λg ( t, x ), v ( λ ) = V T ( g λ ). Ob viously g λ ( t, · ) ∈ H so the minim um of v ( λ ) is atta ined in zero and therefore v ′ (0) = 0 . (14) The express ion for the left-hand side is v ′ (0) = 2 T  R T 0 f ( ξ ( t )) e h ( t, ξ ( t ))d t  − 3 × Z T 0 f ( ξ ( t )) e h ( t, ξ ( t ))d t Z T 0 e h ( t, ξ ( t )) g ( t, ξ ( t ))d m ( t ) − Z T 0 f ( ξ ( t )) g ( t, ξ ( t ))d t Z T 0 e h ( t, ξ ( t )) 2 d m ( t ) ! . Hence in view of (14) w e come to (13). Corollary 2. Let f ∈ Lip( C ), m b e absolutely con tin uous w.r.t. the Leb esgue measure and there exist a constan t q > 0 suc h that for all t ˙ m ≥ q . Then h ( t, x ) = f ( x ) / ˙ m is a solution to the problem (12). 3. An illustra t ion Denote E 0 = E( · · · | F 0 ), Q n = 1 n P n − 1 k =0 f ( ξ k ) µ k , G n = 1 n P n k =1 σ 2 k µ 2 k − 1 . W e denote E 0 = E( · · · | F 0 ) and introduce the conditions CP1. F or any r ∈ N and an y uniformly b ounded sequenc e ( α k ) of R-v a lued Borel functions on R r 1 n n − 1 X k = r  α k ( ǫ k − r +1 , . . . , ǫ k ) − E 0 α k ( ǫ k − r +1 , . . . , ǫ k )  P − → 0 , ASYMPTOTICALL Y OPTIMAL ESTIMA TO R 5 1 n n − 1 X k = r  σ 2 k α k ( ǫ k − r +1 , . . . , ǫ k ) − E 0 σ 2 k α k ( ǫ k − r +1 , . . . , ǫ k )  P − → 0 . CP2. F or suc h r and ( α k ) the sequences 1 n n − 1 X k = r E 0 α k ( ǫ k − r +1 , . . . , ǫ k ) , n = r + 1 , . . . ! , 1 n n − 1 X k = r E 0 σ 2 k α k ( ǫ k − r +1 , . . . , ǫ k ) , n = r + 1 , . . . ! con v erge in probability . Denote f 0 ( x ) = x and, for r ≥ 1, f r ( x 0 , . . . , x r ) = af ( f r − 1 ( x 0 , . . . , x r − 1 )) + x r . Then ξ k = f r ( ξ k − r , ǫ k − r +1 , . . . , ǫ k ) , r < k . Lemma 1. L et c ond itions (2) , (3), CP1 and CP2 b e fulfil le d. Supp ose also that lim N →∞ lim n →∞ 1 n n X k =1 E ǫ 2 k I {| ǫ k | > N } = 0 (15) and ther e exist an F 0 -me asur able r andom variable υ such that for al l k σ 2 k ≤ υ . (16) and p ositive numb ers C , C 1 such that | a | C < 1 , (17) f ∈ Lip( C ) , ( h k ) ∈ H ( C 1 ) . Then ( G n , Q n ) d → ( G, Q ) . (18) Pr o of. Denote ξ r k = f r (0 , ǫ k − r +1 , . . . , ǫ k ), µ r k = h k ( ξ r k ), Q r n = 1 n P n − 1 k = r f ( ξ r k ) µ r k , G r n = 1 n P n k = r σ 2 k ( µ r k − 1 ) 2 . W e claim that conditions (2), (3), (15), (16), (17) and the relatio n ( Q r n , G r n ) d → ( Q r , G r ) as n → ∞ (19) imply (18). Let X r denote ( x 1 , . . . , x r ) ∈ R r . Then under the assumptions on f and h k for a n y N > 0 lim r →∞ sup | x |≤ N ,X r ∈ R r | f r ( x, X r ) − f r (0 , X r ) | = 0 , 6 DMYTR O IV ANENKO whence with pro babilit y 1 for an y k lim r →∞ sup | x |≤ N ,X r ∈ R r | f ( f r ( x, X r )) h k ( f r ( x, X r )) − f ( f r (0 , X r )) h k ( f r (0 , X r )) | = 0 , (20) lim r →∞ sup | x |≤ N ,X r ∈ R r | h k ( f r ( x, X r )) 2 − h k ( f r (0 , X r )) 2 | = 0 . These relations w a s prov ed in [5]. Let us prov e that from conditions (2), (3), (1 5), (16) and (1 7) it follo ws that almost surely lim r →∞ lim n →∞ E 0 | Q n − Q r n | = 0 , lim r →∞ lim n →∞ E 0 | G n − G r n | = 0 . (21) By (2 0) for an y N > 0 lim r →∞ lim n →∞ 1 n n − 1 X k = r E | f ( ξ k ) ⊗ µ k − f ( ξ r k ) µ r k | I {| ξ k | ≤ N } = 0 . (22) Denote χ N k = I {| ξ k | > N } , I N k = I { | ǫ k | > (1 − C ) N } , b N k = E 0 | ξ k | 2 χ N k . Due to (17) and b ecause of ( h k ) ∈ H ( C 1 ) E 0 | f ( ξ k ) µ k | χ N k ≤ C C 1 b N k , Hence and fr o m (2), (3), (15)–(17) we get b y Corollary 1 [5 ] lim N →∞ lim n →∞ 1 n n − 1 X k =0 E 0 | f ( ξ k ) µ k | χ N k = 0 . (23) F urther, for k ≥ r , E 0 | f ( ξ r k ) µ r k | = E 0 | f ( f r (0 , ǫ k − r +1 , . . . , ǫ k )) || h k ( f r (0 , ǫ k − r +1 , . . . , ǫ k )) | , whence E | f ( ξ r k ) µ r k | χ N k ≤ C C 1 E r − 1 X i =0 C i | ǫ k − i | ! 2 χ N k . (24) W riting the Cauch y – Bun y ako vsky inequalit y r − 1 X i =0 C i | ǫ k − i | ! 2 ≤ r − 1 X j =0 C j r − 1 X i =0 C i | ǫ k − i | 2 , w e get for an arbitrary L > 0 E  P r − 1 i =0 C i | ǫ k − i |  2 χ N k ≤ (1 − C ) − 1 E r − 1 X i =0 C i ǫ 2 k − i I {| ǫ k − i | > L } + L 2 P {| ξ k | > N } r − 1 X i =0 C i ! . (25) ASYMPTOTICALL Y OPTIMAL ESTIMA TO R 7 In view of (2), (3) Lemma 1 [5] together with ( 17) and (15) implies that lim N →∞ lim n →∞ 1 n n X k =0 P {| ξ k | > N } = 0 . (26) Ob viously , for arbitrary nonnegativ e n um b ers u 0 , . . . , u r − 1 , v 1 , . . . , v n − 1 n − 1 X k = r r − 1 X i =0 u i v k − i ≤ r − 1 X i =0 u i n − 1 X j =1 v j , so conditions (17) and (15) imply that lim L →∞ sup r lim n →∞ 1 n n − 1 X k = r E r − 1 X i =0 C i ǫ 2 k − i I {| ǫ k − i | > L } = 0 , whence in view of (24) – (26) lim N →∞ sup r lim n →∞ 1 n n − 1 X k = r E | f ( ξ r k ) µ r k | χ N k = 0 . Com bining this with (22) a nd (23), w e arriv e at the first relation of (2 1). The pro o f of the second relation of (21) is similar. The details can b e fo und in [5]. F rom (19), and (21) we obtain that the sequence (( Q r , G r ) , r ∈ N) con- v erges in distribution to some limit ( Q, G ) a nd relation (18) holds. Let us ch ec k (19). Condition CP1 implies that lim r →∞ lim n →∞ E 0 | Q r n − E 0 Q r n | = 0 , lim r →∞ lim n →∞ E 0 | G r n − E 0 G r n | = 0 . It remains to note that under condition CP2 for an y r ∈ N the sequences (E 0 G r n ) and (E 0 Q r n ) con v erge in probabilit y . By construction V n ( h 0 , . . . , h n − 1 ) = G n Q − 2 n . The v a lue Q n = 0 is ex- cluded b y the choice of the tuple ( h 0 , . . . , h n − 1 ) minimizing V n . Corollary 3. Let the conditions of Lemma 1 b e fulfilled and Q 6 = 0 a.s. Then V n d → V , where V = GQ − 2 . Ha ving in mind the use of sto c hastic analysis, we in tro duce the pro cesses ˇ a n ( t ) = ˇ a [ nt ] and the flows F n ( t ) = F [ nt ] with con tin uous time. Theorem 3. L et c onditions of L emma 1 b e fulfil le d. Then √ n (ˇ a n ( · ) − a ) d → β ( · ) , wher e β i s a c ontinuous lo c al martingale with quadr atic ch ar acteristic h β i ( t ) = tV , (27) and initial v a lue 0. 8 DMYTR O IV ANENKO Pr o of. Denote Y n ( t ) = 1 √ n P [ nt ] k =1 ǫ k µ k − 1 . Then b ecause of (4) √ n (ˇ a n ( t ) − a ) = Y n ( t ) Q − 1 n . (28) By construction and conditions (2), (3), (17) Y n is a lo cally square inte- grable martingale with quadratic c haracteristic h Y n i ( t ) = n − 1 [ nt ] G [ nt ] . It was pro v ed in [5] that under conditions (2), (3), (15), (16) , (17) a nd (18) √ n (ˇ a n ( · ) − a ) d → Y ( · ) Q − 1 , where Y is a con tinuous lo cal martingale w.r.t. some flo w (F( t ) , t ∈ R + ) suc h that h Y i ( t ) = tG and the random v ariable Q is F ( 0)-measurable (and so do es G , whic h can b e seen from the expression for h Y i ). In view of Lemma 1 it remains to note that V n = h Y n i (1) Q − 2 n and V = h Y i (1) Q − 2 . Remark. This theorem explains the form of functional (7). In the most general case (without conditions CP1 and CP 2 ) t he denominator ( 2 8) in limit is an F(0)-measurable random v a riable, and the n umerator tends to quadratic c haracteristic at the p oint t = 1 of the contin uous local martingale Y . Th us, the n umerator (7) is the quadratic characteristic at t = 1 o f the pre-limit martingale Y n , and the denominator satisfies the law of large n um b ers. Minimizing pre-limit v ariance in ( h k ) ∈ H ( C 1 ), w e lessen the v alue of limited v ariance of the normalized deviation of estimator (4). Let f urt her h k ( x ) = f ( x ) /σ 2 k +1 . Recall that ( h k , k = 0 , n − 1) is a solution to the problem (8). F o r suc h h k w e ha v e Corollary 4. Let the conditions of Corollary 1 and Theorem 3 b e fulfilled. Then V = lim r →∞ l . i . p . n →∞ 1 n n − 1 X k = r E 0 f ( ξ r k ) 2 σ 2 k +1 ! − 1 . Pr o of. Obv iously V n = Q − 1 n . By Lemma 1 Q n d → Q , where Q = lim r →∞ l . i . p . n →∞ E 0 Q r n . T o complete t he pro of it r emains to note that Q r n = 1 n P n − 1 k = r E 0 f ( ξ r k ) 2 σ 2 k +1 . 4. An example Supp ose that f ∈ Lip( C ), h k ∈ H ( C 1 ) condition (17) b e fulfilled. Let also ǫ n = γ n b n ( ξ n − 1 ), where ( γ n ) b e a sequence of indep enden t random v ariables with zero mean and v ariances ς 2 n , | γ k | ≤ C 2 , b n ∈ H ( C 3 ) and C + C 2 C 3 < 1. Let also E ξ 2 0 < ∞ F or F k w e tak e the σ -algebra g enerated by ξ 0 ; γ 1 , . . . , γ k . Then σ 2 k = ς 2 k b k ( ξ k − 1 ) 2 and ( ǫ n ) satisfies (2), (3). ASYMPTOTICALL Y OPTIMAL ESTIMA TO R 9 Denote further b f r ( x 0 , . . . , x r ) = af ( b f r − 1 ( x 0 , . . . , x r − 1 )) + x r b r ( b f r − 1 ( x 0 , . . . , x r − 1 )) , b ξ r k = b f r (0 , γ k − r +1 , . . . , γ k ) , b µ r k = h k ( b ξ r k ) , b Q r n = 1 n n − 1 X k = r f ( b ξ r k ) b µ r k , b G r n = 1 n P n − 1 k = r ς 2 k +1 b k +1 ( b ξ r k ) 2 ( b µ r k ) 2 . Similarly to the pro of of Lemma 1 we obtain lim r →∞ lim n →∞ E 0 | G n − b G r n | = 0 , lim r →∞ lim n →∞ E 0 | Q n − b Q r n | = 0 . Items in b G r n and b Q r n dep ends on γ k − r +1 , . . . , γ k then they satisfy the law of la r ge num b ers in Bernstein’s form. If b esides ǫ n satisfies CP2 and Q 6 = 0 then Theorem 3 asserts (27). If herein f ( x ) ς 2 k b k ( x ) 2 ∈ Lip then e h k ( x ) = f ( x ) ς 2 k b k ( x ) 2 is a solution t o the problem (8) and V = lim r →∞ l . i . p . n →∞ 1 n n − 1 X k = r E 0 f ( e ξ r k ) 2 ς 2 k +1 b k +1 ( e ξ r k ) 2 ! − 1 . Example. Let b n = b , h n = h and γ n b e i.i.d. random v ariables. In view of expressions for b Q r n and b G r n w e may confine ourselv es with the case α k = α . By the Stone – W eierstrass theorem for σ -compact s paces [7, p. 317] α can b e uniformly on compacta appro ximated with finite linear combina- tions of functions of the kind g 1 ( x 1 ) . . . g r ( x r ). By the c hoice of F k and the assumptions on ( γ n ) E 0 g 1 ( γ k − r +1 ) . . . g r ( γ k ) = r Y i =1 E g i ( γ 1 ) . Hence and fr o m the ab o v e assumption on ( γ k ) condition CP2 emerges. Ac kno wledgemen t. The author is grateful to A. Y urachk ivsky for helpful advices. Bibliography 1. Dorogo vtsev A. Y a., Estimation the ory for p ar ameters of r andom pr o c esses (R ussian). Kyiv Un iv ersit y Press. Kyiv (1982). 2. Y urac hkivsky A. P ., Iv anenko D. O., Matrix p ar ameter estimation in an autor e gr ession mo del with non-stationary noise (U kr anian), Th. Prob. Math. Stat. 72 (2005), 158–172. 3. Elsholz, L. E., Differ ential e q uations and c alculus of variations (Russian ), Nauk a, Mosco w (1969). 4. Chung K. L., Williams R. J., Intr o duction in sto chastic inte gr ation (Rus- sian), Mir, Mosco w (1987). 10 DMYTR O IV ANENKO 5. Y urac hkivsky A. P ., Iv anenko D. O., Matrix p ar ameter estimation in an autor e gr ession mo del, Th eory of Stoc hastic Pro cesses 12(28) N o 1-2 (2006 ), 154-16 1. 6. Liptser R. Sh., Shir y aev A. N., The ory of martingales (R ussian), Nauk a, Mosco w (1986). 7. Kelley , J., Gener al top olo gy (R ussian). Nauk a, Mosco w (1981). Dep ar tment of Ma thema tics and Theo retical Rad iophysic, Kyiv Na tional T aras Shevchenko University , Ky iv, Ukraine E-mail: ida@univ.kiev.ua