Approximate Propagation of both Epistemic and Aleatory Uncertainty through Dynamic Systems

A ppr oxim ate Pr opagation of both Epistemic and Aleatory Uncertai nty thr oug h Dynamic Systems Gabriel T er ejanu a Puneet Singla b T arunraj Singh b Pe ter D. Scott a terejanu@buf falo.edu psingla@buf falo.edu tsingh@buff alo.edu peter@buf falo.edu a Department of Computer Science & Engineering b Department of Mechanical & Aerospace Engineering Univ ersity at Buf falo, Buf falo, NY -14260. Abstract – When ignorance du e to the lack of knowledge, modeled as epistemic uncertainty using Dempster -Sha fer structur es on closed intervals, is pr esent in the model pa- rameters , a new u ncertainty pr opagation method is n eces- sary to pr opagate bo th alea tory and ep istemic un certainty . The new framew ork pr opo sed her e, combines both epistemic and aleatory u ncertainty into a secon d-or der uncertainty r epr esentatio n which is p r opagated thr oug h a dynamic sys- tem driven b y white no ise. F irst, a finite pa rametrization is chosen to mod el the aleatory un certainty by choosing a r epresentative ap pr oximation to the pr oba bility de nsity function conditioned on epistemic variables. The epistemic uncertainty is then pr opagated thr ough th e moment evolu- tion equatio ns of the conditiona l pr obab ility den sity func- tion. This way we are able to model the ignorance when the knowledge abou t the system is incomp lete. The output of the system is a Dempster-Shafer str uctu r e on sets of cumu lative distributions which can be comb ined using differ ent rules of combinatio n and eventually transformed into a s ing leton cu- mulative d istrib ution function using Smets’ pig nistic trans- formation when decision making is needed. Keywords: Uncertainty Propagatio n, Epistemic Uncer- tainty , Aleatory Uncertainty , Dempster-Shafer, Moment Evolution, P ign istic T ransfor mation, Ign orance 1 Introd uction The paper presen ts a novel framework to char acterize the response o f stocha stic dyn amic systems with model pa ram- eters governed by epistemic un certainty . W e distinguish be- tween a leatory unc ertainty which arises due to the stochas- tic behavior of the system and epistemic uncertainty which is used to mo del the ig norance in the model paramete rs. In this last case we do not kn ow if the paramete r in discussion is rando m or not, and if it is random what is its und erlying probab ility distribution [16]. Ferson and Ginzburg [5] argues that when dealing with pro pagating un certainty throug h mathem atical mod- els, different calculation methods are required to propagate aleatory and ep istemic uncer tainty . Howe ver , th e segrega- tion of both types of uncertainty during propag ation, as well as the accurate propagatio n o f either one o f th em are two challenges in creating a method to pro pagate both epistemic and aleatory uncertain ty . Finding only the response of a stochastic dynamic sys- tem w ith no u ncertainty in the p arameters, is a very ac- ti ve area o f research. The prob lem in this case is to solve for th e evolution of the p robability d ensity f unction (p df), p ( t, x ) , corresp onding to the state x of the dynamic system. The e volution of th e pdf is g overned by the Fokker -Planck- K olmog orov (FPK) equation with analytical solution known only for stationary pdf s of a limited class of dy namic sys- tems [ 7, 18]. D if fer ent appr oximate methods exists in the literature to represent the pdf with a finite number of pa- rameters such as finite-difference tech niques, Monte Carlo methods, Gaussian Mixtures models [20] and Gaussian clo- sure methods. These method s provid e ap proximatio ns to the pdf wh en the prior pd f is precisely k nown, the pro cess noise is p er- fectly characterize d and the param eters in the model are e x- actly known. This is also known as the Bayesian dogma o f pr ecision [22], ho wever , in practice, these precise values are difficult to obtain due to the amo unt of infor mation av ail- able, incomplete knowl ed ge of the system or systematic un- derestimation of uncertainty which arises in the elicitation process. This becomes of great importance when doin g haz- ard risk assessment and decision making. The frame work pro posed here, prop agates both epistemic and aleato ry uncertain ty throug h a dyna mic system driven by white n oise using a two-level hierarchical model. On the first le vel we model the aleato ry uncertainty by ch oos- ing a representative appr oximation for the p df with a finite number o f par ameters. In this p aper we model th e pdf con- ditioned on epistemic v ariab les as a Gaussian distrib ution. On the secon d level is the ep istemic un certainty that we have in the mome nts o f the stochastic solution, in this case the first two momen ts. The epistemic uncertainty is m odeled in th is work using finite Dempster-Shafer structures where the focal elemen ts are closed intervals. The closed intervals are pro pagated through the mo ment ev olution equation s us- ing a recent pro posed m ethod based on Polyn omial Chaos - Bern stein Form [21]. Thus the framework p rovides the means to propagate both epistemic an d aleatory uncertainty separately [5] as well as combine them when a decision has to be made. After the pr opagation we obtain finite Dempster-Shafer structures, only this time the focal elements are sets of dis- tributions rep resented b y p robability boxes [6]. Th e struc- tures obtained character ize the uncertainty in t he quantity of interest, namely the cumu lati ve distribution fun ction (cdf) of the r esponse of the system, and they be used with classic decision theor y b y using the pig nistic transformatio n [19] whenever a decision is required. In Section 2 the prob lem to be solved is defined, followed by a backgrou nd of Dem spter-Shafer stru ctures on closed intervals and decision making un der ignor ance in Section 3. The ap proxima te uncertain ty p ropagatio n fram e work is developed in Section 4 with resolutions f or the two po sed problem s. I n Section 5 a pr oof o f concep t example is pre - sented and the conclusion s and futur e w ork are discussed in Section 6. 2 Pr oblem Statement Consider th e fo llowing first ord er model with stochas- tic forcin g and u ncertain initial condition mode led u sing a Gaussian pdf: ˙ x ( t ) + f ( x, α , t ) = Γ( t ) (1) p ( t 0 , x 0 ) = N ( x 0 ; µ 0 , σ 2 0 ) (2) The Gaussian white noise process Γ( t ) ha s the autoco rre- lation function E [Γ( t )Γ( τ )] = q 2 δ ( t − τ ) , a nd α is a vector of n parameters. The variables of the mo del can be segregated in two sets: the set of aleato ry variables giv en by a ( t ) = { x ( t ) , Γ( t ) } and the set of independe nt epistemic vari- ables given by e ( t ) = [ e S , e D ( t )] , where e D ( t ) = { m 1 ( t ) , m 2 ( t ) . . . m k ( t ) } is a time-variant vector of epis- temic mom ents used to characterize the probability distribu- tion of the respo nse, p ( t, x | e D ) , and e S = [ α , q 2 ] is a time- in variant vector of epistemic variables. Given the initial con- dition, the first two moments ar e given by m 1 (0) = µ 0 and m 2 (0) = σ 2 0 + µ 2 0 . If a part of the model parameters, α , are characterized by aleatory uncertainty then they can b e treated as additional state variables and they can be au gmented to the aleato ry vector a ( t ) . The evidence about the model param eters in e ( t ) is modeled here u sing De mpster-Shafer structures on closed intervals an d the uncer tainty of the aleatory v ariables in a ( t ) is quantified by probab ility distributions. When the model parame ters are k nown, th e response of the system is characte rized only b y aleatory uncertainty which is rep resented here by the cumulative d istribution function , P ( X ≤ x f | e 0 ) = Z x f −∞ p ( t, x | e 0 )d x (3) where e 0 = e (0) = [ e S , e D (0)] an d p ( t, x | e 0 ) is o btained in general by solving the FPK equation. Giv en the un certain model param eters, we ar e interested in solving the following three pro blems: 1. The m ain o bjective is to fin d th e ind uced Dempster- Shafer structure about the respo nse of the system, m [ P ( X ≤ x f | e 0 )] . Hen ce, we are looking to deter- mine the focal elements of the response as well as their correspo nding pro bability masses. 2. Construct a cu mulative density function u sing the pig- nistic transformatio n in ord er to use the e xpec ted u tility theory for decision making. The last problem represents a seco ndary objectiv e and is included h ere in o rder to present th e practicality of the ap- proach . 3 Backgrou nd A DS- structure on closed inter vals is a collection of interval-valued f ocal elements and th eir associated b asic probab ility assignm ents [6]. W e write the u ncertainty of the epistemic variable x is repre sented by the following DS- structure as: x ∼  ([ x 1 , x 1 ] , p 1 ) , ([ x 2 , x 2 ] , p 2 ) , . . . ([ x n , x n ] , p n )  (4) Giv en two n ondecreasin g function s F a nd F , where F , F : R → [0 , 1] and F ( x ) ≤ F ( x ) for all x ∈ R , we can r epresent the imprecision in th e cu mulative distribution function , F ( x ) = P ( X ≤ x ) , b y the probab ility box ( p- box) [ F , F ] as fo llows: F ( x ) ≤ F ( x ) ≤ F ( x ) [6]. A Dempster-Shafer structure on clo sed intervals can in- duce a unique p-box, while the in verse is not uniquely deter- mined. Many Dempster-Shafer s truc tures exist for the s ame p-box . Given the following bod y of evidence, Eq.(4), the cumulative b elief fun ction (CBF) an d the cumulative plau - sibility function (CPF) are defined by: C B F ( x ) = F ( x ) = X x i ≤ x p i (5) C P F ( x ) = F ( x ) = X x i ≤ x p i (6) Thus the cu mulative distribution function is b ounded as follows: C B F ( x ) ≤ P ( X ≤ x ) ≤ C P F ( x ) (7) In order to compute e xpec tations using the interval-based belief fu nctions, one ne eds to build a probab ility d ensity function given the belief structu re. Based on Smets’ pig- nistic transfor mation one can d efine the pign istic p robability density function p B et ( x ) [17] as a finite mixture of continu- ous uniform distributions: p B et ( x ) = n X i =1 p i x i − x i I ( x, [ x i , x i ]) (8) where the indicator functio n I ( x, [ x i , x i ]) is gi ven by: I ( x, [ x i , x i ]) =  1 , for x ∈ [ x i , x i ] 0 , otherwise Using the above defined pign istic proba bility den sity function p B et ( x ) one can compute the expe ctations n eeded in the decision making pro cess. Example the expected v alue of x gi ven the pign istic pro bability density fun ction s i given by: E B et ( X ) = Z + ∞ −∞ xp B et ( x )d x = 1 2 n X i =1 p i ( x i + x i ) (9) The pig nistic transfor mation co nstructs a sing leton pd f and m akes the expe cted u tility the ory applicab le, h owe ver it ignores the ignoran ce [9]. One possible way to in corpo- rate the ign orance in to the decision process is to co nstruct a scalar measures which quantifies the total amou nt of ig- noranc e and take a d ecision only if th e level of ignoran ce is relativ ely low . Defin e the normalized in tegral of the d egree of ignoran ce: N I D I = 1 x − x Z x x [ C P F ( x ) − C B F ( x )]d x (10) where x = min 1 ≤ i ≤ n x i and x = max 1 ≤ i ≤ n x i . The N I D I is a scalar measur e with values ranging between 0 . 0 and 1 . 0 and it summ arizes the confidence in the pignistic pdf. A v alue equal with 1 . 0 denotes that we are dealing with interval u ncertainty wh ere we o nly know the boun ds of th e variable, a nd a value closed to 0 . 0 mean s th at we know the pdf of x precisely . Thus, one can be mo re c onfident in ap- plying the expected utility theor y if the amount of igno rance if low (eq. less than 0 . 1 ) and can try to defer the decision making an d g ather more e vide nce if the le vel of ignor ance is not tolerable. 4 Pr oposed Ap proach 4.1 T ime ev olution of moments In order to keep both types of uncertainty s egregated dur- ing prop agation, one can use the mom ent ev olutio n eq ua- tions associated with Eq.(1), wh ich is a set of ordinar y dif- ferential equ ations ( ODE) governed o nly by epistemic un- certainty . Thu s, after the apply ing the It ˆ o ’ s lemma to Eq.(1) [11], the time evolution of the moments conditioned o n the epistemic variables is given by dE[ ϕ ( x ) | e 0 ] d t = − E[ ϕ x ( x ) f ( x, α , t ) | e 0 ] + 1 2 E[ q 2 ϕ xx ( x ) | e 0 ] (11) where ϕ ( x ) = x k . The nonlin earity of th e model is assumed to be p olyno- mial, ho wever the general cas e can be handled using numer- ical integration schemes such as Gaussian quadrature. T hus, when the nonline arity in the model is poly nomial, f ( x, α , t ) = n X i =1 α i x i , (12) the final f orm for th e ev olu tion of the mo ments cond itioned on the epistemic variables, is given by ˙ m k | e 0 = − k n X i =1 α i m i + k − 1 | e 0 + 1 2 k ( k − 1) q 2 m k − 2 | e 0 . ( 13) Observe, that in Eq.(13) the lower order moments depend on th e high er order moments, thu s we hav e an infinite hier- archy of mome nt equatio ns. T o truncate the in finite chain, one can use mom ent closure sch emes which assume a spe- cific class of proba bility distributions to express high er ord er moments in terms of lo wer ord er ones. In this paper we are using the Gaussian closure method, hence we are in terested in propag ating only the first two mo- ments, m 1 and m 2 , un der the assumptio n that p ( t, x | e D ) ≈ N ( x ; µ, σ 2 | e D ) . T o express the mo ments of ord er k > 2 in terms of the lo wer order mom ents one can use the follo wing relation [1]: m k = E  ( X − µ ) k  − k − 1 X i =0 ( − 1) k − i k i ! m i µ k − i (14) where m 0 = 1 , µ = m 1 , σ 2 = m 2 − m 2 1 , and the cen tral moments of the Gaussian distribution are given by E  ( X − µ ) k  =  σ k k ! 2 k/ 2 ( k/ 2)! , k is even 0 , k is odd After using th e Gau ssian closur e, the time-variant vec- tor o f ep istemic variables is giv en o nly by the first two mo- ments, e D ( t ) = [ m 1 ( t ) , m 2 ( t )] T , an d the system of ODEs (13) is transforme d into the following system: ˙ e D = g ( e D , e S ) (15) e D (0) ∼  ([ e D 1 (0) , e D 1 (0)] , p D 1 ) , ([ e D 2 (0) , e D 2 (0)] , p D 2 ) , . . .  e S ∼  ([ e S 1 , e S 1 ] , p S 1 ) , ( [ e S 2 , e S 2 ] , p S 2 ) , . . .  Under the Gau ssian closure assumption , p ( t, x | e 0 ) = p ( t, x | e D ) where e D is obtain giv en e 0 and the Eq. (15). W e are interested in finding the i nd uced Dempster -Shaf er structure in e D ( t ) at time t > 0 . Given the knowledge about elements of e 0 as independe nt DS structures on closed intervals, we need to find the be lief structure associated with e D ( t ) . Th e DS-structure describing the uncer tainty for e D ( t ) can be obtained using Y a ger’ s conv olutio n ru le for DS-structures under the assumption of independ ence [24]: m [ e D ( t )]( C ij ) = X ˙ e D = g ( e D , e S ) e D (0) ∈ A i e S ∈ B j m [ e D (0)]( A i ) | {z } p Di m [ e S ]( B j ) | {z } p S j (16) where C ij = [ e Dij ( t ) , e Dij ( t )] , A i = [ e Di (0) , e Di (0)] an d B j = [ e S j , e S j ] . Here the o nly un known is the collection of focal elements C ij of e D ( t ) . Thus the prob lem of finding the mapping of a body of ev- idence on closed intervals is reduced to interval pro pagation [12]. This prob lem can b e solved u sing th e advanced tech- niques developed in the inter val an alysis field [1 0]. How- ev er, due to the dependence p roblem the obtained b ounds are conservati ve which is detrimental to the belief structu re, since the evidence is assigned automatica lly to other ele- ments wh ich ar e no t in th e bo dy of e vide nce. This p roblem becomes more acute wh en th e uncertainty has to be p ropa- gated over a perio d of time. The dep endence prob lem in in terval arithm etics can be av oided by using T ay lor models and the remainder differen- tial algebra for bou nding th e range of the response of ord i- nary dif feren tial equations und er both initial value and para- metric inter val unc ertainty [15, 2, 13]. In Ref.[4] are pre- sented se veral examples in pro pagating u ncertainties r epre- sented by probab ility boxes using T ay lor mod el method s and interval arithmetics. Also an excellent revie w on interval methods for initial value problems is p resented in Ref.[14]. A method based on polyn omial chaos an d th e Bern stein form is p resented in Ref .[21] to pr opagate DS-structures on closed intervals through nonlinear f unctions. Giv en a f unc- tion of random variables with co mpact sup port probability distributions, the intuition is to quantify the uncertainty in the respon se using polynom ial chao s expansion and discard all the inf ormation provided a bout the rand omness of the output and extract only th e bo unds o f its comp act support. T o solve for the bounding range of polynom ials, we h av e propo sed to transfo rm the polynomial chao s expansion into the Bernstein f orm, and use the ran ge enclosure pro perty of Bernstein po lynomials to fin d the minimum and maximum value of the resp onse [3]. The PCE is mathem atically attractive due to the func- tional rep resentations of the stoch astic variables. I t separ ates the determ inistic part in th e polyno mial coef ficients and th e stochastic part in the ortho gonal polyno mial basis. This b e- comes particularly usefu l in characterizing th e un certainty of the r esponse o f a d ynamical system re presented b y o rdi- nary d ifferential equation s with uncertain p arameters such as in Eq.(15). The result is a set of deterministic differen- tial equations which can be solved numerically to obtain the ev olution of the polyno mial coefficients. 4.2 Interv al Uncertainty pr opagation thr ough ODEs using PCE and the Bern stein F orm Giv en the initial v alue problem in Eq.(1 6 ) with in terval- valued initial co ndition and parameters, ˙ e k D = g k ( e D , e S ) (17) e D (0) ∈ [ A , A ] e S ∈ [ B , B ] our goal is to approxim ate the range of the state variables e k D at time t , where e k D is the k - th compon ent of e D ( t ) . Here, intervals of gener al typ e h av e b een used in ord er to reser ve the indexes for later use. The problem can be transformed into findin g the stochas- tic respon se under the assumptio n th at both the initial con- dition and the m odel parameters ar e uniformly d istributed. Thus we defin e e i D (0) ∼ U ( A i , A i ) and e j S ∼ U ( B j , B j ) , where e i D (0) and e j S is the i -th componen t of e D (0) and the j -th componen t of e S respectively . W e expand b oth the uncertain arguments and the resp onse of the system using the finite dimensiona l W iener-Aske y polyno mial chaos [23]: e l S = p − 1 X j =0 e l S j ψ j ( ξ l ) where ξ l ∼ U ( − 1 , 1) (18) e k D = P − 1 X i =0 e k Di ψ i ( ξ ) where P = ( r + p )! r ! p ! (19) Here r is the number o f u ncertain in put variables and is equal with the sum of th e size of the vector e D (0) and e S , and p is the o rder of the polynom ial expansion. The basis function ψ j is the j -th degree Legendre p olynom ial and ψ i is a multidimension al Legen dre polynomial and the po ly- nomial coefficients are initialize such that they match their initial uniform assumption. W e are inter ested in finding the p olynom ial coefficients e k Di of the response after t sec. Su bstituting Eqs.(18)-(19) in Eq.(17) and using the Galerkin pro jection an d the orthog o- nality pr operty of the p olynom ials one obtain s a system of P deterministic d ifferential equation s which can be solved numerically to obtain th e PC expan sion coefficients of the k -th momen t. ˙ e k Di = < g k , ψ k > < ψ 2 k > (20) where < . , . > represents the inn er pro duct op erator and can be ev aluated in gen eral using sampling or quadr ature technique s. In this particular case sin ce the no nlinearity is of polyno mial type, the inner prod uct between different L egen- dre b asis f unctions can be computed a priori , speed ing this way the numerical integration of the ODE. After t sec, by integratin g Eq.( 20), on ob tains the poly - nomial coefficients which define the stochastic re sponse of the system. H owe ver , we are only interested in finding the bound s that enclose th e respo nse of the system. In Ref.[21] it is shown that by b ringing the p olynomia l chaos expansion, Eq.(19), to a B ern stein form using the Garloff ’ s metho d [8], one can e ffi ciently find the range of the compact support thanks to the enclosing proper ty of Bernstein poly nomials. 4.3 DS structur es on Probability Sets By p ropagatin g the DS structu res throu gh the mo ment ev olution eq uations we obtain, at time t , an induced DS structure for th e momen ts that character ize the prob ability distribution of the response. Since in this paper we ha ve ch o- sen to approximate the con ditional pdf of the response using Gaussian den sity functio ns, we are only interested in the DS structures f or the first two cen tral m oments: the m ean, µ , and the variance, σ 2 . T hus, u sing Eq.(1 6 ) the following DS structure is obtained for the first two moments: e D ( t ) ∼  ([ e D 1 ( t ) , e D 1 ( t )] , p D 1 ) , ([ e D 2 ( t ) , e D 2 ( t )] , p D 2 ) , . . .  (21) For each f ocal elemen t we obta in a pair of two intervals that bound the range of the mean and the variance of a Gaus- sian density function . Consider n ow th e f ollowing Gau ssian density func- tion, N ( x ; µ, σ 2 ) , with un certain pa rameters, µ and σ 2 , giv en by two intervals: [ µ , µ ] and [ σ 2 , σ 2 ] . Let us de- note the cu mulative distribution fu nction N ( x f ; µ, σ 2 ) = R x f −∞ N ( x ; µ, σ 2 )d x . In Ref.[25] it is shown that all the normal cd fs are bo unded by two fu nctions, N ( x f ) ≤ N ( x f ; µ, σ 2 ) ≤ N ( x f ) , that can be computed analytically . N ( x f ) =  N ( x f ; µ , σ 2 ) x f ≥ µ N ( x f ; µ , σ 2 ) x f < µ (22) N ( x f ) =  N ( x f ; µ, σ 2 ) x f ≥ µ N ( x f ; µ, σ 2 ) x f < µ Using the above envelope pr operty and the D S structure in Eq.( 21), we obtain the fo llowing indu ced DS structure for the response of the system, P ( X ≤ x f | e 0 ) ∼   [ N 1 ( x f ) , N 1 ( x f )] , p D 1  , (23)  [ N 2 ( x f ) , N 2 ( x f )] , p D 2  , . . .  Thus, we are modeling a system with secon d order uncer- tainty an d o ur cred al set is defined as a DS struc ture over p-boxes. Having a stru cture like th is makes the pro blem o f decision m aking difficult, sinc e we are dealing with many en velopes of cd fs instead o f a sing leton cdf o r just an im- precise proba bility represented using a p-box. Howe ver, the following subsection pr esents how the pignistic transfo rma- tion can be used whenever decision making is needed. 4.4 Constructing a singleton CDF and Deci- sion Making using DS struct ures on Pr ob- ability Sets In order to compute e xpec tations using the interval-based belief fu nctions, one ne eds to build a probab ility d ensity function given the belief structu re. Based on Smets’ pig- nistic transfor mation one can d efine the pign istic p robability density function p B et ( x ) [17] as a finite mixture of contin- uous u niform distributions. Th us, gi ven the DS structure in Eq.(23), we can constru ct a singleton cdf using the pignistic transform ation as shown in Eq.24. P B et ( X ≤ x f ) = E B et  P ( X ≤ x f | e 0 )  (24) = 1 2 n D X i =1  N i ( x f ) + N i ( x f )  p Di Here, n D is the total nu mber o f fo cal elements in the body of evidence Eq.( 23 ). Notice that this is different from the law of total pr obability where a pro bability density fun ction is constructed for the epistemic v ariab les either e 0 or e D . In bo th cases the pignistic tr ansformatio n is applied much earlier in computin g the quan tity of interest. Furthermo re o ne can con struct an ign orance fun ction by applying Eq.(10) fo r each x f . Th is associates a p oint-wise measure of confidence in constructing the pignistic cdf. I g F ( x f ) = Z N ( x f ) N ( x f )  C P F P ( X ≤ x f | e 0 ) ( z ) − C B F P ( X ≤ x f | e 0 ) ( z )  d z (25) where N ( x f ) = min 1 ≤ i ≤ n D N i ( x f ) and N ( x f ) = max 1 ≤ i ≤ n D N i ( x f ) C P F P ( X ≤ x f | e 0 ) ( z ) = X N i ( x f ) ≤ z p D i C B F P ( X ≤ x f | e 0 ) ( z ) = X N i ( x f ) ≤ z p D i Similar to Eq.(10), we can co nstruct a scalar measure to summarize th e total amoun t of ig norance by integrating the above i gn orance function, N I i g F = 1 x max − x min Z x max x min I g F ( x f )d x f . (26) Giv en the pignistic cd f, x min and x max are the p th and (100 − p ) th perc entiles respectively (eq. p = 0 . 05 ). The N I i g F is a n umber between 0 . 0 an d 1 . 0 and can be used similarly as N I D I , in Section 3, to make decisions using the pign istic cdf in E q. (2 4), if it’ s value is small ( eq. less than 0 . 1 ) otherwise defer the decision if it is permitted an d gather more evidence. Again, a value of 1 . 0 d enotes that we are dealing with interval uncertainty and a value of 0 . 0 means that we now the cdf precisely . 5 Numerical Simulation Consider th e following linear d ynamic system d riv en b y Gaussian white noise: ˙ x + a 1 x = Γ 1 ( t ) (27) p ( t 0 , x 0 ) = N ( x 0 ; 1 . 1 , 2 . 42) (28) where the autocorr elation function of the noise is E [Γ 1 ( t )Γ 1 ( τ )] = q 2 1 δ ( t − τ ) , and both a 1 , q 1 are ep istemic variable, described by the follo wing body of e vide nce: a 1 ∼  ([0 . 86 , 0 . 9] , 0 . 2) , ([0 . 89 , 0 . 96] , 0 . 8)  (29) q 1 ∼  ([0 . 2 , 0 . 3] , 0 . 6) , ([0 . 3 , 0 . 4] , 0 . 4)  (30) W e are interested in finding the quantity of interest P ( t = 2 , Y ≤ − 0 . 5) as well as constru ct the entire cdf, P ( t = 2 , Y ≤ y ) , after t = 2 sec. Here, the exact cond itional probab ility d ensity fun ction is Gaussian, p ( t, x | a 1 , q 1 ) = N ( x ; µ ( t ) , σ 2 ( t ) | a 1 , q 1 ) (31) because the m odel is linear a nd the initial condition and the process noise are normally distributed. Thus, we ar e interested in finding the indu ced DS struc- tures for th e moments of th e no rmal distribution in Eq.(31). The time e volution of mom ents through which the epistemic uncertainty is propaga ted is given by: ˙ m 1 = − a 1 m 1 (32) ˙ m 2 = − 2 a 1 m 2 + q 2 1 m 1 (0) = 1 . 1 and m 2 (0) = 2 . 42 Using the PCE - Bernstein form described in Section 4.2 to propag ate interval uncer tainties throu gh the ODE in Eq.(32), and so lving for the central momen ts we find t he DS structure for µ an d σ 2 after t = 2 sec, shown here in T ab le 1 and graphic ally represen ted in Fig.1a. The DS structure over p-b oxes is ob tained by using th e en velope pr operty in Eq.(22). The focal elemen ts an d their bpa for P ( t = 2 , Y ≤ y | a 1 , q 1 ) are presented in Fig.1b. From this structu re it is easy to obtain the D S structure for P ( t = 2 , Y ≤ − 0 . 5 | a 1 , q 1 ) as it is shown in Fig.1c. Now , using the pign istic p df for P ( t = 2 , Y ≤ − 0 . 5 | a 1 , q 1 ) one can construct an estimate for the q uantity of interest using Eq.(24) as well as provid e a measur e of co nfidence fo r th is estimate: P Bet ( t = 2 , Y ≤ − 0 . 5) = 1 . 07% with N I DI = 1 . 56% (33) Eq.(24) can be used to construc t a singleton cdf for y to take an action u sing th e expected u tility th eory . Also using Eq.(25) we can obtain an ignoran ce function to indicate the point-wise confid ence in the constructed cdf as well as pro- vide a scalar mea sure of trust, Eq.(26), in using this cd f in decision m aking. Both the constructed cdf and the ignor ance function are presented in Fig.1f. Just fo r comp arison purp oses the pr obability theory h as be used to obtain the total probab ility of y und er the as- sumption that the ep istemic variables v ary rand omly . Us- ing Lap lace’ s Principle of In sufficient Reasoning an d we can transform the epistemic variables a 1 and q 1 into the aleatory variables a ∗ 1 and q ∗ 1 . For th is, we u se the p ignistic tr ansfor- mation to find the following pign istic pdfs: p B et ( a ∗ 1 ) = 0 . 2 U (0 . 86 , 0 . 9) + 0 . 8 U (0 . 8 90 . 96) (34) p B et ( q ∗ 1 ) = 0 . 6 U (0 . 2 , 0 . 3) + 0 . 4 U (0 . 3 , 0 . 4 ) W ith this transforma tion the uncertainty in the mo del in Eq.(27) is represented solely by p robabilities. Thus an esti- mate for the quantity of interest is calculated as in Eq.35. P M C ( t = 2 , Y ≤ − 0 . 5) = Z V P ( t = 2 , Y ≤ − 0 . 5 | a ∗ 1 , q ∗ 1 ) p Bet ( a ∗ 1 ) p Bet ( q ∗ 1 )d a ∗ 1 d q ∗ 1 (35) = Z V  Z − 0 . 5 −∞ N ( x ; µ, σ 2 | a ∗ 1 , q ∗ 1 )d x  p Bet ( a ∗ 1 ) p Bet ( q ∗ 1 )d a ∗ 1 d q ∗ 1 = 0 . 85% In Fig.1e it is plotted the h istogram for P ( Y ≤ − 0 . 5 | a ∗ 1 , q ∗ 1 ) given 1 million samples drawn fr om the pd fs in Eq .(34). Also the emp irical cdf f or P ( Y ≤ − 0 . 5 | a ∗ 1 , q ∗ 1 ) is sho wn in Fig.1d and the total cumulative pr obability for y is presented in Fig.1f. The pro babilistic estimate fo r the qua ntity of in terest dif- fers fro m the one giv en by the present app roach due to the assumption that the mod el parameter s vary rand omly . Since we ha ve used the pignistic transformation to obtain a pdf for the mod el param eters and then propag ate this pdf through the dynam ic system, the conc entration in the pro bability mass seen in Fig.1e can not be exp lained gi ven the evidence we h av e started with. This argument is nice ly explained fo r a simple example by Ferson in Ref.[5]. More over the N I D I measure indicates how m uch can we trust our estimate given our lack of knowledge about the model parameter s. The threshold that indicates if th e estimate can be trusted can only depend on the mag nitude of the conseque nces is th e state of the system drops below − 0 . 5 . The two cdf for y in Fig.1f are slightly dif fer ent, howev er for the cdf constructed using the present metho d we also in- clude an ignoranc e function as a poin t-wise measure of con- fidence and a numb er , N I ig F , which similar with N I D I . Here, while the pro babilistic approach o ffers no other alter- native ju st to use th e expected utility theory to take an action , our ap proach thro ugh the u se o f the ignoranc e function and the N I i g F measur e can postpone the decision mak ing if the amount of ign orance is high. Th e use of ig norance functio n and N I ig F re mains to b e stud ied in th e decision making problem , and is not the purp ose of the cu rrent paper . T able 1: The induced DS structure for { µ, σ 2 } m a 1 ([0 . 86 , 0 . 9] ) = 0 . 2 m a 1 ([0 . 89 , 0 . 96 ]) = 0 . 8 m q 1 ([0 . 2 , 0 . 3]) = 0 . 6 m { µ,σ 2 }  [0 . 182 , 0 . 197 ] , [0 . 055 , 0 . 090]  = 0 . 12 m { µ,σ 2 }  [0 . 161 , 0 . 186] , [0 . 047 , 0 . 084 ]  = 0 . 48 m q 1 ([0 . 3 , 0 . 4]) = 0 . 4 m { µ,σ 2 }  [0 . 182 , 0 . 197 ] , [0 . 082 , 0 . 129]  = 0 . 08 m { µ,σ 2 }  [0 . 161 , 0 . 186] , [0 . 076 , 0 . 122 ]  = 0 . 32 0.16 0.17 0.18 0.19 0.2 0.04 0.06 0.08 0.1 0.12 0.14 0.12 0.48 0.08 0.32 µ σ 2 (a) DS-structure for { µ, σ 2 } −1 −0.5 0 0.5 1 1.5 0 0.2 0.4 0.6 0.8 1 y P(t=2, Y <= y | a 1 , q 1 ) 0.32 0.48 0.12 0.08 quantity of interest (b) DS-structure for P ( t = 2 , Y ≤ y | a 1 , q 1 ) 0 0.005 0.01 0.015 0.02 0.025 0.03 1 2 3 4 0.48 0.12 0.32 0.08 P(t=2, Y <= −0.5 | a 1 , q 1 ) Focal element (c) DS-structure for P ( t = 2 , Y ≤ − 0 . 5 | a 1 , q 1 ) 0.005 0.01 0.015 0.02 0.025 0 0.2 0.4 0.6 0.8 1 P(t=2, Y <= −0.5 | a 1 , q 1 ) CDF CDF Bet CBF CPF CDF MC P MC (t=2, Y <= −0.5) = 0.85% P Bet (t=2, Y <= −0.5) = 1.07% NIDI = 1.56% (d) P-box and pigni stic cdf for P ( t = 2 , Y ≤ − 0 . 5 | a 1 , q 1 ) 0 0.005 0.01 0.015 0.02 0.025 0.03 0 500 1000 1500 2000 2500 3000 3500 4000 P(t=2, Y <= −0.5 | a 1 * , q 1 * ) #samples (e) Histogram for P ( t = 2 , Y ≤ − 0 . 5 | a ∗ 1 , q ∗ 1 ) −1 −0.5 0 0.5 1 1.5 0 0.2 0.4 0.6 0.8 1 y P(Y <= y) CDF Bet IgF CDF MC NIigF = 4% (f) Constructe d CDF for y and Ignoranc e Functi on Figure 1: Num erical simulation: DS structures, pignistic cdfs and ignoran ce fun ction after t = 2 sec 6 Conclusions A new fram e work has been presented to p ropagate both aleatory and epistemic uncertain ty th rough d ynamic systems with stochastic forcing. This has b een achieved by u sing a second-o rder uncertainty m odel to prop agate both types of uncertainty . First an app roximation to the probability density function of the respon se is assumed a nd it is used in the momen t closure scheme to find th e correspon ding time evolution o f the mom ents. The ep istemic uncertain ty is mapped thro ugh these equations and the final response of the system is mod- eled as a Dempster Shafer structure on prob ability boxes. The paper in corpora tes a previous work of th e au thor to propag ate interval u ncertainties through ordinary differen- tial equations using the poly nomial chaos expansion and the Bernstein fo rm. Howe ver , other me thods such as T aylor models an d interval arithmetic can be used to prop agated the focal elements throug h the mome nt equations. It is shown that the Demster Shafer stru ctures o btained can be used in constructing estimates of quantity of inter- ests, s uch as pro babilities o f failure, and also in constructing entire cumulative density function s which can be used in decision makin g. In ad dition, an ign orance function and a scalar measure can be comp uted in ord er to hint the confidenc e that on e can have in th e estimates given the initial bod y of evidence concern ing the model parameter s. For now one may ch oose not to pur sue with the decision making if the scalar measure is over a certain thresho ld. The use of the ig norance functio n in c onjunction w ith the expected utility theory is set as future work. Acknowledgment: This work was supp orted under Contract No. HM1582- 08-1-0 012 fr om ONR. Refer ences [1] Milton Abramowitz and Irene A. Ste gun. Handbook of Math- ematical Functions with F ormulas, Grap hs, and Mathemati- cal T ables . Dov er Publications, 1972. [2] M. Berz and K. Makino. V eri fied integration of odes and flows usin g dif ferential algebraic methods on high-order tay- lor models. Reliable Computing , 4:3613 69, 1998. [3] G. T . Cargo and O. Shisha. T he bernstein form of a polyno- mial. Jou rnal of Resear ch of Nat. Bur . Standar ds , 7 0B:79–81, 1966. [4] Joshua A. E nszer , Y oudon g Lin, Scott Ferson, George F . Corliss, and Mark A. Stadtherr . Propagating uncertainties i n modeling nonlinear dynamic systems. [5] S. Ferson and L. R. Ginzbu rg. Different methods are needed to propagate ignorance and variability . Reli ability Engineer- ing and System Safety , 54:133–1 44, 1996. [6] Scott F erson, Vladik Kreinovich, Le v Ginzb urg, David S . Myers, and Kari Sentz. Constructing probability boxes and dempster-sh afer structures. T echnical rep ort, Sandia National Laboratories, 2003. [7] A. T . Fuller . Analysis of nonlinear stochastic systems by means of the fokk er-planck equation. International Journ al of Contr ol , 9:6, 1969. [8] J. Garlof f. Con ver gent bounds for the range of multiv ariate polynomials. Interval Mathematics , 212 :37–56, 1985. [9] R. Haenni. Ignoring ignorance is ignorant. T echnical r eport, Center for Junior Research Fello ws, Unive rsity of K onstanz, 2003. [10] Luc Jaulin, Michel Kieffer , Ol iv ier Didrit, and Eric W alter . Applied Interval Analysis . Springer , 2001. [11] Andre w H. Jazwinski. Stochastic Proce sses and Filtering Theory . Academic Press, 1970. [12] Philipp Limbourg. Dependa bility Modelling under Uncer- tainty: An Impr ecise Proba bilistic Appr oach . Springer , 200 8. [13] Y . Lin and M. A. St adtherr . V alidated solutions of initial v alue problems for parame tric odes. Applied Nu merical Ma themat- ics , 57:1145 1162, 2007. [14] N. S. Nedialkov , K. R. Jackson, and G. F . Corliss. V al- idated solutions of initial value problems for ordinary dif- ferential equations. Applied Mathematics and Computation , 105:2168 , 1999. [15] N. S. Nedialkov , K. R. Jackson, and J. D. P ryce. An effec- tiv e high-order interval method for validating ex istence and uniqueness of the solution of an ivp for an ode. Reliable Computing , 7:44946 5, 2001. [16] W ill iam L. Oberkampf, Jon C. Helton, Cliff A. Joslyn, Stev en F . W ojtkie wicz, and Scott Ferson. Challenge prob- lems: Uncertainty in system response given uncertain param- eters. Reli ability Engineering and System Safety , 85:11–19, 2004. [17] Simon Petit-Renaud and Thierry Denoeux. Nonparametric regression analysis of unce rtain and imprecise data using be- lief function s. Internation al Journal of Appr oximate Reason- ing , 35(1):1 – 28, 2004. [18] H. Risken. The F okker-Planc k Equation: Methods of Solution and Applications . Springer , 1989, 1-12, 32-62. [19] Ph. Smets and R. K ennes. The transferable belief model. Ar- tificial Intelligenc e , 66:191–23 4, 1994. [20] Gabriel T erejanu, Puneet Singla, T arunraj Singh , and Peter D. Scott. Uncertainty propagation for nonlinear dynamical sys- tems using gaussian mixture models. Jou rnal of Guidance, Contr ol, and Dynamics , 31:1623 –1633, 2008. [21] Gabriel T erejanu, Puneet Singla, T arunraj Singh , and Peter D. Scott. Approx imate interv al method for epistemic uncertainty propagation using polynomial chaos and ev idence theory . In Accepted - The 2010 A merican Contro l Confer ence, B alti- mor e, Maryland , 2010. [22] Peter W alley . Statistical Reasoning with Impr ecise Pro ba- bilities (Mono graphs on Statistics and Applied Prob ability) . Chapman and Hall, 1991 . [23] Dongbin Xiu and Geor ge Em K arniadakis. The wiener -askey polynomial chaos for stochastic dif ferential equations. SIAM J . Sci. Comput. , 24(2):619–644, 2002. [24] Ronald R. Y ager . Ari thmetic and other operations on dempster-sh afer structures. Int. J . Man-Mach . Stud. , 25(4):357–3 66, 1986. [25] Jianzhong Zhang and Daniel Berleant. En velope s around cu- mulativ e distribution functions f rom i nterv al parameters of standard continuous distributions. In North A merican Fuzzy Information Pr ocessing Society , pages 407–412 , 2003.