Decisiveness of Stochastic Systems and its Application to Hybrid Models

J.-F . Raskin and D. Bresolin (Eds.): 11th International Symposium on Games, Automata, L ogics, and Formal V erification (GandALF’20). EPTCS 326, 2020, pp. 149–165, doi:10.4204/EPTCS.326.10 c  Bouyer , Brihaye, Randour , Rivière & V andenhov e This work is licensed under the Creativ e Commons Attribution License. Decisiveness of Stochastic Systems and its A p p l i c ation to Hybrid Models ∗ Patricia Bouyer LSV , CNRS & ENS Paris-Saclay , Univ ersit é Paris-Saclay , France Thomas Brihaye UMONS – Univ ersité de Mons, Belgium Mickael Randour F .R.S.-FNRS & UMONS – Univ ersité de Mons, B elgium Cédric Rivière UMONS – Univ ersit é de Mons, Belgium Pierre V andenhov e F .R.S.-FNRS & UMONS – Univ ersité de Mons, B elgium LSV , CNRS & ENS Paris-Saclay , Univ ersité Paris-Saclay , France In [3], Ab d ulla et al. introd uced the concep t o f decisiveness, an interesting tool fo r lifting goo d prop- erties of fin ite Markov ch ains to de n umerab le on es. Later [10], this co ncept was extended to mo r e general stochastic transition systems (STSs), allowing the de sign of various verification a lg orithms for large classes of ( infinite) STSs. W e further improve the und erstanding an d utility of dec isiveness in two ways. First, we pr ovide a g eneral criterio n for provin g the dec isiveness of gen eral STSs. This criter ion, which is v ery natural b u t whose proof is rather techn ical, (strictly) gene r alizes all known criteria from the literatur e. Second, we focus on sto c h astic h y brid systems (SHSs), a stoch astic extension of hybrid systems. W e establish th e decisiv eness of a large class of SHSs and, und er a fe w classical hyp otheses from mathematical logic, we show how to decide reachability pro blems in th is class, even tho ugh they ar e und e c idable fo r general SHSs. T his p rovides a d ecidable stochastic e xtension o f o-minimal hybrid sy stem s [1 6, 22 , 3 2]. 1 Introd uction Hybrid and stochastic models. V ariou s kinds of mathematical m odels hav e been propos ed to repres ent real-l ife systems . In this article, we focus on models combini ng hybri d and stoc hastic aspects. W e outline the main featu res of these mode ls to motiv ate our approac h. The idea of hybrid systems originat es from the ur ge to study systems subject to both discr ete and contin uous phenomena, such as digital computer systems interacti ng with analog data. These systems are transit ion systems w ith two kinds of transitions : continuo us tra nsitions , where some continuo us v ariables e vo lve over time (e.g., according to a dif ferent ial equ ation), an d discre te transitions , w here the system chang es modes and v ariables can be reset . Much of the research about hybrid systems focuses on non- determin istic hyb rid sy stems, i.e., systems modeling uncertainty by considering all possible beha viors (e.g., dif ferent possibi lities for discrete transitio ns at a gi ven time, arbitrary long time between discre te transit ions). A typical questi on concerns the safety of such systems—if a system can reach an undesirab le state, it is said to be unsafe ; if not, it is safe —such a specification is called qualitative . Ho wev er , this is limiting for two reaso ns. First, it does not tak e into account that some behavi ors are more likely to ∗ Research supported by F .R. S .-FNRS under Grant n ◦ F .4520.18 (ManySynth), and ENS Paris-Saclay visiti ng professorship (M. Randour , 2019). Pierre V andenh ove is an F .R.S-FNRS Research Fellow . Mickael Randour is an F .R. S-FNRS Research Associate. 150 Decisi veness of Stocha stic Systems and its Application to Hybrid Mod els occur than others. S econd, risks cannot necess arily be av oided, and it is unrealisti c to pre ven t undesira ble outcome s altogethe r . Therefore, we want to make probabilitie s an inte gral part of our models, in order to be able to quanti fy the probability that they beha ve accord ing to the specifica tion. W e thus consider the class of stocha stic hybrid systems (SHSs, for short), hybrid syste ms in which a stochast ic semantics replac es non-determini sm. Goals. Our interest lies in the formal analys is of continuous -time SHSs, and more specifically in r eachabil ity qu estions, i.e., concern ing th e likeli hood that some set of states is reached. The questions we seek to answer are both of the qualitat iv e kind (is some regio n of the state space almost sur ely reached, i.e., reached with probability 1? ) and of the quantitati ve kind (what is the probabilit y that some states are e ventu ally reached?). Suc h questi ons are crucial, as verify ing that a system works safel y often reduces to verify ing that some un desirabl e state of the sy stem is nev er re ached (or w ith a lo w pro bability ), or that some desirable state is to be reached with high probabi lity [8]. W e want to giv e algorithms that decide, for an SHS H and reachabilit y proper ty P , whether P is satisfied in H . Such an endeav or f aces multiple challe nges; a first obvious one being that e ven for rather restricte d clas ses of non-d eterministi c hybrid systems, reachabil ity problems are undecid able [24, 25]. W e want to define an d consider a class of S HSs for which some reach ability problems are decidable. Methods and contrib utions. O ur methodol ogy consists of two main step s. In a fi rst step, we fol- lo w the appr oach of Bertrand et al. [10]: we study general stoch astic tra nsition syst ems (STSs) through the decisivenes s concep t (Secti on 2). T he class of S TSs is a very versa tile class of systems encompas sing many well-k no wn families of stoc hastic systems, such as Marko v chains, generaliz ed semi-Mark ov pro- cesses , stocha stic timed automata, stochastic Petri nets, and stoch astic hy brid systems. D ecisi veness was introd uced in [3] to study Marko v chains, and exten ded to STSs in [10]. An STS is said to be decis ive with respect to a set of st ates B if e xecu tions of the system almost surely reac h B or a state fro m which B is unreachab le. Decisiv e ST S s bene fit from useful properties that make possible the design of verificatio n algori thms related to reachability prope rties. Our first co ntrib ution is to pro vide a criterion to check the decisi veness of STSs (Proposit ion 2.8 ), which generaliz es the decisi venes s criteria from [3, 10]. This genera lization was mentio ned as an op en prob lem in [10]. In a secon d step, w e focus on stoc hastic hybri d systems , which we introduce in Section 3.1. Reach- ability pro blems in hybrid sys tems are notorious ly undecidable [24, 25] . Our contrib utions rega rding SHSs are split into two parts . First, we aim to use the decisi veness idea to get closer to the deci dability frontier . Albeit desirable, the dec isi venes s of a cla ss of SHSs is not suf fi cient to ha ndle algorithmic questio ns about each SHS, as we need an effe cti ve way to apprehend their uncountab le state space. In this regard, an often-used techni que is to consider a finite abs tract ion of th e system, that is, a finite partitio n of the state space that preser ves the properti es to be verified (a well-kno wn example is the region graph for timed automata [5]). T o find such an abstracti on of SHSs, we borro w id eas from [16, 32]: we consider SHSs with str ong re sets (Section 3.2), a syntac tic conditio n that de couples th eir con tinuous be ha vior from their discrete beha vior . W e sho w th at SH Ss with adequately p laced str ong r esets (at least one per c ycle of their disc rete grap h) ( i ) hav e a finite abstraction (P roposi tion 3.11), and ( ii ) are decisiv e (Propositio n 3.9), which can be pro ved usin g our ne w criterio n. Second, in S ection 3.3, we sho w , with strong resets, how to effect iv ely compute a finite abstra ction and use it to perform a reachability ana lysis of the original system. W e proceed by assuming that the compone nts of our systems are definable in an o-minimal structu r e . The main difficul ty here is that “ a Bouyer , Brihaye, Randou r , Riv ière & V andenho ve 151 satisfa ctory theory of m easur e and inte grat ion seems to be lac king in the o-minimal conte xt ” [9]; in an o-minimal structu re, the primiti ve functio n of a definable function is in general not definable, which complica tes definabilit y quest ions rega rding proba bilities. W e therefor e restrict the possi ble probabi lity distrib utions, using propertie s of o-minimal structu res to kee p our class as lar ge as possible. When the theory of the structu re is decidable (as is the case for the ordere d fi eld of real numbers [43]), the r eachability pr oblems are then decidable . This prov ides a stochas tic extens ion to the theory of o- minimal hybrid systems [32]. W e study qualitativ e an d q uantitati ve problems. Due to space constrain ts, all the missing proofs and some technica l details and additio nal results are omitted from this ve rsion and can be found in th e full art icle [14]. Related work. O ur results combine previ ous work on stochastic systems and hybrid systems. About stoch astic syste ms, we b uild on the work of [3, 10]. Our work also takes inspira tion from re- search about stochasti c timed automata , a su bclass of stoc hastic hy brid systems whic h alre ady combines stocha stic and timed aspects; model checking of stochas tic time d automata has been studied in [7, 11, 12] and conside red in the conte xt of the deci siv eness property in [18]. Fundamenta l results about the decid - ability of the reachabi lity proble m for hybrid systems can be found in [5, 24, 25]. T he class of o-minimal hybrid sy stems , of which we introdu ce a stochas tic extensi on, has been stu died in [16, 22, 32]. The lit erature about S H Ss often follo ws a more practical or nu merical appro ach. A first introductio n to the model was prov ided in [29]. An ext ensi ve re vie w of the unde rlying theory and of many applicat ions of S HSs is pro vided in books [19, 17], and a re vie w of dif ferent possible seman tics for this model is pro vided in [34]. Applicatio ns of SHSs are numerous: a few exa mples are air traf fic management [40, 41], communic ation networ ks [26], biochemical processes [33, 42]. The softwar e tool U P P A A L [20] implements a model of SHS similar to the on e studied in this article and uses numerical methods to compute rea chability probab ilities, through numerica l solv ing of differe ntial equations and Monte Carlo simulatio n. Reachab ility problems ha ve also been conside red in an alternati ve semantics with discr ete time ; a numerical appro ach is for instance provid ed in [1, 2]. Notations. W e write R + = { x ∈ R | x ≥ 0 } for the set of no n-neg ati ve real numbe rs. L et ( Ω , Σ ) be a measurabl e space. W e write Dist ( Ω , Σ ) (or Dist ( Ω ) if there is no ambiguit y) for the set of pr obabil ity distrib utions ov er ( Ω , Σ ) . The comple ment of a set A ∈ Σ is deno ted by A c = Ω \ A . For A ∈ Σ , we say that two pro bability distrib utions µ , ν ∈ Dist ( Ω ) are (qualita tively) equival ent on A if for each B ∈ Σ , if B ⊆ A , µ ( B ) > 0 if and only if ν ( B ) > 0. 2 Decisiv eness of Stochastic T ransition Syst em s In this section, we define stochast ic transi tion syst ems (STSs, for short) as in [10]. W e then describe the concept of decisiv eness , fi rst defined in the spe cific case of Markov chains [3 ], and then extended to STS s [10]. Decisiv e stochast ic systems benefit from “nic e” pro perties making their qualit ati ve and quanti tati ve analysi s more accessib le. The first contrib ution of our work is a new decisi veness criterion (Proposi tion 2.8), which generalizes existin g crite ria fr om the literatur e [3, 10]. It is an intuiti ve criterio n, which was conjecture d in [10] bu t could not be prov ed. W e finish with a brief section on the notion of abstr action between STSs, which will be useful to apply our re sults to stoc hastic hybrid sys tems. 152 Decisi veness of Stocha stic Systems and its Application to Hybrid Mod els 2.1 Stochastic T ransition Systems and Decisiveness [10] Definition 2.1 (Stochastic transitio n system) . A stoc hastic transitio n system (STS) is a tuple T = ( S , Σ , κ ) consisti ng of a measur able space of state s ( S , Σ ) , and a functio n κ : S × Σ → [ 0 , 1 ] (sometimes called Mark ov k ernel ) such that for e very s ∈ S , κ ( s , · ) is a prob ability measure and for ev ery A ∈ Σ , κ ( · , A ) is a measura ble function. The secon d cond ition on κ implies in pa rticular that for a measura ble set B ∈ Σ , the set { s ∈ S | κ ( s , B ) > 0 } is measu rable. W e interpre t ST Ss as sys tems generati ng runs, with a proba bility measure ov er these runs. W e fix an STS T = ( S , Σ , κ ) . From a state s ∈ S , a probabil istic transitio n is perfor med accor ding to distrib ution κ ( s , · ) , and the system resumes from one of the successor states; this process generat es random sequenc es of states. A run of T is an infinite sequence s 0 s 1 s 2 . . . of sta tes. T o formally provi de a probabilisti c semantic s to STS s, we define a probability measure over the set of runs of T . From an initial distrib ution µ ∈ Dist ( S ) , we can define in a classical way a unique prob ability measure Prob T µ on the σ -algeb ra genera ted by all the cylinder s , using Carathéod ory’ s extension theo rem. Expre ssing pr operties of runs. W e use stand ard L TL notati ons [39] to exp ress prop erties of runs of T . If B , B ′ ∈ Σ , we write in particul ar F B (resp. F ≤ n B , B ′ U B , B ′ U ≤ n B , G B , G F B ) for the set of runs that visit B at some point (resp. visit B in les s than n steps, sta y in B ′ until a first visit to B , stay in B ′ until a first visit to B in less than n steps , always stay in B , visit B infinitely often). W e are intereste d in two kinds of reachabi lity prob lems. Definition 2.2 (Qualitati ve and quanti tati ve reachabili ty) . Let B ∈ Σ be a measurab le set of targ et states, and µ ∈ Dist ( S ) be an initial distrib ution. The qualitativ e r eachabil ity pr oblems consi st in decidin g whether Prob T µ ( F B ) = 1, and whethe r Prob T µ ( F B ) = 0. The quantita tive r eachabil ity pr oblem consis ts in deci ding, gi ven ε , p ∈ Q with ε > 0, whether | Prob T µ ( F B ) − p | < ε . T ra nsfor ming probability distributions. Another useful way to reflect on S TSs is as transformers of prob ability distrib utions on ( S , Σ ) . Definition 2.3 (S T S as a transformer ) . F or µ ∈ Dist ( S ) , its tr ansformat ion thr ough T is the probability distrib ution Ω T ( µ ) ∈ Dist ( S ) defined fo r A ∈ Σ by Ω T ( µ )( A ) = Z s ∈ S κ ( s , A ) d µ ( s ) . The v alue Ω T ( µ )( A ) is the prob ability to reac h A in on e step, fro m the in itial distrib ution µ . Attracto rs. W e will be part icularly intereste d in the exi stence of attr actor s for STSs. Definition 2.4 (Attractor) . A set A ∈ Σ is an attracto r for T if for eve ry µ ∈ Dist ( S ) , Prob T µ ( F A ) = 1. While S is always an attractor for T , we will later search for attractors with more inte resting prop- erties. The definiti on of attrac tor actually implies a seemingly strong er statement: an attractor is almos t surely vis ited infinit ely often from an y initi al distrib ution [10, Lemma 19]. Bouyer , Brihaye, Randou r , Riv ière & V andenho ve 153 Decisi veness. Bef ore introdu cing dec isi vene ss, we giv e the definition of an avoid- set : f or B ∈ Σ , its avoid-set is written as e B = { s ∈ S | Prob T δ s ( F B ) = 0 } (where δ s is the Dirac distrib ution at s ). The a vo id-set e B corresp onds to the set of states from w hich runs almost surely stay out of B ad infinitum . One can show that the set e B belongs to the σ -algeb ra Σ [10, Lemma 14]. W e can no w define the concept of decisi veness as in [10]. Definition 2.5 (Decisi veness) . Let B ∈ Σ be a measurable set. W e say that T is decisi ve w .r .t. B if for e very µ ∈ Dist ( S ) , Prob T µ ( F B ∨ F e B ) = 1. Intuiti vely , the decisi vene ss prope rty states that, almost surely , either B will e ventual ly be visited, or states fro m which B can no longer be reache d will e ven tually be visi ted. Example 2.6 (Random walk) . W e conside r the STS T (random walk) from Figure 1. W e want to fi nd out whether T is decisi ve w .r .t. B = { 0 } . W e assume that the initial di strib ution is giv en by δ 1 (the Dirac distrib ution at 1). B y the theory on random walks, if 1 2 < p < 1, the walk will almost surely div er ge to ∞ . This entails that Prob T δ 1 ( F B ) < 1. Moreo ver , sin ce p < 1, the re is a path w ith positiv e prob ability from ev ery state to 0, so e B = / 0. Therefore, Prob T δ 1 ( F B ∨ F e B ) = Prob T δ 1 ( F B ) < 1 , which means that T is not decisi ve w .r .t. B . If p ≤ 1 2 , state 0 is almost surely reached from any state, that is, for any initia l distrib ution µ ∈ Dist ( S ) , we hav e that Prob T µ ( F B ) = 1. Hence, in this case, STS T is dec isi ve w .r .t. B . . . . 0 1 2 1 p p 1 − p 1 − p 1 − p Figure 1: STS T represen ting a random walk on N . A m ajor interest of the decisi venes s concep t lies in the design of simple procedu res fo r the qualitat iv e and quantit ati ve ana lysis of stochasti c systems. Indeed, as present ed in [3 , 10], it allo ws, under ef fec- ti vene ss hypot heses, to verify qualitati ve proper ties or to compute arbitr ary close approxi mations of the probab ilities of vari ous prop erties, lik e reac hability , repeate d reacha bility , and ev en ω -reg ular properties. The reader is referred to [10, S ections 6 & 7] for more details, but we briefly recall the approximatio n scheme for reachab ility prope rties in orde r to il lustrate the usefulne ss of the decisi veness prop erty . This scheme will be appl ied to a specific class of STSs in Section 3.3. Let B ∈ Σ be a measurable set and µ ∈ Dist ( S ) be an initial distrib ution. T o compute an approximation of Prob T µ ( F B ) , we define two sequences ( p Y es n ) n ∈ N and ( p No n ) n ∈ N such that for n ∈ N , p Y es n = Prob T µ ( F ≤ n B ) and p No n = Prob T µ ( B c U ≤ n e B ) . These sequence s are non-de creasing and con ver ge respecti vely to Prob T µ ( F B ) and Prob T µ ( B c U e B ) . Ob- serv e moreo ver that for all n ∈ N , we hav e that p Y es n ≤ Prob T µ ( F B ) ≤ 1 − p No n . The main idea behin d decisi veness of ST Ss lies in the followin g property [3, 10]: if T is decisi ve w .r .t. B , then lim n → ∞ p Y es n + p No n = 1. T herefo re, for any giv en ε > 0, for some n sufficie ntly large , p Y es n ≤ Prob T µ ( F B ) ≤ p Y es n + ε . In situations where p Y es n and p No n can be ef fecti vely approximated arb itrarily closel y and T is deci si ve w .r .t. B , we can thus appro ximate Prob T µ ( F B ) up to any desired error bound. 154 Decisi veness of Stocha stic Systems and its Application to Hybrid Mod els 2.2 A New Criterion for Decisiveness Our go al is to pro vide ne w suf ficient condit ions for the decisi veness of S TSs. T o this end, we expose the follo wing crucial techni cal lemma. Lemma 2.7. Let B ∈ Σ , and A ∈ Σ . Suppose that ther e is p > 0 such that for all ν ∈ Dist ( A ) , we have Prob T ν ( F B ) ≥ p. T hen for any µ ∈ Dist ( S ) , Prob T µ ( G B c ∧ G F A ) = 0 . This result seems rather intuiti ve: if we visit A infinitely often, and after ev ery passage throug h A we hav e a probability bound ed from belo w to reach B , then the proba bility to stay in B c fore ver is 0. An equi v alent statement for Marko v chains has been used w ithout proof in [3, Lemmas 3.4 & 3.7]. A weake r vers ion of this sta tement is giv en as part of the proo f of [10, Proposit ion 36], where it is said that this gene ral case was not kn o wn to be true or f alse. This weake r version assu mes that there is a un iform upper bou nd k such that for all ν ∈ Dist ( A ) , Prob T ν ( F ≤ k B ) ≥ p to obtain a similar co nclusion . W e ha ve remov ed the need for this con straint. A possibl e proof of this resu lt consists of rega rding a stochastic transition system as a stoc hastic proces s, and resorting to results from martinga le theory . More precisel y , using Lévy’ s zer o-one law , we obtain that infinite runs that ne ver reach B are the same (up to a set of probability 0) as the infinite runs s 0 s 1 . . . for w hich the probabil ity to reach B gi ven s 0 . . . s n con ver ges to 0 as n gro ws to infinity . Runs th at visit A infinite ly often cannot bot h a vo id B and ha ve a probabili ty to vis it B that co n ver ges to 0 (since for e very visit to A , the proba bility to visit B is bounded from belo w by p > 0) . Therefore, such runs will almost su rely visit B . W e can no w state our main contrib ution to de cisi ven ess. Pro position 2.8 (Decisi ven ess criterio n) . Let B ∈ Σ be a measurabl e set, and A ∈ Σ be an attrac tor for T . W e den ote A ′ = A ∩ ( e B ) c the set of states of A fr om which B is rea chabl e with a positive pr obabili ty . Assume that ther e exi sts p > 0 such that for all ν ∈ Dist ( A ′ ) , Prob T ν ( F B ) ≥ p. Then T is deci sive w .r .t. B. W ith prob ability 1, ev ery ru n visits att ractor A infinitely ofte n, b ut the hypothes es imply a dichotomy between run s. Some run s will rea ch a state of A from which B is almost surely non-rea chable (i.e., in e B ). The ot her ru ns will go infinitely ofte n throug h states of A such that the probabi lity of re aching B is lo wer bound ed by p (i.e., in A ′ ), and w ill almost surely visit B by L emm a 2.7. This almos t-sure dichotomy between run s is requ ired to sho w decisi veness. This criterio n strictly generali zes those used in the literatur e. The crite rion in [3, Lemma 3.4] assumes the exist ence of a finite attractor; the criteria in [10, Proposition s 36 & 37] assume some finitenes s proper ty in an abstraction (see nex t sect ion), which we do not. In [3, Lemma 3.7], a similar kind of proper ty as ours is required from all the states of the STS s, not only from an attractor . W e can actual ly pro ve the same result using a sligh tly more gene ral notion of attractor (see [14, P ropos ition 11]). 2.3 Abstractions of Stochastic T ransitions Systems Decisi veness and abst ractions are deepl y intertwined concep ts, so w e briefly reca ll this notio n [10] and related propert ies. W e let T 1 = ( S 1 , Σ 1 , κ 1 ) and T 2 = ( S 2 , Σ 2 , κ 2 ) be two ST Ss, and α : ( S 1 , Σ 1 ) → ( S 2 , Σ 2 ) be a measura ble func tion. W e say that a set B ∈ Σ 1 is α -close d if B = α − 1 ( α ( B )) . T o mean that B is α - closed , we also say that α is compa tible with B . Fol lowin g [13, 21], w e define a natur al way to transf er measures from ( S 1 , Σ 1 ) to ( S 2 , Σ 2 ) through α : the pushforwar d of α is the function α # : Dist ( S 1 ) → Dist ( S 2 ) su ch that α # ( µ )( B ) = µ ( α − 1 ( B )) fo r e very µ ∈ Dist ( S 1 ) an d for e very B ∈ Σ 2 . Definition 2.9 ( α -abstra ction) . STS T 2 is an α -abstr action of T 1 if for all µ ∈ Dist ( S 1 ) , α # ( Ω T 1 ( µ )) is qualit ati vely equi v alent to Ω T 2 ( α # ( µ )) . Bouyer , Brihaye, Randou r , Riv ière & V andenho ve 155 Informal ly , the two STSs then hav e the same “qualitati ve” single steps. Later , we may speak of abstr action instead of α -abstra ction if α is clea r in th e conte xt. The objec tiv e of the notion of abstra ction is that by fi nding an α -abstra ction T 2 which is someho w simpler than T 1 (for exa mple, with a smaller state space ), we should be able to use T 2 (with initial distrib ution α # ( µ ) ) to analyze some prope rties of T 1 (with initial distri bu tion µ ). T o do so, we need to kno w which prop erties are preserv ed through an α -abstra ction. A s a first observ ation, positi ve probab ility of reachability pro perties is preserve d. Stronger condition s are required to study almost-su re reacha bility proper ties thro ugh α -abstra ctions. W e select a ke y prop erty of abstractio ns about tha t matter which will be use ful in the subsequen t section s. Definition 2.10 (Sound α -abstra ction) . W e say that T 2 is a sound α -abstra ction of T 1 if for all B ∈ Σ 2 , Prob T 2 α # ( µ ) ( F B ) = 1 implie s Prob T 1 µ ( F α − 1 ( B )) = 1. Sound abst ractions preserve almost-su re reachab ility properties from T 2 to T 1 . A m ajor result from [10] is that if T 1 is decisi ve w .r .t. B and T 2 is a so und α -abstra ction, then for all µ ∈ Dist ( S 1 ) , Prob T 1 µ ( F α − 1 ( B )) = 1 if and only if Prob T 2 α # ( µ ) ( F B ) = 1. Additio nal results about abstractions are pro- vided in [14, Section 2.4]. 3 A pplication to Stochastic Hybrid Systems W e choose to restrict ou r atte ntion to a sto chastic e xtension of the w ell-studied hyb rid systems . A hybrid system is a dynamical system combining discrete and conti nuous trans itions. It can be defined as a non- determin istic automaton with a finite number of continuou s variab les, whose ev olution is desc ribed via an infinite transition syste m. Hybrid systems ha ve bee n widely studied si nce their introdu ction in the 1990s (e.g., [4, 23]). They are effect iv ely used to model vario us time-depen dent reacti ve systems; sys tems that need to take into account both continuous factors (e.g., spee d, heat, time, dista nce) and discrete factors (e.g., e vents, ins truction s) are ub iquitous . W e define hybrid systems an d gi ve them a fully st ochastic se mantics, yieldin g the cla ss of stoch astic hybrid systems (SHSs). W e then sho w that under classic al definabil ity hypothe ses, we can obtain decid- ability results for qualitati ve and quantitati ve reac hability problems in a class of SHS s (e ven though such problems are undec idable in full generality ), using decisi veness (along with our new decisi ve- ness criteri on) as a key tool. This provi des a stochasti c extens ion to the study of o-minimal hybrid systems [16, 22, 32]. 3.1 Hybrid Systems with Pr obabilistic Semantics W e proceed with the definiti on of a (non-determini stic) hybrid syste m . Definition 3.1 (Hybrid syste m) . A hybrid system (HS) is a tuple H = ( L , X , A , E , γ , I , G , R ) where: L is a finite set of locatio ns (discrete states); X = { x 1 , . . . , x n } is a finite set of continu ous variables ; A is a finite alphabet of events ; E ⊆ L × A × L is a finite set of edges ; for each ℓ ∈ L , γ ( ℓ ) : R n × R + → R n is a continu ous func tion describing the dynamics in location ℓ ; I assigns to each location a subset of R n called in varian t ; G assigns to each edge a subset of R n called guar d ; R assigns to each edge e and v aluation v ∈ R n a su bset R ( e )( v ) of R n called r eset . For ℓ ∈ L , e ∈ E , we usual ly denot e γ ( ℓ ) an d R ( e ) by γ ℓ and R e . W e den ote the numbe r of va riables | X | by n . Given a hyb rid system H , we define S H = L × R n as the sta tes of H . W e distingui sh two kinds of transitions between sta tes: 156 Decisi veness of Stocha stic Systems and its Application to Hybrid Mod els • there is a switch-tr ansition ( ℓ, v ) a − → ( ℓ ′ , v ′ ) if there exists e = ( ℓ, a , ℓ ′ ) ∈ E such that v ∈ I ( ℓ ) ∩ G ( e ) , v ′ ∈ R e ( v ) ∩ I ( ℓ ′ ) ; • there is a delay -tran sition ( ℓ, v ) τ − → ( ℓ, v ′ ) if there ex ists τ ∈ R + such that for all 0 ≤ τ ′ ≤ τ , γ ℓ ( v , τ ′ ) ∈ I ( ℓ ) and v ′ = γ ℓ ( v , τ ) . Informal ly , a switch-tra nsition ( ℓ, v ) a − → ( ℓ ′ , v ′ ) means that an edge e = ( ℓ, a , ℓ ′ ) can be tak en w ithout violat ing an y constrain t: the val ue v of the continuou s var iables is an element of the in v ariant I ( ℓ ) and of the guard G ( e ) , and there is a possibl e reset v ′ of the vari ables which is an element of the in varian t I ( ℓ ′ ) . A delay-t ransition ( ℓ, v ) τ − → ( ℓ, v ′ ) means that some time τ elapse s without changing the discrete locatio n of the system—the only co nstraint is that all the v alues taken by the contin uous v ariable s dur ing this time are in th e in va riant I ( ℓ ) . Giv en s = ( ℓ, v ) ∈ L × R n a state of the hyb rid syste m, and τ ∈ R + , we den ote by s + τ = ( ℓ, γ ℓ ( v , τ )) the ne w state aft er some del ay τ , w ithout changing the loc ation. W e consider semantics gi ven by mixed transi tions , i.e., transitio ns that consist of a dela y-transit ion (some time elapses) follo wed by a switch-transi tion (an edge is taken and the locatio n cha nges). A m ixe d transit ion is denoted by ( ℓ, v ) τ , a − → ( ℓ ′ , v ′ ) if and only if ther e ex ists v ′′ ∈ R n such that ( ℓ, v ) τ − → ( ℓ, v ′′ ) a − → ( ℓ ′ , v ′ ) . W e usuall y assume that there is a bijection between the edges E and the alpha bet of ev ents A , and we omit mention ing this alphabet. If e = ( ℓ, a , ℓ ′ ) ∈ E , we can thu s deno te e − → (resp. τ , e − → ) for switch- transit ions (resp. mixed transitions ) inste ad of a − → (res p. τ , a − → ). Example 3.2. W e provid e in Figure 2 an examp le of a hy brid sys tem (first studied in [7]). There are tw o contin uous var iables ( x and y ) and fiv e locatio ns, each of them equipped with the same simple dy namics: ˙ x = ˙ y = 1 (i.e., γ ℓ (( x , y ) , τ ) = ( x + τ , y + τ ) for ev ery location ℓ ∈ L ). L ocatio ns ℓ 2 and ℓ 4 ha ve the same in va riant, which is { ( x , y ) | y < 1 } ; the other in var iants are simply R 2 . Guards are written nex t to the edge to which they are related: for instance , G ( e 4 ) = { ( x , y ) | y = 2 } . The notatio n “ x : = 0” is used to denote a determin istic reset (in this case, the val ue of x is reset to 0 after taking the edge). For instance, R e 1 ( x , y ) = { ( x , 0 ) } (the va lue of x is preserv ed and y is reset to 0). If nothing else is written ne xt to an edg e e , it means that there is no reset on e , i.e., that R e ( v ) = { v } for all v ∈ R n . A n ex ample of mixed tran sitions of this system can be ( ℓ 0 , ( 0 , 0 )) 0 . 4 , e 0 − − − → ( ℓ 1 , ( 0 . 4 , 0 . 4 )) 0 . 6 , e 1 − − − → ( ℓ 2 , ( 1 , 0 )) 0 . 2 , e 2 − − − → ( ℓ 0 , ( 0 , 0 . 2 )) 1 . 5 , e 3 − − − → ( ℓ 3 , ( 1 . 5 , 1 . 7 )) . ℓ 0 ℓ 1 ℓ 2 ℓ 3 ℓ 4 x = 0 0 ≤ y < 1 y < 1 y < 1 y = 2 y : = 0 e 4 1 < y < 2 e 3 x > 2 x : = 0 e 5 y = 1 y : = 0 y < 1 x > 1 x : = 0 e 0 e 1 e 2 Figure 2: Example of a hybrid sys tem with two co ntinuou s vari ables. Each location is equippe d with the dynamic s ˙ x = ˙ y = 1. W e giv e more v ocab ulary to refer to hybrid systems. If there is a switch-transit ion ( ℓ, v ) e − → ( ℓ ′ , v ′ ) , we say that ed ge e is enabled at state ( ℓ, v ) —this means that edge e can be taken with no delay from state ( ℓ, v ) . Giv en a state s = ( ℓ, v ) and an edge e = ( ℓ, a , ℓ ′ ) of H , we define I ( s , e ) = { τ ∈ R + | s τ , e − → s ′ } as the set of delays after which edge e is enab led from s , and I ( s ) = S e I ( s , e ) as the set of delay s after which any edge is ena bled from s . For instance, in the hybrid system from Figure 2, for s = ( ℓ 0 , ( 0 , 0 . 2 )) , we ha ve I ( s , e 0 ) = [ 0 , 0 . 8 ) , and I ( s ) = [ 0 , 1 . 8 ) \ { 0 . 8 } . Bouyer , Brihaye, Randou r , Riv ière & V andenho ve 157 W e say that a state s ∈ L × R n is non-bloc king if I ( s ) 6 = / 0. In the sequel, we only consider hybrid systems such that all states are non-bloc king, thereby justifying why conside ring sol ely mixed transitio ns is doab le—such a tra nsition is a vailab le from any state. W e no w replac e the non-d eterministi c elements of hybrid systems with stochastic ity . Definition 3.3 (S tochastic hybrid system) . A stoch astic hybrid system (SHS) is de fined as a tuple H = ( H ′ , µ L , η R , θ ) , where: • H ′ = ( L , X , A , E , γ , I , G , R ) is a hybrid system, w hich is referred to as the underlyi ng hyb rid system of H . W e re quire guard s, in varian ts and resets to be Borel sets. • µ L : L × R n → Dist ( R + ) assoc iates to each state a proba bility distrib ution on the time delay in R + (equip ped with the cla ssical Borel σ -algeb ra) before lea ving a location. Gi ven s ∈ L × R n , the distrib ution µ L ( s ) will also be denoted by µ s . W e require that for ev ery s ∈ L × R n , µ s ( I ( s ) ) = 1, i.e., the probabi lity that an edge is enabled afte r a delay is 1. • η R associ ates to each ed ge e and va luation v a probab ility distri bu tion on the set R e ( v ) ⊆ R n . Giv en e ∈ E and v ∈ R n , the distrib ution η R ( e )( v ) will also be de noted by η e ( v ) . • θ : L × R n → D ist ( E ) assigns to each state of H a probab ility distrib ution on the edges. W e requir e that θ ( s )( e ) > 0 if and only if ed ge e is enabled at s . For s ∈ L × R , we deno te θ ( s ) by θ s . This distrib ution is only define d for stat es at which an edge is enabled . Remark 3.4. The term “stoc hastic hybri d system” is used for a wide variet y of stochastic exten sions of hybrid systems through out the literature. In this work, we consider stochasti c delays, stochas tic resets, a stochastic edge choice, and stochas tic initia l distrib utions. The way this proba bilistic semantics is added on top of hybrid systems is very simil ar to ho w timed automat a are con ver ted to stoc hastic timed automata in [7, 11, 12]. A lthoug h dynamics appear to be deter ministic, the mode l is power ful en ough to emulate stochast ic dynamics by assuming th at e xtra variab les are solely used to influence the conti nuous flow of the other v ariables. These v ariables can be chos en sto chastica lly in eac h loca tion thro ugh the reset m echan ism. T his is for e xample suf ficient to con sider a stochastic exten sion of the r ectangu lar automata [24], whose v ariables ev olve accordin g to slopes inside an interv al (such as ˙ x ∈ [ 1 , 4 ] ). Our model is very close to the one of the software tool U P P A A L [20]. In Section 3.3, we will identify the need to restri ct the definition of some componen ts of SHSs to ensure their definability; this is ho wev er not req uired for the results of Secti on 3.2. When ref erring to an SHS, we make in particula r use of the same terminology as for hybrid systems (e.g., ru ns, enable d edg es, allo w ed dela ys I ( · ) ) to describ e its underlying hybrid sy stem. In order to apply the theory dev eloped in Section 2 , we giv e the semantics of an SHS H as an STS T H = ( S H , Σ H , κ H ) . The set S H is the set L × R n of state s of H , and Σ H is the σ -algeb ra produ ct betwee n 2 L and the Borel σ -algeb ra on R n . T o define κ H , we first explain briefly the role of each probability distrib ution in the definition of SHS. S tarting from a stat e s = ( ℓ, v ) , a delay τ is chosen randomly , acc ording to the distrib ution µ s . F rom state s + τ = ( ℓ, v ′ ) , an edge e = ( ℓ, a , ℓ ′ ) (enabled in s + τ ) is selected, following distrib ution θ s + τ (such an edge is almost surely av ailable, as µ s ( I ( s ) ) = 1 by hypothes is). The next state will be in location ℓ ′ , and the values of the continuou s vari ables are stocha stically reset accordin g to the distr ibu tion η e ( v ′ ) . W e can thus define κ H as follo w s: for s = ( ℓ, v ) ∈ S H , B ∈ Σ H , κ H ( s , B ) = Z τ ∈ R + ∑ e =( ℓ, a ,ℓ ′ ) ∈ E  θ s + τ ( e ) · Z v ′′ ∈ R n 1 B ( ℓ ′ , v ′′ ) d ( η e ( γ ℓ ( v , τ )))( v ′′ )  d µ s ( τ ) 158 Decisi veness of Stocha stic Systems and its Application to Hybrid Mod els where 1 B is the characte ristic funct ion of B . It gi ves the probabili ty to hit set B ⊆ S H from state s in one step (representi ng a mixed transit ion). The fu nction κ H ( s , · ) defines a pro bability di strib ution for all s ∈ S H . Definition 3.5 (STS indu ced by an SH S) . For an SH S H , we define T H = ( S H , Σ H , κ H ) as the STS induce d by H . Thanks to the stochast icity of our models, we can reason about both qualitati ve and quantitat ive reacha bility problems, as de fined in Definitio n 2.2. Undecidability . W e pro vide a proof that qualitati ve and quan titati ve reachabilit y problems for SHSs with “simple” featu res are undec idable. The un decidabi lity in itself is not surp rising, as some of the undeci dability proofs from the literature about non-dete rministic hybrid systems [24, 25] can be transla ted direc tly to our stoc hastic setting. Our goal with this new proof is to get as close as po ssible to the class that we will later (in Section 3.3) sho w to be deci dable, in order to outline as well as possib le an und ecidabil ity frontier . W e prove undecid ability e ven when constrain ed to purely continuous distri bu tions on time delays from any state, and very simple guards, resets, an d dynamics (that can be defined in simple math ematical structu res). This require s a distinct proof from [24]. The result from [25] is clos e to the one we want to achie ve, and we tak e inspirati on from its proof. The proof cons ists of reducin g the halting pr oblem for two-coun ter machines to deciding whether a measurable set in an SHS is reach ed w ith probab ility 1. Pro position 3.6 (Undecida bility of SHS s) . The qualitativ e rea chab ility pr oblems and the appr oximate quant itative re acha bility pr oblem ar e undecidab le for stoc hastic hybri d systems with pur ely continuou s distrib utions on time delays, guar ds that ar e linear comparis ons of variables and consta nts, and using positi ve inte ger slopes for the flow of the continuous variab les. The appr oximat e qua ntitative pr oblem is mor eover undec idable for any fix ed pr ecision ε < 1 2 . Although the proof is cente red on sho wing the undecida bility of qualitati ve reachab ility problems, we get as a by -produc t the undeci dability of the ap proximatio n. Indeed , as the systems used throu ghout the proof reac h a tar get set B with a probab ility that is eit her 0 or 1, the abili ty to app roximate Prob T H µ ( F B ) with ε < 1 2 would be suffici ent to solv e the qualitati ve problem. As the proof sh ows that these qua litati ve proble ms are undecid able, we obtain that the approximate quantitati ve pro blem is undecid able as well. Our re sult also impli es that deci ding whether a state lies in e B is undecidab le. 3.2 The Cycle-Reset Hypothesis The literatu re about non-det erministic hybrid systems suggests that to obtain su bclasses for w hich the reacha bility pro blem becomes decidable, one must set sharp restric tions on the continuous flow of the v ariables and/or on the discret e transition s (via the r eset mechanism). In this decidable spect rum lie for instan ce the rect angular initialized automata , which are quite permissi ve to ward the continuo us e vo lution of the v ariables, but nee d strong hypo theses about the discre te transitions [24]. Our approach lies at on e end of this spectrum: we restrict the discrete beha vior by conside ring str ong r esets , i.e., resets that for get about the pre vious v alues of the v ariables , decouplin g the discrete beha vior from the continuo us beha vior . W e sho w that one strong re set per cycle of the graph is sufficie nt to obtain valu able results, and we name this prope rty cycle-r eset . This point of vie w has alread y been studied in [16, 22, 32] for non- determin istic hybrid systems, and in [11] for stoc hastic timed automata. N ote that pre vious work about finite abstrac tions of non- determinis tic hyb rid systems (called time-abs tracti on bisimulation s ) does not Bouyer , Brihaye, Randou r , Riv ière & V andenho ve 159 ext end in ge neral to “stochastic ” abstractions (as de fined in Section 2.3), as there may be transitions that ha ve probability 0 to happen. Definition 3.7 (Strong reset, cycle-r eset S HS) . Given H a stoch astic hybrid system and e an edge of H , w e say that e has a str ong re set (or is str ongly r eset ) if the re ex ist R ∗ e ⊆ R n and η ∗ e ∈ Dist ( R ∗ e ) such that for all v ∈ G ( e ) , R e ( v ) = R ∗ e and η e ( v ) = η ∗ e . W e say that an SH S H is cycle-r eset if for e very simple c ycle of H , there e xists a strongl y reset edge. If an edge e is strongly reset, it stoc hastical ly resets all the continuous va riables when it is taken , and the st ochastic reset does not depend on their value s. W e sho w two independen t and ve ry con venie nt results of cycle-rese t SHS s: such SH S s are decis ive w . r .t. any measur able set (the proof of this statement relies on the decis iv eness criteri on from Proposit ion 2.8), and admit a finit e abstra ction . Decisi veness of cycle -re set SHSs. W e m oti v ate this section with an example of a simple non- decisi ve SHS, whic h we will use to sho w that ou r decisi veness result is tight. Example 3.8. W e ad d a stoch astic layer to the hybri d system of Example 3.2, pic tured in Figure 2. The distrib utions on the time delays in location s ℓ 0 , ℓ 2 and ℓ 4 are uni form distrib utions on the interv al of allo w ed delays . For instance, at state s = ( ℓ 0 , ( x , y )) , the distrib ution µ s follo ws a uniform distrib ution on [ 0 , 2 − y ) . In locations ℓ 1 and ℓ 3 , the distrib utions on the delays are Dirac distrib utions . Reset and edge distrib utions are simply modeled as Dirac distrib utions. It is pro ved in [12, Section 6.2.2] that this SHS is not dec isi ve w .r .t. B = { ℓ 2 } × R 2 . T he pro of is quite technic al and we do not recall it here; intuiti vel y , at each passa ge through location ℓ 0 , the v alue of y increase s but stays bou nded from abov e by 1, which decrea ses the probabi lity to take edge e 0 (and thus reach B ); this decrease is too fast to ensure that B is almost su rely reache d. Pro position 3.9. Every cycl e-r eset SHS is decisive w .r .t. any measur able set. Placing (at least) one strong reset per simple cyc le is an easy syntactic way to guara ntee that almost surely , infinitely m any strong resets are performed , which is the actual sufficie nt prope rty used in the proof. As the re are only finitel y many edges, we can find a probab ility lower bo und p on the prob ability to reach B after any strong rese t, as required in the criteri on of Proposition 2.8. Notice that as sho wn in Example 3.8, ha ving indepe ndent flows for each va riable and rese tting each v ariable once in each cyc le is not suf fi cient to obtai n decisi venes s; var iables need to be reset on the same disc r ete transi tion in each cyc le. Existence of a fin ite abstraction. W e show that c ycle-res et SHS s admit a finite α -abstra ction. W e first gi ve a simple ex ample sho wing that with out one strong reset per cycle, so me simple sys tems do not admit a finite α -abstra ction compatible with the locatio ns. Example 3.10. Consider the SHS of F igure 3. The self-loo p edge of ℓ 0 is the only edge not being strong ly reset. W e ass ume that we want to ha ve an abs traction compatible with s ∗ : = { ℓ 1 } × R . In order to obta in an abstra ction, we must split { ℓ 0 } × R n in s 0 : = { ℓ 0 } × ( − ∞ , 0 ) and { ℓ 0 } × [ 0 , + ∞ ) , as all the states of s 0 can reach s ∗ with a positi ve probab ility in one step, but none of the states of { ℓ 0 } × [ 0 , + ∞ ) can. Then, { ℓ 0 } × [ 0 , + ∞ ) must also be split into s 1 : = { ℓ 0 } × [ 0 , 1 ) and { ℓ 0 } × [ 1 , + ∞ ) because the states of s 1 can all reach s 0 with a positi ve probab ility in one step, but none of the states of { ℓ 0 } × [ 1 , + ∞ ) can. By iterating this ar gument, we find tha t the smallest α -abstra ction compatible with { ℓ 1 } × R is counta bly infinite, and the par tition that it induces is composed of s ∗ , s 0 and s i = { ℓ 0 } × [ i − 1 , i ) for i ≥ 1. The un derlying hybr id sys tem actuall y belong s to the class of up datable timed automata [15 ], and the abstracti on almos t coincide s w ith the re gion grap h of the automaton . T he reason ing w e ha ve used 160 Decisi veness of Stocha stic Systems and its Application to Hybrid Mod els is very close to the classical bisimulatio n algo rithm for non-det erministic sys tems, and is exp lained in more deta il in [14, S ection 2.4]. x < 0 x : = 0 ˙ x = 1 x ≥ 0 x : = x − 1 ℓ 0 ℓ 1 ˙ x = 1 x ∈ R x : = 0 Figure 3: The time delays are giv en by e xponen tial distrib utions from an y state ; resets are Dirac distrib u- tions. The smallest abs traction compati ble with { ℓ 1 } × R is cou ntably infinite. The cycle -reset assumptio n is suf ficient to guaran tee the existen ce of a finite abstractio n, as formu- lated in the nex t propos ition. Pro position 3.11. Let H be an SHS, and B ∈ Σ H . If H is cycle- r eset, it has a finite and sound abstr action compatible with B and w ith the locatio ns. The pro of consists of sho wing that a stochas tic adaptation of the classical bis imulation algori thm terminate s under the cycle-res et assumption . Combined with our decisi veness result, we can eve n sho w that th is abstra ction is sound . 3.3 Reachability Analysis i n Cycle-Reset Stochastic Hybrid Systems Our goal in this section is to perfo rm a reachability analysis of cycle-res et SHSs. A fi rst hurdle to circumv ent is that arbitrary SHSs are in general dif fi cult to apprehend algo rithmically : for instance , the contin uous ev olution of their v ariables may be defined by solu tions of systems of diff erential equations, which w e do not know how to solv e in general . T o mak e the problem more accessibl e, we follo w the approa ch of [16, 32] for no n-determin istic hybri d sys tems by as suming that some key component s of our systems are defina ble in a mathe matical structure . W e identify a lar ge class of SHSs for which the finite abstra ction from Section 3.2 is computab le, w hich makes qualita tive reachab ility proble ms decidabl e; namely , cyc le-r eset o-min imal SHSs defined in a decidable theory . W e then identify sufficie nt hypoth eses for the appr oximate quan titative problem to be decida ble, in the form of a finite set of probabiliti es that ha ve to be app roximately computabl e. Qualitati ve analysis. W e assume that the reader is familiar with basic model-th eoretic and logica l terms—the reader is referred to [27] for an introduc tion to the main concepts. In what follo ws, by defina ble , w e mean de finable without para meters. Definition 3.12 (Definable SHS ) . G i ven a structure M , an SHS H is said to be defined in M if for e very location ℓ ∈ L , γ ℓ is a function definable in M and I ( ℓ ) is a set definable in M , and for ev ery edge e ∈ E , the set G ( e ) is definable and there exists a fi rst-order formula ψ e ( x , y ) such that v ′ ∈ R e ( v ) if and only if ψ e ( v , v ′ ) is true. Note that we requir e that the flow of the dynamical system in each location is definable in M , and not tha t it is the solutio n to a de finable system of diffe rential equatio ns. Our go al is to pro ve tha t the class of cycle-r eset o-minimal SHSs , i.e., cycle-r eset SHS s defined in an o-minimal structur e (in troduced in [44, 38]), with addit ional assumptions on the probability distrib utions , is decid able. W e will use that the finite sound abstractio n from Proposit ion 3.11 is then comput able, Bouyer , Brihaye, Randou r , Riv ière & V andenho ve 161 and that decisi ven ess (Proposition 3.9) gi ves us a strong link between reachabili ty properties of the abstra ctions and the ori ginal systems. Definition 3.13 (O-min imality) . A tota lly ordered struc ture M = h M , <, . . . i is o-minimal if ev ery defin- able su bset of M is a fi nite union of poi nts and ope n interv als (poss ibly unbounde d). In other words, the defina ble subse ts of M are ex actly the ones that are definable with parameter s in h M , < i . Some well-kno wn structu res are o-minimal, such as the ordered addit iv e group of ration als h Q , < , + , 0 i , the ordered additi ve group of reals R lin = h R , <, + , 0 , 1 i , the order ed fi eld of reals R alg = h R , < , + , · , 0 , 1 i , th e orde red fi eld of reals with th e exponent ial function R exp = h R , <, + , · , 0 , 1 , e x i [45]. There is no general result about the decidabilit y of the theories of o-minimal structures . A well-kno wn case is the T arski-Seide nber g theorem [43], which asserts that there exists a quantifier -elimina tion algorithm for sentences in the first-ord er language of real closed fields. This resul t implies the decid ability of the theory of R alg . Howe ver , it is not kn own whether the the ory of R exp is decidable. Its decidabil ity is implied by Sch anuel’ s conjec tur e , a fa mous unsolve d proble m in tran scenden tal number theo ry [36]. In this w ork, we define an o-mini mal SHS as an SHS defined in an o-min imal structure . The o-minimal ity of M implies that defina ble subsets of M n ha ve a v ery “ni ce” structure, describe d notabl y by the cell decompos ition theor em [31]. In particula r , ev ery subse t of R n definable in an o- minimal structure belon gs to the σ -algeb ra of Borel sets of R n [30, Proposit ion 1.1]. Moreo ver , to help with issues related to the definability of probabili ties, w e will use the property that in o-minimal structu res, definable sets with positi ve Lebesgue measure coincide with definable sets with non-emp ty interio r [30, Remark 2.1]. This is very hel pful, as the property of ha ving an empty interior is de finable. Remark 3.14. The definit ion of o-min imal (non-d eterminist ic) hybrid sys tem in the literatu re usually assumes that all edges are strongl y reset . O-minimal hybrid systems w ere fi rst introduce d in [32], and furthe r studied notab ly in [16]. The strong reset hypothesis was relax ed in [22] to “one strong reset per cyc le”. Let M = h R , <, + , . . . i be an o-minimal structure whose theor y is decid able, such as R alg . Let H = ( H ′ , µ L , η R , θ ) be a cycle-res et o-min imal SHS defined in M . Let µ ∈ Dist ( S H ) be an initial distrib ution. W e mak e the fol lo wing assumption s, which we deno te by ( † ) : • for all s = ( ℓ, v ) ∈ L × R n , if I ( s ) is finite, µ s is equi va lent to the uniform disc rete distrib ution on I ( s ) ; if I ( s ) is infini te, µ s is equiv alent to the L ebesgu e m easure on I ( s ) ; • the initial distrib ution µ is either equiv alent to the discrete measur e on some finite definab le support D , or equi valent to the Lebesgue measu re on a definable support D ; • for e ∈ E , v ∈ R n , we ask that R e ( v ) is finite or has pos iti ve Lebes gue measure; η e ( v ) is resp. either equiv alent to the discrete measure or the Lebesgue measu re on R e ( v ) . The first requirement was alr eady a stand ard assumption in the case of stochas tic timed automata [7, 11, 12]. Hypothesi s ( † ) is easily satisfied: for instan ce, exp onential distri bu tions (resp. uniform distrib u- tions on [ a , b ] ) are equiv alent to the L ebesgu e measure on R + (resp. [ a , b ] ). W e summarize what these ideas enta il in the next pro position . Pro position 3.15. Let H be a cycle-r eset o-minimal SHS defined in a structu r e whose theory is decid- able . L et B ∈ Σ H be defina ble and µ ∈ Dist ( S H ) . W e assume that assumpt ion ( † ) holds. Then one can decide whet her Prob T H µ ( F B ) = 1 and whether Prob T H µ ( F B ) = 0 . In particular , we can decide the qualitati ve reachability problems for cyc le-reset SHSs defined in R alg and sa tisfying ( † ) . Assuming Schanuel’ s conje cture [36], we could exten d this result to R exp . 162 Decisi veness of Stocha stic Systems and its Application to Hybrid Mod els Appr oximate q uantitati ve analysis. Unde r strengthe ned numerica l hypoth eses, we can solve the approx imate quan titati ve reac hability pro blem in cycl e-reset S HSs. Let H be a cycle- reset S H S, B ∈ Σ H and µ ∈ Dist ( S H ) . Our goa l is to apply the approximation sch eme desc ribed in [10 ] in order to approx imate Prob T H µ ( F B ) . T o do so, we require that T H is decisi ve w .r .t. B , which is implied by the cyc le-reset hyp othesis (Propo sition 3.9 ), and the ability to compute for all m ∈ N , an arbitra rily close approx imation of p Y es m = Prob T µ ( F ≤ m B ) and p No m = Prob T µ ( B c U ≤ m e B ) . Using the cycle- reset hypothesis, w e can ex press these probabili ties as sums and products of a finite number of probabili ties of paths with bounded length b ∈ N , w here b is the length of the longest path without enc ounterin g a str ong reset. Details on this computat ion are av ailable in [14, S ection 5.2] . 4 Conclusion Summary . W e presen ted in S ection 2 how to solve reachabil ity pro blems in stochasti c transition systems via the decisivenes s notion, introduce d in [3, 10]. W e no tably solv ed in Lemma 2.7 a technical questi on that was open in [10], which helped formulate a gen eral suf ficient cond ition for decisi venes s in Proposit ion 2.8, encompa ssing known decisi vene ss criteri a from the literature. In Section 3, we considered stocha stic hybrid systems (SHSs). W e sho wed that SHS s w ith one str ong r eset per cyc le ( cycle -r eset ) are decisi ve (using our new decisi ven ess criterion), and ad mit a finite abstra ction. W e then id entified assumption s, using ideas from logic , leading to the effe cti ve comput abil- ity of this abstraction , and to the decidabilit y of qualitati ve and quantitati ve reacha bility pr oblems. These assumpti ons pertain to the definability of the dif ferent components of the SHSs in a decidab le math- ematical structure. Combined with the previo us decisi ven ess results, this abstra ction can be used to decide qu alitati ve reachability problems. In parti cular , these results apply to cycle -reset SHSs defined in R alg = h R , <, + , · , 0 , 1 i with well-be ha ved distrib utions. Pos sible extensio ns and futur e work. W e identify some pos sible exten sions of our re sults. A first directi on of study is to find other classes of decisi ve stochast ic systems that can be encom- passed by our decisi veness criterio n (Proposition 2.8). In that respect, a good candidat e is the class of stoc hastic r e gene rativ e P etri nets [28, 37]. An ap plicatio n of de cisi ven ess resul ts to stoc hastic P etr i nets was briefly discussed in [10 , Section 8.3], b ut under sev ere con straints; w e may be able to relax part of these con straints with the generalized criter ion. In S ection 3.3, w e circumve nt the issue of the definability of measures by using a specific property of the o-minimal structures (namely , that the Lebesgue measure of a definabl e set is posi tiv e if and only if the interi or of that set is non-empty). Ho weve r , more power ful results exi st about the co mpatibility of o- minimal structur es and m easure theory [30]: some o-minimal structure s are closed under integrat ion with respec t to a giv en measure (then called tame measure). This considera tion may help ex tend our result s to a larger class tha t is less restrict iv e w .r .t. probability dis trib utions, or in which actual prob abilities may be defina ble. Acknowledgments. W e would lik e to thank the anon ymous re vie wers for their v aluable advice, which no tably helpe d simplify the pro of of Lemma 2.7. Bouyer , Brihaye, Randou r , Riv ière & V andenho ve 163 Refer ences [1] Alessandro A b ate, Joost-Pieter Katoen , Joh n L ygeros & Maria Pr a ndini ( 2 010): A ppr o ximate Mode l Check- ing of Stochastic Hyb rid S ystems . Eur . J. Control 16(6) , pp. 6 24–64 1, doi:10 .3166 /e jc. 16.624-641 . [2] Alessandro Abate, Maria Prandini, John L ygero s & Shankar Sastry ( 2 008): Pr ob abilistic r e a chability and sa fe ty for contr olled d iscrete time stochastic hybrid systems . Automatica 44(11 ), pp. 2724 – 2734 , doi:10.1 016/j.au tomatica.2008.03.027 . [3] Parosh A. Abdulla, Noo mene Ben Henda & Richard May r (20 07): Decisive Markov Cha ins . Log. Methods Comput. Sci. 3(4), doi:10 .2168 /LMCS-3(4:7)20 07 . [4] Rajeev A lu r , Costas Courcou betis, Nicolas Halbwachs, Thoma s A. Henz in ger, Pei-Hsin H o , Xavier Nicollin, Alfredo Oliv ero, Joseph Sifakis & Sergio Y ovine (1995) : The Alg orithmic Analysis o f Hyb rid Systems . Theor . Comput. Sci. 138(1 ), pp. 3–34, doi:1 0.1016 /0304 - 3975(94)00202-T . [5] Rajeev Alur & David L. Dill (1994 ): A Th eory of T imed Automata . Theor . Compu t. Sci. 126(2 ), pp . 183–2 35, doi:10.1 016/0 3 04-3975(94)90010-8 . [6] Rajeev Alur & Geo rge J. Pappas, editors (200 4): Hyb rid Systems: Computation and Contr ol, 7th Interna- tional W orkshop, HS CC 2004, Philad e lp hia, P A, USA, Mar ch 2 5 -27, 2004, Pr oceedings . Lecture Notes in Computer Science 2993, Spring er , doi:10 . 1007/b 96398 . [7] Christel Baier , Nath a lie Bertr and, Patricia Bouyer, Thomas Brihaye & Marcu s Grö ß er ( 2 008): Almost- Sur e Mod el Checking of Infin ite P aths in On e-Clock T imed Automata . In: Proceeding s of the T wenty- Third Annua l IEEE Symp osium on Logic in Compu te r Science , LI CS 2008 , 24- 27 June 2008, Pitts- burgh, P A, USA , IEE E Co m puter Society , pp. 2 17–22 6, d oi:10.1 109/LICS.2 0 08.25 . A v ailable at https ://iee explore.ieee.org/xpl/conhome/4557886/proceeding . [8] Christel Baier & Joo st-Pieter Katoen (2 008): Principles of mod el checking . MI T Pr e ss. [9] Alessandro Berarducci & Margar ita Otero (20 04): An a dditive mea sur e in o-minima l expansions of fields . Q. J. Math. 55(4) , pp. 411– 419, doi:10 .1093 /qmath/hah 010 . [10] Nathalie Bertrand , Patricia Bouye r , Tho mas Brihaye & Pierre Carlier (20 18): When ar e stochastic transition systems ta meable? J. Log . Algeb r . Meth. Program . 99, p p. 41–9 6, doi:10 .1016 /j. jlam p.2018.03.004 . [11] Nathalie Bertr and, Patricia Bo u yer, Thomas Brihaye & Nicolas Markey (2 0 08): Quantitative Model-Checking of On e-Clock T imed Automata under Pr ob abilistic Seman tics . In: Fifth In terna- tional Conf erence on the Quan titati ve Evaluation of Systems (QEST 20 08), 14-17 September 2 008, Saint-Malo, Fran ce , IEEE Comp uter Society , p p. 55–6 4, doi:10.1 109/QEST .20 08.19 . A vailable at https ://iee explore.ieee.org/xpl/conhome/4634932/proceeding . [12] Nathalie Bertrand, Patricia Bou y er , Th omas Brihay e, Quen tin Menet, Christel Baier , Mar cus Größ e r & Marcin Jurdzinski (2014) : Stochastic T imed Automata . Log. Methods Comp u t. Sci. 10(4) , doi:10.2 168/LMCS-1 0(4:6)2014 . [13] Vladimir I. Bog achev (2007 ): Measur e theory . 1, Springer Science & Business Media, doi:10.1 007/9 7 8-3-540-34514-5 . [14] Patricia Bouy er , Thoma s Brihaye, Mickael Rando ur , Cédric Ri v ière & Pierre V an denhove (2 020): Decisive- ness of Stochastic S ystems an d its Applica tio n to Hybrid Models ( F ull V ersion) . CoRR . [15] Patricia Bouyer , Catherine Dufourd , Emmanuel Fleury & Antoine Petit (2004) : Up datable timed automata . Theor . Compu t. Sci. 321(2 -3), pp. 2 91–3 4 5, doi:10.1 016/j.tcs.20 04.04.003 . [16] Thom a s Brihay e, Chr istian Mich aux, Cédric Rivière & Christoph e Troestler (20 04): On O-Minimal Hyb rid Systems . In Alur & Pappas [6], pp . 219 –233, doi:1 0 .1007 /978-3 -540-24743-2_15 . [17] Luminita Manue la Bujorianu (2012 ): Stochastic r ea chability analysis of hybrid systems . Sp r inger Science & Business Media, doi:1 0.100 7/978- 1-4471-2795-6 . 164 Decisi veness of Stocha stic Systems and its Application to Hybrid Mod els [18] Pierre Carlier (20 17): V erificatio n of Stochastic T imed Automata. (V é rification des automa tes tem- porisés et stochastiques) . Ph.D. thesis, Uni versity of Paris-Saclay , France. A vailable at https ://tel .archives- ouvertes.fr/tel- 01696130 . [19] Christos G. Cassandras & Joh n L yger os (2007 ) : Stochastic Hybrid Systems . CRC Press, doi:10.1 201/9 7 81420008548 . [20] Alexandre Da vid, Deh u i Du, Kim G. Larsen, Axel Legay , Mariu s Mikucionis, Danny Bøgsted Poulsen & Sean Sedwards ( 2012) : Statistical Model Checking for S tochastic Hybrid Systems . In Ezio Barto cci & Luca Bortolussi, editor s: Proceeding s First Interna tio nal W orkshop on Hy brid Systems an d Biology , HSB 2012, Newcastle Upon T yne, UK, 3r d Septem ber 2 012 , EPTCS 92, pp. 122– 136, doi:10 .4204 /EPTCS.92.9 . [21] Daniel Gb urek, Christel Baier & Sascha Klüppelholz (201 6): Composition of Stochastic T ransition Systems Based on Spans and Coup lings . In: 43rd Internatio nal Colloq uium on Auto mata, Langua ges, and Pro gram- ming, ICALP 20 16, July 11-15 , 20 16, Rome, Italy , LIPIcs 55, Schloss Dagstuhl - L e ibniz-Zentr um fuer Inform atik, pp. 102:1 –102:1 5, do i:10.423 0/LIPIcs.ICALP .2 016.102 . [22] Raffaella Gentilini (200 5 ): Reachab ility Pr o blems o n Extended O- Minimal Hybrid Automata . In Paul Petters- son & W ang Y i, editors: Formal Mod eling an d Ana ly sis of T imed Systems, Third I n ternation a l Con ference, FORMA TS 20 0 5, Upp sala, Sweden , Sep te m ber 26-2 8, 200 5, Procee d ings , Lecture Notes in Compu ter Sci- ence 3829, Sp ringer, pp. 16 2–17 6 , d oi:10.1 007/11 603009_14 . [23] Thom a s A. Henzinger (1996) : The Theory of Hybrid Automata . In: Proceeding s, 11th An- nual I EEE Sym posium on Logic in Co m puter Scienc e , Ne w Brunswick, New Jersey , USA, July 27-30 , 19 9 6 , IEEE Computer Society , pp. 278 –292, do i:10.110 9/LICS.199 6.561342 . A vailable a t https ://iee explore.ieee.org/xpl/conhome/4265/proceeding . [24] Thom a s A. Henz in ger, Peter W . K opke, Anuj Pur i & Pravin V ar aiya (1 998): What’s Decida ble ab o ut Hybrid Automata? J. Com put. Syst. Sci. 57(1) , pp. 9 4–124 , doi:10.1 006/jcss.19 98.1581 . [25] Thom a s A. Henzinger & Jean-François Raskin (2000): Robust U n decidab ility of T imed and Hybrid Sy stems . In L ynch & K r ogh [35], p p. 145– 159, doi:10. 1007/3 -540-46430-1_15 . [26] João P . Hespan ha (2004) : Stochastic Hybrid Systems: Applicatio n to Communication Networks . In Alur & Pappas [6], pp . 387–4 01, doi:10.1 007/9 7 8-3-540-24743-2_26 . [27] W ilfrid Hodge s (1997 ): A Shorter Model Theo ry . Cambrid ge University Press. [28] András Hor váth, M a rco Paolieri, Lorenzo Ridi & Enrico V icario ( 2 012): T ransient analysis of no n-Markovian mo dels using stochastic state classes . Perform. Eval. 69(7- 8), pp . 31 5–335 , doi:10.1 016/j.p ev a. 2 011.11.002 . [29] Jianghai Hu, Joh n L y geros & Shankar Sastry (2000 ) : T owar ds a Theory of Stochastic Hybrid Systems . In L ynch & Kro gh [ 35], pp . 1 60–17 3, doi:10.1 0 07/3-5 40-46430-1_16 . [30] T obias Kaiser ( 2012) : Fir st order tamene ss of mea sur e s . Ann. Pure Appl. Logic 163(1 2), p p. 1903– 1927, doi:10.1 016/j.ap al.2012.06.002 . [31] Julia F . Knight, Anand Pillay & Charles Steinhorn (19 86): Definable sets in order ed structur es. I I . T ransactions of the American Mathem atical So ciety 295(2 ), pp. 5 93–6 0 5, doi:10.1 090/S00 02-9947-1986-0833698-1 . [32] Gerardo Lafferriere, Geo rge J. P appas & Shank a r Sastry ( 2000) : O-Minimal Hybrid Systems . Math. Contro l Signals Syst. 13(1) , pp. 1–21 , do i:1 0.100 7/PL0000 9858 . [33] Xiangfang Li, Oluwaseyi Omo tere, Lijun Q ia n & Ed ward R. Dougherty (20 17): Review of stochastic hybrid systems with ap plications in biological systems mode lin g an d analysis . EURASIP J. Bioinformatics and Systems Biology 2017, p . 8 , doi:10.1 186/s13 637-017-0061-5 . [34] John L ygeros & Maria Prand ini (201 0): Stochastic Hybrid Systems: A P owerful F ramew ork fo r Complex, Lar ge Sca le Applica tions . Eur . J. Contro l 16(6) , pp. 583– 594, doi:10 .3166 /ejc.16.5 83- 594 . Bouyer , Brihaye, Randou r , Riv ière & V andenho ve 165 [35] Nancy A. L ynch & Bruce H. Krog h, ed itors (2000) : Hybrid Systems: Compu tation and Contr ol, Third Internation al W orksho p, HSCC 20 00, Pittsbur g h , P A, USA, Mar ch 23-25 , 200 0, Pr oc eedings . Lecture Notes in Computer Science 1790, Spring er , doi:10 . 1007/3 -540-46430-1 . [36] Angus Macin tyre & Ale x J. W ilkie (1996): On the dec idability of th e r eal exponen tial fi eld . In: Kreiseliana: About and aro und Geo rg Kreisel , A. K. Peters, p p. 4 4 1–46 7 . [37] Marco Paolieri, Andr ás Horváth & En rico V icario ( 2016) : Pr o b abilistic Mod el Checking of Regenerative Concurr ent Systems . IEEE Trans. Software Eng. 42(2) , pp. 153– 169, doi:10 .1109 /TSE.2015 .2468717 . [38] Anand Pillay & Cha rles Steinh orn (19 86): Defin able sets in or der ed structur es. I . T ransactions of the Am er- ican Mathematica l Society 295(2 ), pp. 565 –592 , d oi:10.23 07/200 0052 . [39] Amir Pnueli ( 1977) : The T emporal Logic of Pr ograms . In: 18th Annual Symposium on Foundations of Computer Science, Providence, Rhod e Island, USA, 31 Oc to ber - 1 Novem- ber 1977 , IEEE Computer Society , pp. 46–5 7 , do i:10.11 0 9/SFCS.1977. 32 . A v ailable at https ://iee explore.ieee.org/xpl/conhome/4567914/proceeding . [40] Maria Prandini & Jiangh a i Hu (2009 ): Application of Rea chability Analysis for S tochastic Hy- brid Systems to A ircr aft Con flict Pr e d iction . IEEE Trans. Automat. Contr . 54(4) , p p. 913–9 17, doi:10.1 109/T A C.20 0 8.2011011 . [41] Maria Prand ini, Jiangh ai Hu, John L ygeros & Shan kar Sastry (200 0): A pr oba bilistic appr oach to air craft con- flict detection . IEEE Trans. Inte llig ent Transportation Systems 1(4), pp. 199– 220, doi:10 . 1109/6 979.8 98224 . [42] Abhyu dai Singh & Joao P . Hespanha (2 010): Stochastic hybrid systems for study ing bio chemica l pr o- cesses . Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 368(1 930), pp. 4995 –501 1 , doi:1 0.109 8/rsta.201 0.0211 . [43] Alfred T arski (1951 ): A decision method for elemen ta ry a lgebra and geometry . Univ ersity o f California Press, Berkeley , Californ ia. [44] Lou van den Dr ies (1 984): Remarks o n Tar ski’ s p r oblem con cerning (R, +, *, exp) . Studies in Logic an d the Foundations of Mathematics 112(C), pp. 97–1 21, doi:1 0.101 6/S0049- 237X(08)71811-1 . [45] Alex J. W ilk ie (199 6): Model completeness r esults for expansions of the order ed fie ld of real numbers b y r estricted Pfaffian function s a nd the exponential functio n . Journal o f the American Math e m atical Society 9(4), pp. 1051 –1094 , doi:1 0.109 0 /S0894- 0347-96-00216-0 .