Towards Integrating Formal Verification of Autonomous Robots with Battery Prognostics and Health Management

T o w ards In tegrating F ormal V erification of Autonomous Rob ots with Battery Prognostics and Health Managemen t ? Xingyu Zhao 1 , Matt Osb orne 1 , Jenn y Lantair 1 , V alen tin Robu 1 , Da vid Flynn 1 , Xiao wei Huang 2 , Michael Fisher 2 , F abio Papacc hini 2 , and Angelo F errando 2 1 Sc ho ol of Engineering and Ph ysical Sciences, Heriot-W att Univ ersity , Edinburgh EH14 4AS, U.K. { xingyu.zhao,mho1,jl153,v.robu,d.flynn } @hw.ac.uk 2 Departmen t of Computer Science, Univ ersity of Liv erp ool, Liv erp ool L69 3BX, U.K. { xiaowei.huang,mfisher,fabio.papacchini,angelo.ferrando } @liverpool.ac.uk Abstract. The battery is a k ey comp onen t of autonomous rob ots. Its p erformance limits the rob ot’s safety and reliability . Unlike liquid-fuel, a battery , as a c hemical device, exhibits complicated features, including (i) capacit y fade ov er successive recharges and (ii) increasing discharge rate as the state of c harge (SOC) goes down for a giv en pow er demand. Exist- ing formal verification studies of autonomous rob ots, when considering energy constraints, formalise the energy comp onen t in a generic manner suc h that the battery features are ov erlo ok ed. In this pap er, we mo del an unmanned aerial vehicle (UA V) insp ection mission on a wind farm and via probabilistic mo del c hecking in PRISM show (i) ho w the battery features ma y affect the verification results significan tly in practical cases; and (ii) how the battery features, together with dynamic environmen ts and battery safet y strategies, jointly affect the v erification results. Po- ten tial solutions to explicitly integrate battery prognostics and health managemen t (PHM) with formal verification of autonomous rob ots are also discussed to motiv ate future work. Keyw ords: F ormal v erification · Probabilistic mo del c hecking · PRISM · Autonomous systems · Unmanned aerial v ehicle · Battery PHM. 1 In tro duction Autonomous rob ots, such as unmanned aerial vehicles (UA V) (commonly termed drones 3 ), unmanned underw ater vehicles (UUV), self-driving cars and legged- rob ots, obtain increasingly widespread applications in many domains [14]. Ex- ? Supp orted b y the UK EPSR C through the Offshore Rob otics for Certification of Assets (ORCA) [EP/R026173/1], Rob otics and Artificial In telligence for Nuclear (RAIN) [EP/R026084] and Science of Sensor System Softw are (S4) [EP/N007565]. 3 W e ha ve used the word “drone” interc hangeably with the abbreviation UA V as a less formal naming conv ention throughout the pap er. 2 X. Zhao et al. treme environmen ts – a term used b y UK EPSR C 4 to denote en vironments that are remote and hazardous for humans – are the most promising domains in whic h autonomous rob ots can be deplo yed to carry out a task, suc h as explo- ration, insp ection of oil/gas equipment on the seab ed, maintenance of offshore wind turbines, and monitoring of n uclear plan ts in high radiation conditions [26]. Ho wev er, autonom y p oses a great challenge to the assurance of safety and reliabilit y of rob ots, whose failures may cause b oth a detrimen t to human health and well-being and huge financial losses. Th us, there are increasing demands on regulation of autonomous rob ots to build public trust in their use, whilst the developmen t, v erification and certification of autonomous rob ots is inher- en tly difficult due to the sheer complexity of the system design and inevitable uncertain ties in their op eration [9,6,8,34]. F or instance, [21] shows the infeasibil- it y of demonstrating the safety of self-driving cars from road testing alone, and b oth [23] and [21] argue the need for alternative verification metho ds to sup- plemen t testing. F ormal techniques, e.g. model chec king and theorem pro ving, offer a substantial opp ortunit y in this direction [12]. Indeed, formal metho ds for autonomous rob ots ha ve received great attention [6,28], b oth in controller syn thesis, see e.g. [12,30], and in verifying safet y and reliability when the control p olicy is giv en, see e.g. [42,18,11]. The battery as the p o wer source of autonomous rob ots plays a k ey role in real-life missions [41]. Ho wev er to the b est of our kno wledge, most existing formal v erification studies of autonomous robots, when considering energy constraints, formalise the energy component in a generic and simplified manner such that some battery features are ov erlo ok ed: – Capacit y fading : Ov er successive rec harges, unlike a liquid-fuelled system whose tank v olume normally remains unchanged, the charge storage capacity of a battery will diminish ov er time. – Increasing discharge rate : In one discharge cycle, since the v oltage drops as the battery is b eing discharged, for a constant p o wer output (a pro duct of the v oltage and the curren t), the current increases meaning an increased disc harge rate o ccurs. This is different to a liquid-fuelled system in which a constan t p o w er output t ypically means a constant rate of fuel consumption. Th us, usual assumptions, lik e (i) a fixed battery capacit y regardless the n umber of recharges and (ii) constan t energy consumption for a giv en action regardless the stage in a discharge cycle, become p otentially problematic. On the other hand, the battery prognostics and health managemen t (PHM) comm unity has been dev eloping techniques to accurately forecast the battery b eha viour in b oth a life-cycle and a discharge-cycle. W e b elieve suc h battery PHM results should b e in tegrated in to formal studies (either controller syn thesis or verification) of rob ots to refine the analysis. T o tak e a step forward in this direction, in this pap er, our main work is as follo ws: – W e formalise a UA V insp ection mission on an offshore wind farm, in which the mission scenario and choice of mo del parameters are based on a real 4 h ttps:/epsrc.ukri.org/files/funding/calls/2017/raihubs F ormal V erification of Autonomous Robots and Battery PHM 3 industry surv ey pro ject. The UA V takes a sequence of actions and follows a fixed insp ection route on a 6 × 6 wind farm. It autonomously decides when to return to the base for recharges based on the health/states of the battery . Uncertain ties come from the dynamic environmen t which causes different lev els of p ow er demand. – W e explicitly consider the tw o battery features in our mo delling and sho w (i) how different battery safety strategies, dynamic environmen ts (i.e. differ- en t levels of p o wer demand) and the battery c hemical features join tly affect the formal v erification results; and (ii) the v erification results could b e ei- ther dangerously optimistic or to o p essimistic in practical cases, without the mo delling of the battery features. – W e discuss imp ortan t future work on explicitly integrating battery PHM with formal v erification, given the trend that adv anced PHM algorithms are mostly based on real-time readings from sensors deploy ed on the battery . The organisation of the pap er is as follows. In the next section, we presen t preliminaries on probabilistic mo del chec king and battery PHM. The running example is described in Sec. 3. W e show our probabilistic mo del and verification results in Sec. 4 and 5, respectively . Sec. 6 summarises the related work. F uture w ork and contributions are concluded in Sec. 7. 2 Bac kground 2.1 Probabilistic Mo del Chec king Probabilistic mo del c hecking (PMC) [25] has b een successfully used to analyse quan titative properties of systems across a v ariet y of application domains, includ- ing rob otics [28]. This inv olv es the construction of a probabilistic mo del, com- monly using Discrete Time Marko v Chain (DTMC), Contin uous Time Marko v Chain (CTMC) or Marko v Decision Pro cess (MDP), that formally represent the b eha viour of a system o ver time. The properties of in terest are usually sp ecified with e.g., Linear T emp oral Logic (L TL) or Probabilistic Computational T ree Logic (PCTL), and then systematic exploration and analysis is p erformed to c heck if a claimed prop ert y holds. In this pap er, we adopt DTMC and PCTL whose definitions are as follows. Definition 1 . A DTMC is a tuple ( S, s 1 , P , L ), where: – S is a (finite) set of states; and s 1 ∈ S is an initial state; – P : S × S → [0 , 1] is a probabilistic transition matrix such that P s 0 ∈ S P ( s, s 0 ) = 1 for all s ∈ S ; – L : S → 2 AP is a lab elling function assigning to each state a set of atomic prop ositions from a set AP . Definition 2 . AP is a set of atomic propositions and ap ∈ AP , p ∈ [0 , 1] , t ∈ N and   ∈ { <, ≤ , >, ≥} . The syntax of PCTL is defined by state formulae Φ and 4 X. Zhao et al. p ath formulae Ψ . Φ ::= tr ue | ap | Φ ∧ Φ | ¬ Φ | P . /p ( Ψ ) Ψ ::= X Φ | Φ U ≤ t Φ | Φ U Φ where the temp oral op erator X is called “next”, U ≤ t is called “b ounded un til” and U is called “un til”. Also, F Φ is normally defined as true U Φ which is called “ev entually”. State formulae Φ is ev aluated to b e either true or false in each state. Satisfaction relations for a state s are defined: s | = true , s | = ap iff ap ∈ L ( s ) s | = ¬ Φ iff s 6| = Φ s | = Φ 1 ∧ Φ 2 iff s | = Φ 1 and s | = Φ 2 s | = P . /p ( Ψ ) iff P r ( s | = Ψ )   p P r ( s | = Ψ )   p is the probabilit y of the set of paths starting in s and satisfying Ψ . Given a path ψ , if denote its i -th state as ψ [ i ] and ψ [0] is the initial state. Then the satisfaction relation for a path formula for a path ψ is defined as: ψ | = X Φ iff ψ [1] | = Φ ψ | = Φ 1 U ≤ t Φ 2 iff ∃ 0 ≤ j ≤ t ( ψ [ j ] | = Φ 2 ∧ ( ∀ 0 ≤ k < j ψ [ k ] | = Φ 1 )) It is worth while mentioning that b oth DTMC and PCTL can be augmented with rew ards/costs [7], which can be used to mo del, e.g. the energy consumption of rob ots in a mission. Indeed, this is the typical wa y used in existing studies, and differs from our mo delling of battery in this study . After formalising the system and its requiremen ts in DTMC and PCTL, re- sp ectiv ely , the v erification fo cus shifts to the c hec king of r e achability in a DTMC. In other w ords, PCTL expresses the constraints that must b e satisfied, concern- ing the probability of, starting from the initial state, reaching some states la- b elled as, e.g. unsafe, success, etc. Automated to ols ha v e b een dev eloped to solv e the reachabilit y problem. W e use PRISM [24] whic h emplo ys a symbolic mo del c hecking algorithm to calculate the probability that a path formulae is satisfied. More often, it is of interest to kno w the actual probability that a path formula is satisfied, rather than just whether or not the probability meets a required b ound. So the PCTL definition can b e extended to allow numeric al queries by the form P =? ( Ψ ) [25]. In general, a PRISM mo dule contains a num b er of lo cal v ariables which constitute the state of the mo dule. The transition b eha viour of the states in a mo dule is described by a set of commands which tak e the form of: [ Action ] Guard → P rob 1 : U pdate 1 + ... + P r ob n : U pdate n ; As described by the PRISM manual 5 , the guard is a predicate ov er all the v ari- ables (including those b elonging to other mo dules. Thus, together with the ac- tion labels, it allo ws modules to sync hronise). Each up date describes a transition 5 h ttps://www.prismmo delc heck er.org/manual/ F ormal V erification of Autonomous Robots and Battery PHM 5 Fig. 1. (Left) The non-linear dynamics of voltage and current vs SOC for a constant p o w er demand from [38]. (Right) Cited from [1], the “real data” curve showing a Lithium-ion battery capacit y fade and its PHM predictions (thick green line). whic h the mo dule can mak e if the guard is true. A transition is specified b y giv- ing the new v alues of the v ariables in the mo dule, p ossibly as a function of other v ariables. Each up date is also assigned a probabilit y (in our DTMC case) whic h will b e assigned to the corresp onding transition. 2.2 Battery Mo delling and PHM Electric batteries exhibit non-linear charge and disc harge characteristics due to a n umber of factors. The v oltage v aries with the state of charge (SOC) because of changing c hemical prop erties within the cell, such as increasing electrolyte resistance, non-linear diffusion dynamics and W arburg inductance [15]. Fig. 1- (Left), derived from the exp erimen tal test in [38], shows suc h non-linear results of voltage and curren t vs SOC profile for a constan t p ow er demand. A constant p o wer demand means that an increase in curren t is drawn as the voltage falls with SOC. F or our study , we are interested in a UA V with a battery capacit y of around 11Ah and nominal voltage of 22V. The energy supply is 180Wh from a lithium polymer battery . F or a 22V battery the voltage at full c harge is ∼ 25V and will drop to ∼ 20V at a safe threshold of 30% SOC. The easiest wa y to measure a c hange in SOC is b y integrating the curren t disc harge ov er time from a known initial SOC, called Coulomb coun ting [17]: S O C ( k + 1) = S O C ( k ) − I ( k ) × ∆t Q max (1) where Q max is the maxim um SOC, I ( k ) is the time dep endan t curren t, S O C ( k ) is the SOC p ercentage at the discrete time step k , ∆t is the time step interv al. Although this simplification do es not tak e in to consideration inaccuracies in the battery initial SOC estimation or account for the internal losses, it is proposed as a first approximation to mo del the p o wer usage and disc harge c haracteristics as discrete states using the known battery c haracteristics. 6 X. Zhao et al. Batteries also degrade ov er successive rec harges due to decreased lithium- ion concentrations so that o v er 1000 discharge cycles a 20% loss in capacit y ma y o ccur [36]. Fig. 1-(Righ t) sho ws a t ypical lithium-ion battery capacity fade c haracteristics the (“real data” curv e), cited from [1]. W e can observ e, after the first 80 disc harge cycles, the battery’s capacity drops from 1.85(Ah) to 1.55(Ah). There is a growing interest in the use of PHM tec hniques to reduce life-cycle costs for complex systems and core infrastructure [13]. Batte ry health manage- men t is also a critical area in regards to the safe and reliable deplo ymen t of UA Vs. Numerous studies into battery PHM techniques hav e b een carried out, e.g. the use of Neural Nets [10], Unscented Kalman Filters [17,19], Unscen ted T ransform [4], Hardy Space H ∞ Observ ers [41] and Physics Based mo dels [16]. Although w e are assuming a hypothetical/generic battery PHM metho d in this pap er to pro vide parameters in our latter mo delling, it is envisaged that adv anced PHM tec hniques can b e integrated in our future v erification framework. 3 The Running Example As UA V technology impro ves, energy companies are lo oking to adopt the tech- nology to reduce main tenance and op erating costs. The r esident dr one idea is to station a UA V at lo cations where aerial surv eys are conducted rep eatedly . The adv an tages of suc h resident drone insp ection system are the p ossibilit y of increased av ailability for data collection (e.g. to feed in techniques reviewed in [37]), reduced manual lab our, impro ved safety and more cost effective mainte- nance strategies. W e mo del a typical application of suc h system in this pap er, based on a survey utilising commercial tec hnologies. A simplified wind farm drone insp ection mission as a 6 × 6 grid of turbines with a UA V located at the cen tre is considered. Wind turbines are t ypically dis- tributed b et ween 5 and 12 turbine blade diameters apart, so a square distribution of turbines is mo delled, eac h 500 meters apart, as sho wn in Fig. 2-(Left). Fig. 2. (Left) A fully autonomous UA V inspection mission in a 6 × 6 wind farm. (Righ t) The fixed controller of a UA V insp ection mission on the wind farm. Intersections and cell spaces represen t wind turbines and transp ortation channels resp ectiv ely . F ormal V erification of Autonomous Robots and Battery PHM 7 The drone mission requires the drone take-off and land at the base station, fly a distance determined by the n um b er of grid spaces to a turbine, carry out an insp ection and return to the base or contin ue the mission with a single battery c harge. F or this mission a drone transit velocity of ∼ 10 m/s is assumed. An insp ection is exp ected to take 15 minutes and the take-off and landing time is estimated less than 1 minute. The battery rec harge time is around 1.5 hours. 4 The Mo delling in PRISM Our formal mo del of the running example presen ted in section 3 is a product (via parallel comp osition in PRISM) of four mo dules – Dr one , Grid , Envir onment and Battery . Dep ending up on the mo del parameters used, a typical instance of our mo del has roughly 100 , 000 states and 170 , 000 transitions. In what follows, we in tro duce the mo dules separately and describ e key assumptions, constants, and v ariables used in each mo dule. Given the page limits, we only show some typical PRISM commands in the mo dules and omit some sophisticated synchronisation and parallel composition among mo dules. The complete sources co de in the PRISM language can b e found in our rep ository 6 . 4.1 The Dr one Mo dule The Dr one mo dule is essentially a finite state machine describing the b ehaviour of the UA V during the insp ection mission, as shown in Fig. 3. The UA V b egins the mission in a fully charged state (S0) at the base. Once the UA V successfully tak es off (S1), it ma y either directly land due to violation of the b attery safety str ate gy (see section 4.4), or fly to the target cell (S2) and then carry out an insp ection of the wind turbine (S3). Dep ending up on the battery safety strategy and the battery SOC left after the inspection, the UA V will either fly back to the base for recharging (S4), stay in the same cell if there are more than one wind turbines to b e insp ected, or fly to the next target cell (S2) if all wind turbines of the current cell hav e b een inspected. Once landed at the base (S5), the UA V will declare success of the mission if all wind turbines on the wind farm hav e b een inspected, or recharge and con tinue the abov e work-flo w otherwise. The dotted lines in Fig. 3 represen t ev en ts where the battery SOC falls to 0, leading to an out-of-battery state (S6). Note, the transition f rom S0 to S6 means that the fully charged capacity is not sufficien t to do the next insp ection at the target cell and thus the UA V declares the mission failed without further actions. It is w orthwhile to men tion that, realistically , there should b e some prob- abilit y of failure for eac h action, e.g. 10 − 4 for landing. Ho wev er, since we are only interested in the particular failure mo de of out-of-battery here, w e simplify our mo del b y setting the failure probability of each action to 0. Thus, the only source of uncertain ty w e consider is from the dynamic en vironmen t which causes differen t lev els of pow er demand for each action. W e will discuss this in Sec. 4.3. 6 h ttps://x-y-zhao.github.io/files/V eriBatterySEFM19.prism. 8 X. Zhao et al. Fig. 3. A finite state mac hine of the UA V b eha viour mo delled b y the Dr one mo dule. 4.2 The Grid Mo dule W e formalise the wind farm as a 5 × 5 grid as sho wn in Fig. 2-(Righ t) in whic h the in tersections represent wind turbines and the cell spaces (labelled by co ordinates [ x, y ]) represen t transp ort channels. In this study , w e assume a given con trol p olicy (CP) of the UA V as follo ws: – CP1: The UA V will follow the snake shap ed route, as sho wn b y the red arro ws in Fig. 2-(Righ t), to carry out the insp ection in eac h cell. – CP2: Dep ending up on the co ordinates of the cell, there can b e 1, 2 or 4 app ointe d wind turbines to b e insp ected within a cell. F or instance, at cell [0,0], the wind turbine lo cated at the left-bottom corner is the only app ointed one; b oth the t wo bottom corners at cell [2,0] need to b e insp ected; and for cell [2,2], all 4 wind turbines around it should b e insp ected. Indeed, it would b e unwise (i.e. requiring more energy) to fly to cell [0,0] to insp ect the green dotted wind turbine in Fig. 2-(Righ t), rather than fly with the shortest route to cell [1,1]. – CP3: Dep ending up on the battery safety strategy and the remaining SOC, the UA V may suspend the mission and return to the base for recharging. It will resume the mission at the cell where the mission was susp ended. A part of the PRISM commands in this mo dule are shown in Fig. 4. Note, the transitions probabilities are simplified to 1. 4.3 The Envir onment Module W e explicitly consider the en vironmental dynamics due to its primary impact on the battery’s p o wer demand. F or simplicit y , only one ma jor factor – wind sp eed – was considered when developing the Envir onment mo dule. W e formalise tw o F ormal V erification of Autonomous Robots and Battery PHM 9 Fig. 4. Some PRISM commands of the Grid mo dule. lev els of wind sp eed, and use a parameter p wsp c to capture the dynamics of the wind. Environmen tal assumptions (EA) are listed b elo w: – EA1: The UA V will only attempt to tak e off in a lo w wind speed condition. – EA2: The change of wind sp eed (either from low to high or the other wa y around) o ccurs, with a probabilit y of p w sp c , b efore eac h action is taken in the Dr one mo dule. Fig. 5. The PRISM commands of the Envir onment module. F rom EA2, we know that the higher the p w sp c is, the more dynamic the en vironment, and it is this assumption that introduces uncertaint y in the energy consumption for a given action. The PRISM commands are sho wn in Fig. 5. 4.4 The Battery Mo dule Fig. 6 shows tw o abstracted state machines of the Battery module which run in parallel to describe the battery behaviour. The battery features of capac- 10 X. Zhao et al. it y fading (ov er successiv e rec harges) and increasing discharge rate (in a single disc harge cycle) are captured b y battery assumptions (BA) as follows: – BA1: After eac h rec harge, the battery’s fully charged capacity (i.e. c f ul l in Fig. 6) cannot be recov ered to the new battery’s capacity . Rather, it decreases with a rate which can b e obtained from battery PHM exp erimen ts, e.g. as observ ed from the results in [1], for the first 100-ish disc harge cycles, at eac h rec harge, the capacity will fade at an av erage rate of 0.2%. – BA2: F or a given wind sp eed, the UA V is working at a constan t p o wer demand for all actions. Since we considered 2 lev els of wind sp eed in the Envir onment module, there are 2 levels of constan t p o wer demand as w ell. – BA3: In one discharge cycle, the battery’s v oltage is essen tially a non-linear function of its SOC. W e use a step-wise function to appro ximate the non- linear function – high voltage V 2 (SOC > 0.75), medium voltage V 1 (0.75 ≥ SOC ≥ 0.25) and low v oltage V 0 (SOC < 0.25). In line with the BA2 and BA3, w e use the following Eq. (2) to estimate the battery consumption (Ah) for an action j (denoted as c act in Fig. 6) under differen t levels of v oltage V i and a p o wer demand lev el k : C j = E spec · t j V i · T k (2) where E spec is the specified battery energy , T k is the total running time under a constan t pow er level k , V i is the lev el of v oltage, and t j is the estimated execution time of action j . F or instance, a t ypical UA V battery with a sp ecified energy of 180Wh ( E spec = 180) can fly 30 min utes at the normal level of w orkload ( T k = 0 . 5 and k repre- sen ts the normal level of pow er demand). The sp ecified normal w orking voltage is 22V ( V 1 = 22) (with a maxim um lev el of 25V, V 2 = 25, and minimum level of 20V, V 0 = 20). The a verage time for insp ecting a wind turbine is 15 min utes (if action j represents the insp ection, then t j = 0 . 25). Via Eq. (2) and those esti- mated parameters ab o ve, we obtain the last ro w in T able 1 (results are rounded to one decimal place). Similarly for the battery consumption of each action at the high p o wer demand lev el (high wind sp eed en vironment), the v alues can b e calculated in the same wa y but are not shown in this pap er. T able 1. Battery consumption (Ah) of actions under different levels of v oltage in low wind sp eed environmen t (i.e. the normal level of pow er demand). lo w v oltage medium v oltage high voltage tak e-off/land 0.3 0.2 0.1 transp ort p er cell 0.5 0.4 0.3 insp ection p er wind turbine 4.5 4.0 3.6 F ormal V erification of Autonomous Robots and Battery PHM 11 So far, in our mo delling, instead of assuming a fixed battery consumption for eac h action, we hav e 6 p ossibilities of the battery consumption after an action (3 voltage levels × 2 p o wer demand lev els). Engineers are a ware of the higher risks associated from op erating with a lo wer SOC battery , th us there are requiremen ts on the battery PHM to provide w arnings when the SOC falls b elo w a certain threshold [35] (and to recommend that the mission b e discon tinued), e.g. a t ypical 30% threshold is adopted by NASA in [19]. In line with that battery safety strategy (BS), we also define a parameter safe t as a safety threshold: – BS1: before each of the actions, take-off, fly-to-target and insp ect, the UA V will c heck if the SOC will fall below safe t after a sequence of actions to p erform an in tended insp ection. If there is sufficien t SOC, then tak e the action, otherwise return for recharging. An instance of BS1 is that, b efore flying to the target cell, the UA V will predict the remaining SOC (based on the current battery health/state and wind conditions) after flying to the target and p erforming one insp ection. If there is no safe battery life remained (i.e. SOC < safe t ) after the intended insp ection, the UA V will go bac k for rec harging. Some typical PRISM commands of this mo dule and associated formulas are sho wn in Fig. 7. Fig. 6. Abstracted state machines of the Battery mo dule. 5 Results The main prop erties of in terest and their corresp onding PCTL formulas are: – The probability of mission success 7 : P =? [ F ( s = 7)]; – The exp ected mission time: R { “mt” } =? [ F ( s = 7) | ( s = 6)]; 7 Since we fo cus on the particular failure mo de of out-off-battery in our mo del, rigor- ously this should b e the probability of seeing no out-off-battery failures in a mission. 12 X. Zhao et al. Fig. 7. Some PRISM commands of the Battery mo dule and global formulas. Note, the k ey v ariables c ful l vary (the fully-c harged capacit y considering capacity fading) and c ftt , c ins etc. (the battery consumption of each action, whic h are generically denoted as c act in Fig. 6) should b e obtained dynamically from the PHM system in reality , whilst we make simplified assumptions in the source codes. – The exp ected n umber of recharges: R { “rc” } =? [ F ( s = 7) | ( s = 6)]. W e use the PRISM tool [24] to c heck the properties given different mo del parameters in later subsections. Indeed, we may only b e concerned with the exp ected mission time (or num ber of recharges) given the mission is successful. Ho wev er, PRISM can only solve the “reachabilit y rew ard” prop erties when the target set of states is reached with probability 1, thus our target state here is ( s = 7) | ( s = 6). In our later numerical examples, we only sho w the expected mission time when the probability of the mission failing is very small so that its con tribution to the av erage mission time is negligible. Note, this limitation of PRISM has b een studied in [29]. 5.1 Effects of Battery Safety Strategies and Dynamic Environmen ts F or a typical new battery with 11Ah capacit y , we highligh t the verification results of four represen tative cases, as shown in T able 2, b y setting the ab o ve men tioned mo del parameters (cf. BS1 and EA2) as: – #1, the common case and baseline: safe t = 0 . 3, p wsp c = 0 . 1. F ormal V erification of Autonomous Robots and Battery PHM 13 – #2, a risky battery strategy: safe t = 0 . 25, p wsp c = 0 . 1 – #3, a more dynamic environmen t: safe t = 0 . 3, p wsp c = 0 . 3. – #4, a risky battery strategy in a more dynamic environmen t: safe t = 0 . 25, p wsp c = 0 . 3. T able 2. V erification results of some t ypical cases with a new battery capacity of 11Ah. No. of states No. of transitions Prob. mission success Exp. mission time Exp. no. of rec harges #1 108,688 163,076 1 4700.20 42.65 #2 117,765 177,278 0.91 3885.95 34.59 #3 108,688 163,076 1 7482.86 72.16 #4 117,765 177,278 0.89 5621.66 53.07 The example of #1 represen ts the common case that serves as a baseline in T able 2. Case #2 represen ts the use of a relativ ely risky strategy b y reducing the battery safety threshold from 0.3 to 0.25. Indeed, in T able 2, w e see a decreased probabilit y of mission success (from 1 to 0.91), whilst the expected mission time and n umber of recharges also significantly reduce, which is the b enefit of taking more risk. Comparing case #3 and #1, giv en a fairly safe battery strategy (i.e. safe t = 0 . 3), a more dynamic environmen t will significan tly increase the mission time and num b er of recharges. Because the more dynamic the environmen t is, the more often the UA V decides to go bac k for recharges for battery safety reasons. Note, the probabilit y of mission success remains (i.e. 1), since the battery strategy is conserv ativ e enough to guarantee a safe trip bac k to base in all p ossible circumstances. On the contrary , if we adopt a risky battery strategy in a more dynamic en vironment (#4), then not only the exp ected mission time increases but also the probabilit y of mission success decreases (cf. #4 and #2), b ecause there are cases that the UA V do es not reserve enough battery to fly bac k to base due to a sudden change of en vironments. 5.2 Comparison of Mo dels, Disregarding the Battery F eatures Most existing v erification studies of autonomous rob ots, when considering en- ergy constrain ts, formalise the energy comp onent in a generic/simplified manner suc h that battery features are ov erlo ok ed. In this section, we illustrate the dif- ference b et w een a simplified battery mo del and our relatively adv anced mo del, considering the battery chemical features. Fig. 8 shows the probability of mission success, via differen t mo dels, as a function of the new battery’s capacity (Ah). The solid curv e lab elled as “ad- v anced” represen ts our prop osed mo del considering the battery features (BA1 and BA3). The other curves represen t the basic mo dels without considering bat- tery features, e.g. the “basic high” curve is the case when there is no capacity fading and the battery alwa ys works at a high level of v oltage. 14 X. Zhao et al. Fig. 8. Probability of mission success, via different mo del assumptions on batteries, as a function of the new battery capacit y . In Fig. 8, w e can observ e that, for a given mo del, there is a required minim um new battery capacit y to ha ve non-zero probabilit y to succeed. Indeed, the capac- it y should b e at least enough for the insp ection of the first wind turbine and a safe trip back. Since the battery consumption of eac h action is higher (and high- est) when assuming that the battery is alwa ys w orking at a medium (and low) v oltage lev el, such a required minim um capacit y increases. Similarly , to guaran- tee a successful mission, “basic high” requires that the relativ ely smallest new battery capacit y (around 8.4Ah) due to its ob viously optimistic assumptions, i.e. no capacity fading and alw ays w orking at a high voltage lev el. Note, although the “adv anced” mo del is b ounded by “basic high” and “ba- sic medium”, it is still dangerous to use such simplified bounds to do approx- imation, due to the observed “dip” on the “adv anced” curve in the range of 12Ah–13Ah. That dip of probability of mission success happ ens b ecause, when the new battery’s capacity increases (but is still not big enough), the UA V may decide to take more risk to p erform more actions in one trip (i.e. one disc harge cycle), after which there might not b e enough SOC left for a safe trip back in some edge cases (e.g. a degraded battery w orking at a lo w voltage in a high wind sp eed environmen t). T o eliminate this phenomenon, the simplest wa y is to raise the battery safety threshold, whic h is confirmed by our extra experiments. An example path of a failed mission is presented in Fig. 9, in whic h the new battery’s capacity is 12.8Ah (thus within the “dip” range in Fig. 8). After 22 rec harges at step #148, the UA V flies to the target cell [0,4] and the wind speed c hanges to high ( w sp = 2). A t step #150, the predicted SOC after the in tended insp ection is higher than the safety threshold of 0.3. So instead of returning to the base, the UA V contin ues the insp ection in high-sp eed wind. Although after managing to return to base, the drone fails to land in a high wind sp eed and at the low er v oltage level. Also, if we naiv ely ignore the capacity fading and/or assuming the battery never works at a low voltage, then the UA V would land F ormal V erification of Autonomous Robots and Battery PHM 15 safely in this example. That’s why we don’t observ e the “dips” on the curves of the basic mo dels in Fig. 8. Fig. 9. A fragment of a failed mission path generated by the PRISM simulator, which is an example of the “dip” in Fig. 8 with the new battery’s capacity as 12.8Ah. Note, only key v ariables of the 4 mo dules are configured to be view ed here. Fig. 10 shows the exp ected mission time (upper graph curv es) from the speci- fied battery models and the differences (low er curves) b et ween them. W e observe that, in the practical range of the new battery’s capacity (i.e. < 13Ah, base on our survey), the basic mo dels could giv e either to o optimistic (500 minutes less) or ov er pessimistic (1500 minutes more) results. Suc h a v ariance of 1 to 3 working da ys will mislead wind farm main tenance activities and thus cause significan t economic loss. Not surprisingly , as the new battery capacit y tends to infinity (i.e. the battery is no longer a bottle neck of the giv en mission), the verification results of all mo dels tends to the same v alue (as do the results in Fig. 8). 6 Related W ork Ho w autonomous rob ots should b e v erified is a new challenging question [9,6], and it has receiv ed great atten tion in recent years, e.g. [11,30,33,42]. When con- sidering energy constrain ts, the energy consumption is usually formalised in a linear w ay that b eing generic for both liquid-fuel and batteries. F or instance, in the analysis of rob ot sw arms [22,27] the authors assume constan t energy cost at eac h time step and a fixed capacit y when obtaining energy from “fo od”. Again, in the mo delling of UA V missions [12,18], a fixed battery capacity and con- stan t battery consumption o ver time is assumed. In [11], energy consumption of UUV sensors is mo delled as a rew ard/cost for eac h state, whic h exhibits a linear behaviour ov er time. Indeed, suc h generic and simplified assumptions do not necessarily mean that they are unrealistic, whilst w e b elieve more rigorous discussions and studies should b e carried out prior to their adoption. The study in [3] highlights the difference b et ween real and ideal batteries, with a case study on controlling an energy-constrained rob ot. But it fo cuses on another battery feature – “reco very effect” (e.g. a smart phone might shutdo wn due to an out-of-battery failure, but then b ecome liv e again after an idle perio d). 16 X. Zhao et al. Fig. 10. The upp er graph curv es sho w the exp ected mission time, for the specified battery mo del, as a function of the new battery capacity . The low er graph curves sho w the differences of the exp ected mission times from the sp ecified mo dels. Bey ond the scop e of rob otics systems, battery behaviour does dra w attention for verification. F or instance, in [39], the battery of a satellite is described by the Kinetic Battery Mo del whic h is formalised as a timed automata to precisely mo del the discharge behaviour. Ho w ever they leav e out the capacit y fading fea- ture as future work. Similarly in [20], the Kinetic Battery Model is used for analysing wireless sensor proto cols. F or smartphones, [5] uses runtime verifica- tion to chec k whether the actual battery consumption is within the exp ected limits that are derived from battery consumption profiles for smartphone apps. 7 Discussions, Conclusions and F uture W ork In this pap er, we formalise a UA V insp ection mission of an offshore wind farm, and then do probabilistic mo del chec king in PRISM to sho w (i) ho w the battery’s non-linear features significantly affect the verification results in most practical cases; and (ii) ho w battery safety strategies, dynamic en vironments and battery features jointly affect the v erification results. Most existing formal v erification studies of rob ots mak e simplified linear as- sumptions on energy consumption, which is indeed preferable in the case that the capacit y is far b eyond the total battery cost of the whole mission (i.e. when there is no rec harges and the battery’s working voltage is fairly stable due to a considerable SOC margin remaining at the end of the mission). In contrast, our work shows how suc h a simplification can significan tly affect the verification results in the case that there are multiple recharges in the autonomous mission. Th us, w e b eliev e our w ork highlights this risk and calls for more rigorous dis- F ormal V erification of Autonomous Robots and Battery PHM 17 cussions prior to any battery assumptions made in future formal verification of rob ots, especially when recharges are exp ected in the mission scenarios. Moreo ver, we b eliev e that battery PHM techniques should b e explicitly in- tegrated in to the formal verification of robots. Although in this pap er we use a h yp othetical/generic battery PHM technique to provide the parameters used in the Battery mo dule, it is clear ho w PHM can aid the rigorous mo delling of battery for formal verification. F or now, b oth the battery PHM exp erimen ts and formal v erification are assumed to b e carried out in the lab, i.e. prior to the mission. T o improv e the accuracy , an app ealing idea is to integrate both at run time, since there is a trend of doing online battery PHM based on real-time readings from the sensors deplo yed on the battery , e.g. [2,40]. Whilst there will b e a scalabilit y issue if running both online battery PHM and formal verification algorithms at runtime. A compromised solution, in our example, is to inv oke the formal v erification and PHM during the recharging at the base (where substan- tial computing resources can be used) with newly collected log-data from recen t fligh ts. Thus the verification result will b e up dated with the up-to-date data. W e plan to implement this solution in our future w ork. Apart from highligh ting the need of in tegrating battery PHM techniques, this w ork only serv es as a first approximation 8 of the verification of the residential drone insp ection mission. More rigorous verification/planning of the mission is needed in future, e.g. b y gradually refining the fundamen tal PRISM mo del based on observ ations from v arious sources of data [31,32]. In summary , our main contributions are: – W e formalise a UA V inspection mission on a wind farm based on a real industry survey pro ject, which can b e reused and extended as an exemplar for future research of similar UA V missions. – W e do a sequence of what-if calculations, via probabilistic mo del chec king, to sho w (i) the importance of considering non-linear battery features in for- mal verification of autonomous rob ots; and (ii) how such battery features, together with the dynamic environmen ts and battery safety strategy , join tly affect the verification results. – W e discuss the need of explicitly integrating battery PHM techniques into formal verification of rob ots, and prop ose a p oten tial solution which forms imp ortan t future work. References 1. Andoni, M., T ang, W., Robu, V., Flynn, D.: Data analysis of battery storage systems. CIRED - Op en Access Pro ceedings Journal 2017 (1), 96–99 (2017) 2. Barr ´ e, A., Suard, F., G ´ erard, M., Riu, D.: A real-time data-driven metho d for battery health prognostics in electric vehicle use. In: Pro c. of the 2nd Europ ean Conf. of the Prognostics and Health Managemen t Society. pp. 1–8 (2014) 8 It is a first appro ximation in the sense of, e.g. the simplification of tw o levels of wind sp eed and the round estimations of battery consumption in T ab. 1. 18 X. Zhao et al. 3. Bok er, U., Henzinger, T.A., Radhakrishna, A.: Battery transition systems. In: Pro c. of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. pp. 595–606. POPL ’14, ACM, San Diego, California, USA (2014) 4. Daigle, M., Go ebel, K.: Improving computational efficiency of prediction in model- based prognostics using the unscented transform. Annual Conference of the Prog- nostics and Health Management So ciet y (2010) 5. Espada, A.R., del Mar Gallardo, M., Salmer´ on, A., Merino, P .: Run time v erification of expected energy consumption in smartphones. In: Model Checking Softw are. LNCS, vol. 9232, pp. 132–149. Springer International Publishing, Cham (2015) 6. F arrell, M., Luck cuck, M., Fisher, M.: Rob otics and integrated formal metho ds: Necessit y meets opp ortunity . In: Pro c. of the 14th Int. Conf. on Integrated F ormal Metho ds. LNCS, vol. 11023, pp. 161–171. Springer, Cham (2018) 7. Filieri, A., T am burrelli, G.: Probabilistic verification at runtime for self-adaptive systems. In: C´ amara, J., de Lemos, R., Ghezzi, C., Lopes, A. (eds.) Assurances for Self-Adaptiv e Systems: Principles, Mo dels, and T echniques, LNCS, vol. 7740, pp. 30–59. Springer Berlin Heidelb erg, Berlin, Heidelb erg (2013) 8. Fisher, M., Collins, E., Dennis, L., Luc k cuck, M., W ebster, M., Jump, M., Page, V., P atchett, C., Dinmohammadi, F., Flynn, D., Robu, V., Zhao, X.: V erifiable self- certifying autonomous systems. In: 2018 IEEE Int. Symp. on Soft ware Reliability Engineering W orkshops (ISSREW). pp. 341–348 (2018) 9. Fisher, M., Dennis, L., W ebster, M.: V erifying autonomous systems. Communica- tion of the ACM 56 (9), 84–93 (Sep 2013) 10. Gao, D., Huang, M., Xie, J.: A no vel indirect health indicator extraction based on c harging data for lithium-ion batteries remaining useful life prognostics. SAE In ternational Journal of Alternativ e P ow ertrains 6 (2), 183–193 (2017) 11. Gerasimou, S., Calinescu, R., Banks, A.: Efficien t runtime quan titative v erification using caching, look ahead, and nearlyoptimal reconfiguration. In: Pro c. of the 9th In t. Symp. on Soft ware Engineering for Adaptive and Self-Managing Systems. pp. 115–124. SEAMS 2014, ACM, New Y ork, NY, USA (2014) 12. Giaquin ta, R., Hoffmann, R., Ireland, M., Miller, A., Norman, G.: Strategy syn- thesis for autonomous agents using PRISM. In: NASA F ormal Metho ds. LNCS, v ol. 10811, pp. 220–236. Springer In ternational Publishing, Cham (2018) 13. Go ebel, K., Celay a, J., Sank araraman, S., Ro ychoudh ury , I., Daigle, M., Saxena, A.: Prognostics: The Science of Making Predictions. CreateSpace Indep enden t Pub- lishing Platform, 1st edn. (2017) 14. Guio c het, J., Mac hin, M., W aeselynck, H.: Safet y-critical adv anced robots: A sur- v ey . Rob otics and Autonomous Systems 94 , 43 – 52 (2017) 15. Hariharan, K.S.: Mathematical Modeling of Lithium Batteries F rom Electro c hemi- cal Mo dels to State Estimator Algorithms. Green Energy & T ech., Springer (2018) 16. He, W., Pec ht, M., Flynn, D., Dinmohammadi, F.: A ph ysics-based electrochem- ical model for lithium-ion battery state-of-c harge estimation solv ed by an opti- mised pro jection-based method and moving-windo w filtering. Energies 11 (8), 2120 (2018) 17. He, W., Williard, N., Chen, C., P ech t, M.: State of c harge estimation for electric v ehicle batteries using Unscen ted Kalman Filtering. Micro electronics Reliability 53 (6), 840–847 (2013) 18. Hoffmann, R., Ireland, M., Miller, A., Norman, G., V eres, S.: Autonomous agent b eha viour mo delled in PRISM – A case study . In: Boˇ sna ˇ cki, D., Wijs, A. (eds.) Mo del Chec king Softw are. LNCS, vol. 9641, pp. 104–110. Springer In ternational Publishing, Cham (2016) F ormal V erification of Autonomous Robots and Battery PHM 19 19. Hogge, E.F., Bole, B.M., V azquez, S.L., Celay a, J.R., Strom, T.H., Hill, B.L., Small- ing, K.M., Quach, C.C.: V erification of prognostic algorithms to predict remaining flying time for electric unmanned v ehicles. In ternational Journal of Prognostics and Health Managemen t 9 (1), 1–15 (2018) 20. Iv ano v, D., Larsen, K.G., Sch upp, S., Srba, J.: Analytical solution for long battery lifetime prediction in nonadaptiv e systems. In: Quan titative Ev aluation of Systems. LNCS, vol. 11024, pp. 173–189. Springer, Cham (2018) 21. Kalra, N., Paddock, S.M.: Driving to safet y: How man y miles of driving would it tak e to demonstrate autonomous vehicle reliabilit y? T ransp ortation Research P art A: Policy and Practice 94 , 182 – 193 (2016) 22. Kon ur, S., Dixon, C., Fisher, M.: Analysing rob ot swarm behaviour via probabilis- tic mo del chec king. Robotics and Autonomous Systems 60 (2), 199 – 213 (2012) 23. Ko opman, P ., W agner, M.: Autonomous vehicle safet y: An interdisciplinary chal- lenge. IEEE In telligent T ransp ortation Systems Magazine 9 (1), 90–96 (2017) 24. Kwiatk owsk a, M., Norman, G., P arker, D.: PRISM 4.0: V erification of probabilistic real-time systems. In: Pro c. 23rd Int. Conf. on Computer Aided V erification. LNCS, v ol. 6806, pp. 585–591. Springer (2011) 25. Kwiatk owsk a, M., Norman, G., P arker, D.: Probabilistic mo del chec king: Adv ances and applications. In: F ormal System V erification: State-of-the-Art and F uture T rends, pp. 73–121. Springer International Publishing, Cham (2018) 26. Lane, D., Bisset, D., Buckingham, R., Pegman, G., Prescott, T.: New foresight re- view on robotics and autonomous systems. T ech. Rep. No. 2016.1, Llo yd’s Register F oundation, London, U.K. (2016) 27. Liu, W., Winfield, A.: Mo deling and optimization of adaptive foraging in swarm rob otic systems. The Int. Journal of Rob otics Researc h 29 (14), 1743–1760 (2010) 28. Luc kcuc k, M., F arrell, M., Dennis, L., Dixon, C., Fisher, M.: F ormal sp ecifica- tion and verification of autonomous robotic systems: a survey . arXiv preprint arXiv:1807.00048 (2018) 29. M¨ arck er, S., Baier, C., Klein, J., Kl ¨ uppelholz, S.: Computing conditional probabil- ities: implementation and ev aluation. In: Cimatti, A., Sirjani, M. (eds.) Softw are Engineering and F ormal Metho ds. LNCS, v ol. 10469, pp. 349–366. Springer In ter- national Publishing, Cham (2017) 30. Norman, G., Park er, D., Zou, X.: V erification and control of partially observ able probabilistic systems. Real-Time Systems 53 (3), 354–402 (Ma y 2017) 31. P aterson, C.A., Calinescu, R.: Observ ation-enhanced QoS analysis of comp onen t-based systems. IEEE T rans. on Softw are Engineering (2019). h ttps://doi.org/10.1109/TSE.2018.2864159, (Early Access) 32. P aterson, C., Calinescu, R., W ang, D., Manandhar, S.: Using unstructured data to impro ve the contin uous planning of critical pro cesses inv olving humans. In: 14th In t. Symp. on Softw are Engineering for Adaptive & Self-Managing Systems (2019) 33. P athak, S., Pulina, L., T acchella, A.: V erification and repair of control p olicies for safe reinforcement learning. Applied Intelligence 48 (4), 886–908 (Apr 2018) 34. Robu, V., Flynn, D., Lane, D.: T rain rob ots to self-certify as safe. Nature 553 (7688), 281–281 (2018) 35. Saxena, A., Royc houdhury , I., Cela ya, J., Saha, B., Saha, S., Go eb el, K.: Require- men ts flo wdown for prognostics and health management. In: Infotec h@Aerospace. American Institute of Aeronautics and Astronautics (2012) 36. Sp otnitz, R.: Sim ulation of capacit y fade in lithium-ion batteries. Journal of Po w er Sources 113 (1), 72–80 (Jan 2003) 20 X. Zhao et al. 37. Stetco, A., Dinmohammadi, F., Zhao, X., Robu, V., Flynn, D., Barnes, M., Keane, J., Nenadic, G.: Machine learning metho ds for wind turbine condition monitoring: A review. Renew able Energy 133 , 620 – 635 (2019) 38. T raub, L.W.: Calculation of constant pow er lithium battery discharge curv es. Bat- teries 2 (2) (2016) 39. W ognsen, E.R., Hansen, R.R., Larsen, K.G.: Battery-aw are scheduling of mixed criticalit y systems. In: Lev eraging Applications of F ormal Methods, V erification and V alidation. Sp ecialized T echniques and Applications. LNCS, vol. 8803, pp. 208–222. Springer Berlin Heidelb erg, Berlin, Heidelb erg (2014) 40. Zhang, C., Allafi, W., Dinh, Q., Ascencio, P ., Marco, J.: Online estimation of battery equiv alen t circuit mo del parameters and state of charge using decoupled least squares tec hnique. Energy 142 , 678 – 688 (2018) 41. Zhang, F., Liu, G., F ang, L., W ang, H.: Estimation of battery state of charge with H ∞ observ er: Applied to a rob ot for insp ecting p o wer transmission lines. IEEE T ransactions on Industrial Electronics 59 (2), 1086–1095 (F eb 2012) 42. Zhao, X., Robu, V., Flynn, D., Dinmohammadi, F., Fisher, M., W ebster, M.: Prob- abilistic mo del c hecking of rob ots deploy ed in extreme environmen ts. In: The 33rd AAAI Conf. on Artificial In telligence (In Press). Honolulu, Haw aii, USA (2019)