Unified Simulation and Test Platform for Control Systems of Unmanned Vehicles

1 Unified Simulation and T est Platform for Control Systems of Unmanned V ehicles Xunhua Dai, Chenxu K e, Quan Quan and Kai-Y uan Cai Abstract —Control systems on unmanned vehicles are safety- critical systems whose requir ements on reliability and safety are ever -increasing . Currently , testing a complex autonomous control system is an expensiv e and time-consuming pr ocess, which requir es massive repeated experimental testing during the whole development stage. This paper pr esents a unified simulation and test platf orm f or v ehicle autonomous control systems aiming to significantly impr ove the dev elopment speed and safety lev el of unmanned vehicles. First, a unified modular modeling framew ork compatible with different types of vehicles is proposed with meth- ods to ensure modeling credibility . Then, the simulation software system is developed by the model-based design framework, whose modular programming methods and automatic code generation functions ensure the efficiency , credibility , and standardization of the system development process. Finally , an FPGA-based real- time hardware-in-the-loop simulation platf orm is proposed to ensure the comprehensiv eness and credibility of the simulation and test results. In the end, the proposed platform is applied to a multicopter control system. By comparing with experimental results, the accuracy and credibility of the simulation testing results are verified by using the simulation credibility assessment method pr oposed in our previous work. T o verify the practica- bility of the pr oposed platform, several successful applications are presented for the multicopter rapid pr ototyping, estimation algorithm verification, autonomous flight testing, and automatic safety testing with automatic fault injection and result evaluation of unmanned vehicles. Index T erms —Unmanned vehicles, Modeling, Safety testing, Control system, HIL, U A V . I . I N T R O D U C T I O N Unmanned vehicles (e.g., cars, boats, fixed-wing aircraft, multicopters, robotics, and helicopters) are becoming increas- ingly popular in both civil and military fields [1]. For all unmanned vehicles, safety is always the most basic require- ment, and the concern over potential safety issues remains the biggest challenge for the commercialization of unmanned vehicles. For most commercial unmanned vehicles, there is usually not enough space or payload to carry more hardware redundancy (such as backup engines, actuators or motors) due to the limitation of cost and performance, so the software redundancy is often applied in control systems to ensure safety . As a result, among all components on an unmanned vehicle, the control system is the most complex and important compo- nent that undertakes responsibilities for both reliable operation under nominal conditions and safety decision under failure scenarios. Efficient simulation and test methods [2] for control systems of unmanned vehicles are urgently needed for the ev er-increasing system complexity and safety requirements. The authors are with School of Automation Science and Electrical Engi- neering, Beihang Univ ersity , Beijing 100191, China. According to [3], more than 80% de velopment tasks of an autonomous control system are in the middle level (see Fig. 1) to guarantee safety under various possible faults. Howe ver , most faults are rare to be encountered in practice, so massive repeated e xperimental tests are essential to ensure that control systems can correctly detect and handle unexpected faults. Although different unmanned vehicles have dif ferent shapes, configurations, or running en vironment, they hav e a similar system structure presented in Fig. 1 and share many common model features and fault modes. The common faults for unmanned vehicles include actuator f aults (e.g., block ed, failed or unhealthy), sensor and communication faults (e.g., loss of signal, delays, GPS failed, and transmitting interference) [4], en vironment faults (e.g., obstacles, collisions, and wind disturbances) [5] and vehicle model faults (e.g., vibration and loss of weight). Thus, a unified simulation and test platform compatible with different types of v ehicles will be beneficial to share fault mode information and safety design e xperience to improv e the safety le vel of the whole unmanned vehicle field. Besides, it can help to increase the exchange of safety design experience among different companies, manufacturers and certification authorities, and decrease the repetitive work during testing and assessment processes, which is also benefi- cial to the rapid dev elopment requirements and better response to the rules and regulations of governments. Co n t r o l Sof tware System Sens o r s Ac t u a t o r s Ve h i c l e Body Envi ronmen t Fix ed-wing Mult icopter VT OL Shi p Car He l ic opter Op eratio n S ystem Dri ver I n t erfac e s Low Leve l: Contro l Top Le vel: AI Mi d d le Level: Safet y A ssurance IM U Ba romet er Vis ion GP S Ă Co n t r o l Sig nals Sens o r Da t a For ce & Mo ment Kine matics&D yna mics Ro t a t io n&Thru st Prop e ll e r Bat tery Wh e e l Grou n d Wate r Air Wi nd Ob stacles Moto r Ru dder Ă Ă Fig. 1. System structure of unmanned vehicles. Currently , experimental testing is widely adopted because it can reflect real situations to the utmost extent. Besides, the safety problems of control systems are usually highly coupled with the actual situations. Since there is still no widely rec- ognized safety assessment standard published for unmanned vehicle systems (both unmanned cars and aircraft), many pre- researches are proposed to comprehensiv ely test and assess the 2 safety of unmanned systems based on experimental methods. For example, in [6], the experimental testing and assessing method for the safety of control systems of Unmanned Aerial V ehicles (UA Vs) are studied inspired by the testing methods from the airworthiness of manned aerial v ehicles. Howe ver , for unmanned vehicles, the experimental testing methods are usu- ally high-cost, inefficient, dangerous, and regulatory restricted. W ith the e ver-increasing complexity of control systems, the experimental testing methods become increasingly inefficient in re vealing potential safety issues and covering the testing cases. Besides, in experimental testing, it is usually hard to obtain the true states of unmanned vehicles (usually estimated by sensors or human perceptions) and to ensure consistency of en vironment variables (e.g., wind, temperature, and road condition). For example, the sensor failures may cause the vehicle state estimation inaccuracy and unreliable to reflect the true states of vehicles, so the test results in these cases are not suitable for quantitativ e assessment of vehicle safety . As a result, the e xperimental testing results usually need to be analyzed by experienced engineers to e valuate the safety lev el of a control system, which is not efficient, precise, and automatic enough for modern complex control systems. Due to the abov e disadvantages of experimental testing methods, new simulation and test methods (e.g., the real-time simulation methods [7], high-precision modeling and system identifi- cation methods [8], model-based safety assessment methods [9]) are becoming the trend for both manned and unmanned vehicles. Although experimental testing cannot be completely abandoned, simulation testing techniques are taking on more and more safety testing and assessment tasks [10]. Simulation methods for v ehicle control systems can be divided into Softw are-In-the-Loop (SIL) simulation and Hardware-In-the-Loop (HIL) simulation. As shown in Fig. 2(b), by running the control algorithms in the same com- puter with the vehicle simulation model, SIL simulation can quickly test the control algorithms with the simulation speed much faster than the real world. Howe ver , the software and hardware operating en vironment of the SIL simulation is dif- ferent from the real vehicle system, whose control algorithms are running on specific hardware (e.g., embedded systems, and industrial computers). In summary , SIL simulations can accel- erate the test speed, but it is based on the expense of losing simulation credibility . T o improve the simulation credibility , as shown in Fig. 2(c), HIL simulation is proposed to reflect the real operating en vironment of the control algorithms by using real control systems and real-time simulation computers. The HIL simulation requires the simulation model runs in real- time with the control systems, which makes it con venient to communicate with other external hardware. Currently , there are many popular simulation software supporting HIL sim- ulations for accelerating the development efficienc y , such as Airsim , SwarmSim , and other systems for U A Vs [11], [12], [13], and Car sim , Apollo , and other systems for autonomous cars [14], [15], [16]. Besides, HIL simulation systems are also widely used in robotics studies to verify their algorithms [17], [18] or train the autonomous vehicle control systems [19]. These HIL simulation systems hav e been proven to be con venient and efficient in accelerating the dev elopment speed of control systems of unmanned vehicles, but there are still some problems. First, these HIL systems focus more on providing testing environments for the upper-lev el algorithms such as control algorithms, Computer V ision (CV) algorithms and Artificial Intelligence (AI) algorithms of control systems; they usually cannot simulate the low-le vel software (e.g., operating system, and driv ers) and hardware (e.g., sensor chips and high-speed analog circuits) due to computing capability limitations. Secondly , they require to modify the code of control systems to disable the original hardware driv ers and add interface programs to exchanging data with the simulation computer , which may affect the operating en vironment and performance of the original system (unstable and unreliable). Thirdly , the lack of widely recognized assessment methods makes it hard for people (e.g., users, manufacturers, and certi- fication authorities) to belie ve the credibility of the simulation testing results for assessments and certifications. Co n t r o l Syste m Real W orld Envi ronmen t Ve h i c l e Body Ac t u a t o r s Pro ces sor Co r e Al g o rithm s D r i v e s Sens o r s&Com mu nic a t ion Circuit s (a) R e a l Ve hicl e Sy s t e m Co n t r o l Syste m Pro ces sor Co r e Al g o rithm Driv ers Sens o r s & Commu nic a t ion Circuit s (d) F P GA -based HIL S i m ulation High -per f o r m an c e Compute r Ve h i c l e Simu l a tion Sys tem (bod y,e nvir onme nt,act uat ors) Co r e Algo ri t hms (b) S o f tware-in- the-lo op S imulati o n Ve h i c l e Simu l a t i o n Sys tem Inter face Pro gra m Co n t r o l Syste m Real- tim e Com puter Proces s o r Sens o r s&Co mmu nic a t ion Circuit s (c) H ard wa r e-in-the-lo op (HIL) Simulati o n Ve h i c l e Simu l a t i o n Subs y s t e m CP U ı 1kHz Sens o r s Simu l a tion Subs y s tem FPGA ı 100MH z Dri ver s Co r e Al g o rithm s Dri ves ķ ĸ Ĺ ķ ķ ĺ For ce Mo ment ĸ Ĺ ĺ Ac tuator Sta tes ĸ ķ ĸ Inter f a c e Pro gra m ķ ĸ ķ ĸ ķ A ctu ator C o ntrol Ou t p u t s Ĺ C ont rol Ele c t r o n i c Sig nals ĸ V ehi cle Moti o n S t a t es ĺ S e n sor El ectron ic Sig nals CP U ı 1k Hz Real -tim e Co mputer ķ ĸ ķ ĸ Fig. 2. Comparisons between common simulation methods. In the past, limited by the real-time response performance of simulation computers, it is hard for Central Processing Units (CPUs) [20] on simulation computers to simulate sensors or other electronic chips with high-speed interfaces or high- frequency analog circuits. F or example, a nanosecond-le vel real-time update frequency (100MHz) is required to reliably simulate a sensor chip with the high-speed Serial Peripheral Interface (SPI), which is a difficult task for traditional CPU- based Commercial-Off-The-Shelf (CO TS) simulation com- puters whose reliable update frequencies are usually within microsecond level. In recent years, with the utilization of Field Programmable Gate Array (FPGA) [20], CO TS FPGA- based simulation computers (e.g., OP AL-R T r /OP series and NI r /PXI series) start to hav e the simulation performance of nanosecond-lev el real-time update frequency [7], [21]. This makes it possible to simulate almost everything (including vehicle motion, sensor chips, electric circuits, and high-speed interfaces) outside the main processor of the control systems as presented in Fig. 2(d). Besides, the tested control systems can be treated as black boxes in HIL simulation systems with no need for accessing the code or adding interface programs. Therefore, by simply replacing the sensor models, 3 the HIL simulation system can also be applied to perform comprehensiv e tests for different brands of control systems. For all simulation methods, the primary challenge is to ensure simulation credibility [22], namely making people believ e the simulation results are as real as experiments in the real world. The simulation credibility (compared with real systems) mainly determined by two aspects: platform credibility and model credibility . The platform credibility can be further divided into hardware credibility and software credibility . As previously mentioned, the hardware credibility can be guaranteed by the FPGA-based HIL simulation method, while the software credibility and the model credibility are still challenges for simulation methods. The Model-Based Design (MBD) [23] method is an ef fective means to solve the abo ve credibility problems by using modular visual programming technology and automatic code generation technology to stan- dardize the modeling, developing and testing procedures of complex control systems. The MBD method can ensure the software credibility by eliminating the disturbance factors such as manual programming negligence and nonstandard de velop- ment process. For example, the MathW orks r /MA TLAB (the most widely used MBD software) can ensure the generated code meet the requirements of standards and guidelines such as DO-178C. In MBD methods, the whole simulation systems can be divided into many small subsystems (modules), such as kinematic modules, GPS modules, ground modules, and propeller modules. Certification authorities can verify and validate these modules to build a standard product model database for companies to develop the vehicle prototype and the corresponding vehicle simulation system. Then, the model credibility can be guaranteed by using well-v alidated standard component models. What is more, with MBD method, the same component model can be applied to different types of vehicles and control systems, which may significantly improve the design, v erification, validation, and certification process of the unmanned vehicle filed. For ensuring model credibility , in our pre vious research [24], a credibility assessment method is proposed based on experiments to assess the model credibility of HIL simulation platforms from multiple aspects, such as key performance indices, time-domain characteristics, and the frequency-domain characteristics. By combining the above methods, the credibility of the simulation platform can be guaranteed from the model, de velopment process, and platform hardware aspects. In this paper , a unified simulation and test platform is proposed for control systems of unmanned vehicles based on FPGA-based HIL simulation and MBD methods, aiming to significantly improve the test efficienc y and safety lev el of control systems on unmanned vehicles. The main research contents and the corresponding contributions are listed as follows. (i) Unified Modeling Method. There are so many in common among dif ferent types of unmanned vehicles. The y should not be treated separately as more and more composite vehicles (e.g., multicopter + fixed-wing and car + fixed-wing) emerged. Therefore, a unified modeling method is proposed for all types of unmanned vehicles along with parameter measurement and identification methods to validate the obtained model and ensure simulation credibility . The corresponding content is presented in Section II . (ii) Real-time HIL T est Platform with MBD. A real-time HIL test platform is built for the testing and assessment of control systems. The platform is capable of simulating any real- world situation outside the control software with advantages in obtaining the true states and controlling the testing v ariables. The utilization of MBD methods can ensure that the testing results are credible and standard-compliant. Besides, the HIL simulation method and MBD method ensure the platform can be easily applied to different brands of control systems, which is conv enient for both companies and manufacturers. The corresponding content is presented in Section III . The significant advantages of the proposed unified HIL sim- ulation test platform are reflected in four aspects: extensibility , comprehensiv eness, verification, standardization. (i) Extensibility . By changing the parameters (e.g., weight, size, and aerodynamic coefficients) of specific subsystem modules, it is easy to extend a simulation system to other vehicle systems with similar structures. Moreover , by replacing a whole subsystem module (e.g., a propeller module to a tire module), the vehicle simulation system can be extended to other types of vehicles. (ii) Comprehensiveness . The current simulation systems mainly focus on functional testing, i.e., whether the vehicles can work properly in normal situations. Howe ver , unmanned vehicles are safety-critical systems, and most of the effects are focused on safety testing, i.e., whether the v ehicles can work safely when accident and faults happen. With the modular programming method, the fault modes, the aging process, and the probabilistic reliability property can be modeled for each subsystem module to improve the comprehensiveness of the simulation platform. Mathematically , the fault injection simu- lations (or other safety simulations) can be realized by online changing the module parameter or functional expressions of a subsystem module while the simulation program is running. (iii) V erification . In practice, it is difficult to verify and validate the simulation accuracy and credibility of a complex simulation system. Howe ver , it is relativ ely simple to verify a small subsystem. Therefore, the modular programming method can divide a complex simulation system into many small subsystems, and verify it from lower levels to the top level. More importantly , if all subsystems used in a simulation system are well-verified modules from certification authorities, the verification efficiency can be significantly improved. (iv) Standardization . A standard certification framework is urgently needed for unmanned vehicles to improv e the testing and certification ef ficiency . The modular programming method is a feasible way to solve this problem with the certification framework presented in Fig. 3. In this frame work, the manufactures should pro vide the product hardware along with a simulation model which should be fully verified and certificated by authority agencies based on the simulation data and e xperimental data. That coincides with the idea of Digital T win [25] for the efficient design and testing of complex systems. Then, the vehicle companies can use the certified models for simulation system dev elopment and prototype design. Finally , the simulation results and experimental results 4 Prod u c t Ha rdwar e Simu lati on Mod e l Manuf a c t u r e r s Ve hicle C ertifi c a t io n Mod e l C ertificati o n Au thorit y A genc ie s Expe rimenta l Testi n g De sign and HI L Simu l a t i on Ve hicle C om panie s Ve rified Mod e ls Simu lati on Da ta Expe r i mental Da ta Simu lation Resu l ts Expe r i mental Resu l ts Prod u c ts Certi f ic ation Repo r t Ve hicle Proto t y pe Fig. 3. Certification framework for unmanned vehicles. can be applied for the certification of the unmanned vehicle. In Section IV , the successful application of the testing platform on multicopter vehicles with the PX4 autopilot sys- tem indicates the ef fectiveness of the proposed modeling and simulation methods. T o verify the practicability of the pro- posed platform, several successful applications are presented for the multicopter rapid prototyping, estimation algorithm verification, autonomous flight testing, and automatic safety testing with automatic fault injection and result e valuation of unmanned vehicles. In the end, Section V presents the conclusions. I I . U N I FI E D M O D E L I N G M E T H O D As sho wn in Fig. 1, different types of vehicles (e.g., cars, aircraft, and boats) have many common features in modeling and simulation. T o make the maximum utilization of these common features, a unified simulation testing system is de- veloped with the system structure presented in Fig. 4 for all unmanned vehicle systems. In this section, the unified modeling methods for the simulation system in Fig. 4 will be introduced in detail. The hardware structure and development process for the simulation system in Fig. 4 will be introduced in Section III. A. Overall V ehicle Model 1) Model Abstraction: In practice, a complex dynamic system usually consists of many small subsystem systems (e.g., body system and propulsion systems). Every subsystem includes input signals u ∗ , output signals y ∗ , parameters Φ ∗ , dynamic states x ∗ , and and dynamic state and output functions f ∗ ( · ) and h ∗ ( · ) , which can be mathematically described by the following dynamic equations  ˙ x ∗ = f ∗ ( x ∗ , Φ ∗ , u ∗ ) y ∗ = h ∗ ( x ∗ , Φ ∗ , u ∗ ) . (1) For simplicity , a dynamic subsystem in Eq. (1) is further simplified to the following input/output form as y ∗ = S ∗ ( u ∗ ) . (2) The system description form in Eq. 2 will be applied in the following content to describe the connection relationships among different subsystems. The subscript symbol “ ∗ ” in Eqs. (1)(2) can be replaced by different abbreviation words to represent different dynamic systems, such as the control software system S ctrl and the sensor simulation subsystem S sens . 2) Main F rame work of Simulation System: As presented in Fig. 4, the whole simulation system can be di vided into three main subsystems: the v ehicle simulation subsystem S vehi (generating vehicle states according to the control signals), the 3D environment simulation subsystem S 3d (generating vision data according to the vehicle states), and the sensor simulation subsystem S sens (generating sensor signals according to the vehicle states and vision data). Besides, there is a control system S ctrl (generating control signals according to the sensor data) to be tested. The above connection relationships among the above four systems are mathematically described by y ctrl = S ctrl ( u ctrl ) , u ctrl = y sens y vehi = S vehi ( u vehi ) , u vehi = { y ctrl , y 3d } y 3d = S 3d ( u 3d ) , u 3d = y vehi y sens = S sens ( u sens ) , u sens = { y vehi , y 3d } (3) which is consistent with the connection relationships in Fig. 4. In the following, the unified modeling methods for the above three main subsystems will be introduced sequentially . B. V ehicle Simulation Subsystem The vehicle simulation subsystem S vehi in Fig. 4 can be further divided into four main subsystems: the actuator subsys- tem S act , the environment subsystem S en v , the force&moment subsystem S fm , and the vehicle body subsystem S body . As shown in Fig. 5, the connection relationships of the four subsystems are mathematically described as y body = S body ( u ctrl ) , u ctrl = y fm y fm = S fm ( u fm ) , u fm = { y body , y act , y en v } y en v = S en v ( u en v ) , u en v = { y body , y 3d } y act = S act ( u act ) , u act = { y ctrl , y body , y en v } (4) where y body contains vehicle motion states (e.g., position, velocity , and attitude), y fm denotes all the forces and moments acting on the vehicle, y en v includes environment parameters (e.g., gravity , air density , terrain, and obstacle distribution) , y act denotes actuator states (e.g., rotating speed of rotors, and deflection angle of control surface). By combining the output signals in Eq. (4), the output set y vehi for the vehicle simulation subsystem S vehi is giv en by y vehi , { y body , y fm , y en v , y act } . The ke y modeling methods for the four subsystems in Eq. (4) will be introduced as follows. 1) V ehicle body subsystem: As shown in Fig. 5, the vehicle body subsystem S body computes the vehicle states y body ac- cording to the force and moment y fm acting on the vehicle. In practice, based on the flat-earth assumption (ignoring the curvature of the earth in a small range) and the rigid-body assumption (the body is rigid and not flexible), most v ehicle body subsystem S body can be described by Six-Degree-of- Freedom (6-DOF) equations [26, pp. 25-54],[27, pp. 99-143]        e ˙ p = R e b · b v b ˙ v = − b ω × b v + (1 /m ) · b F ˙ R e b = R e b ·  b ω  × J · b ˙ ω = − b ω ×  J · b ω  + b M (5) 5 Co ntrol S oftware Syste m Force & Mo ment Subs y s tem Ac tuator Control Force &Mo nent Ve hicle Ae rodyn amics Grou n d Suppor t & P h y sic a l Co llision Grav i ty Ve hicle B ody Subsy stem Ve locity Posit i o n Quat e r n i o n An gular Ve locity Dy namics Envi r o n m en t Subsyste m Grav i ty Air De n s i ty Wi nd Magn etic Terra i n & Ob stacle ... Prop u l s io n Unit Subs y s tem Batte ry ES C Moto r Prop e ll e r . . . Ac tuator Sub system ... ... Ve hicle S i m ula tion Subsyst e m (CP U) Ve hicle Sta tes Co ntrol Sig nals Sens o r Da ta Subs ystem GPS Baro me t er Gy rosco p e Ac celerom e t e r Visio n ... 3D E n v i r o n ment Sub s y s t e m Sens o r Pr o d uct Subs ystem Ub lox N8N MS5 6 1 1 LiDAR MPU 6050 Cam e r a ... Co mmunic ation Subs y s t e m ... Rada r Ob stacle s Back g r ound L ight View P o i n ts ... Sens o r Si mulatio n Subs ystem (F PGA) Othe r E xternal Ha rdware S yst e ms Testi n g Programs Mod e l P aramet ers Fault I n jection Safet y A ssessme n t 3D D i s p l a y Real- Time Sim ulatio n Com puter ... A c t u a t o r S u b s y s t e m #1 #2 ... Servo Systems Engi n e &Tir e s #3 #1 #2 ... #3 #1 #2 ... #3 Prop u l si o n Uni ts Host Co mputer Sens o r S i gnals SPI I2C CAN PWM AD/D A UAR T 3D D a t a He alth State s Ve hicle Sta tes Exter nal Sig nals Mi ssion Co mmand s 3D Da ta Co ntrol S y s t e m Fig. 4. System structure of the simulation test platform for unmanned vehicles. Stea dy S tate Fun ctio n 1 s Dyn ami cs ss,1 v 1 v Con trol Signal Vec tor n s ss, n v n v … ... i s i v ctrl y Actu ator Sub system act y Actu ator Stat e Vec tor Stea dy S tate Fun ctio n Dyn amics Actu ator Con trol Force Aero dyn am ic Force Forc e of Grav ity Con tact Force å grou F aero F act F grav F fm y Forc e &Mom ent Subsystem Total Forc e and Moment Veh icle Body Subsystem Env iron men t Sub sys tem body y Veh icle States Env ironment Para met ers And Sta tes env y body y fm y env y body y Fig. 5. The structure of the vehicle simulation subsystem. where the right-superscript symbols “e” and “b” denote the North-East-Down (NED) earth frame and the Head-Right- Down body frame [26, pp. 25-54] respectiv ely; e p ∈ R 3 is the position vector defined in the earth frame; b v ∈ R 3 and b ω ∈ R 3 are the v elocity vector and the angular velocity v ector defined in the body frame; b F ∈ R  and b M ∈ R  are the force vector and the moment vector defined in the body frame; R e b ∈ R 3 × 3 is the rotation matrix to transform a vector from body frame to earth frame; J ∈ R 3 × 3 and m are the moment of inertia matrix and the mass of the vehicle. The 6-DOF dynamic equations in Eq. (5) can be applied to describe the vehicle body module y body = S body ( y fm ) in Eq. (1), where the state set is x body ,  e p , b v , R e b , b w  , the input set is u body = y fm ,  b F , b M  , the parameter set is Φ body , { J , m } , and the output set is y body ,  x body , e v , b ˙ v , b ω , R e b , · · ·  . 2) En vir onment subsystem: As shown in Fig. 5, the en viron- ment subsystem S en v generates en vironment parameters y en v (e.g., air density , temperature, terrain, wind, and magnetic field) based on the position of the vehicle e p ∈ y body . In practice, the W orld Geodetic System (WGS84) model [28] is widely used to describe the shape of the earth, which can con vert the position vector e p to the earth Latitude-Longitude- Altitude (LLA) global position e p g , [ µ ι h ] T , where µ, ι (unit: degree) are the latitude and longitude, and h (unit: m) is the altitude. Then, the acceleration of gravity g can be estimated by the WGS model [28] based on the vehicle global position e p g . Similarly , the air density and temperature are estimated by the International Standard Atmosphere (ISA) model [29], and the magnetic field vector is estimated by the W orld Magnetic Model (WMM) [30]. Besides, according to the Military Specification MIL-F-8785C [31], the wind velocity disturbance vector e v wind ∈ R 3 (defined in the earth frame) can be described by the following superposition form e v wind = e v turb + e v cons + e v sheer + e v gust (6) where e v turb denotes the atmospheric turbulence field, e v cons denotes the prev ailing wind field, e v sheer denotes the wind shear field, and e v gust denotes the wind gust field. There many widely used mathematical models for the wind components in Eq. (6). For example, the wind turb ulence e v turb can be described by the Dryden Wind T urbulence Model [31]. 3) Actuator subsystem: As shown in Fig. 5, the actuator subsystem S act outputs actuator state y act according to the control input y ctrl from the control system. In practice, it is dif ficult to obtain the mathematical model of an actuator system because it is usually composed by complex mechanical components along with programmable control units, such as the Electronic Speed Controller (ESC) for U A V brushless motors, and the Electronic Control Unit (ECU) for car en- gines and steering systems. These control units have feedback control to ensure that the actuator steady output δ ss ,i satisfy the preprogrammed function of the input control signal σ i ∈ y ctrl under dif ferent operating environment. According to [27], a complex actuator system can be linearized to a steady-state process f ss ,i ( · ) and a dynamic response process G ss ,i ( s ) around the rated operation condition. For example, a motor- propeller system with an ESC can be simplified as a first-order or second-order inertial process G ss ,i ( s ) and a steady-state function f ss ,i ( σ i ) as δ i = G ss ,i ( s ) · f ss ,i ( σ i ) . (7) Notew orthy , f ss ,i ( · ) can be measured by static testing, and G ss ,i ( s ) can be measured by system identification methods through frequency-response testing [32]. By using Eq. (7), it is easy to obtain the actuator output signal δ i ( t ) (propeller rotating speed) under the giv en control signal σ i ( t ) (throttle control signal). Then, the control force and torque generated by the state of an actuator δ i can be obtained by the ground friction model, aerodynamic model, or other mechanical mod- els [27], [33], [34]. 6 4) F or ce & moment subsystem: As sho wn in Fig. 5, the force&moment subsystem S fm outputs the force and moment y fm ,  b F , b M  to the vehicle body subsystem S body . The total force b F and moment b M acting on a vehicle can be divided into many components from different sources. T aking the force vector b F ∈ y fm (defined in the body frame) as an example, it can be described by the following superposition form b F = b F aero + b F grav + b F cont + X b F act ,i (8) where b F aero ∈ R  denotes the aerodynamic force vector , b F grav ∈ R  denotes the force of gravity vector , b F cont ∈ R  denotes the contact force vector from ground supporting or physical collision, and b F act ,i ∈ R  denotes the control force vector generated by an actuator . Notew orthy , the above force vectors should be all transformed to the body center and projected to the body frame. The aerodynamic force vector b F aero is a nonlinear function determined by the relati ve speed of the surrounding air e v rel as e v rel , e v wind − e v where e v wind is the wind speed from the en vironment sub- system in Eq. (6) and e v is the vehicle speed from the body subsystem in Eq. (5). The high-precision aerodynamic modeling method has been well studied in [26], which is compatible with all types of vehicles such as multicopters [27], helicopters [33] and cars [34]. The contact force b F cont caused by the ground supporting or physical collision can be modeled by simplifying the vehicle body shape to a cuboid or a cylinder and simplifying the contact surface to a spring-loaded system. By adjusting the spring stiffness, it is con venient to simulate physical contact on objects with different surface hardness. The terrain and obstacle information comes from the en vironment subsystem output y en v , which further comes from the 3D en vironment subsystem S 3d . The actuator force vector b F act ,i can be unified described by the following nonlinear expression b F act ,i = f act ,i ( Φ fm , y en v , y body , δ i ) (9) where δ i ∈ y act is the instantaneous state of an actuator (the rotating speed of a propeller , deflection angle of a servo system, or dri ving torque of a tire) from the actuator module S act . Notew orthy , the expression of f act ,i ( · ) is also related to the vehicle state y body and the en vironment state y en v , and the methods to obtain the force model f act ,i ( · ) have been well studied in [27], [33], [34]. As shown in Fig. 6, the different types of vehicle simulation models are mainly distinguished by the actuator types and configurations. In a similar way , the actuator force models for different types of vehicles in Fig. 6 can be easily obtained with proper actuator system modeling methods. C. 3D Envir onment Simulation Subsystem The 3D en vironment subsystem S 3d in Eq. (3) aims to generate vision data y 3d based on the vehicle states y vehi from Prop e ll er Ele vator #2 Elevato r #1 Ru d der Ail e r o n #1 Ail eron #2 Pro peller # 3 Prop e ll er #1 Pro peller # 2 Pro peller # 4 Tir e # 3 Tir e # 1 Tir e # 4 Tir e # 2 Pro peller # 1 Pro peller # 2 Fig. 6. Actuator force models for different types of unmanned vehicles. the vehicle simulation subsystem. The vision data will be sent to the sensor subsystem to generate data for vision sensors such as camera, radars, laser range finder, etc. Currently , many widely used 3D en vironment engines can be applied for vehicle vision modeling. For example, the Simulink 3D Animation T oolbox provides interfaces to con- veniently access the video stream for image process and controller design in Simulink; the Airsim [11] is developed by Microsoft r to generate high-fidelity visual and physical simulation en vironment using Epic Games r /Unreal Engine 4 (UE4). Both Simulink 3D Animation T oolbox and Airsim can be applied to different types of vehicles, including aircraft and cars. There are also man y 3D simulation en vironments exclusi vely dev eloped for specific types of vehicles. For ex- ample, the Gazebo HIL simulator [16] for visual simulation of autonomous cars, and the FlightGear [35] for aircraft simulations. These 3D simulation engines can be applied to dev elop the 3D en vironment subsystem to generate vision data (cameras), obstacle distance information (rangefinders, radars), or point cloud data (Lidar). W ith the dev elopment of GPU performance and 3D modeling technology , the obtained vision data y 3d will be more and more high-fidelity and realistic in the future, which will significantly improve the credibility of visual simulations. D. Sensor Simulation Subsystem The sensor subsystem S sens in Eq. (3) describes the process to transform the vehicle state y vehi and vision data y 3d to the electric signals y sens for the control system. Concretely , it can also be divided into three small subsystems, including the sensor data subsystem S data , the sensor product subsystem S prod and the communication subsystem y sens . As shown in Fig. 4, their connection relationships are mathematically described as y data = S data ( u data ) , u data = u sens = { y vehi , y 3d } y prod = S prod ( u prod ) , u prod = y data y sens = y com = S com ( u com ) , u com = y prod (10) where y data contains ideal data for sensors (e.g., the accelera- tion of accelerometers, the magnetic field of magnetometers, and the image or point cloud data of vision sensors), and y prod 7 is the sensor signals after adding detailed product features (e.g., noise lev el, temperature drift, failure mode, and camera distortion), and y sens is the binary electrical signals transmitted to the control system for position and attitude estimation. 1) Sensor Data Subsystem: As described in Eq. (10), the sensor data subsystem S data generates the sensor data y data (applicable for a class of sensor products) based on the vehicle state y vehi and vision data y 3d . T ransformations usually required to obtained the sensor data from the vehicle states and vision data. For example, accelerometers measure the specific force (the dif ference between the acceleration of the aircraft and the gravitational acceleration) [36, p. 122] instead of the vehicle acceleration b ˙ v ∈ y vehi . Similar computation expressions are applied to other types of sensors, such as the GPS Modules (longitude and latitude obtained from the vehicle position), electronic compasses (the magnetic field intensity obtained from the attitude and global position of the vehicle), optical flow sensors (relati ve velocity obtained from image stream), etc. 2) Sensor Pr oduct Subsystem: The sensor product subsys- tem S prod is dev eloped to add product features (e.g., noise, vibration, and calibration) to the sensor data y data obtained from the above sensor data subsystem S data . Giv en the same sensor data, dif ferent sensor products may obtain different results due to product features, so the senor product subsystem is necessary . In most cases, giv en the ideal sensor data x m ∈ y data , the noise feature is mainly reflected in the noise n a and bias drift b a of the measured value x 0 m , which can be described as [27, p. 151] ( x 0 m = x m + b a + n a ˙ b a = n b (11) where n a ∼ N  0 , σ 2 a  and n b ∼ N  0 , σ 2 b  are zero-mean Gaussian noise vectors for inertial sensors. The standard devi- ation parameters σ a , σ b can be found in the datasheet document of a sensor product or obtained by system identification with the actual sensor output signals. If the system is not affected by vibrations, σ a can be modeled as a constant v alue. When the vibration feature of a sensor is considered, the measuring noise n a may also be affected by the vibration from may sources (e.g., engines, motors, and fuselage).Therefore, the standard deviation σ a is not always constant v alue, which should be modeled based on the actual system characteristics. The calibration feature is mainly determined by the working en vironment of the installation configuration a sensor, which can be described as [27, p. 149] x 00 m = T e K e ( x 0 m + p e ) (12) where p e is a constant vector for the position installation deviation, T e is a rotation matrix for the installation deviation, K e is a diagonal matrix for the scale deviation, and x 00 m ∈ y prod is the final output data of a sensor . Eqs. (11)(12) are applicable to most types of sensors to simulate the properties of real sensor products. There are also man y methods to add product features for vision sensors. For example, the methods to add camera SCK CS Clo ck s i g n a l Tran s m issi on enab le si g n al ... ... Qu e r y comma nd rec e i v e d Reco g n i t ion, comp utation, dat a pre parati o n Co mmunica t io n mo d e l s i mu lates the SPI pro tocol (8-b i t p er exc h a nge) Sens o r dat a retur n e d MOS I ... Q 1 2~7 Q 1 1 Q 1 8 Q 2 1 Q 2 8 Q 3 1 Q 2 2~7 MIS O ... R 1 1 R 1 8 R 2 1 R 2 8 R 3 1 R 2 2~7 R 1 2~7 Co n t r o l s ystem query co mmand s Sens o r retur n e d dat a Fig. 7. Communication subsystem model for SPI buses. features (e.g., blurs, distortions, and noises) to an ideal image is introduced in [37]. Other environment factors (e.g., lighting, reflection, and fogging) can be simulated by the latest 3D engines, such as UE4. 3) Communication Subsystem: The communication subsys- tem is developed to transform the sensor data with product features y prod to binary electronic signals y sens for the control system. The outputs y prod of the above sensor product sub- system S prod are decimal numerical signals, but binary elec- tronic signals are required for the communication requirements between the control system and other hardware. There are many communication interfaces and protocols widely used in the vehicle control systems, such as SPI, Inter-Inte grated Circuit (I 2 C) [38], Controller Area Network (CAN), Univ ersal Asynchronous Receiv er-T ransmitter (UAR T), and Pulse-W idth Modulation (PWM). The mathematical model for a communication interface is usually simple, but it is hard to be simulated by a CPU- based simulation computer due to the extremely high real-time update frequency and bandwidth requirements. T aking the SPI interface as an example, the SPI interface uses four signal wires to exchange information between the master de vice (the main processor of the control system) and the slav e device (on- board sensors). As shown in Fig. 7, the sensor has to finish the command recognition, measured data computation, and output data preparation within a small interval (after the pre vious byte data recei ved and before the following byte data to be sent). In a real sensor chip, the above process is instantaneously realized by analog circuits whose time consumption can be treated as infinitely small. Ho wever , for a real-time simulation system, it usually requires a real-time update frequency at the nanosecond lev el to ensure that the sensor signals are correctly computed, prepared, and transmitted. For the performance requirements, this paper uses FPGA system to simulate all sensor communication features (e.g., data transmission, chip recognition, programmable setting functions), which ensures all the sensor hardware related lo w-level test cases can be simulated by the simulation platform. E. Model Identification and V alidation The mathematical model of a subsystem can be classified into three types: the static models, the dynamic models, and the statistical models. (i) The static models are described by 8 constant parameters that can usually be precisely measured by professional equipment. These parameters include the mass, moment of inertia, air density , aerodynamic coef ficients, etc. (ii) The dynamic models are described by dynamic equations that are usually difficult to be measured by equipment directly . System identification methods [8], [32] are widely used in obtaining dynamic parameters in aircraft and vehicle systems. (iii) The statistical models are described by statistical values or stochastic processes (e.g., the sensor noise, and the atmo- spheric turbulence). The statistical models can be obtained by spectral analysis or statistical analysis of the output signals with enough sample data. F or example, the expectation and variance of the White Gaussian noise of a sensor product in Eq. (11)(12) can be obtained by its output signals [39]. In practice, by using advanced modeling methods with enough experimental data, an y vehicle system can be modeled arbitrarily close to the real system. Ho wev er , a high-fidelity mathematical model usually indicates more computational complexity and cost (higher dimension and nonlinearity), so abstraction and simplification methods are important for simu- lation systems. The simulation credibility is the most important index in the trade-of f between precision and complexity . In our previous research [24], by feeding the same signals to both the simulation system and the real system and analyz- ing the simulation errors, a simulation credibility assessment method is proposed to assess the simulation credibility of a simulation system from multiple aspects, such as system performance, time-domain response, and frequency-domain response. A normalized assessment inde x is proposed to score the simulation credibility of a subsystem under a uniform assessment criterion. The proposed method is ef ficient to verify and v alidate the simulation effect of each subsystem and the whole simulation system, which is used in this paper to ensure the credibility of the obtained vehicle subsystem models. I I I . Real-time HIL T est Platform with MBD This section presents the hardware structure and develop- ment frame work of the proposed HIL test platform in Fig. 4 successiv ely . A. Hardwar e Structur e of the HIL Platform The platform hardware consists of three parts. 1) Real-time Simulation Computer: A real-time simulation computer (also called real-time simulator) is a special type of computer , running Real-Time Operation System (R TOS) to ensure the simulation systems run at the same speed as the actual physical system. The latest CO TS simulation computers usually provide a CPU-based system and an FPGA-based sys- tem for dif ferent requirements of real-time update frequenc y; the CPU-based system is better at running complex simulation models with moderate frequency requirements (usually smaller than 100KHz); the FPGA-based system is better at running simple simulation models with extremely high frequency (usu- ally larger than 100MHz). By combining the advantages of the abov e two systems, the hardware structure of the proposed test platform is presented in Fig. 4, where the CPU-based system is applied to run the vehicle simulation subsystem presented in Section II-B and the FPGA-based system is applied to run the sensor simulation subsystem presented in II-D. 2) Contr ol System: The control system is the test object of the proposed test platform. The control system computes control signals for dri ving the actuators according to the vehicle states measured and estimated by different sensors. T o ensure the control system can work normally in the proposed test platform, the original sensors should be blocked (or remov ed), and the sensor pins on the control system should be reconnected to the FPGA-based system of the real-time simulation computer . Then, the control system can receiv e the simulated sensor chip signals for full-function operation. The abov e process only needs to kno w the brands and models of sensors used in the control system, and has no requirement to access the source code or internal hardware structure. Therefore, it is practical to perform black-box testing for different control system products from different unmanned vehicle companies. 3) Host Computer: A high-fidelity 3D simulation environ- ment is also essential for training or testing the top-level algorithms, including computer vision, machine intelligence, and decision making systems. Therefore, a host computer with high-performance Graphics Processing Units (GPUs) and realistic 3D engines are used in this paper to generate vision signals to the real-time simulation computer . The latest high- end consumer GPUs (such as NVIDIA GTX2080) have been powerful enough to generate high-resolution (lar ger than 4K) video streaming with update frequency more than 100Hz, which is capable of simulating most vision sensors. Besides, the host computer also takes responsibility for running other auxiliary programs such as model parameter configuration program, 3D display program, ground control program, etc. Notew orthy , for dif ferent data bandwidth and real-time re- quirements, the connection and communication among the simulation computer and the host computer can be realized by optical fibers, network cables, serial ports, etc. B. Development F ramework with MBD 1) Modular Pr ogramming: In practice, developing a com- plex simulation software through hand-coding is a dif ficult and unreliable task due to too many mathematical operations. Any coding mistake, logical mistak e, or unknown vulnerability may lead to wrong or inaccurate simulation results. The types and amounts of unmanned v ehicles will be far more than manned vehicles, and the development cycles are required to be much shorter , so the traditional hand-coding methods are no longer suitable for developing simulation software for unmanned v ehicles. As a result, modular (also described as graphical or visual) programming methods ha ve been widely used in many MBD tools (e.g., MathW orks r /Simulink and NI r /LabVIEW). The whole simulation system presented in Section II has been di vided into many simple and independent subsystems, and each subsystem can be easily realized by a visual module or block in the above MBD tools, which makes it easy to develop and test a complicated vehicle simulation system. 9 Simu l i n k Environm ent Ve hicle S ubsyste m Co de Ge neration Sens o r Subs y s t e m LabV IEW En vironmen t Ve hicle S tates Co ntrol S i g n a l s Sens o r Subs y s t e m Ve hicle Subs y s t e m SPI/I 2 C (10MH z) Seria l (1 0 0 Hz) PWM (250H z) Proce ssor (Co n t r o l Softwa r e ) Co ntrol S y s t e m (FPGA ı 100mHz ) (CP U ı 1k Hz) Real- Time Sim ulatio n Com puter Ne twork Visio n Ve hicle S ubsys tem Host Co mputer Mod e l P arameter Inter f a c e s Fault Injection Prog r a m 3D E n v i r o n ment Mod e l ... V i s i o n UE4 E nvi ronmen t Ve hicle Co ntrol Syste m 3D Envi r o n me nt Subs y s t e m Co de Ge neration Co de Ge nerati on Sta tes Co ntrols Sens o r Da ta Co de Gen e r a t ion Fig. 8. Code generation and deployment framework for simulation software of HIL testing platforms. 2) Automatic Code Generation: Modular programming and automatic code generation are two of the most significant features of MBD tools such as Simulink, LabVIEW , and UE4. Simulink is better at developing comple x simulation systems (such as unmanned vehicle simulation systems), and LabVIEW is better at de veloping hardware-closed simulation systems (such as sensors, circuits and communication signals) and deploying the simulation program to the real-time simu- lator . For example, the Aerospace Blockset in Simulink [40] provides many demos for quickly de veloping vehicle simula- tion systems, such as aircraft and multicopter . There are many 3D engines for developing high-fidelity vehicle 3D simulation en vironments. Among these engines, UE4 provides modular visual programming functions, so the UE4 en vironment is selected to dev elop 3D simulation scenes for the HIL platform. In order to take full advantages of both MBD tools, the dev elopment process for simulation system software is sho wn in Fig. 8. The development process is divided into the follo w- ing steps: (i) dev eloping and verifying the vehicle simulation model in Simulink Environment; there are many powerful ver - ifying tools in Simulink such as requirements traceability , code cov erage check, document generation, etc., which guarantee the simulation software meets the standards and guidelines such as DO-178C; (ii) compiling the vehicle simulation sub- system into code, and importing it to the LabVIEW en viron- ment; (iii) building the sensor subsystem in LabVIEW and building the interfaces to communicate with other systems; (iv) generating code and ex ecutable files to deploy them to the real- time simulation computer; (v) dev eloping the 3D simulation en vironment in UE4 and deplo y it to the host computer . The whole development process in Fig. 8 is ef ficient and reliable because all coding and deploying operations are automatically finished by MBD tools without much human intervention. Therefore, it is con venient to replace some models and reb uild the simulation system for different types of vehicles or control systems. I V . V E R I FI C A T I O N A N D A P P L I C A T I O N In this section, a real-time HIL test platform is first de- veloped for multicopter U A Vs based on the proposed test platform. Then, the simulation accuracy is verified through a series of experiments. In the end, several successful applica- tions are presented to verify the feasibility and practicability of the proposed methods in this paper . Pixh awk Aut opilo t Groun d S tation Prog ram Unre al Engin e 4 3D Envir onm ent Model Faul t In jecti o n Safety Testin g P r o gram Sens o r Mod el ˄ FPG A | 10 0MHz ˅ Vehi cle Mod el ˄ CPU | 5kHz ˅ Sens ors Blocked CPU:N I ® PXIe- 8133 FPGA:N I ® PXIe-7846R High-p e r f o r m an ce Works tation PC Simuli n k + La b VI EW Pixhaw k ® Auto pilot Futaba ® Radi o Co ntrol Vehi cle Simulat ion States Test Ca s e s | Vehicl e Para mete rs | 3D V ision Sens o r Signa l s Control S i g nals Exte rnal Devi ce Signal s Cont rol Comm ands Controll e r States Fig. 9. Hardware composition of the real-time HIL test platform Four videos hav e been published to demonstrate the devel- opment, experimental verification, and application process in this section for the proposed platform. The first video gives an overall introduction of the proposed platform along with sev eral applications on different multicopter control systems: https://youtu.be/nXAoLdPzz I The second video presents se veral demos of applying the proposed HIL platform to different types of vehicles (cars, copters, aircraft) with multi-vehicle traffic en vironment simu- lation: https://youtu.be/xQUnkqH29qU The third video presents the automatic fault injection, safety testing, and safety assessment demos: https://youtu.be/MHieyE3hbHY The fourth video introduces the MBD modeling and de vel- opment process of the platform with experiments to quantita- tiv ely verify its simulation credibility: https://youtu.be/ChNtkb5rrQs W e hav e also published the MA TLAB source code and the detailed modeling tutorial to Github: https://github .com/XunhuaDai/CopterSim Readers can use it to rapidly develop SIL or HIL simulation systems for dif ferent types of unmanned vehicles by modifying the aerodynamic model and the actuator model. A. Platform Implementation 1) Hardwar e Composition: Based on the hardware struc- ture presented in Fig. 4, a real-time HIL test platform is de- veloped as shown in Fig. 9. The real-time simulation computer adopted in the platform is the NI r /PXI simulator with CPU board: PXIe-8133 (Intel Core I7 Processor , PharLap ETS Real- T ime System) and FPGA I/O Module: PXIe-7846R. The host computer is a high-performance workstation PC with profes- sional GPU. The control system is Pixhawk autopilot, which is one of the most popular open-source autopilot hardware systems for small unmanned vehicles. All the onboard sensors (IMU, magnetometer , barometer , etc.) and external sensors (GPS, rangefinder , camera, etc.) of the Pixhawk hardware are blocked, and the sensor pins are reconnected to the FPGA 10 I/Os to receiv e the simulated sensor signals through interfaces including SPI, PWM, I 2 C, UAR T , UBX, etc. In the real- time simulation computer , the update frequency of the v ehicle simulation model is up to 5 kHz, and the update frequency of sensor simulation model is up to 100 MHz, whose perfor- mance is fast enough for HIL simulations of most commercial control systems. The communication among between the host computer and the real-time simulation computer is realized by network cables with TCP and UDP protocols. 2) Software Development: The simulation software of the real-time HIL test platform is dev eloped based on the MBD method in Section III-B. The Simulink is selected to de velop the v ehicle simulation model because it is the most profes- sional and widely used software for vehicle dynamic system dev elopment; the LabVIEW is selected to de velop the sensor simulation model because it is efficient and conv enient in real- time simulation system development; the UE4 (version: 4.22) is selected to de velop the 3D en vironment model because it is one of the most realistic 3D engines in the dev elopment of simulation systems, games and VR systems. The Simulink, LabVIEW , and UE4 dev eloping en vironments all provide con venient modular visual programming en vironments (see Fig. 10) and automatic code generation technology for the dev elopment of simulation systems, which are perfect for the implementation of the model-based design method. After the simulation models are all de veloped, the code generation and deployment framew ork in Fig. 8 is applied to deploy the simulation software to the real-time HIL test platform presented in Fig. 9. 3) High-fidelity 3D Simulation En vir onment: The fidelity of the 3D simulation model is important for testing the vision- related functions of control systems, such as visual data pro- cessing, obstacle av oidance, safety decision-making, etc. W ith UE4, it is easy to develop high-fidelity 3D scenes for different types of vehicle in dif ferent environments. F or example, as shown in Fig. 11, we hav e de veloped se veral 3D simulation scenes in UE4 for the HIL test platform. According to the comparisons with experiments, the display effect presented in Fig. 11 has been realistic enough for most unmanned vehicle systems to simulate the real indoor or outdoor scenes. B. Experiments and V erification 1) Platform F eature V erification: The Pixhawk autopilot supports for running different types of flight control software systems (e.g., PX4, Ardupilot, and other embedded control software systems) in it, and also supports for controlling different types of unmanned vehicles (e.g., multicopters, small cars, and fixed-wing aircraft). Besides, the Pixhawk autopilot has a series of a v ailable hardware configurations for certain performance requirements, such as Pixhawk 1, Pixhawk 4, etc. T o verify the proposed modeling method and test platform, we first apply the proposed unified modeling method in Section II to develop the simulation models for different types of un- manned vehicles, including multicopters, small cars, and fixed- wing aircraft. Then, these simulation models are deployed to the HIL test platform with the MBD framework presented in Section III. Finally , a series of tests are performed for Co n t rol Inp u t Ter rain & Ob s t a c l e Input Ac t u a t or Subs y s t e m For c e & Mom e n t Sub s ystem Envi ronmen t Subs y s t e m Fault Inje ctio n Sub s ystem Ve h i c l e Bo d y Subs y s t e m Ou t p u t Inte r face Subs y s t e m (a) S imuli n k g r aphic a l progra mming for v ehicle mode l (b) L a b VI E W g raphi cal p r o gram ming f o r sen s o r model (c) U E4 gra p hic al pro grammi n g for 3D e n vir o n ment M ode l Ve h i c l e Sta t e s GP S Ch i p Sig n a l s GP S D a ta Sub system Fig. 10. Modular visual programming environments in Simulink, LabVIEW , and UE4. ( a ) P l a t e a u i c e l a ke sc e ne f or qua dc opt e r s ( b ) F or e s t s c e ne f o r he xa c opt e r s ( c ) N e i ghbo r hoo d st r e e t sc e n e f or c a r s ( d ) Mou nt a i n sc e n e f or f i xe d - w i n g a i r c r a f t Fig. 11. 3D simulation scenes dev eloped by UE4 for different types of unmanned vehicles. the Pixhawk systems with v arious combinations of hardware configurations, software systems, and vehicle types. In our experimental tests, the four adv antages of the proposed method (including extensibility , comprehensiveness, verification, and standardization) concluded in Section III-B are verified with the proposed modeling method and simulation test platform. 2) Experimental Setup: For simplicity , a quadcopter control system will be selected as the representative tested object in this section to perform quantitativ e verification for the proposed methods. The control system is the most widely used open-source autopilot system for small-scale unmanned 11 (a) F 450 q u a dcop t e r c o mpone n t s (b) In doo r t est e quipm ent f o r attitu d e res p o nse identi f ic ation (c) B i f il a r pe n d ulum me t h o d for mo me n t of i n e r t ia m ea surem ent (d) Me a s uring device s fo r p r opulsion sys tems o f sm all airc raft Fig. 12. V erification equipment for the proposed HIL test platform. vehicles, and the detailed configuration is Pixhawk 1 hardware (MCU: STM32F427, sensor: MPU6000, MS5611, LSM303D, L3GD20H, Ublox-M8N, etc.) with the PX4 control software. The quadcopter U A V is selected because it is the most repre- sentativ e unmanned vehicle type that covers the model char- acteristics (e.g., aerodynamics, ground collision, kinematics, and dynamics) and operating environments (e.g., near -ground, mid-air , indoor , outdoor , ho vering flight, and forward flight) of most unmanned vehicles. The experimental setup is presented in Fig. 12, where an F450 quadcopter airframe (diagonal length: 450mm, weight: 1.4kg, propulsion system: DJI E310, battery: LiPo 3S 4000mAh) is selected as the experimental subject whose component diagram is shown in Fig. 12(a). In order to test the attitude dynamics and aerodynamics of the quadcopter , an indoor test bench (see Fig. 12b) is developed with the quadcopter fixed to a stiff stick (through the center of mass) with smooth bearings to minimize friction. The quadcopter is free to smoothly rotating along one axis, which makes it possible to perform sweep-frequency testing for system identification and uniform rotation testing for roll damping coefficient measurement. Figs. (c)(d) present the test benches to measure the moment of inertia and the propulsion system parameters of small-scale unmanned vehicles, and the detailed measuring methods can be found in our pre vious w ork [27, pp. 121-143]. Besides, lots of outdoor flight tests are also performed to obtain the aerodynamic coef ficients of the tested quadcopter and v erify the platform simulation results with actual flight results. The experimental process and results hav e also been presented in the video mentioned in the last subsection. 3) Simulation validation: A series of comparativ e experi- ments are performed with the experimental test benches (see Fig. 12) and the HIL simulation test platform (see Fig. 9) Acc e l e r a t ion z-a xis (m/s 2 ) An gular s p e e d x-axis ( rad/s ) Time (s) Time (s) (a) A cce lerom eter n o i s e &vibration test (b) G yro sco pe noise& vibration test Co n t r o l Signa l Mo tor s t e a d y sp eed (rad/s ) Mo tor s pee d (rad/s) Pro pell er th r u s t ( N) (c) M otor &E SC in p u t/output mo d e l t est (d) P r o peller a e r o dyna mic model test Ma gnitu d e ( dB) (e) B ode magn i tu de plot fo r p i t ch cha nne l swee p f reque n c y t est Phas e (deg) Expe r i mental r e s u l ts Simu l a tion re sults Fre quency ( rad/s) Fre quenc y (rad/ s) (f) B ode pha s e p l o t f or p itch ch a n nel s w e ep f reque n c y t est Mo del: y = 1148 x - 1 4 1 .4 Mo del: y = 0.0 0 0 011 x 2 Fig. 13. Comparison experiments to verify the simulation accuracy of the HIL test platform. to verify proposed modeling method and test platform, and sev eral representative comparison results are presented in Fig. 13. In Figs. 13(a)(b), the accelerometer and gyroscope data obtained from the IMU sensor (MPU 6000 in Pix- hawk) are presented to verify the sensor modeling methods in Section II-D. In the simulation and the e xperiment, the motor throttle is stepped to 50% at 21.5s to test the vi- bration effect caused by the high-speed rotating motors and propellers. In Figs. 13(c)(d), the propeller actuator modeling method in Section II-B is verified with the propulsion system test benches in Fig. 12(d). Besides, sweep frequency tests are performed for the quadcopter pitch channel with the test bench presented in Fig. 12(c), and the magnitude and phase curves of the Bode plots obtained by the CIFER r software [32] are presented in Figs. 13(e)(f), respectiv ely . From the comparativ e results in Fig. 13, the following conclusions can be obtained. (i) The FPGA-based HIL test platform allows using the same control system in both experiments and sim- ulations, which minimizes the disturbing factors for result analysis and makes it con venient to acquire control system data for comparison and assessment. (ii) From the perspectiv e of qualitativ e analysis, the obtained HIL test results are close to the e xperimental results in Fig. 13 from both time-domain 12 Calculate T otal Weig  kg Frame Size  mm Altitude  m Aero Design medium Min. Battery Capacity 15% Max. T akeoff Throttle 85% FCU Max. Tilt Limit No Limit FCU & Attaches Current A Motor Brand: Model: Propeller Brand Model: ESC Brand Model Battery Brand Model Quadcopter-  mm-  kg-  min- '-, Mathematical Model ( Doc )  0.5  Air T emperature 25 °C CFP 9x4.7 E310 DJI 2312 KV960 DJI DJI ACE LiPo 3S-1 1.1V-25C-4000mAh : g = 9.8 m/s² : J xx = 1.704e-2 kg.m² J yy = 1.704e-2 kg.m² J zz = 3.176e-2 kg.m² : d = 0.225 m Propeller Thrust Coef. (T p /!²) : C T = 9.286e-6 N/(rad/s)² Propeller Moment Coef. (M p /!²) : C M = 1.189e-7 N.m/(rad/s)² Throttle " to Motor steady speed ! ss ( ! ss =C R *"+! b ) Motor-Propeller Inertia : C R = 691.33 rad/s : ! b = 162.59 rad/s : J m = 8.18e-5 kg.m² : T m = 0.0119 s Motor Response Time Constant Air-Drag D Coef. (D=Cd*V²) : C d = 6.697e-2 N/(m/s)² Air-Torque M Coef. (M=Cm*w²) : C m = 9.174e-3 N.m/(rad/s)² Multicopter Mass Acceleration of Gravity MultiCopter Inertia Matrix J=diag(J xx , J yy , J zz ) Distance of Motor to Center : m = 1.4 kg Fig. 14. Screenshot of the online toolbox flyev al.com. and frequency-domain aspects, which verify the simulation accuracy and credibility of the proposed modeling method and test platform. (iii) The quantitative simulation credibility assessment method proposed in our previous work [24] is applied in this paper to assess and improve the simulation credibility of the HIL platform, which ensures that a high matching degree (the credibility index in [24]) larger than 90% (where 60% presents the minimum accuracy requirement, and 100% presents a perfect match) is obtained by analyzing the results between the test platform and real experimental system from the quantitativ e perspectiv e. C. Method Applications This subsection presents se veral successful applications with the proposed test platform to increase the de velopment, testing, and validation efficienc y of multicopters. 1) Rapid Pr ototyping: T o take maximum adv antage of the proposed test platform with MBD method, a component model database is dev eloped for the rapid dev elopment of electric multicopters. The database cov ers the common prod- ucts on the market for multicopter propulsion systems with model parameters obtained by their product specifications and e xperimental data. With this model database, an online toolbox is released (URL: https://flyev al.com/) based on our previous studies [41], [42], [43] for the automatic design and performance estimation of multicopter UA Vs, and a screenshot of the online toolbox is presented in Fig. 14. Users can select component products from the database to quickly assemble a multicopter to estimate its flight performance and model parameters. The toolbox has been released for more than two years, and the user feedback indicates that it can significantly improv e the multicopter model de velopment efficienc y with decent simulation accuracy . (a) R eal F 450 Q uadcop ter Time (s) Pitch a n gle ste p r e s ponse (rad) (c) H igh-p r e c i sion Mo d e l (b) We bsite Es ti m ated M o d el Fig. 15. Lev el flight testing results for simulation validation. The level flight testing is an effecti ve way to assess the simulation accuracy of the whole simulation model because it is the joint effect of all component models in the HIL test platform. Figs. 15(a)(b)(c) present the representativ e level flight testing results from a real quadcopter , an estimated model from the toolbox in Fig. 14, and a high-precision model calibrated with experimental data in Fig. 13. The quadcopter is commanded to step from hovering mode to le vel flight mode (the desired pitch angle is 15 degree, i.e., 0.262 rad) at 0.2s. The curve data are collected from the log data in Pixhawk after flight tests. It can be observed from Fig. 15 that the high-precision model curve almost coincides with the real experimental curve, and the estimated model curve is slightly dif ferent from the experimental curv e, but the error is acceptable because it re veals most dynamic and aerodynamic characteristics of the quadcopter . 2) Algorithm Comparison: Another important advantage of the proposed HIL test platform is that it can obtain the true states of the simulated vehicle, which is significant for comparing performance dif ference of control algorithms. In Fig. 16, two simulations are performed on the HIL test platform with two different estimation filter algorithms in the Pixhawk autopilot. They are the Extended Kalman filter algorithm in Fig. 16(a) and the complementary filter algorithm in Fig. 16(b), respectiv ely . It can be observed from the result in Fig. 16 that the extended Kalman filter has a better estima- tion effect than complementary , which is consistent with the theoretical analysis. This conclusion is hard to obtain through experiments because a higher more precise external measuring devices (e.g., differential GPS or visual positioning systems with centimeter-le vel precision) is required to measure the true states of the vehicle. These external measuring devices are usually expensiv e and restrained. For example, the dif ferential GPS is easy to be disturbed by flight en vironment factors, and its data frequency is too low (usually 5Hz), and the visual positioning system cannot be used outdoors. Therefore, the proposed test platform is a better way to acquire the true states of the vehicle for comparativ e analysis and performance assessment. 3) Normal T esting: The proposed HIL test platform makes it possible to comprehensiv ely test the control system only with computers, which is significant in reducing cost and time relative to outdoor experiments. Fig. 17 presents an autonomous mission flight test with the proposed HIL test 13 Est imation value i n Pixh a wk True va lue from HIL s y stem Time (s) Time (s) Ve h i c l e posit i o n x-aix s (m ) Ve h i c l e positio n x-a i x s ( m) (a) F l ig ht e valua t io n f or P i x h a wk wit h E xtende d Ka l m an filt er (b) F li g ht e valua t io n f or P i x h a wk wit h c o m p l e me n t ary f ilte r Fig. 16. Comparing estimation performance of different filter algorithms in turning flight stage. ( c ) A u t o m a t i c M i s s i o n F l i g h t w i t h G r o u n d C o n t r o l P r o g r a m ( b ) F r o n t C a m e r a V i e w f r o m 3 D E n g i n e ( a ) S i m u l a t i o n C o m p u t e r & P i x h a w k Fig. 17. Autonomous mission flight testing with the proposed HIL test platform. platform, which usually should be performed by outdoor flight tests in traditional test methods. Besides, more comparativ e simulation tests and experiment demonstrate that all flight tests (indoor or outdoor , manual or automatic) can be tested on the HIL test platform if the vehicle modeling is comprehensiv e and accurate enough. 4) Automatic Safety T esting: W ith the fault modes (e.g., sensor failure, wind disturbances, and motor fault) being well modeled, the proposed HIL test platform can also be applied to perform automatic safety testing for control systems. W e hav e developed an automatic safety testing framework (see Fig. 18) based on the HIL test platform. First, a test case database should be de veloped to store the vehicle command script, the fault injection triggering time, the test stop time, and the desired vehicle flight performance. In each testing case, the control system and the simulation models are automatically reinitialized to default states, and then the vehicle control system automatically controls the vehicle model in the HIL test platform to the desired state. At this moment, a fault case is injected to the HIL test platform to simulate a vehicle failure. After a period of time, the vehicle and control system states are automatically analyzed and recorded to a report. Finally , the Test Ca s e Da t a b a s e Co n t r o l Syste m Sens o r D ata & C om ma n ds Real -tim e Simu lation Co mp u ter Co n t r o l Signa ls & S t a te Info Evalu ation Repo r t Evalu ation Prog r a m Tes t Case Test Resu lt s Ne x t Te s t Cas e Fig. 18. Automatic safety testing and validation framework. next testing case is tested in the same way until all the testing cases are tested and ev aluated. Some automatic test examples hav e been presented in the attached video in the previous subsection. The test results demonstrate that the platform can simulate the vehicle failure situations realistically , and the test efficienc y is significantly improv ed with the automatic safety test method. V . C O N C L U S I O N A N D F U T U R E W O R K This paper presents a unified simulation and test platform, aiming to significantly improve the dev elopment speed and safety le vel of unmanned vehicle control systems. A unified modular modeling frame work is proposed by abstracting the common features of different types of unmanned vehicles. The application examples demonstrate this framew ork is ef ficient in developing a vehicle simulation model with compatibility for the future safety assessment and certification standards. Another key problem solved in the paper is to develop a HIL simulation test platform to ensure simulation credibility . W ith the model-based design method, the developing process of the simulation software can be automatized and standardized, which ensures dif ferent de velopers can obtain the same simula- tion software with credibility guaranteed by the automatic code generation tools. W ith the FPGA-based real-time hardware- in-the-loop simulation technology , the operating en vironment of the control algorithms is guaranteed by using the same control system in both e xperiments and simulations. After the credibility of the simulation test platform is well guaranteed, we can focus on v erifying and v alidating the simulation models with quantitative assessment method proposed in our pre vious work. The proposed test platform is applied to a multicopter control system, where the accuracy and fidelity of the simula- tion testing results are verified by comparing with experiments. The successful applications present the advantages in the mul- ticopter rapid prototyping, estimation algorithm verification, autonomous flight testing, and automatic safety testing with automatic fault injection and result e valuation of unmanned vehicles. Since the test platform can provide high-fidelity and cred- ibility simulation results, it will help to improve the training efficienc y of artificial intelligence algorithms. Besides, the air - worthiness for unmanned aerial vehicles require more formal, quantitativ e, and efficient testing methods to assess the safety 14 lev el, and we will apply the proposed test platform for the safety assessment of unmanned vehicle systems. R E F E R E N C E S [1] H. N. Nguyen, S. Park, J. Park, and D. Lee, “ A novel robotic platform for aerial manipulation using quadrotors as rotating thrust generators, ” IEEE T ransactions on Robotics , vol. 34, no. 2, pp. 353–369, 2018. [2] O. Goury and C. Duriez, “Fast, generic, and reliable control and sim- ulation of soft robots using model order reduction, ” IEEE T ransactions on Robotics , vol. 34, no. 6, pp. 1565–1576, 2018. [3] H. Lipson and M. Kurman, Driverless: intelligent cars and the road ahead . Mit Press, 2016. [4] R. Mebarki, V . Lippiello, and B. Siciliano, “Nonlinear visual control of unmanned aerial vehicles in gps-denied en vironments, ” IEEE T ransac- tions on Robotics , vol. 31, no. 4, pp. 1004–1017, 2015. [5] K. Cole and A. M. Wickenheiser , “Reactive trajectory generation for multiple vehicles in unknown en vironments with wind disturbances, ” IEEE T ransactions on Robotics , vol. 34, no. 5, pp. 1333–1348, 2018. [6] C. M. Belcastro, D. H. Klyde, M. J. Logan, R. L. Newman, and J. V . Foster , “Experimental flight testing for assessing the safety of unmanned aircraft system safety-critical operations, ” in 17th AIAA A viation T echnolo gy , Integr ation, and Operations Confer ence . AIAA 2017-3274, 2017. [7] S. S. Noureen, V . Roy , and S. B. Bayne, “ An overall study of a real- time simulator and application of R T -LAB using MA TLAB simpower- systems, ” in 2017 IEEE Green Energy and Smart Systems Conference , 2018, pp. 1–5. [8] M. B. Tischler , “System identification methods for aircraft flight control dev elopment and validation, ” in Advances In Aircr aft Flight Contr ol , 2018, pp. 35–69. [9] O. Lisagor , T . Kelly , and R. Niu, “Model-based safety assessment: Revie w of the discipline and its challenges, ” ICRMS’2011 - Safety First, Reliability Primary: Proceedings of 2011 9th International Conference on Reliability , Maintainability and Safety , pp. 625–632, 2011. [10] D. Jung and P . Tsiotras, “Modeling and hardware-in-the-loop simulation for a small unmanned aerial vehicle, ” in AIAA Infotech@Aerospace 2007 Confer ence and Exhibit . AIAA 2018-2768, May . 2007. [11] S. Shah, D. Dey , C. Lovett, and A. Kapoor, “ Airsim: High-fidelity visual and physical simulation for autonomous vehicles, ” in Field and service r obotics , 2018, pp. 621–635. [12] D. B. W orth, B. G. W oolley , and D. D. Hodson, “SwarmSim: a frame- work for modeling swarming unmanned aerial vehicles using hardware- in-the-loop, ” The J ournal of Defense Modeling and Simulation , pp. 1–20, 2017. [13] A. I. Hentati, L. Krichen, M. Fourati, and L. C. Fourati, “Simulation tools, en vironments and framew orks for U A V systems performance analysis, ” 2018 14th International Wir eless Communications & Mobile Computing Confer ence (IWCMC) , pp. 1495–1500, 2018. [14] P . Sarhadi and S. Y ousefpour , “State of the art: hardware in the loop modeling and simulation with its applications in design, dev elopment and implementation of system and control software, ” International Journal of Dynamics and Control , vol. 3, no. 4, pp. 470–479, 2015. [15] J. R. Sylnice and G. H. Alf ´ erez, “Dynamic ev olution of simulated autonomous cars in the open world through tactics, ” in Pr oceedings of the Futur e T echnologies Conference , 2018, pp. 257–268. [16] Y . Chen, S. Chen, T . Zhang, S. Zhang, and N. Zheng, “ Autonomous vehicle testing and validation platform: Integrated simulation system with hardware in the loop, ” in 2018 IEEE Intelligent V ehicles Symposium (IV) , 2018, pp. 949–956. [17] A. Franchi, C. Secchi, H. I. Son, H. H. B ¨ ulthoff, and P . R. Giordano, “Bilateral teleoperation of groups of mobile robots with time-v arying topology, ” IEEE Tr ansactions on Robotics , vol. 28, no. 5, pp. 1019– 1033, 2012. [18] Y . Liu, J. M. Montenbruck, D. Zelazo, M. Odelga, S. Rajappa, H. H. Bulthoff, F . Allgower , and A. Zell, “ A distributed control approach to formation balancing and maneuvering of multiple multirotor U A Vs, ” IEEE T ransactions on Robotics , vol. 34, no. 4, pp. 870–882, 2018. [19] W . Li, C. W . Pan, R. Zhang, J. P . Ren, Y . X. Ma, J. Fang, F . L. Y an, Q. C. Geng, X. Y . Huang, H. J. Gong, W . W . Xu, G. P . W ang, D. Manocha, and R. G. Y ang, “AADS: Augmented autonomous dri ving simulation using data-driv en algorithms, ” Science Robotics , vol. 4, no. 28, 2019. [20] H. Saad, T . Ould-Bachir, J. Mahseredjian, C. Dufour, S. Dennetiere, and S. Nguefeu, “Real-time simulation of MMCs using CPU and FPGA, ” IEEE T ransactions on P ower Electr onics , vol. 30, no. 1, pp. 259–267, 2015. [21] S. Mikkili, A. K. Panda, and J. Prattipati, “Review of real-time simulator and the steps in volv ed for implementation of a model from MA T - LAB/SIMULINK to real-time, ” Journal of The Institution of Engineers (India): Series B , vol. 96, no. 2, pp. 179–196, 2015. [22] U. B. Mehta, D. R. Eklund, V . J. Romero, J. A. Pearce, and N. S. Keim, “Simulation credibility: Advances in verification, validation, and uncertainty quantification, ” N ASA Ames Research Center , Moffett Field, CA United States, T ech. Rep. J ANNAF/GL-2016-0001, Nov . 01, 2016. [23] I. MathW orks, “Model-based design - MA TLAB & Simulink, ” https://www .mathworks.com/solutions/model- based- design.html, Accessed May 24, 2019. [24] X. Dai, C. Ke, Q. Quan, and K.-Y . Cai, “Simulation credibility assessment methodology with FPGA-based hardware-in-the-loop platform, ” arXiv e-prints , vol. arXiv:1907.03981, pp. 1–8, 2019. [Online]. A vailable: https://arxiv .org/abs/1907.03981 [25] A. M. Miller, R. Alvarez, and N. Hartman, “T ow ards an extended model-based definition for the digital twin, ” Computer-Aided Design and Applications , vol. 15, no. 6, pp. 880–891, 2018. [26] B. L. Ste vens and F . L. Lewis, Air craft Contr ol and Simulation 2nd Edition . W iley , New Jersey , 2004. [27] Q. Quan, Intr oduction to Multicopter Design and Control . Springer, Singapore, 2017. [28] R. W . Smith, Department of Defense W orld Geodetic System 1984: its definition and relationships with local geodetic systems . Defense Mapping Agency , 1987. [29] M. Cavcar , “The international standard atmosphere (ISA), ” Anadolu University , T urkey , vol. 30, p. 9, 2000. [30] A. Chulliat, S. Macmillan, P . Alken, C. Beggan, M. Nair , B. Hamilton, A. W oods, V . Ridley , S. Maus, and A. Thomson, The US/UK world magnetic model for 2015-2020 . BGS and NO AA, 2015. [31] D. J. Moorhouse and R. J. W oodcock, “Background information and user guide for MIL-F-8785C, military specification-flying qualities of piloted airplanes, ” U.S. Air Force Wright Aeronautical Labs, Wright-Patterson AFB, OH, T ech. Rep. AFW AL-TR-81-3109, July . 1982. [32] R. K. Remple and M. B. Tischler , Aircr aft and r otor craft system identification: engineering methods with flight-test examples . American Institute of Aeronautics and Astronautics, 2006. [33] G. Cai, B. M. Chen, and T . H. Lee, Unmanned r otorcr aft systems . Springer Science & Business Media, 2011. [34] R. Rajamani, V ehicle dynamics and control, Second Edition . Springer Science & Business Media, 2011. [35] A. I. Hentati, L. Krichen, M. Fourati, and L. C. Fourati, “Simulation tools, en vironments and framew orks for U A V systems performance analysis, ” 2018 14th International Wir eless Communications and Mobile Computing Confer ence, IWCMC 2018 , pp. 1495–1500, 2018. [36] R. W . Beard and T . W . McLain, Small unmanned air craft: Theory and practice . Princeton university press, 2012. [37] M. Ku ˇ ci ˇ s and P . Zem ˇ c ´ ık, “Simulation of camera features, ” Proceedings of the 16th Central Eur opean Seminar on Computer Graphics , pp. 117– 123, 2012. [38] F . Leens, “ An introduction to I2C and SPI protocols, ” IEEE Instrumen- tation & Measur ement Magazine , vol. 12, no. 1, pp. 8–13, 2009. [39] N. El-Sheimy , H. Hou, and X. Niu, “ Analysis and modeling of inertial sensors using allan variance, ” IEEE T ransactions on instrumentation and measur ement , vol. 57, no. 1, pp. 140–149, 2008. [40] S. Gage, “N ASA HL-20 lifting body airframe modeled with Simulink and the aerospace blockset, ” https://www .mathworks.com/help/aeroblks/ nasa- hl- 20- lifting- body- airframe.html, Accessed May 24, 2019. [41] D. Shi, X. Dai, X. Zhang, and Q. Quan, “ A practical performance ev aluation method for electric multicopters, ” IEEE/ASME T ransactions on Mechatr onics , vol. 22, no. 3, pp. 1337–1348, 2017. [42] X. Dai, Q. Quan, J. Ren, and K.-Y . Cai, “ An analytical design optimization method for electric propulsion systems of multicopter U A Vs with desired hov ering endurance, ” IEEE/ASME T ransactions on Mechatr onics , vol. 24, no. 1, pp. 228–239, 2019. [43] X. Dai, Q. Quan, J. Ren, and C. Kai-Y uan, “Efficiency optimization and component selection for propulsion systems of electric multicopters, ” IEEE T ransactions on Industrial Electr onics , vol. 66, no. 10, pp. 7800– 7809, 2019.