Feedback Control for Autonomous Riding of Hovershoes by a Cassie Bipedal Robot

F eedback Contr ol f or A utonomous Riding of Hov ershoes by a Cassie Bipedal Robot Shuxiao Chen*, Jonathan Rogers*, Bike Zhang*, and K oushil Sreenath Abstract — Motivated towards achieving multi-modal locomo- tion, in this paper , we de velop a framework f or a bipedal robot to dynamically ride a pair of Hovershoes o ver various terrain. Our developed control strategy enables the Cassie bipedal robot to interact with the Hovershoes to balance, regulate forward and rotational velocities, achieve fast turns, and move over flat terrain, slopes, stairs, and r ough outdoor terrain. Our sensor suite comprising of tracking and depth cameras for visual SLAM as well as our Dijkstra-based global planner and timed elastic band-based local planning framework enables us to achieve autonomous riding on the Hovershoes while navigating an obstacle course. W e present numerical and experimental validations of our w ork. I . I N T R O D U C T I O N While locomotion using legs is efficient when trav eling ov er rough and discrete terrain, wheeled locomotion is more ef ficient when tra veling over flat continuous terrain [7]. Humans are able to optimize locomotion efficiency by using multiple locomotion modalities that comprise of not only being able to walk and run, but also being able to ride various micro-mobility platforms, such as Segways and Hov ershoes. Enabling legged robots to autonomously ride on v arious personal mobility platforms will of fer multi- modal locomotion capabilities, improving the efficienc y of locomotion ov er v arious terrains. Autonomous robots with multi-modal locomotion capabil- ities can hav e a big impact in the real-world from package deliv ery to security and surveillance to search and rescue missions [16]. In order to address this problem, we use the Cassie bipedal robot developed by Agility Robotics to autonomously ride Hovershoes, see Fig. 1. A. Challenges There are multiple challenges in dev eloping an au- tonomous framew ork for Cassie to ride Hovershoes. First, Cassie is a complex, underactuated robot with 20 degrees- of-freedom (DOFs), making it a dif ficult system to control. Second, the Hovershoes are a sensitiv e and highly dynamic platform that serv e as two decoupled mo ving platforms under each of Cassie’ s feet and are hard to coordinate. Third, the algorithms we can dev elop neither have access to the internal states of the Hov ershoes nor can the y directly specify * These authors contributed equally to this work. This work was supported by National Science Foundation Grant IIS- 1834557 and Berkeley Deep Driv e. S. Chen, J. Rogers, B. Zhang and K. Sreenath are with the De- partment of Mechanical Engineering, Univ ersity of California, Berkeley , CA, 94720, USA, { shuxiao.chen, jr1397, bikezhang, koushils } @berkeley.edu Fig. 1: Snapshot of an experiment of the bipedal robot Cassie autonomously riding on Hovershoes. Experimental videos are at https://youtu.be/b2fKBb 0iT o. the torque inputs on the Hovershoes, since Cassie can only indirectly interact with the Hovershoes through the contact forces. Fourth, the Cassie-Hov ershoes system needs to be robust to kinematic and dynamic variances arising from (i) manual initialization of the Hov ershoes that result in an initial relative translation and orientation between the Hov ershoes; (ii) manual placement of Cassie’ s feet on the Hov ershoes resulting in off-center foot placement; and (iii) differences in the dynamic characteristics between the Hov- ershoes due to manufacturing. Fifth, the Cassie-Hovershoes system needs to be able to autonomously avoid obstacles, resulting in fast perception and planning demands. W e solve these challenges by designing controllers that assume the dynamics are decoupled along various degrees- of-freedom, strategically choosing a minimalist sensor suite for state estimation, and assuming kinematics is sufficient for path planning. This motiv ates the simplest approach to solving the problem - using PD controllers on decoupled dynamics, using a VIO and depth sensor for state estimation and object detection, and simple global and local kinematic planners. In the future, we will look at more complex controllers, estimators and planners. B. Related W ork 1) Multi-Modal Locomotion: There are some examples of multi-modal locomotion. For instance, DRC-HUBO+ is a bipedal robot that can have wheeled attachments at the knee joint of each leg [12], enabling both legged and wheeled locomotion. Boston Dynamics’ Handle robot—designed for box handling in warehouses—is another legged robot with wheels for feet [1]. ANYmal is yet other wheeled-legged Fig. 2: Segway , Hoverboard, and Hovershoes are sho wn from left to right. The complexity in terms of user control increases along the arro w head direction. robot used for trav eling on irregular terrain [2]. While these robots can offer multi-modal locomotion, the integrated le g- wheel design is neither optimized for legged nor for wheeled mobility , resulting in complex and hea vy feet. Micro-mobility platforms—such as Segways and Hover - boards—allow legged robots without wheeled feet to also hav e multi-modal locomotion capabilities. This is advan- tageous since it keeps the legs lighter for faster motions during le gged movem ent, while also having the option of wheeled mobility by riding on these platforms when needed. Segways and Ho verboards are relati vely easy to ride since the platforms for either foot are connected for increased stability and turning is accomplished through simply leaning on a bar or differentially actuating the two connected platforms. Howe ver , these micro-mobility systems have limited abilities since they hav e a single platform for the user’ s feet, as discussed next. Hov ershoes offer more versatile mov ements since each of the user’ s feet can mov e independently . Howe ver , Hov er- shoes increase the riding complexity as they are significantly more sensiti ve and have no internal controller for turning. Though the Hovershoes are the most complex to ride, they offer the greatest versatility as each foot can move indepen- dently to a void obstacles and go on uneven terrain, which is why we selected this platform for our research. W e illustrate the degree of complexity of riding these products in Fig. 2. 2) Controls: There has been research on dev eloping con- trols for balancing on unstable environments such as seesaws and Bongo Boards with robust lateral stabilization [14]. Howe ver , these en vironments do not translate; but rather, they only tilt. There has been recent work on balancing on platforms that both tilts and translates, a Hov erboard, through sequential online learning control [10]. Addition- ally , a balancing controller for Cassie worked in riding a Segway by regulating the center-of-mass (COM) position [5]. Howe ver , these control strategies, though, are only for rigidly connected wheeled platforms and do not work on Hov ershoes, which are not connected and add additional degrees-of-freedom for more v ersatility . Our research aims to dev elop a control algorithm for a bipedal robot to balance on a highly dynamic and decoupled platform under each foot. 3) Planning: Generic planning algorithms are primarily categorized into sampling-based and search-based methods. Sampling-based planning methods are computationally eco- nomical and offer high update rates. Nev ertheless, feasible trajectories are not always guaranteed and there is some stochasticity in the choice of the plan with the same en- vironmental layout potentially resulting in different plans [9]. In contrast, search-based methods produce deterministic solutions, howe ver , they may incur redundant computation due to their iterative searches [4]. 4) V ision: An accurate odometry source is of vital im- portance to robot movements and velocity tracking. As a benchmark study rev eals, VINS-Mono, a monocular VIO algorithm, with loop-closure detection offers decent accuracy and rob ustness and so does OKVIS, a stereo algorithm [3], [15]. Besides, there are also mature back-end mapping solutions to collaborate with the front-end VIO, e.g. ORB- SLAM2 which introduces a lightweight framework and R T AB-Map that uses OctoMap to achieve good memory management [13], [11]. For walking tasks on legged robots, there has been re- search on utilizing the contact between the feet and the static en vironment to optimize for odometry [6]. Howe ver , since in our case the feet are always in contact with the Hovershoes and the motion is relativ ely smooth, some standard robust VIO and SLAM solutions are suf ficient for our scenario [8]. C. Contribution The contrib utions of our work thus are: 1) Dynamics: W e dev eloped a dynamical model of the in- tegrated Cassie-Hovershoes system for numerical v alidation. 2) Controls: W e designed a control strategy for bipedal robots to robustly balance on independently mobile and decoupled wheeled platforms. 3) Autonomy: W e dev eloped a frame work comprising of a vision system for odometry and obstacle detection, a path planner for online trajectory generation, and a control strategy for the Cassie bipedal robot to autonomously ride the Hov ershoes and a void obstacles. 4) Real-world Experiments: W e demonstrated the robust- ness of our controller for Cassie to robustly ride on Hover - shoes subject to external perturbations on real-world terrain (flat and rough ground as well as stairs), track commanded translational and rotational velocities, perform turning ma- neuvers, and ride the Hov ershoes in a wave pattern. 5) Simplicity: W e presented a solution that uses PD controllers designed assuming decoupled dynamics, an al- most minimal sensor-suite for pose estimation and object detection, and kinematic planners to successfully solve the complex problem of making a bipedal robot autonomously ride a pair of Hovershoes. D. Organization The rest of the paper is organized as follows. Section II presents the dynamical model of the Cassie-Hov ershoes system. Section III describes our proposed controller design. Section IV presents the simulation results. The perception and planning are described in Section V. Section VI demon- strates experimental results with discussion. Section VII discusses shortcomings of our work. Finally , Section VIII summarizes the work and provides thoughts on future work. I I . D Y N A M I C A L M O D E L O F C A S S I E O N H OV E R S H O E S Having established the need for multi-modal locomotion, described the challenges, and outlined our solution, we now present a dynamical model of Cassie and Hovershoes that also captures the interaction between each other . A. Dynamical Model of Cassie Cassie is a highly dynamic, under-actuated bipedal robot. Cassie has twenty DOFs as listed in (1): q = [ q x , q y , q z , q yaw , q pitch , q roll , q 1 L , q 2 L , q 3 L , q 4 L , q 5 L , q 6 L , q 7 L , q 1 R , q 2 R , q 3 R , q 4 R , q 5 R , q 6 R , q 7 R ] T , (1) where, ( q x q y q z ) and ( q yaw q pitch q roll ) are the Cartesian coordinates of the pelvis and the Euler Angles in the Z-Y - X order, and ( q 1 L , ..., q 7 L ) , ( q 1 R , ..., q 7 R ) are the generalized coordinates of the left and right legs, respectively . These correspond to the DOFs for each leg and are defined in (2).           q 1 q 2 q 3 q 4 q 5 q 6 q 7           =           hip roll hip yaw hip pitch knee pitch shin pitch tarsus pitch toe pitch           . (2) Fig. 3a sho ws the generalized coordinates of Cassie’ s pelvis and right leg. The generalized coordinates of Cassie’ s left leg are similar to the right leg states. Each of Cassie’ s legs has se ven DOFs with fi ve of them being actuated: q 1 , q 2 , q 3 , q 4 , and q 7 . The corresponding motor torques are u 1 , u 2 , u 3 , u 4 , and u 5 . The other two DOFs, q 5 and q 6 , are passive, corresponding to stiff springs. The dynamics of Cassie can then be expressed in the following Euler-Lagrange dynamics: D ( q ) ¨ q + H ( q , ˙ q ) = Bu + J T s ( q ) τ s + J T c ( q ) F c , (3) where q is the generalized coordinate vector as defined in (1), D ( q ) is the mass matrix, H ( q , ˙ q ) contains the centripetal, Coriolis, and gravitation terms, B is the motor torque matrix, u is the motor torque vector of dimension 10 corresponding to the actuators on the two legs, J s ( q ) is the Jacobian for the spring torques, τ s is the spring torque v ector , J c ( q ) is the Jacobian for the ground contact forces, and F c is the ground contact force vector . B. Dynamical Model of Hovershoes The Hov ershoes is a highly sensitiv e wheeled platform. It has an internal controller that regulates the pitch to zero when there is no e xternal force. The internal parameters and states of the hov ershoe are unknown. Here, we dev elop a closed- loop dynamical model of the Hovershoe. W e consider θ , ψ , u θ , u ψ to be the pitch and yaw angles and torques in the body X-Y -Z frame, and x , y , v to be the x, y position and (a) (b) Fig. 3: (a) Kinematic model of Cassie showing the robot’ s general- ized coordinates in the body frame. (b) Hovershoe model where u θ and u ψ change the pitch and yaw of the Hovershoe in the X-Y -Z body frame, respecti vely . scalar speed in the global X-Y frame. Our dynamical model then is, J θ ¨ θ = − c 1 θ − c 2 ˙ θ + u θ , (4) J ψ ¨ ψ = − c 3 ˙ ψ + u ψ , (5) ˙ x = v cos ( ψ ) , (6) ˙ y = v sin ( ψ ) , (7) m ˙ v = c 4 θ . (8) Here, the parameters ( c 1 , c 2 , c 3 ) correspond to the Hover - shoe internal controller, c 4 corresponds to the contact model between the Ho vershoe and the ground. The same dynamics is used for the left and right Hovershoes. The Hovershoe pitch dynamics is giv en by (4), where J θ is the moment of inertia of the Hov ershoe about the y-axis, c 1 is the coefficient for the stiffness term, c 2 is the coefficient for the damping term, and u θ is the input torque from a rider’ s toe about the y-axis. The “feedback” with θ and ˙ θ terms in this equation account for the Hovershoe’ s internal stabilization controller that drives θ to zero when u θ is zero. The yaw dynamics of the Hovershoe is described by (5), where J ψ is the moment of inertia of the Hov ershoe about the z-axis, c 3 is the coefficient for the ground contact damping term, and u ψ is the input torque from the rider’ s toe about the z-axis. Here, we only hav e a ˙ ψ term since there is no internal Hov ershoe controller driving ψ to zero when u ψ is zero. Figure 3b shows u θ and u ψ on the Hovershoe. The global translational dynamics of the Hov ershoe are captured by (6)-(7), where v is the speed and m the mass of the Hovershoe. Finally , the acceleration dynamics of the Hov ershoe is captured by (8) as a function of Hovershoe pitch angle. Note that we did not conduct system identification to determine the parameters of the model. Since computing the tilt angle and applied torque on an accelerating Hov ershoe was hard, we instead focused on capturing the structure of the Hov ershoe model for use in our control design. C. Contact Model between Cassie and the Hovershoes The Cassie robot interacts with the Hovershoes through feet contacts. The contact force is F c =  F 1 , · · · , F 4  T in (3) Fig. 4: Autonomy framework where ˙ ψ est is yaw rate of change estimated from VIO, v est is the estimated velocity from VIO, ˙ ψ d is the desired yaw rate of change gi ven from the path planner , and v d is the desired velocity given from the path planner. The user provides a goal location as input. corresponding to the four contacts (front and back contact locations for either foot) with each F i having an x, y , and z components in the local Hovershoe frame, applied on Cassie. The two torques u θ and u ψ , with respect to y-axis and z-axis, respecti vely are computed as,  u θ u ψ  =  0 1 0 0 0 1  4 ∑ i = 1 r i × − F i , (9) where, r i is the relative position of the i t h contact locations with respect to each Hovershoe position, and i = { 1,2 } for left hov ershoe and i = { 3,4 } for right hovershoe. The contact forces F i are themselves determined through a compliant ground model with stick-slip friction and are then used in F c in (3), while u θ , u ψ from (9) are used in (4)-(5). I I I . C O N T R O L D E S I G N Having presented the dynamical model of the Cassie with Hov ershoes system in Section II, we now proceed to discuss our control strategy that will be implemented on Cassie in order to achieve our goal of driving the Hovershoes to desired locations or at desired speeds while Cassie still balances on them. It must be noted that we can neither directly measure the Hovershoes internal states nor can we directly command input torques on the Hov ershoe. Our only means of interaction between the Cassie and Hov ershoe are through the contact forces. Fig. 4 illustrates our autonomy framew ork detailing the interactions between the vision, planner , controller , and the Cassie with Hovershoes system. Our proposed control strategy includes a Hovershoe X- Y controller , a velocity controller , and a turning controller , all integrated with a balancing controller . The X-Y controller regulates the relati ve position of each Ho vershoe with respect to Cassie. The velocity controller tracks a desired transla- tional velocity and the turning controller maintains a desired rotational v elocity . The balancing controller incorporates the output of the three controllers and regulates the position of Cassie’ s COM. Note that the gains of our proposed controller are empirically determined. A. Hovershoe X-Y Contr oller Each Hov ershoe serves as a highly dynamic, moving platform for each foot of Cassie. A low-le vel controller that regulates the relativ e x-y position of each Hov ershoe was designed for Cassie to keep the Hovershoes together . In the absence of this controller , the Hov ershoes drift apart resulting in Cassie falling off of the moving Hovershoes. The Hovershoe X-Y controller is also critical since it makes the system robust to variances that occur when the Hov ershoes are initialized with a non-zero relativ e translation or orientation, any off-center placement of Cassie’ s feet on the Hovershoes, and any dynamic variances between the Hov ershoes. The x-axis controller keeps the relativ e x-coordinate of the Hovershoes at zero in the local Cassie frame. If one of the Hovershoes is ahead of the other , then Cassie needs to bring one Hov ershoe back and the other forward by pitching the Hov ershoe accordingly with its toe (recall from (8) that increasing the pitch of the Hov ershoe results in forward motion). This is accomplished through an inner-outer loop controller . The outer loop computes the desired dif ference in the pitch of either toe, see (10)-(11), while the inner loop computes the desired toe pitch torque to realize the toe pitch difference, see (12)-(13). Here x foot L and q 7 L are the left foot x-position and toe pitch angles respectiv ely . A similar notation is used for the right leg. The computed toe pitch torque gets added to the left and subtracted from the right toe torques that will be specified by the nominal balancing controller . q des ∆ toe = − K x p ( x foot L − x foot R ) , (10) q ∆ toe = q 7 L − q 7 R , (11) e ∆ toe = q ∆ toe − q des ∆ toe , (12) u ∆ toe 5 = − K ∆ toe p e ∆ toe − K ∆ toe d ˙ e ∆ toe . (13) The y-axis controller is designed to keep the y-coordinate of the left and right toe at y offset and − y offset , respectiv ely . Due to the Hovershoes’ nonholonomic constraint, the Hov- ershoe can not mo ve sideways directly along the y-axis. The y-coordinate can only be changed when the Hov ershoes yaw and mo ve forward, so that the le gs can come closer or move apart depending on the yaw angle of each Hovershoe. The y-axis controller has an inner-outer loop as well. The outer loop determines each hip’ s desired yaw angle through the difference between each toe’ s y-coordinate and desired y offset , see (14)-(15). The hip ya w is used since the toe does not ha ve an indi vidual motor for toe yaw . In the inner loop, the desired hip yaw angle is con verted into hip yaw motor torque, see (16). e y = y foot ± y offset , (14) q des 2 = − K y p e y , (15) u y 2 = − K q 2 p ( q 2 − q des 2 ) . (16) In the abov e equations, ± refers to addition to one leg and subtraction for the other . T o achiev e a wa ve pattern with the Hov ershoes, we can specify a time-varying y offset ( t ) . B. V elocity Contr oller Once Cassie can balance on the Ho vershoes and maintain a relativ e position between the Hovershoes while the y are moving, we can then design higher-le vel controllers for Cassie to perform speed regulation and turning maneuvers. These high-level controllers are required for interfacing with the path planner in the autonomous system. The velocity controller maintains the velocity of Cassie on the Hovershoes at a specified set point giv en either by an operator or by the path planner . V elocity control is accomplished through regulating the x -COM position of Cassie, where the desired x -COM position is proportional to the error in velocity: COM des x = − K vel p ( v est − v d ) (17) Here, v est is the v elocity of Cassie estimated from our vision system and v d is the desired velocity . C. T urning Contr oller The turning controller maintains a desired yaw rate for Cassie’ s pelvis that is specified either by an operator or by the path planner . For the two Hov ershoes to turn along a curve, the Hovershoes not only need to constantly yaw but the outer Hovershoe needs to go faster than the inner Hovershoe. Furthermore, for fast turns, Cassie will also need to bank by leaning into the turn. Leaning into a turn ensures Cassie remains balanced on the Hovershoes during fast turning maneuvers and does not tip ov er . This is accomplished by a three-part controller . The Hovershoes are turned by yawing Cassie’ s hips proportional to the yaw rate error , see (18)- (19). The outer Hovershoe is made to go faster than the inner Hovershoe by differentially actuating Cassie’ s toes, see (20) where the computed toe dif ference torque will be added to one toe and subtracted from the other in the balancing controller . Finally , Cassie is made to lean into the turn by changing the desired COM position along the y-axis when there is a large angular and/or tangential velocity , see (21)- (22). e ˙ ψ = ˙ ψ est − ˙ ψ d , (18) u turn 2 = − K yaw p e ˙ ψ , (19) u turn 5 = − K pitch p e ˙ ψ , (20) θ tilt = arctan ˙ ψ est v est 9 . 81 (21) COM des y = − K shift p ( L θ tilt ) . (22) Here, ˙ yaw vision is the actual v elocity of Cassie computed from our vision system and L is the distance of Cassie’ s COM to the Hov ershoes platform plane. D. Integr ating with a Nominal Balancing Contr oller Our proposed controller is based on a nominal balancing controller for Cassie from [5]. The nominal balancing con- troller works by regulating the COM position error and the torso orientation. In order to mov e the COM along the x- axis, the controller modules the toe pitch torque. In order to move the COM along the y-axis, the controller changes the leg length dif ference between the left and right legs. W e modify the nominal torque ˜ u i from the nominal balancing controller as follows: Fig. 5: Simulation visualization with Cassie on the Hovershoes. This is a static balancing task demonstration. 0 5 10 15 20 0 0.02 0.04 0.06 0.08 0.1 0.12 (a) 0 5 10 15 20 -0.02 0 0.02 0.04 0.06 0.08 0.1 0.12 (b) -20 -10 0 10 20 -5 0 5 10 15 20 25 30 35 (c) -20 -10 0 10 -5 0 5 10 15 20 25 30 35 (d) -1 0 1 2 3 4 5 -0.3 -0.2 -0.1 0 0.1 0.2 (e) -1 0 1 2 3 4 5 -0.3 -0.2 -0.1 0 0.1 0.2 (f) Fig. 6: Simulation results for the Hovershoes (left column) and Cassie-Hov ershoe system (right column) for (a,b) forward velocity tracking, (c,d) going around in a circle, and (e,f) following a wave pattern. u 1 = ˜ u 1 , (23) u 2 = ˜ u 2 + u y 2 + u turn 2 , (24) u 3 = ˜ u 3 ( C OM des y ) , (25) u 4 = ˜ u 4 ( C OM des y ) , (26) u 5 = ˜ u 5 ( C OM des x ) ± u ∆ toe 5 ± u turn 5 + u damping 5 . (27) In particular , the hip yaw torque u 2 in (24) comprises of the yaw torque from the nominal balancing controller ˜ u 2 , as well as torques from the y-axis controller u y 2 , and turning controller u turn 2 . The toe torque u 5 in (27) comprises of the Fig. 7: The left picture sho ws the real scene and the right demon- strates the se gmented ground and obstacles. nominal balancing controller toe torque ˜ u 5 as well as the toe pitch difference torque u ∆ toe 5 from the x-axis controller , the turning controller torque u turn 5 , and a damping controller u damping 5 that suppresses the oscillation of the toe pitch joint. As mentioned earlier , ± refers to a torque that is added to one leg and subtracted from the other . I V . S I M U L A T I O N R E S U L T S In order to verify the proposed controller design presented in Section III on the dynamical model from Section II, we ran two numerical simulations. First, we considered the Hov ershoes (without Cassie) and provided the controllers full state knowledge of the Hovershoes and allo wed the controllers to directly actuate the Hovershoes. In the next simulation, we considered the Cassie-Hovershoes integrated system where the controller did not hav e access to the Hov ershoe state and could only indirectly actuate the Hover - shoes through Cassie. W e discuss these simulation results in the following subsections. The simulations ran in MATLAB 2018a installed on a laptop (Intel Core i5, RAM 8GB). A. Hovershoes Simulation W e modeled each Hovershoe in Matlab with the dynamics specified in (4)-(8). In the simulation, we assumed perfect state kno wledge of the Ho vershoes, and the controllers would provide the input torques: u θ and u ψ . W e verified the velocity controller by setting a desired velocity of 0 . 1 m / s and verifying the tracking as shown in Fig. 6a. W e verified the turning controller by setting a desired rotational velocity and ensuring the Hovershoes turned in a circle as shown in Fig. 6c. Finally , we verified the wave pattern controller by setting the y offset to a sinusoidal wav e, resulting in Fig. 6e. B. Cassie-Hovershoes System Simulation W e next simulated the Cassie-Hovershoes integrated sys- tem dynamics (3)-(8), with our designed controller . Fig. 5 sho ws the visualization of Cassie and the Hov ershoes balancing in the simulation. W e used the same set points as the previous Hov ershoes simulation to test our velocity , turning, and wav e pattern controller and obtained a similar performance, see Figs. 6b, 6d, and 6f. V . P E R C E P T I O N A N D P L A N N I N G Having numerically validated our control strategy , we will next pave the path for experiments by presenting our perception and path planning strategies and also detailing the communication between perception, planning, and control. Fig. 8: Hardware infrastructure diagram for the autonomous Cassie system. The vision and path planning computer and the controller computer are onboard Cassie, while the monitoring computer is connected wirelessly and used to see signals and tune parameters. A. P erception In order to modularize our set-up to be simple and easy to manage, and at the same time decrease the computational load, we opted for an integrated VIO sensor, the Intel RealSense T265, which is comprised of a pair of fish-eye lenses is combined with an on-board implementation and extension of an algorithm utilizing stereo VIO with loop- closure detection to produce reliable odometry [8]. Addi- tionally , a rigidly attached depth camera, Intel RealSense D435i, w as used for the obstacle mapping. The D435i camera processed and matched the stereo images using dedicated on- board hardware. By designing the vision set-up as abo ve, we are able to incorporate the entire perception, planning, and control pipeline using the CPUs on Cassie, and av oid the need of power-hungry GPUs. The transformation between the two cameras was known and published internally via R OS. The depth camera was slightly tilted to wards the ground to cov er more ground space near Cassie. The depth maps were con verted to point clouds, which were then filtered and vox elized to a horizontal resolution of 5cm and vertical resolution of 10 cm to reduce the memory usage and accelerate the computation. After transforming the point clouds to the base frame of the Hov ershoes, a ground segmentation with a tuned tolerance to noise was applied to remove the ground plane and register the obstacles onto an global occupancy map. Fig. 7 shows an example of our segmentation. The map registration was achiev ed by using the R T AB-Map R OS package, but the built-in appearance-based loop closure detector was turned off. In order to increase the refresh rate for the local costmap, the segmented 3-D obstacles were further con verted and merged to a pseudo-laser scan to decrease the data rate. Based on the above perception infrastructure, we next detail how we can achieve real-time planning in order to circumvent obstacles. B. Motion Planning T o create real-time motion plans for the Cassie- Hov ershoes system, we require the planner to be reliable and kinematically feasible. For the global planner , we employed (a) (b) (c) (d) Fig. 9: Cassie riding the Hovershoes on (a) different heights, (b) rough terrain, (c) up a 7 ◦ incline, and (d) down a 13 ◦ incline. Fig. 10: Cassie robustly descending three stairs with heights 79%, 93%, 93% of the wheel radius respectiv ely . a search-based method, Dijkstra’ s algorithm, for its guarantee of a solution. The costmap decay rate was tuned to confine the Dijkstra’ s behavior to follow a path that stays in the middle of two potentially incoming obstacles. For the local planner , the optimization-based T imed Elastic Band (TEB) planner was applied on top of the Dijkstra’ s algorithm as the TEB planner looks 1 . 75 m ahead of the current position and generates a short-term, kinematically feasible path for Cassie. In order to emulate the forward non-holonomic property of the Hov ershoes, customized car- like optimization weights were used to encourage forward motion. The optimization process was adjusted to adapt the computing po wer of Cassie and the resulting planner fre- quency could reach up to slightly less than 50 Hz. Ho wev er , 10 Hz proved to be sufficient for Cassie to autonomously navigate through a narro w corridor with obstacles. C. Communication between P er ception, Planning & Contr ol Fig. 8 sho ws a diagram of the communication setup and hardware infrastructure. Please note that the router can be replaced with a W i-Fi hotspot. In addition, between the real-time controller and vision system, we use the UDP to communicate to each other . Finally , the vision and path planning hav e a separate computer other than the controller because only the controller requires a real-time computer . W e use a gantry during experiments for safety purposes only . V I . E X P E R I M E N TA L R E S U LT S After dev eloping our control strategy , path planning algo- rithm, and perception system, we now present the results of our autonomous framework. First, we demonstrate the robustness of our controller . Then, we show the performance of our velocity and turning maneuver . Last, we demonstrate that Cassie successfully navigating an obstacle course au- tonomously while riding the Hovershoes. A. Robustness Once Cassie could balance and perform the tasks in simu- lation, we transferred the controller to Cassie and checked the robustness of the controller subject to disturbances that could 0 5 10 15 20 25 30 35 40 45 0 0.5 1 1.5 2 0 5 10 15 20 25 30 35 40 45 -0.2 0 0.2 0.4 0.6 Fig. 11: T op: velocity response to three successiv e step inputs; Bottom: yaw rate of change response to two successive step inputs. cause Cassie to fall. Robustness was checked by kicking the Hov ershoes and pushing Cassie’ s pelvis and tarsus while it was riding the Hovershoes. In addition, Cassie successfully trav ersed over rough terrain with cracks and bumps, as well as riding on up and down slopes with 7 ◦ and 13 ◦ inclines respectiv ely , and riding with Hovershoes at dif ferent heights (see Fig. 9). A challenge task was to go down a variety of steps. Cassie successfully went down single stairs that were 79% and 171% of the Hov ershoe wheel radius and a sequence of three stairs that were 79%, 93%, 93% of the Hov ershoe wheel radius, respecti vely . As can be seen in Fig. 10, Cassie remains balanced and the Hovershoes remain lev el after descending each step. B. V ision-based V elocity T racking W e tested the performance of our vision-based velocity controller by providing a step input change in desired ve- locity . W e found that it can track a desired velocity with a small steady-state error . As can be see in Fig. 11, the desired velocity step inputs of 0 . 5 m / s, 1 . 0 m / s, and 1 . 5 m / s, were tracked with small steady-state errors with the root mean square error (RMSE) being 0 . 1467 m / s. C. F ast T urning A similar step input for turning yaw rotational velocity of 0 . 5 rad / s and 0 . 0 rad / s produced the response shown in Fig. 11. Clearly , the tracking is not perfect and has a steady- state tracking error with RMSE is about 0 . 1446 rad / s. This can be improved with better choice of gains or incorporating an integral feedback. Furthermore, Fig. 12 shows Cassie performing a high-speed turn. It is clear from the center graphic of Fig. 12 that Cassie successfully leans into the turn by shifting its COM position along the y-axis in order to pre vent from tipping over during the turn. D. W ave P attern with F eet for Obstacle A voidance Certain obstacles are short enough where Cassie does not need to circumnavigate them in order to av oid collision. W ith the Hovershoes, Cassie can split around the obstacle Fig. 12: Cassie executing a fast turn by leaning, with the lean angle directly dependent on the forward and rotational v elocities. Fig. 13: Cassie showing wa ve pattern with feet task by av oiding an obstacle without going around it. F or certain obstacles, Cassie can stay on a straight trajectory while still missing the obstacle. using the wav e pattern controller . Fig. 13 sho ws that Cassie successfully av oids an obstacle by doing a wav e pattern maneuver with its feet. The decision making of splitting its feet was made by a human operator . E. Obstacle A voidance Finally , combining our controllers with the computer vi- sion obstacle detection system and path planner , we achiev ed autonomous locomotion through an obstacle course. The leftmost graphic of Fig. 14 shows the obstacle course and the rest show Cassie autonomously navigating through the obstacle course. V I I . S H O RT C O M I N G S O F O U R P RO P O S E D W O R K Our proposed method is the simplest control and autonomy package for achieving the task of riding Hov ershoes. In particular , our controller decouples the system dynamics into sev eral simple sub-components, our planner only considers a kinematical model of the system, and our vision system only uses the stereo camera and IMU without using the ad- ditional sensors on the robot. By considering a fully coupled dynamical system and incorporating the dynamics into the planner and taking all available sensing into account, we can further improve our state estimation, controller and planner performance significantly . Finally , our proposed solution assumes Cassie is somehow initialized on the Hovershoes, which is currently done manually and does not truly offer multi-modal locomotion capabilities yet. V I I I . C O N C L U S I O N S In this paper , we hav e presented a framew ork for au- tonomous locomotion of a Cassie bipedal robot over Hov er- shoes. Our de veloped framew ork enables the Cassie bipedal robot to interact with the Hovershoes to balance, regulate forward and rotational velocity , achieve fast turns, and move ov er flat terrain, slopes, stairs, and rough outdoor terrain, as well as autonomously navigate an obstacle course. Fig. 14: Cassie performing obstacle a voidance. The leftmost figure is a picture of the obstacle course. The rest of the pictures are sequential snapshots of the path planner and 3-D map while the robot is autonomously navigating through the course. The future work will focus on combining Cassie walking to achiev e multi-modal locomotion. Currently human oper- ators are needed to initialize Cassie on the Hovershoes. W e would like to develop a method for Cassie to step up on the Hov ershoes without the need of operators. R E F E R E N C E S [1] “Handle: Mobile box handling robots for logistics, ” https://www . bostondynamics.com/handle, accessed: 2019-06-06. [2] M. Bjelonic, C. D. Bellicoso, et al. , “Keep rollin’ - whole-body motion control and planning for wheeled quadrupedal robots, ” IEEE Robotics and Automation Letters , vol. 4, no. 2, pp. 2116–2123, 2019. [3] J. Delmerico and D. Scaramuzza, “ A benchmark comparison of monocular visual-inertial odometry algorithms for flying robots, ” in IEEE Int. Conf. on Robotics and Automation , 2018. [4] E. W . Dijkstra, “ A note on two problems in connexion with graphs, ” Numerische mathematik , vol. 1, no. 1, pp. 269–271, 2018. [5] Y . Gong, R. Hartley , et al. , “Feedback control of a cassie bipedal robot: W alking, standing, and riding a segway , ” in American Contr ol Confer ence , 2019. [6] R. Hartley , J. Mangelson, et al. , “Legged robot state-estimation through combined forward kinematic and preintegrated contact fac- tors, ” in IEEE Int. Conf. on Robotics and Automation , 2018. [7] K. Hashimoto, T . Hosobata, et al. , “Realization by biped leg-wheeled robot of biped walking and wheel-driv en locomotion, ” in IEEE Int. Conf. on Robotics and Automation , 2005. [8] E. S. Jones and S. Soatto, “V isual-inertial navigation, mapping and localization: A scalable real-time causal approach, ” The Int. J. of Robotics Resear ch , vol. 30, no. 4, pp. 407–430, 2011. [9] S. Karaman and E. Frazzoli, “Sampling-based algorithms for optimal motion planning, ” The Int. J . of Robotics Resear ch , v ol. 30, no. 7, pp. 846–894, 2011. [10] K. Kimura, S. Nozawa, et al. , “Riding and speed governing for parallel two-wheeled scooter based on sequential online learning control by humanoid robot, ” in IEEE Int. Conf. on Intelligent Robots and Systems , 2018. [11] M. Labb and F . Michaud, “R T ABmap as an opensource lidar and visual simultaneous localization and mapping library for largescale and longterm online operation, ” Journal of F ield Robotics , vol. 36, no. 2, pp. 416–446, 2019. [12] J. Lim, I. Lee, et al. , “Robot system of DRC-HUBO+ and control strategy of team KAIST in DARP A robotics challenge finals, ” Journal of F ield Robotics , vol. 34, no. 4, pp. 802–829, 2017. [13] R. Mur-Artal and J. D. T ardos, “ORB-SLAM2: an open-source slam system for monocular, stereo and rgb-d cameras, ” IEEE T rans. on Robotics , vol. 33, no. 5, pp. 1255–1262, 2017. [14] U. Nagarajan and K. Y amane, “Balancing in dynamic, unstable envi- ronments without direct feedback of environment information, ” IEEE T ransactions on Robotics , vol. 30, no. 5, pp. 1234–1241, 2014. [15] P . L. T ong Qin and S. Shen, “VINS-Mono: A robust and versatile monocular visual-inertial state estimator , ” IEEE T rans. on Robotics , vol. 34, no. 4, pp. 1004–1020, 2018. [16] S. W aharte and N. T rigoni, “Supporting search and rescue operations with uavs, ” in IEEE Int. Conf. on Emerging Security T echnologies , 2010.