From Bipedal to Quadrupedal Locomotion, Experimental Realization of Lyapunov Approaches Citation Ma, Wen-Loong (2021) From Bipedal to Quadrupedal Locomotion, Experimental Realization of Lyapunov Approaches. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/j1ty-zb28. https://resolver.caltech.edu/CaltechTHESIS:05042021-155258800 Abstract Possibly one of the most significant innovations of the past decade is the hybrid zero dynamics (HZD) framework, which formally and rigorously designs a control algorithm for robotic walking. In this methodology, Lyapunov stability, which is often used to certificate a dynamical system's stability, was introduced to the control law design for a hybrid control system. However, the prerequisites of precise modeling to apply the HZD methodology can often be too restrictive to design controllers for uncertain and complex real-world hardware experiments. This thesis addresses the problem raised by noisy measurements and the intricate hybrid structure of locomotion dynamics. First, the HZD methodology's construction is based on the full-order, hybrid dynamics of legged locomotion, which can be intractable for control synthesis for high-dimensional systems. This thesis studies the general structure of hybrid control systems for walking systems, ranging from 1D hopping, 2D walking, 2D running, and 3D quadrupedal locomotion on rough terrains. Further, we characterize a walking behavior--gait--as a solution (execution) to a hybrid control system. To find these solutions, which represent a "gait," we employed advanced numerical methods such as collocation methods to parse the solution-finding problem into the open- and closed-loop trajectory optimization problems. The result is that we can find versatile gaits for ten different robotic platforms efficiently. This includes bipedal running, bipedal walking on slippery surfaces, and quadrupedal robots walking on sloped terrains. The numerous solution-finding examples expand the applicability of the HZD framework towards more complex dynamical systems. Further, for the uncertain and noisy real-world implementation, the exponential stability of the continuous dynamics is an ideal but restrictive condition for hybrid stability. This condition is especially challenging to satisfy for highly dynamical behaviors such as bipedal running, which loses ground support for a short period. This thesis observes the destabilizing effect of the noisy measurements of the phasing variable. By reformulating the traditional input-to-state stability (ISS) concept into phase-uncertainty to state stability, we are able to synthesize a robust controller for bipedal running on DURUS-2D. This time+state-based controller formally guarantees stability under noisy measurements and stabilizes the 1.75 m/s running experiments. Lastly, robotic dynamics have long been characterized as the interconnection of rigid-body dynamics. We take this perspective one step further and incorporate controller design into the formulation of coupled control systems (CCS). We first view a quadrupedal robot as two bipedal robots connected via some holonomic constraints. In a dimensional reduction manner, we develop a novel optimization framework, and the computational performance is reduced to a few seconds for gait generation. Furthermore, we can design local controllers for each bipedal subsystem and still guarantee the overall system's stability. This is done by combining the HZD framework and the ISS properties to contain the disturbance induced by the other subsystems' inputs. Utilizing the proposed CCS methods, we will experimentally realize quadrupedal walking on various outdoor rough terrains. Item Type: Thesis (Dissertation (Ph.D.)) Subject Keywords: Nonlinear control, nonconvex optimization, legged locomotion Degree Grantor: California Institute of Technology Division: Engineering and Applied Science Major Option: Mechanical Engineering Thesis Availability: Public (worldwide access) Research Advisor(s): Ames, Aaron D. Thesis Committee: Burdick, Joel Wakeman (chair) Ames, Aaron D. Murray, Richard M. Chung, Soon-Jo Defense Date: 5 May 2021 Non-Caltech Author Email: wenloong.ma (AT) gmail.com Funders: Funding Agency Grant Number NSF 1724464 NSF CNS-1239055 NSF 1239055 Record Number: CaltechTHESIS:05042021-155258800 Persistent URL: https://resolver.caltech.edu/CaltechTHESIS:05042021-155258800 DOI: 10.7907/j1ty-zb28 Related URLs: URL URL Type Description https://doi.org/10.1109/LCSYS.2020.3006963 DOI Article adapted for ch.4 https://doi.org/10.1109/LRA.2021.3065174 DOI Article adapted for ch.4 https://doi.org/10.1109/IROS45743.2020.9341181 DOI Article adapted for ch.4 https://doi.org/10.1109/ICRA40945.2020.9196841 DOI Article adapted for ch.4 https://doi.org/10.23919/ACC.2019.8815085 DOI Article adapted for ch.4 https://doi.org/10.1109/IROS40897.2019.8968189 DOI Article adapted for ch.2 and ch.4 https://doi.org/10.1109/ICRA.2019.8793761 DOI Article adapted for ch.2 and ch.3 https://doi.org/10.23919/ECC.2019.8796090 DOI Article adapted for ch.2 https://doi.org/10.1109/CCTA.2017.8062633 DOI Article adapted for ch.2 https://doi.org/10.1145/3049797.3049823 DOI Article adapted for ch.3 https://doi.org/10.1109/CDC.2017.8264603 DOI Article adapted for ch.2 https://doi.org/10.1109/IROS. 2016.7759856 DOI Article adapted for ch.3 https://doi.org/10.1109/ACC.2015.7170931 DOI Article adapted for ch.2 https://doi.org/10.1109/ICRA.2014.6907605 DOI Article adapted for ch.1 and ch.2 https://doi.org/10.1109/ICCPS.2014.6843723 DOI Article adapted for ch.2 ORCID: Author ORCID Ma, Wen-Loong 0000-0002-0115-5632 Default Usage Policy: No commercial reproduction, distribution, display or performance rights in this work are provided. ID Code: 14132 Collection: CaltechTHESIS Deposited By: Wenlong Ma Deposited On: 04 Jun 2021 00:16 Last Modified: 02 Nov 2021 17:58 Thesis Files PDF - Final Version See Usage Policy. 10MB Repository Staff Only: item control page

From Bipedal to Quadrupedal Locomotion, Experimental Realization of Lyapunov Approaches Thesis by Wen-Loong Ma In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in the Department of Mechanical Engineering CALIFORNIA INSTITUTE OF TECHNOLOGY Pasadena, California 2021 Defended May 05, 2021 ii © 2021 Wen-Loong Ma ORCID: 0000-0002-0115-5632 All rights reserved except where otherwise noted iii To my parents and grandfather. iv ACKNOWLEDGEMENTS What a splendid adventure! My goal for graduate school was to understand the whole process of making a robot. Now that I am on the other side of this trek and ready for another voyage, I want to thank my advisor, Aaron Ames, for guiding me through all the challenging years and for constructing the research environment for my ambitious dreams. Beyond the nonlinear control theorem, I learned from him; his passion and boldly rational vision have shown me how to navigate the ocean of knowledge without being disturbed by the fancy buzzwords and hot trends. Further, it was an honor to be part of the AMBER group. I was lucky enough to work with the phase-I students, Ryan, Shishir, Huihua, Ayonga, and Matthew. They are the good examples of hard-working, creativity, and persistence, which set the ideal initial conditions for my research career. Those postmidnight hangouts after deadlines are the most memorable moments of my twenties. I owe much to Shishir for the “on-call” mentorship in developing the theorem of this thesis. I am grateful to have worked with Christian, Eric, Jenna, Noel, Drew, and Maegan to crack open the mysterious real-world robotics experiments and for their friendship. My life in graduate school was made enjoyable because of all the crazy-minded peers I came across. The countless arguments with Fangzhou Xiao, Shaowu Pan, Wei Ding, Honglie Ning, Jin Sima, Yu Sun, etc., have stimulated various angles to reevaluate my research from the perspectives of biology, physics, aerospace, logic, and math. These discussions have widely opened my eyes and gradually broken my subconscious barriers. My gratitude extends to all my collaborators and those academic pioneers, Jessy Grizzle, Paulo Tabuada, Kaveh Akbari Hamed, Koushil Sreenath, Joel Burdick, John Doyle, Richard Murray, etc., for educating me not only knowledge but also the quality of an academic professional. Their light shone across the Pacific Ocean and guided me to this land of possibilities. I am particularly thankful for Kaveh’s and Joel’s coaching in the last few years of my PhD life. Finally, I want to thank the delicious coffee stores in Pasadena, such as Copa Vida, Jameson Brown, Intelligentsia, Ambrose Cafe, and Starbucks, for brewing the refreshing drinks and setting the cosy environment to convert caffeine into control algorithms. v ABSTRACT Possibly one of the most significant innovations of the past decade is the hybrid zero dynamics (HZD) framework, which formally and rigorously designs a control algorithm for robotic walking. In this methodology, Lyapunov stability, which is often used to certificate a dynamical system’s stability, was introduced to the control law design for a hybrid control system. However, the prerequisites of precise modeling to apply the HZD methodology can often be too restrictive to design controllers for uncertain and complex real-world hardware experiments. This thesis addresses the problem raised by noisy measurements and the intricate hybrid structure of locomotion dynamics. First, the HZD methodology’s construction is based on the full-order, hybrid dynamics of legged locomotion, which can be intractable for control synthesis for high-dimensional systems. This thesis studies the general structure of hybrid control systems for walking systems, ranging from 1D hopping, 2D walking, 2D running, and 3D quadrupedal locomotion on rough terrains. Further, we characterize a walking behavior—gait—as a solution (execution) to a hybrid control system. To find these solutions, which represent a “gait,” we employed advanced numerical methods such as collocation methods to parse the solution-finding problem into the openand closed-loop trajectory optimization problems. The result is that we can find versatile gaits for ten different robotic platforms efficiently. This includes bipedal running, bipedal walking on slippery surfaces, and quadrupedal robots walking on sloped terrains. The numerous solution-finding examples expand the applicability of the HZD framework towards more complex dynamical systems. Further, for the uncertain and noisy real-world implementation, the exponential stability of the continuous dynamics is an ideal but restrictive condition for hybrid stability. This condition is especially challenging to satisfy for highly dynamical behaviors such as bipedal running, which loses ground support for a short period. This thesis observes the destabilizing effect of the noisy measurements of the phasing variable. By reformulating the traditional input-to-state stability (ISS) concept into phase-uncertainty to state stability, we are able to synthesize a robust controller for bipedal running on DURUS-2D. This time+state-based controller formally guarantees stability under noisy measurements and stabilizes the 1.75 m/s running experiments. vi Lastly, robotic dynamics have long been characterized as the interconnection of rigid-body dynamics. We take this perspective one step further and incorporate controller design into the formulation of coupled control systems (CCS). We first view a quadrupedal robot as two bipedal robots connected via some holonomic constraints. In a dimensional reduction manner, we develop a novel optimization framework, and the computational performance is reduced to a few seconds for gait generation. Furthermore, we can design local controllers for each bipedal subsystem and still guarantee the overall system’s stability. This is done by combining the HZD framework and the ISS properties to contain the disturbance induced by the other subsystems’ inputs. Utilizing the proposed CCS methods, we will experimentally realize quadrupedal walking on various outdoor rough terrains. vii PUBLISHED CONTENT AND CONTRIBUTIONS Ambrose, E., W. Ma, and A. D. Ames (2021). “Towards the Unification of System Design and Motion Synthesis for High-Performance Hopping Robots”. In: International Conference on Robotics and Automation (ICRA). WM designed the optimization algorithm to simultaneously solve for a gait and the system parameters, and participated in the writing of the manuscript. Ma, W., N. Csomay-Shanklin, and A. D. Ames (2021). “Coupled Control Systems: Periodic Orbit Generation With Application to Quadrupedal Locomotion”. In: IEEE Control Systems Letters 5.3, pp. 935–940. doi: 10.1109/LCSYS.2020. 3006963. WM participated in the conception of the project, designed the optimization algorithm, led the experimental testing and the writing of the manuscript. Ma, W. -L., N. Csomay-Shanklin, S. Kolathaya, et al. (2021). “Coupled Control Lyapunov Functions for Interconnected Systems, With Application to Quadrupedal Locomotion”. In: IEEE Robotics and Automation Letters 6.2, pp. 3761–3768. doi: 10.1109/LRA.2021.3065174. WM proved the main theorem about the coupled control Lyapunov function, developed the control algorithm and conducted the experiments, and led the writing of the manuscript. Ma, W. -L., N. Csomay-Shanklin, and A. D. Ames (2020). “Quadrupedal Robotic Walking on Sloped Terrains via Exact Decomposition into Coupled Bipedal Robots”. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4006–4011. doi: 10.1109/IROS45743.2020.9341181. WM derived the analysis for controlling quadrupedal robots walking on slope, participated in the experimental implementation, and led the writing of the manuscript. Ma, Wen-Loong and Aaron D. Ames (2020). “From Bipedal Walking to Quadrupedal Locomotion: Full-Body Dynamics Decomposition for Rapid Gait Generation”. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 4491–4497. doi: 10.1109/ICRA40945.2020.9196841. WM derived the dynamic decomposition for gait generation, conducted the experiments, and led the writing of the manuscript. Akbari Hamed, Kaveh., Wen-Loong. Ma, and Aaron. D. Ames (July 2019). “Dynamically Stable 3D Quadrupedal Walking with Multi-Domain Hybrid System Models and Virtual Constraint Controllers”. In: 2019 American Control Conference (ACC), pp. 4588–4595. doi: 10.23919/ACC.2019.8815085. W. M designed the optimization algorithm, defined the hybrid structure of quadrupedal dynamics, and participated in the writing of the manuscript. Ma, W., K. A. Hamed, and A. D. Ames (2019). “First Steps Towards Full Model Based Motion Planning and Control of Quadrupeds: A Hybrid Zero Dynamics viii Approach”. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5498–5503. doi: 10.1109/IROS40897.2019.8968189. WM wrote the code for the gait generation of quadrupedal robots for three different motions, conducted outdoor experiments, and led the writing of the manuscript. Ma, W., Y. Or, and A D. Ames (2019). “Dynamic Walking on Slippery Surfaces : Demonstrating Stable Bipedal Gaits with Planned Ground Slippage*”. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 3705–3711. doi: 10.1109/ICRA.2019.8793761. WM designed the controller, the gait optimization method, derived the hybrid structure of walking, conducted the experiments, and lead the writing of the manuscript. Reher, Jake, Wen-Loong Ma, and Aaron D. Ames (2019). “Dynamic Walking with Compliance on a Cassie Bipedal Robot”. In: 2019 18th European Control Conference (ECC), pp. 2589–2595. doi: 10.23919/ECC.2019.8796090. WM designed the compliant walking gait and the feedforward controller, and participated in the experiments and the writing of the manuscript. Ambrose, E., W. Ma, C. Hubicki, et al. (2017). “Toward benchmarking locomotion economy across design configurations on the modular robot: AMBER-3M”. In: 2017 IEEE Conference on Control Technology and Applications (CCTA), pp. 1270–1276. doi: 10.1109/CCTA.2017.8062633. WM designed the optimization algorithm for gait generation, participated in the conception of the project, and designed and conducted the experiments. Ma, Wen-Loong, Shishir Kolathaya, et al. (2017). “Bipedal Robotic Running with DURUS-2D: Bridging the Gap Between Theory and Experiment”. In: Proceedings of the 20th International Conference on Hybrid Systems: Computation and Control. HSCC ’17. Pittsburgh, Pennsylvania, USA: ACM, pp. 265–274. isbn: 978-1-4503-4590-3. doi: 10.1145/3049797.3049823. url: http://doi. acm.org/10.1145/3049797.3049823. WM designed running trajectory using optimization, participated in stability analysis for the continuous dynamics, participated in experimental design, and led the writing of the manuscript. Tabuada, P. et al. (2017). “Data-driven control for feedback linearizable singleinput systems”. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp. 6265–6270. doi: 10.1109/CDC.2017.8264603. WM participated in the conception of the project, designed the experimental procedures, and participated in the writing of the manuscript. Ma, W., A. Hereid, et al. (2016). “Efficient HZD gait generation for three-dimensional underactuated humanoid running”. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5819–5825. doi: 10.1109/IROS. 2016.7759856. WM designed the control and optimization methods for bipedal running, analyzed and prepared the data, and wrote the manuscript. ix Kolathaya, Shishir, Wen-Loong Ma, and Aaron D Ames (2015). “Composing Dynamical Systems to Realize Dynamic Robotic Dancing”. In: Algorithmic Foundations of Robotics XI. Springer, Cham, pp. 425–442. WM designed and programmed the controller, constructed the real-time operating system environment, conducted the experiments, and participated in the writing of the manuscript. Mehra, A. et al. (2015). “Adaptive cruise control: Experimental validation of advanced controllers on scale-model cars”. In: 2015 American Control Conference (ACC), pp. 1411–1418. doi: 10.1109/ACC.2015.7170931. WM participated in the conception of the project, derived the safety controller for simulation, and participated in the writing of the manuscript. Ma, W., H. Zhao, et al. (May 2014). “Human-inspired walking via unified PD and impedance control”. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 5088–5094. doi: 10.1109/ICRA.2014.6907605. WM designed the walking controller, motor control algorithm, constructed the real-time operating system environment, conducted the experiments, and led the writing of the manuscript. Zhao, Hui-Hua et al. (2014). “Human-inspired multi-contact locomotion with AMBER2”. In: Cyber-Physical Systems (ICCPS), 2014 ACM/IEEE International Conference on. Best Paper Award Finalist of ICCPS 2014. IEEE, pp. 199–210. doi: 10.1109/ICCPS.2014.6843723. WM designed and programmed the controller, conducted the experiments, and participated in the writing of the manuscript. x TABLE OF CONTENTS Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Published Content and Contributions . . . . . . . . . . . . . . . . . . . . . . vii Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix List of Illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi Chapter I: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Organization of Dissertation . . . . . . . . . . . . . . . . . . . . . . 3 Chapter II: Constrained Rigid-Body Dynamics . . . . . . . . . . . . . . . . . 6 2.1 Coordinates and Notations . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 Holonomic Constraints and System-Level Degrees of Freedom . . . 8 2.3 Continuous-Time Dynamics . . . . . . . . . . . . . . . . . . . . . . 9 2.4 Underactuated System . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.5 The ODE Form of Control-Affine Systems. . . . . . . . . . . . . . 13 2.6 Hybrid System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.7 Discrete Transitions . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.8 Solution to Hybrid Dynamics . . . . . . . . . . . . . . . . . . . . . 19 2.9 Numerical Optimization . . . . . . . . . . . . . . . . . . . . . . . . 20 Chapter III: Controller Design for Bipedal Running . . . . . . . . . . . . . . 26 3.1 Running Dynamics (Open-Loop) . . . . . . . . . . . . . . . . . . . 26 3.2 Virtual Constraint and Zero Dynamics . . . . . . . . . . . . . . . . 29 3.3 Gait Design via Closed-Loop Optimization . . . . . . . . . . . . . . 35 3.4 Bipedal Running with Input to State Stability . . . . . . . . . . . . . 52 Chapter IV: Coupled Control System and Quadrupedal Applications . . . . . 72 4.1 HZD, from Bipedal to Quadrupedal Robots . . . . . . . . . . . . . . 72 4.2 Coupled Control System and Its Solutions . . . . . . . . . . . . . . 81 4.3 CCS Optimization and Application to Quadrupedal Gaits . . . . . . 92 4.4 Coupled Control Lyapunov Functions . . . . . . . . . . . . . . . . . 111 4.5 Coupled Mechanical Systems . . . . . . . . . . . . . . . . . . . . . 118 4.6 Example 1. Feedback Linearization-Based CLFs . . . . . . . . . . . 122 4.7 Example 2. Quadrupedal Walking with Model-Free CLFs . . . . . . 125 Chapter V: Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . 130 5.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 5.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 xi LIST OF ILLUSTRATIONS Number Page 1.1 Robots developed to control in this thesis. From left to right: AMBER2 (2012-2014), AMBER3 (2015), AMBER3M (2016-2020), DURUS-2D (2015-2016), DURUS (2016), Cassie (2017-now), Vision60 v3.2 & v3.9 (2018-2020). . . . . . . . . . . . . . . . . . . . 4 2.1 Configuration coordinates of AMBER robots, DURUS-2D running robot, DURUS, Cassie, and Vision 60, the quadrupedal robots. . . . 6 2.2 Three fundamental contact problems modelled by holonomic constraints: a) point contact, ℎ(𝑞) ∈ R3 denotes the three-dimensional Cartesian position fixed to the world; b) line contact, ℎ(𝑞) ∈ R3+2 fixed one point of the contact line and the pitch, yaw angles of the body-fixed frame to the world; c) plane contact, ℎ(𝑞) ∈ R6 fixed the 6-dimensional body-fixed frame to the world. . . . . . . . . . . . . 8 2.3 Branched chain structure of a bipedal system—AMBER-3M, and a quadrupedal system—Vision 60. . . . . . . . . . . . . . . . . . . . 10 2.4 Simple examples of overactuation in the presence of holonomic constraints: (a,b) sliding on a track; (c) pushing against a wall; (d) a quadrupedal robot standing on two feet. . . . . . . . . . . . . . . . 11 2.5 Directed graph for two robotic systems. . . . . . . . . . . . . . . . . 15 2.6 A mixed sequence of walking tasks of a bipedal robot: flat ground walking, uphill walking, downhill walking, flat ground walking. On the right is the directed graph, where the arrows represent the corresponding edges. Each edge is associated with transition dynamics. . 20 2.7 Left: the configuration coordinate of the 1D hopper. Right: The optimized trajectory from (2.51). . . . . . . . . . . . . . . . . . . . 24 3.1 (a) The simulated running of the humanoid robot, DURUS, as a result of large-scale HZD optimization. (b) The directed cycle structure of the multi-domain hybrid system model for flat-footed humanoid running. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.2 A block diagram showing the state-based feedback control structure. 34 3.3 Limit cycle of running at 2.0 m/s over 20 steps. . . . . . . . . . . . . 41 3.4 Snapshots of the DURUS running at 2.0 m/s. . . . . . . . . . . . . . 42 xii 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 The computation performance for generating gaits at each running velocity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multiple running gaits with forward velocity from 1.5 m/s to 3.0 m/s. Slippage in the beginning of a step: pre-slip on the left and post-slip on the right. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . On the left: The AMBER-3M with point foot, constrained to a planar rail to walk in a 2D environment on a treadmill. On the right: the model’s configuration coordinates, with 3 global coordinates and 4 local coordinates. . . . . . . . . . . . . . . . . . . . . . . . . . . . Snapshots of one slippery walking step from Experiment 3. In the first three pictures, the left foot (stance foot) is slipping smoothly on the lubricated treadmill. . . . . . . . . . . . . . . . . . . . . . . . . Phase portrait of 50 seconds’ experimental data from Experiment 3, with walking speed (from left to right): 0.26m/s, 0.3m/s, 0.38m/s, 0.42m/s. Solid lines are for the desired values and dashed lines are for the actual measurements. . . . . . . . . . . . . . . . . . . . . . 𝑀𝐶𝑂𝑇 + of the experiments on AMEBR-3M. Those not included for certain speeds are failed experiments. . . . . . . . . . . . . . . . . . The spring-legged planar running biped, DURUS-2D, during take off (left) and while airborne (right). . . . . . . . . . . . . . . . . . . . . (a) The model of DURUS-2D with two linear springs; (b) the directed cycle structure of the multi-domain hybrid system model for DURUS2D running. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A block diagram showing the time-based feedback control structure. Limit cycles of (a) simulation where time-based IO + state-based PD controller was applied for 100 steps; (b) Simulation where white noise was added to 𝜏𝑣 (𝑞); (c) Experimental data. Note the solid lines are the designated gait from optimization. . . . . . . . . . . . . . . . Bounded tube around the designed periodic orbit. . . . . . . . . . . Running tiles of simulation (top) vs. experiment (bottom) for one step. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Experimental setup for DURUS-2D running: 1) control station computer, 2) emergency stop, 3) four Li-Po batteries, 4) tripping harness, 5) treadmill control panel, 6) encoder wheel to measure treadmill speed, 7) treadmill. . . . . . . . . . . . . . . . . . . . . . . . . . . . Left and right foot height (ground clearance). . . . . . . . . . . . . . 43 43 44 46 51 51 52 52 54 59 67 67 68 68 70 xiii 3.20 3.21 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 The actual vs. desired joint angles from the experiment. . . . . . . . The left column is the phase variables which are used to calculate time- and state-based outputs: 𝑦 𝑑,𝑡 (𝑡) = 𝑦 𝑣𝑑 (𝜏𝑣 (𝑡)), 𝑦 𝑑 (𝑞) = 𝑦 𝑣𝑑 (𝜏𝑣 (𝑞)); the right column is output errors (virtual constraints) showing ultimate boundedness. (a, b, c, d) are from two simulations with controller given by (3.78). And (c, d) has a sinusoidal disturbance with 10% amplitude added to 𝜏𝑣 (𝑞); (e, f) are from experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (a) Vision 60 V3.2, 26 kg, 0.45m tall; (b) Vision 60 V3.9, 44 kg, 0.55m tall; (c) Configuration coordinates of both quadrupedal robots. Both robots were manufactured by Ghost Robotics. . . . . . . . . The cyclic directed graph for the multi-domain hybrid dynamics of walking, ambling, and trotting gaits. The dashed lines represent a relabeling map (as given in (3.32)) that flips the left and right legs’ contact attributes. We took these nomenclature from (Muybridge, n.d.). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . On the top left is the snapshot for a 2 × 2-domain walking gait; on the top right is a 2 × 2-domain ambling gait; and on the bottom is a 4 × 2-domain trotting gait. The symbol 0 is a left-right mirror of the contact attributes for domain . The orange-colored legs are the leftside legs whose contact points are highlighted by triangles; the bluecolored legs are the right-side legs with contact points highlighted by inverted triangles. . . . . . . . . . . . . . . . . . . . . . . . . . . . The periodic trajectories designed by NLP (4.7): walk (left), amble (middle), and trot (right). . . . . . . . . . . . . . . . . . . . . . . . Snapshots of the Vision 60 ambling in an outdoor environment, showing a full step of 2 × 2 domains of the amble gait. . . . . . . . . . . The designed gaits (in red) from optimization/simulation vs. the experimental data (in cyan) in the form of phase portrait for amble. HR is short for hip roll, HP is for hip pitch, and k is for knee. . . . . . Conceptual illustration of the full-body dynamics decomposition, where the 3D quadruped — the Vision 60 — is decomposed into two constrained 3D bipedal robots. . . . . . . . . . . . . . . . . . . . . A conceptual illustration of the full-body dynamics decomposition, where the 3D quadruped — the Vision 60 — is decomposed into two constrained 3D bipedal robots. . . . . . . . . . . . . . . . . . . . . 70 71 72 76 77 79 80 80 82 96 xiv 4.9 4.10 4.11 4.12 4.13 4.14 4.15 4.16 4.17 Top: Snapshots showing a full step of the ambling gait on an outdoor lawn. Bottom: The periodic trajectory produced by optimization (CCS-OPT) (in red) vs. the experimental tracking data (in cyan) vs. RaiSim simulation data (in green) in the form of phase portrait (limit cycle) using 18 seconds’ data. . . . . . . . . . . . . . . . . . . . . . 104 A comparison between the solution of bipedal walking dynamics obtained from the decomposition-based optimization and a simulated step of the full-order quadrupedal dynamics using the composed bipedal gaits; here MATLAB ODE45 was used. . . . . . . . . . . . . . 106 Comparison between MuJoCo simulation (animated) and experiments. The upper two are for stepping in place, gait4; the lower two are logged for a full step of the ambling gait. . . . . . . . . . . . 107 (a) The cyclic directed graph for the multi-domain hybrid dynamics on sloped terrain; (b) friction cone (pyramid) condition on some sloped terrain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Phase portraits of the designed gaits (solid lines) vs. experimental data (transparent overlay) for quadrupedal walking on 13◦ (red) and 20◦ ∼ 25◦ (blue) slopes. HR, HP, K are short for hip roll, hip pitch, and knee, respectively. . . . . . . . . . . . . . . . . . . . . . . . . . 109 Full steps of gait tiles for the Vision 60 ambling in outdoor grasslands. The top two is comparison between simulation and experiments for a 13◦ slope. The bottom two are for walking on a ramp with varying slopes of 20◦ ∼ 25◦ . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 The quadrupedal robot, Vision 60 v3.9. . . . . . . . . . . . . . . . . 118 A cart-pole system with two inverted pendula, each is directly actuated by a motor. The mass of the cart and pendula are 2𝑀 and 𝑚, correspondingly. The length of both pendulum is 𝑙. We have the configuration coordinates as (𝜉, 𝜃 1 , 𝜃 2 ) > . . . . . . . . . . . . . . . . 119 The cart-pole simulation. data labeled FL (red) used the feedback linearization controller (4.98); data labeled CLF-QP (blue) used the controller (4.100); dark and light variants of the colors are used to distinguish between the first and second pendula. Both simulations show stability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 xv 4.18 (a) The decomposition of a quadrupedal robot into two bipedal systems with toe indices labeled and (b) the configuration of the quadruped; each leg has a point contact toe and three rotational joints with motors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 4.19 Top: Snapshots showing a two full steps of the walking gait on an outdoor lawn. Bottom: Running gait in the RaiSim simulation environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 4.20 Left three: Experimental (dashed transparent) and simulated (solid transparent) phase portraits for walking plotted against the desired values (solid). Right three: Simulated (transparent) vs. desired (solid) phase portraits for walking (red), hopping (green), and running (blue) behaviors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 xvi LIST OF TABLES Number Page 4.1 Computing performance of gait generation. This is performed on a Linux machine with an i7-6820HQ CPU @2.70 GHz and 16 GB RAM.106 4.2 Average torque inputs in experiments and simulations. . . . . . . . . 106 1 Chapter 1 INTRODUCTION 1.1 Background Legged locomotion has been long described as “controlled falling.” This characterization reveals the key challenge of legged systems — stability. To accomplish stable walking on various terrains, many modelling and control strategies have been proposed and explored since the 1980s. Meanwhile, the majority of control methods are built to provide stability guarantees for certain, linear, or rather simple dynamical systems (Aëšström and Kumar, 2014). Therefore, most work within legged locomotion involves some form of model simplification before designing a stabilizing controller. The most fundamental modeling technique used in robotics are rigid-body dynamics, which is defined as “the movement of systems of interconnected bodies under the action of external forces” (Tsai, 1999). While a robot (modeled by rigid-body dynamics) appears to be the central component of legged locomotion, numerous other subcomponents are coupled with the robot itself through interconnections, such as terrain dynamics, compliant dynamics, impact dynamics, etc. The complexity of controlling such systems grows exponentially as the mathematical model details. There is a famous observation in statistics (Box, 1976): “Since all models are wrong the scientist cannot obtain a "correct" one by excessive elaboration,” which also captures this challenge of robotics. Therefore, a multi-link rigid-body system, with compliant components that are no more complicated than linear coil springs, and rigid contact/impact model is widely accepted as the “correct” model for robotics. We refer to this system as the full-order system in this thesis. Among the approaches used in legged locomotion, a significant subset of the work lies in viewing walking dynamics as a reduction problem, wherein the complex real-world dynamics are assumed to be governed by the evolution of some reduced system. The linear invert pendulum model (LIPM) (S. Kajita, Kanehiro, et al., 2001; S. Kajita, Tani, and Kobayashi, 1990) assumes a bipedal system behaves as an invert pendulum with massless limbs. For high-performance behaviors such as running and hopping, a spring-loaded invert pendulum (SLIP) model is broadly applied to bipedal locomotion (Reinhard Blickhan and Full, 1993; R. Blickhan, n.d.; 2 Rezazadeh et al., 2015). It is also extended to the asymmetric-SLIP model to account for the heavy torso (Ioannis Poulakakis and Jessy W Grizzle, 2009). The centroidal dynamics (D. E. Orin, Goswami, and S.-H. Lee, 2013) captures the dominating effect of center of mass and angular momentum to control legged locomotion, including quadrupedal locomotion (Di Carlo et al., 2018). A prominent result of using simple models is the seminal work of Raibert’s hoppers and quadrupedal robots (M. Raibert and Tello, 1986). To maintain balance (an intuitive description of stability) and capture points, (Koolen et al., 2012) utilized a region to stabilize the LIPM-based systems, and the well-known Zero Moment Point (ZMP) method (Vukobratovic and Branislav Borovac, 2004) gives a robust but restrictive condition to prevent footrolling and further avoid falling. Yet these methods lack guarantees with respect to the full-order dynamics, and to mitigate erratic experimental performance, intensive parameter tuning is required. Other works investigate this dimensional reduction by performing design of locomotion on the passive dynamics of the system. This can improve model fidelity and represent more physical details of the system; relevent methods include hybrid zero dynamics (HZD) methods (E. R. Westervelt, J. W. Grizzle, Chevallereau, et al., 2007; A. Ames, 2014) and other optimization-based approaches (Dalibard et al., 2013). By considering the rigid-body dynamics as a continuous-time system and the rigid impact with the ground as a discrete-time system, walking can be modelled as a hybrid system (Barton and C. K. Lee, 2002). Correspondingly, the HZD framework was then invented as a means to reduce the stability problem of the high-dimensional walking dynamics to the lower-dimensional zero dynamics on the hybrid zero dynamics manifold. From the perspective of Lyapunov, stability of the overall hybrid system is formally guaranteed in (A. Ames, Galloway, et al., 2014b). Many hardware results have been accomplished in the domain of legged locomotion, such as walking (J. Reher et al., 2016; Sreenath, H.-W. Park, I. Poulakakis, and J. W. Grizzle, 2011) and running (Sreenath, H.-W. Park, I. Poulakakis, and J.W. Grizzle, 2013). 1.2 Contribution The contribution of this thesis is twofold. First, it enriches the application of HZD framework in the domains of 2D walking (Ambrose, W. Ma, C. Hubicki, et al., 2017), running (Wen-Loong Ma, Ayonga Hereid, et al., 2016; Wen-Loong Ma, Shishir Kolathaya, et al., 2017), monopedal 3 hopping (Ambrose, W. Ma, and A. D. Ames, 2021), walking with compliance (Jake Reher, Wen-Loong Ma, and Aaron D. Ames, 2019), and quadrupedal locomotion (W. Ma, K. A. Hamed, and A. D. Ames, 2019). Although the full-body hybrid dynamics of walking is believed to be “good enough” to represent the real-world hardware dynamics, we still face many uncertainties that can easily destroy stability, such as sensor noise, low-bandwidth compliant dynamics, etc. Therefore, (WenLoong Ma, Ayonga Hereid, et al., 2016) utilized direct-collocation methods to fast-finding periodic running gaits and formulated input-to-to state stability (ISS) analysis for the end result of robust running on DURUS-2D. See Fig. 1.1 for the development of the robots. Secondly, when solving the control problem of quadrupedal locomotion from the hybrid control perspective (W. Ma, K. A. Hamed, and A. D. Ames, 2019), the computational complexity and experimental robustness is not effective in comparison with state-of-art simple model-based approaches. Therefore, we take inspirations from the seminal work of Raibert’s virtual leg principle (M. Raibert, Blankespoor, et al., 2008), and characterize quadrupedal locomotion as a collection of two connected bipedal subsystems. Through dynamics decomposition (Wen-Loong Ma and Aaron D. Ames, 2020), we were able to efficiently solve for quadrupedal gaits within a few seconds. It is also applied to sloped terrain walking (W. -L. Ma, CsomayShanklin, and A. D. Ames, 2020). These results are theoretically justified using the notion of coupled control systems (CCS) in (W. Ma, Csomay-Shanklin, and A. D. Ames, 2021), and Lyapunov stability analysis was used to synthesize local optimal controllers in (W. -L. Ma, Csomay-Shanklin, S. Kolathaya, et al., 2021) for each decoupled bipedal systems. The goal of this thesis is to provide theoretical analysis for locomotion from Lyapunov’s perspective, and serve as a comprehensive introduction for experimental design of various robotic platforms (see Fig. 1.1 with a few examples). 1.3 Organization of Dissertation This section provides a brief overview of the contents of each chapter. Chapter 2: Constrained Rigid-Body Dynamics. This chapter details the modelling techniques and layout of the notations used for constructing the hybrid dynamics of legged locomotion. Using the conventional rigid-body dynamics and holonomic constraints, we can obtain the continuous-time equations of motion for the walking systems. Further we use plastic impact to describe the ground contact as a sim- 4 Figure 1.1: Robots developed to control in this thesis. From left to right: AMBER2 (2012-2014), AMBER3 (2015), AMBER3M (2016-2020), DURUS-2D (20152016), DURUS (2016), Cassie (2017-now), Vision60 v3.2 & v3.9 (2018-2020). ple discrete-time dynamics. An alternating sequence of continuous and discrete dynamics is then regarded as a hybrid system, which is then represented by a set of differential algebra equations (DAEs). These concepts will be explained using multiple robots including underactuated, fully-actuated, and overactuated systems. We will finish this chapter with an optimization formulation that is used throughout this thesis to find a desired gait for a simple hopping robot, which is defined as the (numerical) solutions to the hybrid system. Chapter 3: Controller Design for Bipedal Running. After formulating the dynamic equations for these legged systems, we are in place to control them. In this section, we introduce some key concepts to control legged locomotion from Lyapunov’s perspective. This includes the virtual constraints, desired outputs, tracking control and Lyapunov stability. We first formally pose the control problem as a trajectory optimization problem, where a closed-loop system can be defined and a solution (gait) to the closed-loop system is found. These will be detailed using DURUS, a 3D underactuated humanoid robot, as an example. We will also use AMBER3M walking on slippery surface as an example the show the scalability of the optimization for high-performance locomotion. Ideally, tracking these solutions on the system with the pre-defined controller equates controlling the system as desired, which is true as we verified in simulation. However, the real-world suffers from 5 uncertainty such as inaccurate measurement. We then utilize the concept of input to state stability to analyze the tracking noise from the phasing variable for bipedal running on DURUS-2D. With the formally provn improved robustness, we can then realize bipedal running experimentally. Chapter 4: Coupled Control System and Quadrupedal Applications. The aforementioned methods and application all build on full-order dynamics of these legged systems. While this is beneficial to realize high-performance robotics, it is not completely necessary. For example, multiple simplified models have been shown effective for quadrupedal locomotion. In this chapter we first show how to use offline trajectory optimization to generate closed-loop trajectory for complex systems such as the quadrupedal robots of interest. Then we construct the concept of coupled control systems, which focuses on the control and optimization problem from a perspective of viewing the system as a set of subsystems coupled through control. In a concrete example, a quadrupedal robot can be viewed as two connected bipedal robots coupled through holonomic constraints. In the construction, we provide the stability conditions in the sense of Lyapunov and synthesize two algorithms. First, we proposed an optimization framework to rapidly generate gaits for the bipedal subsystem that can be reconstructed to quadrupedal gaits. Then we used coupled control Lyapunov functions to control each subsystem to achieve the stability of the full-order system. The result is Vision 60, version 3.2 and 3.9, two robots of different size and weight that can walk robustly indoor and outdoor on rough terrains. 6 Chapter 2 CONSTRAINED RIGID-BODY DYNAMICS To formally construct the control problem of legged locomotion, we first convert the general differential geometry equations (DAE) formulation used in rigid-body dynamics into a control-friendly formulation — the ordinary differential equation (ODE) form 𝑥¤ = 𝑓 (𝑥) + 𝑔(𝑥)𝑢 — in this chapter. For this purpose, all internal variables need to be explicitly and uniquely determined by the state variables and control variables. Further, we employ the trajectory optimization technique to define and find numerical solutions to such systems. These solutions, i.e., gaits, are essentially what the controller is designed to drive the dynamics to in the content that follows. 2.1 Coordinates and Notations Figure 2.1: Configuration coordinates of AMBER robots, DURUS-2D running robot, DURUS, Cassie, and Vision 60, the quadrupedal robots. The target of this section is to define the state coordinates for robotic systems such as those in Fig. 1.1. As shown in Fig. 2.1, we model each robot as a multi-link system. 7 The configuration coordinate is denoted by 𝑞 ∈ Q ⊂ R𝑛 , where 𝑛 is the number of degrees of freedom (DOF) without considering any other constraints (refereed to as states-DOF). Following (Jessy W. Grizzle et al., 2014), we use the floatingbase convention for robotic systems. Each system starts from a body linage as the base (normally picked as the torso link, or the ground for manipulation problems), with 𝜉 ∈ R3 × SO(3) as the global Cartesian position and orientation of a frame attached to the body linkage. Then 𝑘-limbs are connected to the body, each of which has a few children links. These limbs are associated with local coordinates 𝜃, including both prismatic joint length and revolute joint angles. Additionally, each robot is assumed to have 𝑚 actuated joints and 𝑚 ≤ 𝑛. Here, “actuation” specifically refers to those joints included in the configuration coordinates that are driven by actuator modules, which are normally composed of a Brushless (BL) DC motor and a reduction gearbox. Concretely, we denote the input variable as 𝑢 ∈ U ∈ R𝑚 . In summary, we use the following convention: • 𝑞 = (𝜉 > , 𝜃 > ) > ∈ R𝑛 is used exclusively for a robot’s configuration coordinates, where 𝜉 is the floating-base coordinates, 𝜃 is the “shape” coordinates representing the rotaional and prismatic joints’ displacement, and their time¤ 𝜃. ¤ derivative is denoted as 𝑞, ¤ 𝜉, • 𝑦 is for system-level outputs, and 𝜂> = (𝑦 > , 𝑦¤ > ). • Normal form coordinates: (𝜂> , 𝑧> ) > , where 𝑧 is the zero dynamics coordinates. Notations. In this thesis, we denote that the set of natural numbers as N; the set of 𝑛 by 1 real vectors as R𝑛 , the set of non-negative real numbers as R+ , the set of 𝑚 by 𝑛 matrices as R𝑚×𝑛 , an 𝑛 by 1 vector whose elements are 1 as 1𝑛 , a proper dimensional vector whose elements are 0s as 0, an 𝑛 by 𝑛 identity matrix as 𝐼𝑛 , and an 𝑚 by 𝑛 zero matrix as 0𝑚×𝑛 . The Lie derivative of a function 𝑓 (𝑥) along the vector field 𝑔(·) is defined as 𝐿 𝑔(·) 𝑓 (𝑥) , 𝜕 𝑓 (𝑥) 𝑔(·). 𝜕𝑥 (2.1) Plus, some norm operators need to be defined. The Euclidean norm of a vector of proper dimension is |·|, and we take k𝑑 k ∞ , sup (|𝑑 (𝑡)|) . 𝑡≥0 (2.2) 8 The matrix norm induced by the Euclidean vector norm is k·k 2 , and the distance from a point (𝑥, 𝑧) to a periodic orbit O is defined as: k (𝑥, 𝑧)k O , 2.2 inf (𝑥 0 ,𝑧 0 )∈O |(𝑥, 𝑧) − (𝑥 0, 𝑧0)| . (2.3) Holonomic Constraints and System-Level Degrees of Freedom We are interested in scenarios when the robots are shown in Fig. 1.1 interact with the world through rigid contacts, which are modeled as some form of holonomic constraints (for example, unilateral constraints, see (R. M. Murray et al., 1994)). In classical mechanics, holonomic constraints are defined relations between the position variables (Arnold, 1989). In this thesis, we denote a holonomic constraint as a function in the form of ℎ(𝑞) ≡ c, (2.4) where c ∈ R𝑛ℎ is a constant, ℎ : R𝑛 → R𝑛ℎ , and the operator “≡”, identical equality of functions, means that ℎ(𝑞) = 0 is true for all 𝑡 ∈ I that 𝑞(𝑡) is defined on. Figure 2.2: Three fundamental contact problems modelled by holonomic constraints: a) point contact, ℎ(𝑞) ∈ R3 denotes the three-dimensional Cartesian position fixed to the world; b) line contact, ℎ(𝑞) ∈ R3+2 fixed one point of the contact line and the pitch, yaw angles of the body-fixed frame to the world; c) plane contact, ℎ(𝑞) ∈ R6 fixed the 6-dimensional body-fixed frame to the world. There are three fundamental contact problems in rigid-body dynamics: line contact, point contact, and plane contact. See Fig. 2.2. However, many complicated scenarios exist in reality, such as multiple contact points. Moving forward, we require that there be no redundancy when picking the representation of holonomic constraints. Mathematically, this means the Contact Jacobian, 𝐽ℎ (𝑞) , 𝜕ℎ(𝑞) ∈ R𝑛ℎ ×𝑛 , 𝜕𝑞 (2.5) is full rank. That is, Rank(𝐽ℎ ) = min(𝑛 ℎ , 𝑛). A simple example of redundancy is that in the line-contact problem shown in Fig. 2.2(b), either (𝑥 𝐴 , 𝑦 𝐴 , 𝑧 𝐴 , 𝜃 𝑦 , 𝜃 𝑧 ) ≡ 9 0, or (𝑥 𝐴 , 𝑦 𝐴 , 𝑧 𝐴 , 𝑥 𝐵 , 𝑦 𝐵 , 𝑧 𝐵 ) ≡ 0 can represent the contact condition. But the second formulation has one degree of redundancy, which is the result of rigid-body formulation: the distance between point A and B is a constant. Hence, we call Rank(𝐽ℎ ) the degree of contact constraint. Recall the states-DOF 𝑛 in Sec. 2.1, these holonomic contact constraints obviously change the system-level degrees of freedom (abbreviated as system-DOF). We then denote the total degrees of freedom as system-DOF , 𝑛 − Rank(𝐽ℎ ). (2.6) In consistence with this definition, we have 𝑛 ≥ 𝑛 ℎ , and thus Rank(𝐽ℎ ) = 𝑛 ℎ . Furthermore, this equality constraint ℎ(𝑞) is enforced via contact forces (also known as constraint wrench) 𝜆 ∈ R𝑛ℎ . 2.3 Continuous-Time Dynamics The continuous-time dynamics of a robotic system will be expressed as a set of differential algebra equations (DAEs), which is composed of two parts: the equations of motion (EOMs), and some algebraic equations. Concretely, 𝐷 (𝑞) 𝑞¥ + 𝐶 (𝑞, 𝑞) ¤ + 𝑔(𝑞) = 𝐵𝑢 + 𝐽ℎ> (𝑞)𝜆, s.t. ℎ(𝑞) ≡ 0. (2.7) (2.8) We can obtain the EOMs in (2.7) by using the Euler–Lagrange equation, see (R. M. Murray et al., 1994; M. W. Spong, 1989; R. Featherstone, 2008). Here, 𝐷 (𝑞) ∈ R𝑛×𝑛 represents the mass-inertia matrix, and is symmetric positive-definite; 𝐶 (𝑞, 𝑞) ¤ ∈ R𝑛×𝑛 is the Coriolis matrix, and 𝑔(𝑞) ∈ R𝑛 contains the gravity terms. For the sake of notational simplicity, we denote 𝐻 (𝑞, 𝑞) ¤ , 𝐶 (𝑞, 𝑞) ¤ + 𝑔(𝑞) throughout this thesis; the actuation matrix 𝐵(𝑞) ∈ R𝑛×𝑚 maps the motor input 𝑢 from the control space U to the configuration space Q, which is a constant matrix; and 𝐽ℎ ∈ R𝑛ℎ ×𝑛 is the contact Jacobian, with 𝜆 ∈ Rℎ ℎ the contact force. Remark 1. We note a special property of the mass-inertia matrix 𝐷 (𝑞) when using the floating-base coordinate — the branch-induced sparsity. As detailed in (Roy Featherstone, 2010), there is a sparse structure for a multi-limbs robotic system such as a humanoid, or a quadrupedal robot. For example, as shown in Fig. 2.3, the 10 Figure 2.3: Branched chain structure of a bipedal system—AMBER-3M, and a quadrupedal system—Vision 60. inertia matrix for a bipedal system and a quadrupedal system has the structure of  𝐷𝜉 𝐷𝑏 l   𝐷 biped =  𝐷 > 𝐷 left  𝑏l 𝐷 > 0  𝑏r  𝐷𝜉 𝐷𝑏 𝐷𝑏 𝐷𝑏 𝐷𝑏  0 1 2 3   >   0 0  𝐷 𝑏r   𝐷 𝑏0 𝐷 𝐿0 0    . 0 𝐷 0 0 0  , 𝐷 quad =  𝐷 > 𝐿 1  𝑏1    0 0 𝐷 0 𝐷 >  𝐷 right  𝐿 2 𝑏  1  𝐷 > 0 0 𝐷 𝐿 3   𝑏3 0 (2.9) This is particularly useful when formulating the coupled control problem of quadrupedal locomotion in Chapter 4. 2.4 Underactuated System To eventually convert the DAEs given by (2.7)-(2.8) to an ODE form, we need to explicitly solve the internal variable 𝜆 with 𝑞, 𝑞, ¤ 𝑢. But first, we need to systematically define the notion of underactuated, fully-actuated, and overactuated mechanical systems and their degrees. Similar to how the contact force enters the EOMs of (2.7), 𝐽 (𝑞) >𝜆, the joint torque (or linear actuator’s force) enters the system in the same way, 𝐵𝑢. Indeed, both are obtained via the principle of virtual work (Lanczos, 1986) (formally extended to dynamical systems as the Lagrange–d’Alembert principle, see (Arnold, 1989)), which is a specific application of the method of Lagrange multiplier (R. M. Murray et al., 1994, Chapter 6). Therefore, just as a complicated contact situation can cause redundancy among the contacts, the joint torque 𝑢 could also “fight” with the contact 11 force 𝜆 (see Fig. 2.4(c) where the control variable 𝑢 fights with the ground reaction force), which is the overactuation problem. Figure 2.4: Simple examples of overactuation in the presence of holonomic constraints: (a,b) sliding on a track; (c) pushing against a wall; (d) a quadrupedal robot standing on two feet. Define the following matrix as the Actuation Jacobian: " # 𝐵> 𝐵A , ∈ R (𝑚+𝑛ℎ )×𝑛 . 𝐽ℎ (2.10) Note that Rank(𝐵A ) ≤ 𝑛. Definition 1. If Rank(𝐵A ) < 𝑛, we say the full system is under-actuated, and 𝑛U = 𝑛 − Rank(𝐵A ) (2.11) is defined as the degrees of under-actuation for the full system. Definition 2. If Rank(𝐵A ) = 𝑛, and 𝑚 + 𝑛 ℎ > 𝑛, the system is overactuated, and 𝑛O = 𝑚 + 𝑛 ℎ − Rank(𝐵A ) is defined as the degrees of overactuation for the full system. Example. • The system in Fig. 2.4 (a) has dynamics:    1 −1 𝑚 𝑥¥ = 𝑢 2 + 𝜆         𝑚 𝑦¥ = 𝑢 1 − 𝜆 ⇒ 𝐵A = 1 0  ⇒ 𝑛O = 2 + 1 − 2 = 1.      0 1    𝑥¥ − 𝑦¥ = 0    • The system in Fig. 2.4 (b) has dynamics:    1 0 𝑚 𝑥¥ = 𝑢 2         𝑚 𝑦¥ = 𝑢 1 − 𝜆 ⇒ 𝐵A = 1 0 ⇒ 𝑛O = 2 + 1 − 2 = 1.      0 1   𝑦¥ = 0    (2.12) 12 Full system overactuation can be interpreted as: • Control inputs fight with each other (Fig. 2.4(a)); • Control inputs fight with constraint force 𝜆 (Fig. 2.4(b,c)). Definition 3. A system that is neither underactuated nor overactuated is said to be fully actuated. Definition 4. If the actuation matrix 𝐵A is not full rank (degenerate), i.e., Rank(𝐵A ) < min(𝑚 + 𝑘, 𝑛), the system is internally overactuated. The degree of internal overactuation is defined as 𝑛IO = min(𝑚 + 𝑛 ℎ , 𝑛) − 𝑅𝑎𝑛𝑘 (𝐵A ). The internally overactuation is often caused by some closed-chain structure, such as the double-support scenario of the quadrupedal locomotion with two-point contacts on the toes. As shown in Fig. 2.4(d), the red line indicates the closed-chain inside the system. Also, as shown in Fig. 2.4(e), right after an impact, the nonstance foot’s spring needs some time to recover to the normal length while the stance spring is compressing itself. This results in a double-support domain, which also creates an internal chained structure and internal overactuation. To uniquely determine 𝜆 with 𝑞, 𝑞, ¤ 𝑢, we require the dynamics given by (2.7)-(2.8) have no overactuation or internally overactuation. In other words, 𝐽 ℎ , 𝐵, and 𝐵A are all full-rank, and 𝑛O , 𝑛IO = 0. This can be achieved by enforcing additional algebraic equations, such as setting some motor inputs to be 0. The reason is that a violation of this requirement will result in an underdetermined problem when solving for 𝜆. This will be seen in the next section. In summary, we demand our problem to be either underactuated or fully-actuated. Remark 2. It is unnecessary to have 𝐽ℎ , 𝐵, and the actuation matrix full-rank in realworld applications. For example, an all-wheel-drive (AWD) vehicle is apparently overactuated. Although it has no mathematical difference from a rear-wheel-drive vehicle in terms of rigid-body dynamics, the control robustness differs significantly. Further, a degenerate matrix formulation will make the stability analysis later using the traditional control language over complicated. This is beyond the scope of this thesis, but interested readers can find the optimization-based analysis for such problems in the literature of control allocation such as (Johansen and Fossen, 2013). 13 2.5 The ODE Form of Control-Affine Systems. We now convert a DAE system given by (2.7)-(2.8) into a ODE system. We first note that (2.8) is an equality constraint placed on 𝑞(𝑡) ∀ 𝑡 ∈ I. To explicitly solve 𝜆, we take the derivatives of (2.8) to get: 𝐽ℎ (𝑞) 𝑞¤ = 0; (2.13) 𝐽¤ℎ (𝑞, 𝑞) ¤ 𝑞¤ + 𝐽ℎ (𝑞) 𝑞¥ = 0. (2.14) Therefore, (2.7)-(2.8) became    𝐷 (𝑞) 𝑞¥ + 𝐻 (𝑞, 𝑞)  ¤ = 𝐵𝑢 + 𝐽ℎ>𝜆 . (2.15)   𝐽¤ℎ (𝑞, 𝑞) ¤ 𝑞¤ + 𝐽ℎ (𝑞) 𝑞¥ = 0  When compute the constraint wrench in (2.7), we can easily derive the following (𝐽ℎ 𝐷 −1 𝐽ℎ> )𝜆 = 𝐽ℎ 𝐷 −1 𝐻 − 𝐽ℎ 𝐷 −1 𝐵𝑢 − 𝐽¤ℎ 𝑞¤   ⇒ 𝜆 = (𝐽ℎ 𝐷 −1 𝐽ℎ> ) −1 𝐽ℎ 𝐷 −1 𝐻 − 𝐽ℎ 𝐷 −1 𝐵𝑢 − 𝐽¤ℎ 𝑞¤ , (2.16) where we suppressed the arguments of 𝐷 (𝑞), 𝐻 (𝑞, 𝑞), ¤ 𝐽ℎ (𝑞) and 𝐽¤ℎ (𝑞, 𝑞). ¤ Note that if 𝐽ℎ is not full rank, 𝐽ℎ 𝐷 −1 𝐽ℎ𝑇 becomes a singular matrix, in which case we cannot uniquely determine 𝜆. Plugging (2.16) into (2.7), we have   𝐷 𝑞¥ + 𝐻 = 𝐵𝑢 + 𝐽ℎ> (𝐽ℎ 𝐷 −1 𝐽ℎ> ) −1 𝐽ℎ 𝐷 −1 𝐻 − 𝐽ℎ 𝐷 −1 𝐵𝑢 − 𝐽¤ℎ 𝑞¤   ⇒ 𝑞¥ = −𝐷 −1 𝐻 + 𝐷 −1 𝐽ℎ> (𝐽ℎ 𝐷 −1 𝐽ℎ> ) −1 𝐽ℎ 𝐷 −1 𝐻 − 𝐽¤ℎ 𝑞¤ + | {z } drift vector   𝐼𝑛 − 𝐷 −1 𝐽ℎ> (𝐽ℎ 𝐷 −1 𝐽ℎ> ) −1 𝐽ℎ 𝐷 −1 𝐵𝑢. (2.17) Another form often used in the control + robotics literature is the manipulator dynamics:     𝐷 𝑞¥ + 𝐻 − 𝐽ℎ> (𝐽ℎ 𝐷 −1 𝐽ℎ> ) −1 𝐽ℎ 𝐷 −1 𝐻 − 𝐽¤ℎ 𝑞¤ = 𝐼𝑛 − 𝐽ℎ> (𝐽ℎ 𝐷 −1 𝐽ℎ> ) −1 𝐽ℎ 𝐷 −1 𝐵 𝑢. | {z } | {z } , 𝐻¯ (𝑞,𝑞) ¤ ¯ , 𝐵(𝑞) (2.18) Further, we define the state space X = 𝑇 Q ⊆ R2𝑛 with the state vector: " # 𝑞 𝑥, , 𝑞¤ 14 where 𝑇 Q is the tangent bundle of the configuration space Q. Finally, we have the dynamical system that we wish to design controllers for: 𝑥(𝑡) ¤ = 𝑓 (𝑥(𝑡)) + 𝑔(𝑥(𝑡))𝑢, which will be abbreviated as 𝑥¤ = 𝑓 (𝑥) + 𝑔(𝑥)𝑢 (2.19) with " 𝑓 (𝑥) , 𝑞¤ −𝐷 −1 𝐻 + 𝐷 −1 𝐽ℎ> (𝐽ℎ 𝐷 −1 𝐽ℎ> ) −1 𝐽ℎ 𝐷 −1 𝐻 − 𝐽¤ℎ 𝑞¤ " # 0 . 𝑔(𝑥) ,
𝐼𝑛 − 𝐷 −1 𝐽ℎ> (𝐽ℎ 𝐷 −1 𝐽ℎ> ) −1 𝐽ℎ 𝐷 −1 𝐵 2.6 #
, (2.20) Hybrid System Having established the continuous-time dynamics for a particular contact scenario, we are in the position to define the hybrid model for walking. The motivation of using hybrid dynamics is straightforward: for legged locomotion, when the contact condition changes — such as adding or removing some contacts — the input (contact force, motor torque) changes, hence the dynamics change. The combination of these “domains”1 of dynamics results in a hybrid automata (Barton and C. K. Lee, 2002), which is given by the following definition. Definition 5. A hybrid control system is defined to be the tuple: H C = (Γ, D, S, Δ, F), (2.21) where, • Γ , {V, E} is a directed graph (or digraph) (see (Mesbahi and Egerstedt, 2010)), with V the set of domain indices and E the set of edges between two domains. Their elements are denoted as 𝑣 ∈ V, 𝑒 ∈ E, where 𝑒 , (𝑣 → 𝑤) with 𝑣, 𝑤 ∈ V; • D , {D𝑣 } 𝑣∈V is a set of admissible domains, where the state space is defined on; 1 “Domain” is interchangeably used with “phase” in this thesis. 15 • S , {S [𝑒]} 𝑒∈E is a set of guards (or switching surfaces), where S ⊂ D𝑣 ∩D𝑤 , with 𝑣, 𝑤 ∈ V ; • Δ , {Δ[𝑒]} 𝑒∈E is a set of reset maps that maps the pre-edge states to the post-edge states. This will be detailed in the next section; • F , {F𝑣 } 𝑣∈V is a set of control systems that determine the continuous-time dynamics on a domain D𝑣 . Discussion. In application, we denote a specific contact condition as ℎ𝑣 (𝑞) ≡ 𝑐 𝑣 for each domain D𝑣 with 𝑣 ∈ V. Moreover, the satisfaction of this constraint defines the state space, i.e., the continuous dynamics on domain D𝑣 is given by 𝑥¤ = 𝑓𝑣 (𝑥) + 𝑔𝑣 (𝑥)𝑢, 𝑥 ∈ D𝑣 . (2.22) The guard S𝑒 associated an edge 𝑒 , (𝑣 → 𝑤) with 𝑣, 𝑤 ∈ V is normally defined by the change of states from 𝑥 ∈ D𝑣 to 𝑥 ∈ D𝑤 . The control law {𝐹𝐺 𝑣 } 𝑣∈V will be the main topic of the next chapter. Figure 2.5: Directed graph for two robotic systems. We now give two examples of formulating the hybrid model for low-dimensional legged systems. A hopping machine. A canonical example of the hybrid dynamical system is hopping robots. In the simplified vertical hopping machine shown in Fig. 2.5(a), the robot hops up and down only. It then has two dynamic domains, the stance domain Ds , where the foot (bottom link) stays on the ground, and the flight domain Df when the foot left the ground. Formally, this is defined as Ds , {𝑥 ∈ 𝑇 Q | 𝑧foot = 𝑧¤foot = 0}, (2.23) Df , {𝑥 ∈ 𝑇 Q | 𝑧foot ≥ 0}, (2.24) 16 where 𝑧foot is the height of the foot. We then denote 𝜆 foot as the ground reaction force exerting on the foot. Note that 𝜆 foot can be explicitly solved with 𝑥, 𝑢 using (2.16). Therefore, we have the domain indices V , {s, f}, and the set of edges as E , { 𝑒 s→f , 𝑒 f→s }, |{z} |{z} “lift” (2.25) “impact” where 𝑒 s→f marks the event that foot leaves from the ground, and 𝑒 f→s is the event that foot touch the ground from the air. Thus, we can use the boundary condition of each domain to define the guard conditions as: S [𝑒 s→f ] , {(𝑥, 𝑢) ∈ Ds | 𝜆 foot (𝑥, 𝑢) = 0}, (2.26) S [𝑒 f→s ] , {𝑥 ∈ Df | 𝑧foot = 0, 𝑧¤foot < 0}. (2.27) The transition dynamics for each edge 𝑒 ∈ E, are called lift dynamics Δ[𝑒 s→f ], and the impact dynamics Δ[𝑒 f→s ] will be detailed in the next section. AMBER-3M walks on slippery surfaces. The cyclic directed graph of the multidomain hybrid system for walking on a slippery surface is shown in Fig. 2.5(b). The solid lines are for transitions without non-stance foot impact events, and dashed lines are for transitions with impact events. The subscript s is for walking with slippage, 0 is walking without slippage, and the superscript 𝐼 marks that the edge is equipped with an impact event. As shown, we have two types of dynamic domains: the stick domain D0 , where the contact foot stands on the ground without slippage, and the slip domain Ds , where the contact foot slides the ground. Formally, this is given as 𝑧 𝑠 = 𝑧¤𝑠 = 𝑥¤ 𝑠 = 0, 𝑧 𝑛𝑠 ≥ 0, |𝜆0𝑥 | ≥ 𝜇𝜆0𝑧 }, (2.28) Ds = {(𝑥, 𝑢) ∈ 𝑇 Q × U | 𝑧 𝑠 = 𝑧¤𝑠 = 0, 𝑥¤ 𝑠 ≠ 0, 𝑧 𝑛𝑠 ≥ 0, 𝜆 𝑠𝑧 ≥ 0}, (2.29) D0 = {(𝑥, 𝑢) ∈ 𝑇 Q × U | where 𝑥 𝑠 , 𝑧 𝑠 represent the Cartesian position of the stance foot along the horizontal and vertical axes, accordingly; and 𝑧 𝑛𝑠 is the height of the nonstance foot, 𝜆 𝑥 , 𝜆 𝑧 are the ground reaction forces along the horizontal and vertical axes. Hence, we have the domain indices V = {0, s}. Correspondingly, we have the set of edges 𝐼 𝐼 𝐼 𝐼 E = {𝑒 0→s , 𝑒 s→0 , 𝑒 s→s , 𝑒 0→0 , 𝑒 0→s , 𝑒 s→0 , 𝑒 s→s }. (2.30) The superscript “𝐼” denotes transition via impact, whereas its absence denotes stick ↔ slip transitions. We now define the guards, which are switching surfaces or conditions for transition between domains. The first guards are associated with the 17 smooth transitions between sticking and slipping domains: S [𝑒 0→s ] = {(𝑥, 𝑢) ∈ D0 | |𝜆0𝑥 | = 𝜇𝜆0𝑧 }, S [𝑒 s→0 ] = {(𝑥, 𝑢) ∈ Ds | 𝑥¤ 𝑠 = 0, |𝜆0𝑥 | ≤ 𝜇𝜆0𝑧 }, S [𝑒 s→s ] = {(𝑥, 𝑢) ∈ Ds | 𝑥¤ 𝑠 = 0, |𝜆0𝑥 | > 𝜇𝜆0𝑧 }. Note that the last transition above associated with 𝑒 s→s is reversal of slip direction (cf. (Gamus and Yizhar Or, 2015)). The guards corresponding to transitions that involve sticking or slipping impacts are defined as: 𝐼 S [𝑒 0→0 ] = {(𝑥, 𝑢) ∈ D0 | 𝑧 𝑛𝑠 = 0, 𝑧¤𝑛𝑠 < 0, |Λ0𝑥 | ≤ 𝜇Λ0𝑧 }, 𝐼 S [𝑒 s→0 ] = {(𝑥, 𝑢) ∈ Ds | 𝑧 𝑛𝑠 = 0, 𝑧¤𝑛𝑠 < 0, |Λ0𝑥 | ≤ 𝜇Λ0𝑧 }, 𝐼 S [𝑒 0→s ] = {(𝑥, 𝑢) ∈ D0 | 𝑧 𝑛𝑠 = 0, 𝑧¤𝑛𝑠 < 0, |Λ0𝑥 | > 𝜇Λ0𝑧 }, 𝐼 S [𝑒 s→s ] = {(𝑥, 𝑢) ∈ Ds | 𝑧 𝑛𝑠 = 0, 𝑧¤𝑛𝑠 < 0, |Λ0𝑥 | > 𝜇Λ0𝑧 }, where Λ𝑥 , Λ𝑧 are the impact impulses along the horizontal and vertical axes. These guards represent the conditions for sticking or slipping impacts as described above. Note that the overall non-smooth frictional dynamics may have special degenerate cases where the solution is inconsistent, indeterminate, or singular. These rare cases are know as Painlevé paradox (Champneys and Várkonyi, 2016; Yizhar Or, 2014), and lie beyond the scope of this work. We remark that the sequence of motion in Fig. 2.5 that is marked by red color, has been controlled (see Sec. 3.3) and realized on the robot AMBER-3M, in (Wen-Loong Ma, 2019a). 2.7 Discrete Transitions For legged locomotion, a change of contact condition2 will result in either an event of lift or impact, or simultaneously both. We can characterize this “jump” between domains as discrete transitional events, which will be detailed in this section. Impact-transition dynamics. Legged locomotion inevitably includes foot-impacts with the ground. This process can be modeled through complicated terrain dynamics such as the impact on granular media (Li, Zhang, and Goldman, 2013; A. H. Chang et al., 2017) and elastic or inelastic collisions. In this work, for the mathematical 2 This not only includes foot impacts with the ground but also whenever a constraint ((2.8)) changes. For example, a hard stop of the spring vibration. 18 consistency with the assumption that the robots stand on the rigid ground — based on which the holonomic constraints can be defined — we model the foot impact as the perfect inelastic collision (also regarded as plastic impact in this thesis). Therefore, it obeys the conservation of momentum. And the positional variable remains unchanged through the event of an impact, and the velocity terms jump from the pre-impact quantity to the post-impact state. Formally, we model the impact dynamics from domain D𝑤 to domain D𝑣 — the transition dynamics on the edge 𝑒 𝑤→𝑣 with 𝑣, 𝑤 ∈ V — as:    𝑞 +𝑒 = 𝑞 −𝑒  ,   𝑞¤ +𝑒 = Δ[𝑒 𝑤→𝑣 ] (𝑞 −𝑒 ) 𝑞¤ −𝑒  (2.31) where 𝑞 +𝑒 , 𝑞¤ +𝑒 are the pre-event states of the edge 𝑒 , 𝑒 𝑣→𝑤 , i.e., these are the initial conditions of domain D𝑣 ; 𝑞 −𝑒 , 𝑞¤ −𝑒 are the post-event states of the edge 𝑒 = 𝑒 𝑣→𝑤 , i.e., these are the final conditions of domain D𝑤 . Note that we shorted the notation of 𝑒 𝑤→𝑣 to 𝑒 for simplicity in this section. We now detail the transitional map Δ[𝑒 𝑤→𝑣 ]. This map is obtained first via the law of momentum conservation: 𝐷 (𝑞 −𝑒 )( 𝑞¤ +𝑒 − 𝑞¤ −𝑒 ) = 𝐽ℎ 𝑣 (𝑞 −𝑒 ) > Λ𝑒 , (2.32) where Λ𝑒 ∈ R𝑛ℎ𝑣 is the impact impulse exerted on the contact surface. Entering domain D𝑣 means that the post-impact holonomic condition ℎ𝑣 (𝑞 +𝑒 ) ≡ 𝑐 𝑣 also holds. This yields: 𝐽ℎ 𝑣 (𝑞 +𝑒 ) 𝑞¤ +𝑒 = 0. Combining (2.31)-(2.33), we have the impact map as: " # " # −1 " # 𝑞¤ +𝑒 𝐷 (𝑞 −𝑒 ) −𝐽ℎ 𝑣 (𝑞 −𝑒 ) > 𝐷 (𝑞 − ) 𝑞¤ − = , Λ𝑒 𝐽ℎ 𝑣 (𝑞 −𝑒 ) 0 0 (2.33) (2.34) which simultaneously gives the impulse and post-impact states, and this is referred to as the impact dynamics. As can be seen, this jump of states from 𝑞¤ −𝑒 to 𝑞¤ +𝑒 results in a discrete transition. Additionally, removing some subset of the constraints defined on the previous domain, i.e., lifting, does not result in any impulse. Hence from (2.32), we then have the lift transition dynamics as:    𝑞 +𝑒 = 𝑞 −𝑒    𝑞¤ +𝑒 = 𝑞¤ −𝑒  . (2.35) 19 We now unify (2.32) and (2.35) as the transition dynamics using state vectors. This is given as 𝑥 𝑣+ = Δ[𝑒 𝑤→𝑣 ] (𝑥 𝑤− ). (2.36) The more general transition dynamics that involve impact with the slippery condition can be found in (W. Ma, Y. Or, and A D. Ames, 2019). 2.8 Solution to Hybrid Dynamics Having established the multi-domain hybrid dynamics of a legged system as: H ,    𝑥¤ = 𝑓𝑣 (𝑥) + 𝑔𝑣 (𝑥)𝑢  𝑥 ∈ D𝑣 , ∀𝑣 ∈ V   𝑥 + = Δ[𝑒] (𝑥 − )  𝑥 − ∈ S [𝑒], ∀𝑒 ∈ E , (2.37) we hereby define the solutions to a multi-domain hybrid system and provide an optimization method to find the numerical solution. This section is an extension of the single-domain hybrid system given by (Aaron D. Ames et al., 2017). Definition 6. A solution (execution) to the hybrid system H is a tuple: 𝜒 = (V, I, C), (2.38) where, • V = (𝑣 1 , 𝑣 2 , . . .) with 𝑣 𝑖 ∈ V and 𝑖 ∈ N+ , is a finite or countably infinite domain sequence3 , and we denote its cardinality as 𝑁. • I = (𝐼1 , 𝐼2 , . . .) is a sequence of the time interval 𝐼𝑖 with 𝑖 ∈ N+ , for each executed domain 𝑣 𝑖 . For each 𝑣 𝑖 ∈ V, 𝐼𝑖 is defined as: – 𝐼1 = [0, 𝑇1 ], if 𝑇1 ≠ ∞, otherwise 𝐼𝑖 = [0, ∞), – 𝐼𝑖 = [𝑇𝑖−1 , 𝑇𝑖 ], if 𝑣 𝑖−1 , 𝑣𝑖 ∈ V and 𝑇𝑖 ≠ ∞, – 𝐼 𝑁 = [𝑇𝑁−1 , 𝑇𝑖 ), if 𝑖 > 1 and 𝑇𝑖 = ∞ • C = ({𝑥 1 (𝑡), 𝑢 1 (𝑡)}, {𝑥 2 (𝑡), 𝑢 2 (𝑡)}, . . .) is a sequence of continuous trajectories for the states 𝑥𝑖 (𝑡) and control inputs 𝑢𝑖 (𝑡) that satisfies 𝑥¤𝑖 (𝑡) = 𝑓𝑣 𝑖 (𝑥𝑖 (𝑡)) + 𝑔𝑣 𝑖 (𝑥𝑖 (𝑡))𝑢𝑖 (𝑡), ∀ 𝑡 ∈ 𝐼𝑖 . We additionally require that for each 𝑣 𝑖 ∈ V, 3 Note that unlike a set, 𝑣 𝑖 and 𝑣 𝑗 with 𝑖 ≠ 𝑗 can be the same, i.e., the same domain can appear more than once in the solution’s domain sequence. Further, note the sequence of domains V is different from the set of domains V from Def. 5. 20 – 𝑥𝑖 (𝑡) ∈ D𝑣 𝑖 , ∀ 𝑡 ∈ 𝐼𝑖 , – 𝑥𝑖 (𝑇𝑖 ) ∈ S [𝑒 𝑣 𝑖 →𝑣 𝑖+1 ] and Δ[𝑒 𝑣 𝑖 →𝑣 𝑖+1 ] (𝑥𝑖 (𝑇𝑖 )) = 𝑥𝑖+1 (𝑇𝑖 ) if 𝑣 𝑖+1 ∈ V. We regard 𝑥 0 , 𝑥 1 (0) the initial condition of 𝜒. A mixed sequence of walking tasks of a bipedal robot can serve as a good example of this definition. As shown in Fig. 2.6, we can define four types of domain: D0 for standing still, D 𝑓 for walking on a flat surface, D𝑢 for walking uphill, D𝑑 for walking downhill. Hence we have the set of domain as D = {D 𝑓 , D𝑢 , D𝑑 }, and the set of guards S = {S [𝑒 𝑓 → 𝑓 ], S [𝑒 𝑓 →𝑢 ], S [𝑒 𝑢→ 𝑓 ], S [𝑒 𝑢→𝑢 ], S [𝑒 𝑓 →𝑑 ], S [𝑒 𝑑→ 𝑓 ], S [𝑒 𝑑→𝑑 ]}. We now assume there exists a solution to this hybrid system such that the robot starts from standing still, then walk two steps on flat ground, two steps uphill, two steps on the hill, then keeps walking on the flat ground. Therefore according to Def. 6, we have domain sequence as V = (D 𝑓 , D 𝑓 , D𝑢 , D𝑢 , D 𝑓 , D 𝑓 , D𝑑 , D𝑑 , D 𝑓 , D 𝑓 , D 𝑓 , . . .). Correspondingly, we can obtain the sequence of smooth trajectories C. Figure 2.6: A mixed sequence of walking tasks of a bipedal robot: flat ground walking, uphill walking, downhill walking, flat ground walking. On the right is the directed graph, where the arrows represent the corresponding edges. Each edge is associated with transition dynamics. 2.9 Numerical Optimization As given in (2.38), a solution to the hybrid dynamics is not necessarily a physically realizable trajectory. For example, the trajectory for the torque input 𝑢(𝑡) ∀𝑡 may violate the actuator’s physical limitation. This section introduces the numerical optimization formulation of finding the solution to a nonlinear system in (2.37) that respects some inequality constraints. We will apply this formulation throughout this thesis. The history of finding solutions to nonlinear dynamics using numerical optimization is rich in all engineering areas. An exhaustive review of numerical optimization can be found in (Betts, 1998). We want to find a solution as (2.38) that also satisfies some inequality constraints in this section. In particular, we employ direct collocation method (Kelly, 2017; Ayonga Hereid, Shishir Kolathaya, et al., 21 2014) to accomplish this goal. This method essentially transcribes the solutionfinding problem into a set of algebraic equations via collocation schemes. This is an implicit Runge-Kutta method to solve the system dynamics simultaneously. There are many types of collocation schemes, but we will focus on a specific method: the local direct collocation method. Unlike the single shooting method for the initial value problems (IVP) (Kiehl, 1994), where the solution 𝑥(𝑡) ∀𝑡 is obtained explicitly by integrating the dynamics from 𝑡 = 𝑡 0 , direct collocation formulation approximates the solution 𝑥(𝑡) implicitly by piecewise polynomials along the evolution of the nonlinear dynamics. We first present the algorithm to find the solution 𝑥(𝑡) ∀ 𝑡 ∈ 𝐼𝑖 using defect constraints. The basic procedures are: Step1. We begin by defining a sequence of 𝐾 + 1 discrete nodes along the time span of the trajectory: 𝑇𝑖−1 = 𝑡 0 < 𝑡1 < 𝑡2 < · · · < 𝑡 𝐾 = 𝑇𝑖 , (2.39) which forms the basis of our discrete representation of the continuous dynamics, and we denote 𝛿𝑡 𝑘 = 𝑡 𝑘+1 − 𝑡 𝑘 , where 𝑘 ∈ (0, 1, . . . , 𝐾𝑖 ). Note that 𝐾𝑖 depends on the domain 𝑣 𝑖 , but we will omit the subscript 𝑖 for the ease of notation. Step2. We use a piecewise Hermite interpolation (cubic) polynomial to approximate the solution over each subinterval [𝑡 𝑘 , 𝑡 𝑘+1 ] using the following cubic polynomial: 𝑥(𝑡) = 𝐶0 + 𝐶1 𝑠 + 𝐶2 𝑠2 + 𝐶3 𝑠3 (2.40) 𝑡−𝑡 𝑘 𝑠 = 𝑡 𝑘+1 −𝑡 𝑘 ∈ [0, 1] is the normalized time within the subinterval. The four coefficients 𝐶0 , 𝐶1 , 𝐶2 , 𝐶3 are determined by the two boundary values 𝑥 𝑘 , 𝑥 𝑘+1 , which are given as decision variables of the optimization algorithm. Their derivatives 𝑥¤𝑖 and 𝑥¤𝑖+1 are computed by the continuous dynamics in (2.37). Step3. Using these coefficients, we can compute the interpolated value of 𝑥 at the center of the subinterval, i.e., the collocation point, as 𝑥¯𝑐𝑘 = (𝑥 𝑘 + 𝑥 𝑘+1 )/2 + 𝛿𝑡 𝑘 ( 𝑥¤ 𝑘 − 𝑥¤ 𝑘+1 )/8. (2.41) Similarly, the slope of the cubic polynomial at the center point is 𝑥¤̄𝑐𝑘 = −3(𝑥 𝑘 − 𝑥 𝑘+1 )/2𝛿𝑡 𝑘 − ( 𝑥¤ 𝑘 + 𝑥¤ 𝑘+1 )/4. (2.42) 22 Step4. The defect constraint at the center of the subinterval is defined as the difference between the interpolated slope and the first-order derivatives of states at the center of the subinterval computed by the system dynamics, i.e., 𝜁 𝑘 = 𝑓 ( 𝑥¯𝑐𝑘 ) + 𝑔( 𝑥¯𝑐𝑘 )𝑢 𝑐𝑘 − 𝑥¤̄𝑐𝑘 . (2.43) We now have the most important statement in this section: ( 𝑥¯𝑐𝑘 , 𝑥¤̄𝑐𝑘 )𝑇 satisfies the dynamic constraints (C1) if and only if 𝜁 𝑘 = 0. In fact, (2.43) is a nonlinear equality constraint posed on the nodes 𝑡 𝑘 and 𝑡 𝑘+1 . Step5. Stack these defect constraints into a vector as  𝜁0     .   . 𝑍𝑖 ,  .  ,  𝐾−1   𝜁    (2.44) in which 𝑍 becomes a function of the discretized states x𝑖 , (𝑥 0 , 𝑥 1 , . . . , 𝑥 𝐾 ), and control inputs u𝑖 , (𝑢 0 , 𝑢 0𝑐 , 𝑢 1 , 𝑢 1𝑐 , 𝑢 2 , . . . , 𝑢 𝐾 ). Then the goal is to “find” a set of discretized states x𝑖 so that the defect Δ𝑖 = 0 for domain 𝑣 𝑖 ∈ V. As a result, the interpolated piecewise polynomials are an valid approximation of the solution 𝑥(𝑡) ∀ 𝑡 ∈ 𝐼𝑖 . Note that there are two ways to improve the smoothness of control inputs: 𝑢 𝑐𝑘 ≡ 𝑢 𝑘 and 𝑢 𝑐𝑘 ≡ (𝑢 𝑘 + 𝑢 𝑘+1 )/2, but these additional constraints are not required. To find a solution to the hybrid system (2.38), we then encode the defect constraint (2.44) into a nonlinear program as follows: min 𝐽 (X) (2.45) X s.t. (C1) 𝑍𝑖 (x𝑖 , u𝑖 ) = 0 1 (C2) 𝑥𝑖+1 = Δ[𝑒] (𝑥𝑖𝐾𝑖 ) (C3) 𝑝(X)  0 (C4) 𝑏(X) = 0 ∀𝑖 ∈ (1, 2, . . . 𝑁) ∀𝑒 ∈ E 23 where, 𝐽 (X) is a user-defined cost function; 𝑥𝑖1 and 𝑥𝑖𝐾𝑖 are the first and last value of domain 𝑣 𝑖 and 𝑣 𝑖+1 , who are connected through edge 𝑒; 𝑝(·) and 𝑏(·) are user-defined inequality and equality constraints. Further, the decision variable is given as X = (x1 , . . . , x𝑁 , u1 , . . . , u𝑁 , 𝑇1 , . . . , 𝑇𝑁 ). We can then solve this optimization problem using nonlinear solvers such as SNOPT (Gill, W. Murray, and Saunders, 2005), IPOPT (Wächter and Biegler, 2006), GPOPS (Hager et al., 2019), PSOPT (Becerra, 2010),s and FROST (Ayonga Hereid and Aaron D. Ames, 2017). The collocation constraint 𝑍𝑖 (X) = 0 is a stage-3, implicit Runge-Kutta method to solve dynamical systems. When the number of subinterval increases, the approximated solution becomes closer to the exact solution of the continuous dynamics. On the contrary, a shooting method finds a solution to the dynamical system by solving ∫ 𝑡𝑓 𝑥(𝑡) = 𝑓 (𝑥(𝑠)) + 𝑔(𝑥(𝑠))𝑢(𝑠) 𝑑𝑠 + 𝑥(0) 𝑡0 using explicit Runge-Kutta methods. The implementation of this methodology on simple and rigid systems is acceptable, see (Shishir Kolathaya, Wen-Loong Ma, and Aaron D Ames, 2015; H.-H. Zhao et al., 2014). However, it suffers from computational cost to maintain numerical stability when it comes to a highdimensional problem. The collocation method instead simulates the dynamics implicitly because the solution at each sample time is picked by the optimization solver simultaneously. This made it expandable to high-dimensional and nonstiff systems. We will use a simple hopping robot example below to show this implementation. Example. We now present a simple example, where we find the solution to a 1D hopping system using a nonlinear programming (NLP) toolbox —PSOPT(Becerra, 2010). In particular, we will utilize the local discretization methods to parse our problem. The robot of interest is shown at Fig. 2.7, we write the equations of motion as    𝑧¥𝑚 = 𝑚𝑢𝑚 − 𝑔     𝑧¥𝑟 = − 𝑚𝑢𝑟 − 𝑚𝐹𝑠𝑟 − 𝑔      𝑧¥ 𝑓 = 𝑚𝐹𝑠 − 𝑔 𝑓  ∀ 𝑥 ∈ Df (2.46) 24 zm zr zf clearance 0.45 0.4 0.35 height (m) 0.3 0.25 0.2 0.15 0.1 0.05 0 -0.05 0 0.5 1 1.5 time Figure 2.7: Left: the configuration coordinate of the 1D hopper. Right: The optimized trajectory from (2.51). for the flight phase; and    𝑧¥𝑚 = 𝑚𝑢𝑚 − 𝑔     𝑧¥𝑟 = − 𝑚𝑢𝑟 − 𝑚𝐹𝑠𝑟 − 𝑔      𝑧¥ 𝑓 = 0  ∀ 𝑥 ∈ Ds (2.47) for the stance phase. The directed graph is given in Fig. 2.5 (a), and the domain definitions are given in (2.23). The other notation is explained as: the state variable is 𝑥 > = (𝑞 > , 𝑞¤ > ); the configuration coordinate is 𝑞 > = (𝑧 𝑚 , 𝑧𝑟 , 𝑧 𝑓 ); 𝑚 𝑚 = 1.5, 𝑚𝑟 = 3.5, 𝑚 𝑓 = 0.4; is the mass of the motor, the body, and the foot, respectively; 𝐹𝑠 = 𝑘 𝑠 (𝑧𝑟 − 𝑧 𝑓 − 𝐿 0 ) + 𝑐 𝑠 ( 𝑧¤𝑟 − 𝑧¤ 𝑓 ) is the spring force, with 𝑘 𝑠 = 8000, 𝑐 𝑠 = 4, 𝐿 0 = 0.09 the stiffness, damping and the natural length of the coil spring, respectively. We wish to generate a motion that can hop above 0.3 m with minimal motor force inputs. Therefore, we first parse the continuous dynamics given by (2.46) and (2.47) into the collocation (equality) constraints as given by (2.44). Because of the simplicity, the discrete dynamics that is shown abstractly in (2.31) is formulated into the following boundary conditions:    𝑥 f1 − 𝑥s𝐾s = 0       𝑞 1s − 𝑞 f𝐾f = 0     ( 𝑧¤𝑚 )s1 − ( 𝑧¤𝑚 )f𝐾f = 0      ( 𝑧¤𝑟 )s1 − ( 𝑧¤𝑟 )f𝐾f = 0       ( 𝑧¤ 𝑓 )s1 = 0  .𝑇 (2.48) 25 We then constrain the ending time of each domain as    0.1 ≤ 𝑡s𝐾s ≤ 1,     0.1 ≤ 𝑡f𝐾f ≤ 3,      𝑡s𝐾s − 𝑡f0 = 0.  (2.49) To hop higher than 0.3 m, we utilized the following path constraints:        (𝑧 𝑓 )s0 = ( 𝑧¤ 𝑓 )s0 = 0 2 𝑖 ℎmax exp(− 12 ( 𝜏−𝑐 𝜎 ) ) − 0.01 − 𝑧 𝑓 ≤ 0 ∀𝑖 ∈ (0, 1, . . . , 𝐾f ) (2.50) where ℎ𝑚𝑎𝑥 = 0.3, and 𝑐 is where we wish the highest hopping point to appear, and 𝜏= 𝑡 𝑖 − 𝑡f0 𝑡 f𝐾f − 𝑡f0 is a parameterization of the time. We now have the open-loop trajectory optimization posed as Õ 2 (2.51) argmin 𝑢𝑖 X 𝑖 s.t. (2.44)(2.48)(2.49)(2.50). After 1.98 second, we obtained an optimal solution, i.e. the trajectory of the states xs , xf and the inputs us , uf . We show the result in Fig. 2.7. Note that benefit from the simplicity of this EOM in (2.46) and (2.47), we can use automatic differentiation (AD) solvers such as ADOL-C (Griewank, Juedes, and Utke, 1996) to get the Jacobians of the constraints. However, this simple structure is very rare in multi-body dynamics; obtaining the Jacobians using AD for the fullorder dynamics of a humanoid robot demands much more computational resources. Therefore, we will not give the dynamics explicitly, and we will use Wolfram Mathematica to obtain the Jacobians from now on. Additionally, there are many software that can parse a multi-body structure such as the URDF4 into the general rigid-body dynamics format in (2.7), such as the Screws package in (R. M. Murray et al., 1994). 4 http://wiki.ros.org/urdf 26 Chapter 3 CONTROLLER DESIGN FOR BIPEDAL RUNNING In this chapter, we design controllers and trajectories (gaits) for bipedal locomotion using the full-order hybrid dynamics. In particular, we study the gait design, Lyapunov stability, and input-to-state stability analysis for nonlinear control systems and apply them to a high-dynamic behavior, bipedal running, on two bipedal robots—DURUS-2D and DURUS. 3.1 Running Dynamics (Open-Loop) Figure 3.1: (a) The simulated running of the humanoid robot, DURUS, as a result of large-scale HZD optimization. (b) The directed cycle structure of the multi-domain hybrid system model for flat-footed humanoid running. This section details the multi-domain hybrid dynamic model of 3D running on the DURUS robot. Robot Model of 3D Humanoid—DURUS A popular approach for robotic running is to utilize the Spring-Loaded Inverted Pendulum (SLIP) model (R. Blickhan, n.d.; Collins et al., 2005b; Rezazadeh et al., 2015), since the springs can be of assistance to improve energy efficiency and absorb the high-speed plastic impacts to protect the hardware. Inspired by the SLIP model, the three-dimensional DURUS robot (Fig. 2.1 (d)) is designed and built by SRI International for the study of high-efficiency multi-domain bipedal locomotion (Ayonga Hereid, Eric. Cousineau, et al., 2016; Jacob Reher et al., 2016). DURUS is an 80 kg, 23 degrees-of-freedom (DOFs) underactuated humanoid robot with 15 actuated joints and two passive linear springs attached to the end of each ankle joint 27 that compress perpendicular to the foot. The robot’s upper body, which is used to balance the dynamics and better resemble human locomotion, is controlled by three orthogonal waist joints. For the running model of DURUS, we use the generalized floating-base coordinates, 𝑞 = [ 𝑝 𝑏 , 𝜙 𝑏 , 𝜃] > ∈ Q, of the robot, where 𝑝 𝑏 ∈ R3 is the Cartesian position and 𝜙 𝑏 ∈ SO(3) is the orientation of the body base frame 𝑅𝑏 , which is attached to the center of the pelvis link, with respect to the world frame; and as shown in Fig. 2.1 (d), the local joints are represented by 𝜃 = [𝜓𝑤 , 𝜙𝑤 , 𝜃 𝑤 , 𝜓𝑙ℎ , 𝜙𝑙ℎ , 𝜃 𝑙ℎ , 𝜃 𝑙 𝑘 , 𝜃 𝑙𝑎 , 𝜙𝑙𝑎 , 𝑟 𝑙𝑠 , 𝜓𝑟 ℎ , 𝜙𝑟 ℎ , 𝜃 𝑟 ℎ , 𝜃 𝑟 𝑘 , 𝜃 𝑟𝑎 , 𝜙𝑟𝑎 , 𝑟𝑟 𝑠 ] > , with the coordinates corresponding to the waist yaw, roll, pitch angles, left hip yaw, hip roll, hip pitch, knee pitch, ankle pitch, ankle roll angles and spring deflection, and the right hip yaw, hip roll, hip pitch, knee pitch, ankle pitch, ankle roll angles and spring deflection. Hybrid Dynamics for Running Due to the existence of both continuous and discrete dynamics, bipedal robot running is naturally modeled as a hybrid control system. The flat-footed running of DURUS is composed of two continuous domains: a stance domain, where the nonstance foot swings in the air while the stance foot stays on the ground, and a flight domain, where both feet are in the air (see Fig. 3.1 (b)). Therefore, as discussed in (5), the hybrid control system of DURUS running is defined as a tuple: H C = (Γ, D, U, S, Δ, FG), (3.1) where, • Γ = {V, E} is a directed cycle with vertices V = {s, f}, where s represents the stance domain and f represents the flight domain, and the edges E = {s → f, f → s}, • D = {Ds , Df } is a set of admissible domains of continuous dynamics, • U = {Us , Uf } is a set of admissible controls, • S = {Ss ⊂ Ds , Sf ⊂ Df } is a set of guards, • Δ = {Δs→f , Δf→s } is a set of smooth reset maps representing the discrete dynamics, 28 • FG = {( 𝑓s , 𝑔s ), ( 𝑓f , 𝑔f )} is a set of affine control systems 𝑥¤ = 𝑓𝑣 (𝑥) + 𝑔𝑣 (𝑥)𝑢, defined on D𝑣 for all 𝑣 ∈ V, with 𝑥 = (𝑞, 𝑞) ¤ being the system states. The directed cycle Γ is depicted in the Fig. 3.1 (b). The construction of individual elements of (3.1) will be presented in the remainder of this section. Stance Domain. During the stance domain, the stance foot remains flat on the ground. Often we use holonomic constraints to model the foot contact with the ground (Jessy W. Grizzle et al., 2014). Here, we define the holonomic constraints of the stance domain as " # 𝑝 𝑠 𝑓 (𝑞) ℎs (𝑞) , ∈ R6 , (3.2) 𝜙 𝑠 𝑓 (𝑞) with 𝑝 𝑠 𝑓 the position and 𝜙 𝑠 𝑓 the orientation of the stance foot. Given the physical properties of each link, the unconstrained dynamics of the stance domain Ds is given by 𝐷 (𝑞) 𝑞¥ + 𝐻 (𝑞, 𝑞) ¤ = 𝐵𝑢 + 𝐽s> (𝑞)𝐹, (3.3) where, 𝐷 (𝑞) is the inertia matrix, 𝐻 (𝑞, 𝑞) ¤ contains the Coriolis, gravity, and spring forces terms, 𝐽s (𝑞) is the Jacobian of the holonomic constraints, and 𝐹 is a wrench containing the ground constraint forces and moments. The holonomic constraints are guaranteed via enforcing the second-order derivative of ℎs to be zero: 𝐽s (𝑞) 𝑞¥ + 𝐽¤s (𝑞, 𝑞) ¤ 𝑞¤ = 0. (3.4) Thus the affine control system ( 𝑓s , 𝑔s ) can be determined by combining (3.3) and (3.4). The manifold of the stance domain is determined by unilateral constraints, which could be formulated as a vector of admissible conditions, 𝐴s (𝑞, 𝑞, ¤ 𝑢). These conditions include positive non-stance foot height, positive normal ground force, etc. In other words, we have Ds = {(𝑞, 𝑞, ¤ 𝑢) ∈ 𝑇 Q × Us | 𝐴s (𝑞, 𝑞, ¤ 𝑢) ≥ 0}. (3.5) Further, the guard condition of the stance domain is defined as the normal ground force crosses zero, i.e., Ss→f = {(𝑞, 𝑞, ¤ 𝑢) ∈ 𝑇 Q × 𝑈s | 𝐹 𝑧 (𝑞, 𝑞, ¤ 𝑢) = 0}. (3.6) 29 Because there is no impact involved during the transition, the reset map from the stance domain to flight domain, Δs→f , is an identity map. Flight Domain. Since there is no ground contact during the flight domain, the continuous dynamics of the domain is determined by the unconstrained EulerLagrangian equation: 𝐷 (𝑞) 𝑞¥ + 𝐻 (𝑞, 𝑞) ¤ = 𝐵(𝑞)𝑢. (3.7) The admissible conditions of the flight domain are defined so that both feet are above the ground, i.e., 𝐴f (𝑞) = (ℎ 𝑠 𝑓 (𝑞), ℎ𝑛𝑠 𝑓 (𝑞)). Therefore, we have Df = {(𝑞, 𝑞, ¤ 𝑢) ∈ 𝑇 Q × Uf | 𝐴f (𝑞) ≥ 0}. (3.8) Accordingly, the transition from the flight to stance domain occurs when the nonstance foot strikes the ground, i.e., Sf→s = {(𝑞, 𝑞, ¤ 𝑢) ∈ 𝑇 Q × Uf | ℎ𝑛𝑠 𝑓 (𝑞) = 0, ℎ¤ 𝑛𝑠 𝑓 (𝑞, 𝑞) ¤ < 0}. (3.9) Both dynamics in (3.3) and (3.7) can be converted to the following form: 𝑥¤ = 𝑓𝑣 (𝑥) + 𝑔𝑣 (𝑥)𝑢, (3.10) and the derivation was given in (2.20). Transition Map. The reset map from the flight to the stance domain incorporates the impact dynamics when the non-stance foot hits the ground, during which the joint velocities undergo discrete changes due to new contact constraints. Given the pre-impact states (𝑞 − , 𝑞¤ − ), the post-impact states (𝑞 + , 𝑞¤ + ) = Δf→s (𝑞 − , 𝑞¤ − ) are determined by assuming a perfectly plastic impact of the rigid body. Since the position terms do not change through the impact, we have the impact equation, which determines the discrete changes of velocities as " #" # " # 𝐷 (𝑞 − ) −𝐽s> (𝑞 − ) 𝑞¤ + 𝐷 (𝑞 − ) 𝑞¤ − = (3.11) 𝐽s (𝑞 − ) 0 𝛿𝐹 0 where 𝛿𝐹 is a vector of impulsive contact wrenches. 3.2 Virtual Constraint and Zero Dynamics Given the running model, this section introduces the basic concepts used in this chapter, including virtual constraints, phasing variables, and zero dynamics. We will use DURUS-3D running (Fig. 3.1) as an example to concretize these constructions. 30 Desired behavior. To achieve stable locomotion, we first need to decide what to control, which is the feature (actual output, 𝑦 𝑎 ) that we intend to drive to some desired behavior (desired output, 𝑦 𝑑 ) that we will design through trajectory optimization. The actual output is a function of the system’s states, given as 𝑦 𝑎 = 𝑦 𝑎 (𝑥); (3.12) and the desired behavior is represented by a Bézier polynomial 𝑦 𝑑 = 𝛽(𝜏, 𝛼), (3.13) where 𝛼 is the (constant) coefficients for the polynomial, and this trajectory is parameterized by a phasing variable 𝜏, and it monotonically increases over time. Phasing variable. Locomotion gaits, viewed as a set of desired trajectories, are often modulated as functions of a phase variable to eliminate the dependence on the timebased (Villarreal and R. D. Gregg, 2014). We will discuss two types of phasing variables in this manuscript, one is a time-based phasing variable 𝜏𝑡 = 𝜏(𝑡). (3.14) Correspondingly, the desired output (3.13) will be a function of time. We also define a state-based phasing variable as 𝜏𝑠 = 𝜏(𝑞). (3.15) Correspondingly, the desired output (3.13) will be a function of states. In the content that follows, we will omit the state-based subscript 𝑠 since we will not talk about time-based setting until Sec. 3.4. Virtual constraint. To achieve stable locomotion, we first need to establish the following goal 𝑦𝑑 → 𝑦𝑎 as time 𝑡 → ∞, (3.16) which is further described as driving the output (virtual constraint): 𝑦 = 𝑦 𝑎 (𝑥) − 𝑦 𝑑 (𝜏, 𝛼) (3.17) to 0 as 𝑡 → ∞. Since 𝛼 is a set of constant parameters, we will suppress this argument from 𝑦 𝑑 (𝜏, 𝛼) to have 𝑦 𝑑 (𝜏). 31 Output dynamics. The dynamic relation between the output 𝑦, and an input 𝑢, is referred as output dynamics. For the robotic systems considered in this dissertation, we are particularly interested in two types of outputs, the relative degree 1 output: 𝑦¤ = 𝑑𝑦 𝜕𝑦 = 𝑥¤ 𝑑𝑡 𝜕𝑥 𝜕𝑦 𝜕𝑦 = 𝑓 (𝑥) + 𝑔(𝑥) 𝑢, 𝜕𝑥 𝜕𝑥 | {z } | {z } ,𝐿 𝑓 𝑦 (3.18) ,𝐿 𝑔 𝑦 and the relative degree 2 output when 𝐿 𝑔 𝑦 = 0: 𝑦¥ = 𝑑 2 𝑦 𝜕𝐿 𝑓 𝑦(𝑥) = 𝑥¤ 𝜕𝑥 𝑑𝑡 2 𝜕𝐿 𝑓 𝑦(𝑥) 𝜕𝐿 𝑓 𝑦(𝑥) 𝑓 (𝑥) + 𝑔(𝑥) 𝑢. = 𝜕𝑥 𝜕𝑥 | {z } | {z } ,𝐿 2𝑓 𝑦 (3.19) ,𝐿 𝑔 𝐿 𝑓 𝑦 where 𝑓 , 𝑔 is given in (3.10). Rigid-body dynamics have no more than a relativedegree two outputs. All systems of interest in this thesis only have less than two relative degrees. In particular, any output that is only a function of the configuration coordinate 𝑦(𝑞), is relative-degree two. Note that when compliance such as serieselastic actuator is involved, some of the global coordinates (w.r.t. world frame) can have more than two relative degrees. The goal is to design a control scheme 𝑢 to drive the difference between the actual output and the desired output to zero. Before doing so, let us take the bipedal running as an example. Example. For DURUS running Fig. 3.1(a), we first pick the actual outputs 𝑦 𝑎 (𝑞, 𝑞). ¤ In the stance domain Ds , the forward velocity of the center of mass (COM) is chosen as the relative degree one output 𝑎 𝑦 1,s = 𝑣 𝑥𝑐𝑜𝑚 (𝑞, 𝑞) ¤ to regulate the forward velocity of the robot, and the (vector) relative degree two outputs are defined as 𝑎 𝑦 2,s (𝑞) = (𝜃 𝑠𝑘 , 𝜙 𝑠𝑙𝑟 , 𝜃 𝑠ℎ , 𝜓 𝑠ℎ , 𝜙 𝑠𝑎 , 𝜙𝑤 , 𝜃 𝑤 , 𝜓𝑤 , 𝜃 𝑛𝑠𝑘 , 𝜙𝑛𝑠ℎ , 𝜃 𝑛𝑠ℎ , 𝜙𝑛𝑠 𝑓 , 𝜃 𝑛𝑠 𝑓 , 𝜓𝑛𝑠 𝑓 ) > . In the flight domain Df , actual outputs consist of only the relative degree two outputs 𝑎 𝑦 2,f (𝑞) = (𝜃 𝑠𝑘 , 𝜙 𝑠𝑙𝑟 , 𝜃 𝑠ℎ , 𝜙 𝑠 𝑓 , 𝜃 𝑠 𝑓 , 𝜓 𝑠 𝑓 , 𝜙𝑤 , 𝜃 𝑤 , 𝜓𝑤 , 𝜃 𝑛𝑠𝑘 , 𝜙𝑛𝑠ℎ , 𝜃 𝑛𝑠ℎ , 𝜙𝑛𝑠 𝑓 , 𝜃 𝑛𝑠 𝑓 , 𝜓𝑛𝑠 𝑓 ) > . 32 In particular, 𝜙 𝑠𝑙𝑟 = 𝜙 𝑠ℎ − 𝜙𝑛𝑠ℎ is the stance leg roll angle, 𝜙 𝑠 𝑓 , 𝜃 𝑠 𝑓 , 𝜓 𝑠 𝑓 and 𝜙𝑛𝑠 𝑓 , 𝜃 𝑛𝑠 𝑓 , 𝜓𝑛𝑠 𝑓 are the orientations (Euler angles) of the stance and non-stance foot, respectively. Other outputs are simply joint angles, as shown in Fig. 2.1 (d). We want to design a running motion with constant forward velocity, hence the desired velocity of COM is a constant 𝑣 𝑑 , i.e., 𝑎 𝑦 1,s (𝑞, 𝑞, ¤ 𝑣 𝑑 ) = 𝑦 1,s (𝑞, 𝑞) ¤ − 𝑣𝑑 . (3.20) 𝑑 (𝜏(𝑞), 𝛼 ) are represented by And the desired relative degree two outputs 𝑦 2,𝑣 𝑣 seventh-order Bézier polynomials parameterized by a set of parameters 𝛼𝑣 with 𝑣 ∈ {s, f}. The virtual constraints on D𝑣 became: 𝑎 𝑑 (𝜏(𝑞), 𝛼𝑣 ) , 𝑦 2,𝑣 (𝑞, 𝛼𝑣 ) = 𝑦 2,𝑣 (𝑞) − 𝑦 2,𝑣 (3.21) where 𝜏(𝑞) is a monotonic state-based parameterization of time, defined as 𝜏(𝑞) = 𝑝 𝑥𝑏 − 𝑝 0 𝑝1 − 𝑝0 , (3.22) where 𝑝 𝑥𝑏 is the 𝑥-position of the floating base’s frame 𝑅𝑏 , and the parameters 𝑝 1 and 𝑝 0 are the corresponding values, by design, at the beginning and the end of each (cyclic) step. This setting allows the phasing variable to paramertize the desired trajectory over one step from 0 to 1, i.e., 𝜏 ∈ [0, 1]. In particular, the desired outputs of the stance and non-stance foot orientations are set to be zero respectively to keep the feet being flat throughout the step. Note that if an output is defined for both domains, the coefficients of the corresponding desired Bézier polynomials must be the same. As given in (3.18) and (3.19), we can summarize (3.20) and (3.21) as " # " # " # 𝐿 𝑓 𝑦 1,𝑣 𝐿 2𝑓 𝑦 1,𝑣 𝑦¤ 1,𝑣 = 2 + 𝑢 𝐿 𝑓 𝑦 2,𝑣 𝑦¥ 2,𝑣 𝐿 𝑔 𝐿 𝑓 𝑦 2,𝑣 | {z } | {z } L 𝑣 (𝑞,𝑞) ¤ (3.23) A 𝑣 (𝑞) where we call A 𝑣 (𝑞) the decoupling matrix. Feedback Linearization-Based Control As often practiced for nonlinear control systems, we can construct a feedback linearization-based control law for bipedal locomotion. To drive the output: " # 𝑦 1,𝑣 𝑦𝑣 = →0 𝑦 2,𝑣 33 for each D𝑣 with 𝑣 ∈ {s, f}, we can utilize the input-output feedback linearization control law
𝑢 𝑣𝜀 = −A 𝑣−1 L𝑣 + 𝜇𝑣𝜀 , (3.24) which is often shorted as IO controller. The control structure is illustrated in Fig. 3.2. For running dynamics, we have the decoupling matrix A 𝑣 as: " # 𝐿 𝑔s 𝑦 1,s (𝑞, 𝑞) ¤ As = , Af = 𝐿 𝑔f 𝐿 𝑓f 𝑦 2,f (𝑞) 𝐿 𝑔s 𝐿 𝑓s 𝑦 2,s (𝑞) respectively, and " # 0 Ls = , 𝐿 𝑓s 𝐿 𝑓s 𝑦 2,s (𝑞) Lf = 𝐿 𝑓f 𝐿 𝑓f 𝑦 2,f (𝑞). With the given control law, we have the output dynamics become " # 𝑦¤ 1,s = −𝜇s𝜀 , 𝑦¥ 2,f = −𝜇f𝜀 𝑦¥ 2,s (3.25) for the stance and flight domain respectively, where 𝜇𝑣𝜀 can be chosen so that the outputs converge to zero exponentially at a rate of 𝜀 > 0. In particular, we define " # 𝜀𝑦 (𝑞, 𝑞, ¤ 𝑣 ) 1,s 𝑑 𝜇s𝜀 = , (3.26) 2 2𝜀 𝑦¤ 2,s (𝑞, 𝑞, ¤ 𝛼s ) + 𝜀 𝑦 2,s (𝑞, 𝛼s ) 𝜇f𝜀 = 2𝜀 𝑦¤ 2,f (𝑞, 𝑞, ¤ 𝛼f ) + 𝜀 2 𝑦 2,f (𝑞, 𝛼f )). (3.27) Denote some new coordinates as  𝑦 1,s      𝜂s ,  𝑦 2,s  ,    𝑦¤ 2,s    " # 𝑦 2,f 𝜂f , , 𝑦¤ 2,f and 𝜂 = (𝜂s , 𝜂f ) > , we can see the exponential stabilizing effect of the 𝜂-dynamics according to the following definition. Definition 7. A smooth function 𝑉 : R𝑛 → R+ is an exponentially stabilizing control Lyapunov function (ES-CLF) for 𝜂¤ = 𝑓 (𝜂) + 𝑔(𝜂)𝑢, 34 if there exists constants 𝑐 1 , 𝑐 2 , 𝑐 3 > 0 such that 𝑐 1 |𝜂| 2 ≤ 𝑉 (𝜂) ≤ 𝑐 2 |𝜂| 2 (3.28)
inf 𝐿 𝑓 𝑉 (𝜂) + 𝐿 𝑔𝑉 (𝜂)𝑢 + 𝑐 3𝑉 (𝜂) ≤ 0 𝑢 for all 𝑥 ∈ X. If there further exists a constant 𝜀 ∈ (0, 1) such that 𝑐2 𝑐 1 |𝜂| 2 ≤ 𝑉 (𝜂) ≤ 2 |𝜂| 2 𝜀   𝑐3 inf 𝐿 𝑓 𝑉 (𝜂) + 𝐿 𝑔𝑉 (𝜂)𝑢 + 𝑉 (𝜂) ≤ 0 𝑢 𝜀 (3.29) for all 𝜂 ∈ Y, then 𝑉 (𝜂) is a rapidly exponentially stabilizing control Lyapunov function (RES-CLF). and we can pick a Lyapunov candidate as 𝑉 (𝜂) = 𝜂> 𝑃𝜂 for the dynamics in (3.25) under control input (3.26) and (3.27). Remark that these controllers only stabilize the 𝜂−dynamics, with 𝑘 < 𝑛. For the underactuated fullorder system, we need some other conditions to establish its stability property, which is introduced below. Figure 3.2: A block diagram showing the state-based feedback control structure. Hybrid Zero Dynamics Invariance The fundamental theorem gives the condition of the overall system’s stability in the sense of Lyapunov in the HZD condition, (A. Ames, Galloway, et al., 2014a), which will be a basis for the development of the methods in this thesis. We will briefly introduce it here. 35 Any admissible state-based feedback controller, such as (3.24), that has been applied to the control system, FG, yields the following closed-loop continuous system: 𝑥¤ = 𝑓𝑣cl (𝑥) = 𝑓𝑣 (𝑥) + 𝑔𝑣 (𝑥)𝑢 𝑣 (𝑥, 𝛼𝑣 ) forward invariant on the (partial) zero dynamics surface: Z𝑣 , {(𝑞, 𝑞) ¤ ∈ D𝑣 | 𝑦 2,𝑣 (𝑞) = 𝑦¤ 2,𝑣 (𝑞, 𝑞) ¤ = 0}. (3.30) But due to the impact dynamics, the invariance of the zero dynamics surface is not necessarily guaranteed by the controller. Particularly, it is impossible to enforce the relative degree one output to be invariant through impact due to the changes in the velocity at the impact event. Therefore, a submanifold Z𝑣 is impact-invariant if Δ𝑒 (𝑥) ∈ Z𝑣 + , ∀𝑥 ∈ S𝑒 ∩ Z𝑣 (3.31) for each 𝑒 , (𝑣 → 𝑣 + ) ∈ E. The resulting hybrid invariant submanifold is referred Ð as a hybrid zero dynamics (HZD), ℋ| Z . Further, Z = 𝑣∈V Z𝑣 is hybrid invariant if it is invariant over all domains of continuous dynamics and discrete dynamics. In fact, the restricted reduced dimensional dynamics are independent of control input. According to (A. Ames, Galloway, et al., 2014a, Thm.2) if there exists a RES-CLF for the output dynamics in (3.25), which drives the actual output to the desired trajectory exponentially, then there exists an exponentially stable periodic orbit of the hybrid zero dynamics manifold given in (3.30). We can have an exponentially stable orbit for the full-order system. With the feedback controller defined, as given in (3.24), (3.26), and (3.27), the mission is to find the parameter set 𝛼 that 𝑦 𝑎 (𝑞) converges to 𝑦 𝑑 (𝛼, 𝜏) exponentially. In other words, the goal of designing a gait is to find a set of parameters 𝛼 = {𝑣 𝑑 , 𝛼s , 𝛼f } that ensures the existence of a periodic orbit for the system (3.1) and the (partial) hybrid zero dynamics surface, Z𝑣 is invariant through the discrete impact dynamics. The process of finding 𝛼 is then formulated as a nonlinearly constrained optimization problem in Sec. 3.3 subject to the HZD condition. 3.3 Gait Design via Closed-Loop Optimization We can now use optimization tools to design a gait (periodic trajectory), a periodic solution to the closed-loop system. In this section, we first use bipedal running as an example to present the details of closed-loop optimization, including the equality 36 and inequality constraints. Then we study the energy efficiency of these generated running behaviors. We will lastly use the tool developed to realize bipedal walking on slippery surfaces. Closed-Loop Optimization for Running Building on top of (2.45), we first define the dynamic constraints for hybrid systems for the purpose of finding periodic solution to the open-loop dynamics (3.10) as follows. 1. Continuous Dynamic Constraint. We first need the optimization algorithm to find a trajectory of the states and inputs for all time. Follow the setting in Sec. 3.10 and notations from (3.1), we have 𝑁 = 2 for the two-domain running, and pick 𝐾1 = 𝐾2 = 15 for the discretization of horizon. We then have the first equality constraint for continuous dynamics (as in (2.45)): 𝑍 𝑣 (x𝑣 , u𝑣 ) = 0 ∀𝑣 ∈ V , (s, f) . (C.1) 2. Discrete Dynamic Continuity. To guarantee the trajectories C𝑣 , {x𝑣 , u𝑣 } of the neighbor domains belong to the same flow, we have the following equality constraint for discrete dynamic continuity 𝑥f0 − Δ[𝑒 s→f ] (𝑥 s𝐾1 ) = 0 (C.2) In essence, (C.1) and (C.2) pose the hybrid dynamics    𝑥¤ = 𝑓𝑣 (𝑥) + 𝑔𝑣 (𝑥)𝑢  𝑥 ∈ D𝑣 ,   𝑥 + = Δ[𝑒 𝑣 𝑖 →𝑣 𝑖+1 ] (𝑥 − )  where 𝑣 ∈ V , (s, f), into a set of equality constraints. The advantage is that we can pre-compute the symbolic Jacobians of these constraints and hence accelerate the evaluation time, and the problem is made robust against stiff systems (see (Kiehl, 1994; Ascher and Petzold, 1998)). 3. Periodic Continuity. To find a periodic solution, we need to make sure the final condition is “stitched” to the initial condition. Still, since there is a jump in the velocity 37 terms due to the impact, as in (3.11), we will need to relate them through the impact map. But before doing so, a key concept needs to be introduced, the relabelling matrix R : Q → Q. Since all of the legs’ configuration coordinates are labelled as stance “00𝑠 and nonstance “00𝑛𝑠 , and after the impact, the nonstance leg will become a stance leg, we need to mirror (relabel) the states accordingly. For example, the pitch angles of the stance and nonstance joints need to be flipped, and the roll and yaw angles of stance and nonstance legs need to be flipped with a negative sign. We then modify (3.11) with the notation from (C.2) to have an equality constraint for periodic continuity: " #" # " # 𝐷 (𝑞 f𝐾f ) −𝐽s> (R𝑞 f𝐾f ) 𝑞¤ 1s 𝐷 (R𝑞 f𝐾f ) 𝑞¤ f𝐾f = , (C.3) 𝐽s (R𝑞 f𝐾f ) 0 𝛿𝐹 0 or simply, 𝑥 s0 − Δ[f → s]R𝑥f𝐾f = 0. (3.32) To establish Lyapunov stability analysis, we need to have the system in the closedloop form, which is open-loop dynamics with a parameterized feedback controller such as in (3.24). We now present the closed-loop dynamics in the optimization setting for the control problem of interest. 4. Output Dynamics. Since the controller in (3.24) yields output dynamics in (3.25), we can equivalently enforce the output dynamics as    𝑦¤ 1,𝑣 ( 𝑥,  ¤ 𝑥) + 𝜀𝑦 1,𝑣 (𝑥, 𝑣 𝑑 ) = 0 (C.4)   𝑦¥ 2,𝑣 ( 𝑥, ¤ 𝑥) + 2𝜀 𝑦¤ 2,𝑣 (𝑥) + 𝜀 2 𝑦 2,𝑣 (𝑥, 𝛼𝑣 ) = 0  according to (3.26) and (3.27), where 𝑖 = 0, 1, . . . , 𝐾𝑣 , with 𝑣 ∈ (s, f). Remark the here we have introduced the most essential parameter 𝛼𝑣 that represents the desired output (trajectory) of the solution, as given in (3.20) and (3.21). 5. Hybrid Invariance. As established in (3.31), we need the HZD condition to reduce the complexity of stability analysis from the full-order system to the passive dynamics on the hybrid zero dynamics manifold Z𝑣 . Formally, 𝑦 2,𝑣 (𝑥 𝑣0 , 𝛼𝑣 ) = 𝑦¤ 2,𝑣 (𝑥 𝑣0 , 𝑥¤𝑣0 , 𝛼𝑣 ) = 0 𝑦 2,𝑣 (𝑥 𝑣𝐾 𝑣 , 𝛼𝑣 ) = 𝑦¤ 2,𝑣 (𝑥 𝑣𝐾 𝑣 , 𝑥¤𝑣𝐾 𝑣 , 𝛼𝑣 ) = 0 ∀ 𝑣 ∈ V which is a boundary condition. (C.5) 38 We now introduce constraints that are essential to robotics and legged locomotion. 6. Feasibility Conditions. We can use some optimization algorithm, such as direct collocation with (C.1)—(C.5), to find a solution to a dynamical system. However, these solutions might not be feasibly realizable on hardware. For example, although a solution 𝑥(𝑡), 𝑡 ∈ [0, 𝑇] satisfies 𝑥(𝑡) ¤ = 𝑓 (𝑥(𝑡)), there is no guarantee that this trajectory will not penetrate an obstacle on in the way. We can use a set of feasibility conditions (also known as path constraint) to enforce these conditions. These conditions include the electric motors’ torque saturation, each joint’s feasible workspace that is specified by the hardware limitation, given by −umax  𝑢𝑖  umax , qmin  𝑞𝑖  qmax , −vmax  𝑞¤𝑖  vmax . We also need to enforce the friction pyramid condition1 to prevent the stance feet from slipping on the ground. Then, these conditions can be configured in the following condition:    𝑢𝑖 ∈ U, 𝑥 𝑖 ∈ X     𝜇𝐹𝑧 (𝑥 𝑖 , 𝑢𝑖 ) − |𝐹𝑥 (𝑥 𝑖 , 𝑢𝑖 )| ≥ 0      𝜇𝐹𝑧 (𝑥 𝑖 , 𝑢𝑖 ) − |𝐹𝑦 (𝑥 𝑖 , 𝑢𝑖 )| ≥ 0  ∀𝑖 ∈ (0, 1, . . . , 𝐾𝑣 ), ∀𝑣 ∈ V (C.6) where 𝜇 is the static friction coefficient, and 𝐹(·) is the ground reaction force along certain axis. Remark that for experimental success, these conditions can often be manually tuned to be more restrictive, for example, a smaller friction constant and a smaller torque bound can give the actuator more “freedom to save” the robot from failures under disturbance and uncertainty. 7. Foot clearance. For legged locomotion, ground clearance for the nonstance foot is necessary. This is a heuristic constraint that users normally need to tune to get a “naturallooking” gait eventually. Denote the height of the nonstance foot (or feet, 1 A computationally efficient but more restrictive version of the friction cone condition. 39 sometime there can be two feet in the air, such as the flight phase) as ℎnsf,𝑧 (𝑞), we can specify a lower bound ℎ𝑙𝑣 (𝑡) for 𝑣 ∈ V. The foot clearance conditions become: ℎ𝑙𝑣 (𝑡 𝑖 ) − ℎnsf,𝑧 (𝑞𝑖 )  0, ∀𝑖 ∈ (0, 1, . . . , 𝐾𝑣 ), ∀𝑣 ∈ V. (C.7) We now have completed the optimization formulation for finding a solution to the closed-loop dynamics of a running bipedal robot. Formally, argmin 𝐽 (X) (3.33) X s.t. (C.1) 𝑍 𝑣 (x𝑣 , u𝑣 ) = 0 (C.2) 𝑥f0 − Δ[𝑒 s→f ] (𝑥 s𝐾1 ) = 0 (C.3) (C.5) 𝑥s0 − Δ[f → s]R𝑥 f𝐾f = 0    𝑦¤ 1,𝑣 ( 𝑥,  ¤ 𝑥) + 𝜀𝑦 1,𝑣 (𝑥, 𝑣 𝑑 ) = 0   𝑦¥ 2,𝑣 ( 𝑥, ¤ 𝑥) + 2𝜀 𝑦¤ 2,𝑣 (𝑥) + 𝜀 2 𝑦 2,𝑣 (𝑥, 𝛼𝑣 ) = 0  𝐾𝑣 𝐾𝑣 𝑦 02,𝑣 = 𝑦 2,𝑣 = 𝑦¤ 02,𝑣 = 𝑦¤ 2,𝑣 =0 ∀𝑣 ∈V (C.6)   𝑢 𝑖 ∈ U, 𝑥 𝑖 ∈ X     (C.4) (C.7) ∀𝑣 ∈ V ∀𝑖 ∈ (0, 1, . . . , 𝐾𝑣 ), ∀𝑣 ∈ V 𝜇𝐹𝑧 (𝑥 𝑖 , 𝑢 𝑖 ) − |𝐹𝑥 (𝑥 𝑖 , 𝑢 𝑖 )| ≥ 0     𝜇𝐹𝑧 (𝑥 𝑖 , 𝑢 𝑖 ) − |𝐹𝑦 (𝑥 𝑖 , 𝑢 𝑖 )| ≥ 0  ℎ𝑙𝑣 (𝑡 𝑖 ) − ℎnsf,𝑧 (𝑞𝑖 )  0, ∀𝑖 ∈ (0, 1, . . . , 𝐾𝑣 ), ∀𝑣 ∈ V where 𝐽 (X) is the cost function. A common choice is the total energy consumption, Í𝑁 given as 𝑖=0 𝐵𝑢𝑖 , 𝑞¤𝑖 . With this formulation, we can employ some optimization tools such as IPOPT (Wächter and Biegler, 2006) to solve the gait parameters 𝛼𝑣 and initial condition 𝑥(0). We now apply this gait generation method to two behaviors: studying the energy efficiency of running motion and realizing bipedal walking on slippery surfaces. Towards High-Energy Efficiency of Running Marc Raibert’s technical report “Dynamically Stable Legged Locomotion, 1989” (M. H. Raibert et al., n.d.) summarized that “the running speed of a legged system depends upon the frequency and length of its steps” based on the study of his legendary hopper. To better understand this phenomenon in 3D bipedal-legged systems, this section documents a statistical result based on the stable running gaits obtained from an optimization-based gait generation framework, i.e., how the 40 optimizer should respond when asked to generate faster gaits to maximize its energy efficiency. Bipedal running is an essential benchmark for humanoid control for many mathematical and practical reasons. Unlike walking, running is an inherently underactuated control problem (M. Spong, 1998). Whenever the robot leaves the ground, it fundamentally loses its ability to actuate all degrees of freedom and is at the mercy of its ballistic trajectory. It is also a multi-domain hybrid control problem (H. Zhao et al., 2016). Further, the high power demands push the practical limits of humanoid actuators. This section presents 3D running via hybrid zero dynamics (HZD) (A. D. Ames, 2014; E. R. Westervelt, J. W. Grizzle, C. Chevallereau, et al., 2007) on a simulated underactuated model of the humanoid robot, DURUS. The running gaits emerge from a large-scale gait optimization of the full-order system dynamics, a previously developed tool for 3D walking with the DURUS hardware (Ayonga Hereid, Eric. Cousineau, et al., 2016). We report the success of this toolset as a milestone toward 3D running. The earliest example of running controllers was developed using a set of highly successful heuristics. Examples include the Raibert hoppers (M. H. Raibert et al., n.d.) and the ARL-Monopod II (Ahmadi and Buehler, 2006). Decades later, Honda’s humanoid robot, ASIMO (Sakagami et al., 2002), claims running speeds up to 2.5 m/s without revealing its control method. Other methods have been employed to achieve stable running in simulation by constraining the robot’s dynamics to a reduced-order model (W. C. Martin, Wu, and Geyer, 2015) and even achieving simulated high-speed turning (Wensing and D. Orin, 2014). Researchers have also generated running gaits for simulated robots with various degrees of freedom. From simple point-mass models (Srinivasan and Ruina, 2006) to planar hopping models (Xi, Yesilevskiy, and Remy, 2015) to planar humanoid models (Mombaur, 2009) (which even certify stability inside the optimization). Recent work optimized an ATLAS model for 3D locomotion by reasoning about the centroidal dynamics (Hongkai Dai, Valenzuela, and Russ Tedrake, 2014). Graphics researchers even successfully created 3D running controllers after running an extensive evolutionary algorithm (Geijtenbeek, Panne, and Stappen, 2013). Here, we seek a method that generates optimal running gaits within the hybrid zero dynamics framework to leverage its formal guarantees regarding stability. In an effort to embrace underactuation in locomotion with formal control methods, hybrid zero dynamics (HZD) was developed on multiple successful robotic walking implementations (H. Park et al., 2012; A. D. Ames, 2014). It was ultimately imple- 41 mented to produce planar running on the spring-legged robot, MABEL (Sreenath, H.-W. Park, I. Poulakakis, and J. Grizzle, 2013), and is being extended to non-planar cases (Kaveh Akbari Hamed and Jessy W. Grizzle, 2013). While there are formal mathematical underpinnings to HZD, practical implementation requires a gait optimization which considers the full-order dynamics of the system. Traditionally, these gait optimizations have become increasingly unreliable with robots as complex as humanoids. In prior work, the authors presented a collocation-based HZD optimization formulation to produce stable 3D humanoid walking (Ayonga Hereid, Eric. Cousineau, et al., 2016). Here, we further extend this approach to 3D running on DURUS. Moreover, we show that the optimization is sufficiently reliable that we generated 25 gaits at various running speeds. This library of 3D gaits allows us, for the first time, to observe trends in energy costs and running strategies in humanoid running. Prior work has given physical insight into mechanisms of speed adjustment, such as step length and step frequency (M. H. Raibert et al., n.d.). Here, we can assess whether these strategies manifest when optimizing 3D running with humanoid complexity. We also present a cost-of-transport vs. speed curve for 3D running, commonly reported in animal locomotion studies, which we can now tractably generate for humanoid running. With the proposed optimization method, we generated multiple stable 3D running gaits for DURUS with velocities varying from 1.5 m/s to 3.0 m/s. This section will focus on one of the simulated running gaits in detail first, then the statistical analysis of the NLP performance, and all of the running gaits will be summarized. Figure 3.3: Limit cycle of running at 2.0 m/s over 20 steps. Running at 2 m/s. With the constraints configured as explained in (3.33) and the large-scale IPOPT NLP solver developed by COIN-OR, a 3D running gait is solved after 722 iterations and 374 seconds of computation, with dual infeasibility converged to 9.0 × 10−4 , and constraints violation 1.8 × 10−7 . This particular gait runs 42 at 2.0 m/s. Note that we categorize each running gait based on the 𝑥 component of the COM velocity during the flight domain. The specific cost of transport (SCOT) (Collins et al., 2005b) is calculated in simulation as 0.90, the maximum angular velocity of all joints is 4.4 rad/s, peak torque is 446 N m, and peak power is 1.1 kW. A running tile is shown in Fig. 3.4, the limit cycle of each joint is also shown in Fig. 3.3. Only one leg is shown because of the symmetric motion. We have verified the stability of this running gait by numerically computing the eigenvalues of the linearization of the Poincarémap that is restricted to the zero dynamics about the Poincarésection where 𝑝 𝑥𝑏 = 0. The magnitude of its eigenvalues are [0.414, 0.083, 0.031, 0.006, 0.000]. All values smaller than 1 indicate asymptotic stability obtained from this running framework (see (Morris and J. W. Grizzle, 2005) for details). Figure 3.4: Snapshots of the DURUS running at 2.0 m/s. Efficient 3D Running Gaits Generation. We now present a working framework to generate stable running gaits for 3D bipedal robots reliably and efficiently (see (Wen-Loong Ma, 2016b) for the simulated running). For the 25 gaits that the optimization found, we documented the computation time and iterations it needs (see Fig. 3.5). As a result, it takes 609 iterations and 323 seconds on average for the optimizer to find a feasible solution2. Note that the threshold of the dual infeasibility of the NLP is set to be 10−3 . The number of grid points is chosen to be 15 for both the stance and flight domains. All constraints and physical limitations are configured based on the hardware capability, and the constraint violation converged below 10−6 . A significant benefit of this method’s computational efficiency is that it offers the flexibility to refine the running behavior and adjust the model parameters in practice, paving the way to actual experimental realizations. In practice, however, generating candidate gaits reliably will requires some minor heuristic tweaks to the constraints. For instance, by simply modifying the forward velocity constraint and fine-tuning 2 This algorithm runs on a Ubuntu14.04 machine, equipped with an Intelr Xeonr processor E3-1246 V3 and 32 GB of RAM. 1200 800 1000 700 800 600 600 500 400 400 200 300 0 Computation Time(s) Iteration 43 200 1.5 2 2.5 3 Vcom (m/s) Iteration Computation Time Figure 3.5: The computation performance for generating gaits at each running velocity. a few constraints to adjust the running appearance, the suggested gait generation method can find running gaits that satisfy all the physical limitations reliably. Once a reliably solvable formulation is engineered, we can solve for many gaits and inspect the solutions for trends in energy-efficient locomotion. The Specific Cost of Transport, which quantifies the energy efficiency of transportation, is embedded as the objective by the optimizer. Notably, as shown in Fig. 3.6 (a), for a faster running gait, the optimization tended to generate gaits with higher SCOT, which aligns with the sense that for a particular running pattern, faster locomotion requires greater energy cost. Further, as shown in Fig. 3.6 (c) and Fig. 3.6 (b), when the optimizer is commanded to find incrementally faster gaits (while still minimizing SCOT), the optimizer increases both the running frequency and step length to achieve a faster running speed, which agrees with Raibert’s hypothesis and indicates that both are key factors to fast running (M. H. Raibert et al., n.d.). 1.5 1 1.5 2 2.5 Vcom (m/s) 3 0.8 Step Length (m) Running Frequency (Hz) SCOT 2 4 3.5 3 0.75 0.7 0.65 0.6 0.55 1.5 2 2.5 Vcom (m/s) 3 1.5 2 2.5 3 Vcom (m/s) Figure 3.6: Multiple running gaits with forward velocity from 1.5 m/s to 3.0 m/s. Dynamic Walking with Planned Slippage Tremendous progress in realizing robust bipedal robot locomotion has been achieved in the last decade. This is in part due to the successful combination of theoretical 44 modeling and analysis using the framework of hybrid systems (A. Ames, 2014; Jessy W. Grizzle et al., 2014), application of advanced methods of nonlinear control (Khalil, 2002; E. R. Westervelt, J. W. Grizzle, Chevallereau, et al., 2007), as well as careful mechanical design and hardware implementation on various experimental platforms such as AMBER-3M (Ambrose, W. Ma, C. Hubicki, et al., 2017), DURUS (J. P. Reher et al., 2016), and Cassie (Da et al., 2016). Underlying these results, along with the successes for robots using other paradigms such as ZMP (Vulobratocić and B. Borovac, 2004; R. Tedrake et al., 2015) and spring-loaded inverted pendulum (SLIP) based models (Ioannis Poulakakis and Grizzle, 2007; Vejdani et al., 2015), is the assumption that the foot does not slip. Thus, in all of these cases, the foot acts as a stationary pivot point. While this assumption may easily hold in sterile laboratory environments where the floors can be chosen with sufficiently high friction, it becomes impractical on natural outdoor terrains; wherein there is a plethora of slippery or slightly granulated irregular surfaces. Success in challenging the stationary contact point assumption includes multi-contact walking (H.-H. Zhao et al., 2014) and bipedal running (Wen-Loong Ma, Shishir Kolathaya, et al., 2017; Sreenath, H.-W. Park, I. Poulakakis, and J. Grizzle, 2013). Figure 3.7: Slippage in the beginning of a step: pre-slip on the left and post-slip on the right. The section aims to address this fundamental assumption of no slippage by embracing its violation while still demonstrating the ability to achieve stable walking experimentally. In legged robots, foot slippage is often treated as an external disturbance which should be avoided at the gait planning stage (T.-H. Chang and Hurmuzlu, 1993; Shuuji Kajita et al., 2004), or detected and recovered in real-time by feedback control at the experimental implementation stage (Kaneko et al., 2005; 45 Vázquez and Velasco-Villa, 2013). Some of the most famous examples are Boston Dynamics’ robots BigDog (Big Dog Walking 2010) and SpotMini (Spotmini Walking 2016) successfully recovering from slippage. Conversely, legged animals across a wide range of scales show impressive adaptability to slippery surfaces on natural terrains. Stick insects when confronted with a slippery surface modulate their motor outputs to produce normal walking gaits, despite a drastic change in the loads that these limbs experience (Gruhn, Zehl, and Büschges, 2009). Slippage in the bipedal running of Guinea fowl has been studied in (Clark and Higham, 2011), showing that falling on slippery surfaces is a strong function of both speed and limb posture at touchdown. Several works in human biomechanics literature study the conditions that cause slipping (Moyer et al., 2006), its consequences (Tinetti and Williams, 1997) and dynamics (Strandberg and Lanshammar, 1981). Finally, (Spence et al., 2007) has measured feet motion in galloping gaits of horses on outdoor racing terrains and found a significant phase of hoof slippage. Recent theoretical work has incorporated slippage into classic simple planar models of legged locomotion both in passive dynamics and actuated walking — the rimless wheel (Gamus and Yizhar Or, 2015), compass biped (Gamus and Yizhar Or, 2015; Gamus and Yizhar Or, 2013), and SLIP (Yizhar Or and Moravia, 2016). The models use Coulomb’s friction law and account for stick-slip transitions and frictionbounded inelastic impacts, which add complexity to the system’s multi-domain hybrid dynamics. By investigating the influence of friction on both passive dynamics down a slope and open-loop actuated walking, it has been found in (Gamus and Yizhar Or, 2015; Gamus and Yizhar Or, 2013) that upon decreasing the friction coefficient, periodic solutions with stick-slip transitions begin to evolve while their orbital stability decreases until reaching stability loss when the friction is too low. Nonetheless, stability can be recovered when adding simple PD control to track a reference trajectory. In addition, it has been found in (Gamus and Yizhar Or, 2015; Gamus and Yizhar Or, 2013; Yizhar Or and Moravia, 2016) that periodic solutions with slipping impact showed a significant reduction in the energetic cost of transport compared to their no-slip counterparts. Yet, these promising theoretical results have never been tested and implemented experimentally on legged robots. In this section, we bridge this gap by presenting an experimental realization of stable planar bipedal robotic walking on a slippery surface. About the Robot. For the bipedal robot AMBER3-PF (PF is short for point foot, see Fig. 3.8), the configuration space is chosen as 𝑞 ∈ Q ⊆ R𝑛 , where 𝑛 is the 46 number of unconstrained degrees of freedom (DOF), i.e. without considering contact constraints. Using the floating base convention, we have 𝑞 = (𝑞 𝑏 , 𝑞 𝑙 ), where 𝑞 𝑏 ∈ R3 is the global coordinate of the body-fixed frame attached to the base linkage (torso), and 𝑞 𝑙 ∈ R4 is the local coordinate representing rotational joint angles. For planar walking on AMBER3-PF, it is chosen as 𝑞 𝑏 = ( 𝑝 𝑥 , 𝑝 𝑦 , 𝜃 𝑦 ), where 𝑝 𝑥 , 𝑝 𝑦 are the Cartesian positions of the torso and 𝜃 𝑦 is the angle between the torso and world. The local coordinates are chosen as 𝑞 𝑙 = (𝑞 𝑠𝑘 , 𝑞 𝑠ℎ , 𝑞 𝑛𝑠ℎ , 𝑞 𝑛𝑠𝑘 ), each representing the stance knee, stance hip, non-stance hip, and non-stance knee joint angle. Further, the continuous-time state space X = 𝑇 Q ⊆ R2𝑛 has coordinates 𝑥 = (𝑞 > , 𝑞¤ > ) > . The control inputs 𝑢 ∈ U ⊆ R𝑚 represents the actuator torques, with 𝑚 the total number of motors. For AMBER3-PF, we have 4 motors on both knee and hip joints. This indicates under-actuated dynamics for AMBER-3M walking. Figure 3.8: On the left: The AMBER-3M with point foot, constrained to a planar rail to walk in a 2D environment on a treadmill. On the right: the model’s configuration coordinates, with 3 global coordinates and 4 local coordinates. Continuous-time Dynamics for Stick/Slip Domains. The kinematic constraint of zero normal displacement of the stance foot reads as 𝑧 𝑠 (𝑞) = 0. An additional no-slip constraint in tangential direction occurs only in the stick domain, and is given by 𝑥 𝑠 (𝑞) = 𝑥 0 . For a particular continuous domain (𝑞, 𝑞) ¤ ∈ D𝑣 , as introduced in Sec. 2.6 and Fig. 2.5 (b), the dynamics is formulated as 𝐷 (𝑞) 𝑞¥ + 𝐻 (𝑞, 𝑞) ¤ = 𝐵𝑢 + 𝐽𝑥> (𝑞)𝜆 𝑥 + 𝐽𝑧> (𝑞)𝜆 𝑧 (3.34) 47 where 𝐷 (𝑞), 𝐻 (𝑞, 𝑞), ¤ 𝐵(𝑞) are given by the physical parameters of the robot and thus remain the same across all continuous domains. In addition, the Jacobian matrices (constraint gradient vectors) in (3.3) are defined as 𝐽𝑥 (𝑞) = 𝜕𝑥 𝑠 (𝑞) 𝜕𝑞 and 𝐽𝑧 (𝑞) = 𝜕𝑧 𝑠 (𝑞) , 𝜕𝑞 and 𝜆𝑥 , 𝜆 𝑧 are the tangential and normal forces enforcing the contact constraints. In the domain of sticking contact, expressions for the contact forces can be obtained by augmenting the second time-derivative of the holonomic constraints: ! 𝐽 (𝑞) 𝑥 𝐽¤(𝑞, 𝑞) ¤ 𝑞¤ + 𝐽 (𝑞) 𝑞¥ = 0, where 𝐽 (𝑞) = . (3.35) 𝐽𝑧 (𝑞) Eliminating 𝑞¥ from (3.34) and substituting into (3.35), one can solve for the constraint forces under sticking contact (cf. (R. M. Murray et al., 1994; Gamus and Yizhar Or, 2015)): !  −1    𝜆0𝑥 −1 −1 > ¤ 𝐽 𝐷 (𝐻 − 𝐵𝑢) − 𝐽 𝑞¤ , (3.36) = 𝐽𝐷 𝐽 𝜆0𝑧 where the dependencies on 𝑞, 𝑞, ¤ 𝑢 in (3.36) are suppressed for brevity. The forces must satisfy Coulomb’s inequality of dry friction: |𝜆0𝑥 (𝑞, 𝑞, ¤ 𝑢)| ≤ 𝜇𝜆0𝑧 (𝑞, 𝑞, ¤ 𝑢), (3.37) where 𝜇 is the coefficient of friction. When 𝜇 is too low (𝜇 ≤ 0.1 is assumed in this section), slippage of the stance foot in tangential direction begins to evolve, 𝑥¤ 𝑠 = 𝐽𝑥 𝑞¤ ≠ 0. In this case, the equation of motion (3.34) still holds while the tangential constraint in (3.35) is no longer valid. Instead, the following two equations should be augmented with (3.34): 𝐽¤𝑧 (𝑞, 𝑞) ¤ 𝑞¤ + 𝐽𝑧 (𝑞) 𝑞¥ = 0, (3.38) 𝜆 𝑥 = −sgn( 𝑥¤ 𝑠 )𝜇𝜆 𝑧 . (3.39) The tangential force during slippage reaches its maximal magnitude while opposing the slip direction. Note that we do not distinguish here between static and dynamic friction coefficients for simplicity. Combining (3.34) and (3.38) to obtain expressions for the constraint forces during slippage (Gamus and Yizhar Or, 2015):   −1   𝜆 𝑠𝑧 (𝑞, 𝑞, ¤ 𝑢) = 𝐽𝑧 𝐷 −1 (𝐽𝑧 − sgn( 𝑥¤ 𝑠 )𝜇𝐽𝑥 ) > 𝐽 𝐷 −1 (𝐻 − 𝐵𝑢) − 𝐽¤𝑧 𝑞¤ , (3.40) 𝜆𝑥𝑠 (𝑞, 𝑞, ¤ 𝑢) = − sgn( 𝑥¤ 𝑠 )𝜇𝜆 𝑧 . (3.41) 48 The inequality constraints for slippage are 𝜆 𝑧 ≥ 0 and 𝑥¤ 𝑠 ≠ 0. Finally, in both domains the non-stance foot must stay above the ground, 𝑧 𝑛𝑠 (𝑞) ≥ 0. Additionally, for a particular domain 𝑣 ∈ {0, s}, we can convert the dynamics (3.34) and constraint forces in (3.36) or (3.40) into an affine control system in the state space as: 𝑥¤ = 𝑓𝑣 (𝑥) + 𝑔𝑣 (𝑥)𝑢 ∀𝑥 ∈ D𝑣 . (3.42) Discrete Dynamics. The reset maps associated with non-impacting transitions Δ[𝑒 0→s ], Δ[𝑒 s→0 ], Δ[𝑒 s→s ] are simply an identity matrix: 𝑥 + = 𝑥 − , where 𝑥 − , 𝑥 + are the preevent and post-event states. This means that the transition is smooth in state space. In the case of collision of the non-stance foot, the transition involves impact which induces an instantaneous velocity jump 𝑞¤ + = Δ[𝑒] 𝑞¤ − . The impulse-momentum balance reads as follows 𝐷 (𝑞 𝑐 )( 𝑞¤ + − 𝑞¤ − ) = 𝐽 (𝑞 𝑐 )Λ = 𝐽𝑥 (𝑞 𝑐 ) > Λ𝑥 + 𝐽𝑧 (𝑞 𝑐 ) > Λ𝑧 , (3.43) where 𝑞 𝑐 is the robot’s configuration at collision and Λ = (Λ𝑥 , Λ𝑧 ) > are tangential and normal impulses at the colliding foot. (Note that one has to interchange the stance and non-stance variables right before impact, so that 𝐽𝑥 , 𝐽𝑧 are associated with velocities of the colliding foot.) The commonly used model is that of perfectly inelastic impact. Assuming zero tangential and normal contact velocities at the post-impact state gives 𝐽 (𝑞 𝑐 ) 𝑞¤ + = 0. Combining this with (3.43), one obtains the contact impulse and post-impact velocity as: ! 0 Λ 𝑥 Λ0 = = −(𝐽 𝐷 −1 𝐽 > ) −1 𝐽 𝑞¤ − Λ0𝑧   + −1 > −1 > −1 𝑞¤ = I − 𝐷 𝐽 (𝐽 𝐷 𝐽 ) 𝐽 𝑞¤ − where I is the identity matrix and 𝐷, 𝐽 are evaluated at 𝑞 = 𝑞 𝑐 . This is the sticking 𝐼 𝐼 impact law, associated with reset maps Δ[𝑒] for transition edges 𝑒 0→0 , 𝑒 s→0 . This 0 solution holds only if the impulses satisfy the frictional inequality |Λ𝑥 | ≤ 𝜇Λ0𝑧 . Otherwise, a slipping impact occurs where 𝐽𝑧 𝑞¤ + = 0 while 𝐽𝑥 𝑞¤ + ≠ 0. The impulses are thus related as Λ𝑥 = −sgn(𝐽𝑥 (𝑞 𝑐 ) 𝑞¤ + )𝜇Λ𝑧 . Combining this with (3.43), one obtains: Λ𝑧𝑠 = −(𝐽𝑧 𝐷 −1 𝐽˜> ) −1 𝐽𝑧 𝑞¤ −   + −1 ˜> −1 ˜> −1 𝑞¤ = 𝐼 − 𝐷 𝐽 (𝐽𝑧 𝐷 𝐽 ) 𝐽𝑧 𝑞¤ − 49 where 𝐽˜ = 𝐽𝑧 − sgn(𝐽𝑥 (𝑞 𝑐 ) 𝑞¤ + )𝜇𝐽𝑥 . This slipping impact law is associated with reset 𝐼 𝐼 . maps Δ[𝑒] for transition edges 𝑒 0→s , 𝑒 s→s Trajectory Optimization. To generate a slippery walking gait, we formulate this control problem as an implicit trajectory optimization problem. As was done in (3.33), we have Õ min 𝑢𝑖> 𝑢𝑖 𝑖 = 0, 1, 2, ...𝐾 (3.44) 𝛼,𝑥𝑖 ,𝑥¤𝑖 𝑢 𝑖 s.t. (C.1)-(C.7) C.8 slipping feasibility with 𝐾 the total number of collocation points, and the target is to minimize torque inputs. To yield a slipping gait, we additionally include feasibility constraints (C.8) from definitions in (2.28). In our formulation, we pre-specified a specific ordered sequence of transitions, indicated by the red line in Fig. 3.6 (b). Additionally, since a smoother state trajectory is preferred for experiment robustness, we further forced the static parameters to be the same across all domains. It is worthwhile to mention, this constraint is feasible if and only if the transition between domains within one step does not involve any jump in states. This yields a uniform trajectory for the multi-domain walking dynamics. Optimal Gaits. Solving the optimization problem (3.44), we obtained a two-domain slippery walking gait with slippage on the stance foot 3 cm, shown with snapshots in Fig. 3.7. The 𝑀𝐶𝑂𝑇 + from optimization is given as 0.001. The positive only mechanical cost of transport is calculated using 𝑀𝐶𝑂𝑇 + = 𝑃¯ + 𝑚𝑔𝑣 (3.45) with 𝑚 the total mass, 𝑔 the gravitational acceleration, 𝑣 the average walking speed, 𝑁 with 𝑖 ∈ {1, 2, 3...𝑁 } and 𝑁 is the total and 𝑃¯ + is the mean value of 𝑃+ = {𝑃𝑖+ }𝑖=1 number of sample points. The positive only power at sample time 𝑡𝑖 is computed by
Í 𝑃𝑖+ = 4𝑘=1 max 𝑢𝑖 (𝑘) · 𝑞¤𝑖 (𝑘), 0 , with 𝑢𝑖 ∈ R4 and 𝑞𝑖 ∈ R4 torques and velocities of the actuated joints at time 𝑡𝑖 . For a fair comparison against sticky walking, we simulate the slippery gait-based controller in a sticky environment, i.e., the ground has a much higher friction coefficient so that no slipping can happen. After 20 ∼ 30 steps, the walking converged into a new stable patten. with 𝑀𝐶𝑂𝑇 + being 0.0024. which is 140% less energy efficient than walking on a slippery surface. Further at its steady state, the 50 non-stance foot’s velocities changed from (0.563, −0.359)m/s to (0, 0)m/s through the sticky impact. The body kinetic energy changed from 6.87J to 5.84J. However, the original optimal slippery gait has a non-stance foot impact velocity changing from (0.371, −0.237)m/s to (0.251, 0)m/s, and kinetic energy changing from 3.00J to 2.63J. This agrees with the theories on energy efficiency in (Gamus and Yizhar Or, 2015). Experiment. AMBER-3M is a modular testbed to study planar bipedal locomotions. Its robustness and durability were validated in multiple experiments (Ambrose, W. Ma, C. Hubicki, et al., 2017; Tabuada et al., 2017). In this section, we particularly studied the slippery walking behavior on the point foot version (with total mass 21.6kg). As detailed in (Ambrose, W. Ma, C. Hubicki, et al., 2017), planar walking is achieved by constraining the robot on a planar rail structure and walking on a treadmill (Fig. 3.7). Further experimental details will be presented in this section. To begin, we placed a demonstration walking gait designed for a sticky surface on a slippery surface covered by some lubricant. This gait has been shown to be robust over countless trials via public demonstrations, and it seldom fell, i.e., loss of stability. However, a few drops of lubricant easily disabled its walking capability. To clarify, we consider falling and hitting the mechanical limits of the testbed both as failures. Later, we conducted four different experimental setups. For experiment 1, 2, and 3, we increased the amount of lubricant on the treadmill to induce different slippery walking behaviors and completely removed the lubricant for experiment 0. For each fixed environmental setup, we manually increased the treadmill speed to trigger different walking speeds on the slippery surface. We logged 50 seconds’ data (sampling period 3 ms) for each experiment to calculate the energy economy. Fig. 3.10 shows the phase portrait for experiment 3 which has the most slippery surface. The result is AMBER-3M is capable of walking stably on different slipping conditions, including on a sticky surface, proving its robustness and adaptability to uncertainties induced by differences between simulation and experiment. See (Wen-Loong Ma, 2019a) for the walking on slippery surfaces. Energy Economy. Previous research (Ambrose, W. Ma, C. Hubicki, et al., 2017) on AMBER-3M with a circular boom has benchmarked the energy economy of walking controllers. In this research, due to the slippage of stance foot, it became too noisy to measure the movement of the center of mass. In this work, we used the measure (3.45) for 𝑀𝐶𝑂𝑇 + . See Fig. 3.11 for the energy results. Note that we only provide positive only power because AMBER-3M’s hardware cannot do power-regeneration 51 Figure 3.9: Snapshots of one slippery walking step from Experiment 3. In the first three pictures, the left foot (stance foot) is slipping smoothly on the lubricated treadmill. Figure 3.10: Phase portrait of 50 seconds’ experimental data from Experiment 3, with walking speed (from left to right): 0.26m/s, 0.3m/s, 0.38m/s, 0.42m/s. Solid lines are for the desired values and dashed lines are for the actual measurements. of the negative work. While the energy efficiency Fig. 3.11 seems better than (Ambrose, W. Ma, C. Hubicki, et al., 2017), our measure shows experiment energy efficiency is ∼ 10 times worse than simulation, and the efficiency on different surfaces does not vary as much as simulation data. This is not only caused by different external environments such as inconsistency of the lubricated treadmill and real-world uncertainties, but we posit the dominance of the 𝑀𝐶𝑂𝑇 + by nominal energy usage of the robot. That is, due to 52 0.24 0.23 0.22 0.21 0.2 0.19 0.18 0.17 0.16 0.25 0.3 0.35 0.4 0.45 0.5 Figure 3.11: 𝑀𝐶𝑂𝑇 + of the experiments on AMEBR-3M. Those not included for certain speeds are failed experiments. Figure 3.12: The spring-legged planar running biped, DURUS-2D, during take off (left) and while airborne (right). the order of magnitude difference in the simulation and experimental 𝑀𝐶𝑂𝑇 + , the comparatively small fluctuations in the MCOT between different walking cannot be observed with the current experimental setup. Therefore, it is necessary to study differences in the 𝑀𝐶𝑂𝑇 + between slipping and nominal gaits wherein changes in energy usage can be isolated from nominal energy usage and the effects of the environment on the cost of transport. 3.4 Bipedal Running with Input to State Stability The task of controlling the bipedal robots is often a precarious balance between maintaining formal stability guarantees and expanding control capabilities. This duality has been present since the genesis of bipedal control. Beginning in the 53 1960’s, Zero Moment Point (Vukobratovic and Branislav Borovac, 2004) methods were the original foundation of formal biped control. Still, its validity required significant restrictions on the dynamics of the robot (fully-actuated flat-footed contact). In contrast, the Raibert hoppers (M. Raibert and Tello, 1986) exhibited agile bounces and flips that remain impressive today. But their control was built without the a priori confidence of formal methods. Research over the following decades has considerably narrowed this formality gap, with formal approaches rising to the challenge of underactuation (Manchester et al., 2011; H. Park et al., 2012; Sreenath, H.-W. Park, I. Poulakakis, and J. Grizzle, 2013; Ayonga Hereid, Eric. Cousineau, et al., 2016) and highly dynamic robots incorporating formal analysis in their control (Bhounsule et al., 2014; Rezazadeh et al., 2015). Bipedal robotic running, despite the decades that have passed since Raibert’s hopper, remains an extremely difficult control problem. Very few control methodologies have been presented that lead to experimental success with prominent aerial phases (Tamada et al., 2014; Sreenath, H.-W. Park, I. Poulakakis, and J. Grizzle, 2013). With an eye toward viewing bipedal running as a hybrid dynamical system: an alternating sequence of stance and flight domains with instantaneous impacts in between, the notion of hybrid zero dynamics (HZD) was used (J W Grizzle, Abba, and Plestan, 2001; A. Ames, 2014; A. E. Martin, Post, and Schmiedeler, 2014a). HZD, as previously introduced, operates on a principle of dimensional reduction, aimed at simplifying the numerous degrees of freedom present in legged machines while also allowing for underactuation. This framework was used to enable bipedal running on MABEL (Sreenath, H.-W. Park, I. Poulakakis, and J. Grizzle, 2013), a pivotal demonstration showing the intersection of theory and experiment. However, on top of the HZD framework used on MABEL, there are also important expertdriven adjustments to the implementation, like the tuning of control loops, adding feedforward trajectories, and online parameter update routines. One way to interpret this is: the gap between the assumed model and the experimental testbed necessitated modifications in the control implementation needed to realize stable robotic running. We seek to reduce further the need for this expert adjustment with formal stabilizing controllers. In this section, we will study this methodology on the DURUS-2D running robot. First, using the optimization algorithm mentioned in Sec. 3.3, we find some periodic solution to the closed-loop dynamics of running, which is stable in the sense of Lyapunov. However, the resulting gait that was built upon an ideal model and precise 54 sensing cannot guarantee experimental realization. Unlike theoretical simulation, where most variables are either measurable or exclusively solvable, real-world experiments suffer from a wide array of uncertainties. Indeed, uncertainties like unmodeled dynamics, nonlinear stiffness properties, damping effects and actuators, poor signal-to-noise ratio, and even deformations due to impacts are often observed. Therefore, we not only seek a fast optimization approach that yields feasible solutions under the assumed model but also a controller formally guaranteeing robustness under real-world constraints. In this section, we use the notion of input to state stability (ISS) that captures the practical limitations of the actuator inputs in an elegant manner. Specifically, we address the phase-based uncertainty that is typically a high deterrence in tracking parameterized functions. Similar problems involving inaccurate phase determinations were solved (Shishir Kolathaya, Ayonga Hereid, and Aaron D Ames, 2016), where pure time-based parameterizations were used. But this paper will construct time+state-based parameterizations to yield stronger stability conditions. Note that, in order to realize running, a variety of uncertainties need to be considered. So we will use the solutions from (Shishir Kolathaya and D. A. Ames, 2016; Angeli, 1999) to account for the remaining uncertainties. -qnsh -qsh z qnsk qsk -qsf qnsr qsr x ( a) (b) Figure 3.13: (a) The model of DURUS-2D with two linear springs; (b) the directed cycle structure of the multi-domain hybrid system model for DURUS-2D running. Robot and Hybrid Dynamics. Similar to the setup in Sec. 3.3, we have a hybrid model for DURUS-2D running. As shown in Fig. 3.13a, the configuration space Q ⊂ R𝑛 , 𝑛 = 9, of DURUS-2D is defined as 𝑞 = (𝑠 𝑓𝑥 , 𝑠 𝑓𝑧 , 𝜃 𝑠 𝑓 , 𝑟 𝑠𝑝 , 𝜃 𝑠𝑘 , 𝜃 𝑠ℎ , 𝜃 𝑛𝑠ℎ , 𝜃 𝑛𝑠𝑘 , 𝑟 𝑛𝑠𝑝 ) > ∈ Q, 55 where 𝑠 𝑓𝑥 and 𝑠 𝑓𝑧 are the positions of the end points of the stance foot along 𝑥 and 𝑧 directions, 𝑟 𝑠𝑝 , 𝑟 𝑛𝑠𝑝 are the deflections of the springs on stance and nonstance legs, 𝜃  are the joint angles of the stance foot, stance knee, stance hip, nonstance hip, and nonstance knee. In addition, the control inputs are defined as 𝑢 = (𝑢 𝑠𝑘 , 𝑢 𝑠ℎ , 𝑢 𝑛𝑠ℎ , 𝑢 𝑛𝑠𝑘 ) > ∈ R 𝑘 , 𝑘 = 4, which represent the torque applied at knee and hip joints. We have the same hybrid setting as those presented in (3.1) with a small modification of the continuous dynamics. The Equation of Motion (EOM) over a continuous domain D𝑣 , with 𝑣 ∈ V , (s, f), is determined by the Euler-Lagrange equation and holonomic constraints 𝐷 (𝑞) 𝑞¥ + 𝐻 (𝑞, 𝑞) ¤ = 𝐵𝑢 + 𝐽𝑣> (𝑞)𝐹𝑣 , 𝐽𝑣 (𝑞) 𝑞¥ + 𝐽¤𝑣 (𝑞, 𝑞) ¤ 𝑞¤ = 0, (3.46) where 𝐽𝑣 (𝑞) ∈ R𝑛×𝑚 is the Jacobian of the holonomic constraints ℎ𝑣 (𝑞), and 𝐹𝑣 ∈ R𝑚 is a wrench containing the constraint forces or moments, which can be explicitly solved as a function of system states and inputs. The holonomic constraints for each domain are defined as
> ℎs (𝑞) , 𝑠 𝑓𝑥 , 𝑠 𝑓𝑧 , 𝑟 𝑛𝑠𝑝 ≡ 0,
> ℎf (𝑞) , 𝑟 𝑠𝑝 , 𝑟 𝑛𝑠𝑝 ≡ 0, (3.47) meaning, the stance foot must remain on the ground during the stance domain, and stance and nonstance springs must be locked during the flight domain. Further, by defining the state vector 𝑥 = (𝑞, 𝑞) ¤ ∈ R2𝑛 , the EOM can be converted to an affine control system, as given in (3.10). We define the outputs (virtual constraint) of the system on a domain D𝑣 , 𝑣 ∈ V as 𝑦 𝑣 (𝑞) = 𝑦 𝑎 (𝑞) − 𝑦 𝑣𝑑 (𝜏𝑣 ), (3.48) where the actual output is chosen as the four actuated joint angles: 𝑦 𝑎 (𝑞) , (𝜃 𝑠𝑘 , 𝜃 𝑠ℎ , 𝜃 𝑛𝑠ℎ , 𝜃 𝑛𝑠𝑘 ) > ; and 𝑦 𝑣𝑑 is the desired output represented by a set of fifth-order Bézier curves 𝑦 𝑣𝑑 = B (𝛼𝑣 , 𝜏𝑣 ). Therefore, the output has relative degree two with respect to inputs. (3.49) 56 The phase variable 𝜏𝑣 is used to modulates the desired outputs 𝑦 𝑣𝑑 . Normally, in order to make the outputs purely state-based, we can have the phase variable 𝜏𝑣 : Q → R+ , purely a function of the robot configuration: 𝜏𝑣 (𝑞) = 𝜃 𝑠 𝑓 − 𝑝 +𝑣 , 𝑝 −𝑣 − 𝑝 +𝑣 (3.50) with 𝑝 −𝑣 , 𝑝 +𝑣 the (desired) initial and final position of 𝜃 𝑠 𝑓 for D𝑣 . Although, it must be noted that state-based modulation has implementation difficulties due to noisy sensing of underactuated degrees of freedom of DURUS-2D. This motivates the use of a time-based phase variable 𝜏𝑣 : R+ → R+ , 𝜏𝑣 (𝑡) = 5 Õ 𝑝𝑖 𝑡 𝑖 , (3.51) 𝑖=0 where 𝑝𝑖 is a set of power series polynomial coefficients obtained by a curve fitting from 𝜏𝑣 (𝑞) w.r.t. time 𝑡. This has desirable stability properties under sensory perturbations, which will be discussed later. The method of finding some optimal gait parameter 𝛼𝑣 is then formulated as an optimization problem subject to the multi-domain hybrid system model. As was done in Sec. 3.3, we can produce some natural-looking running gaits for the bipedal robot, DURUS-2D (Fig. 3.12), in 43 seconds from a zero initial guess. See Fig. 3.17. State-Based Feedback Controller and Zero Dynamics. To drive the virtual constraints (outputs) 𝑦 𝑣 → 0 exponentially for each domain D𝑣 , 𝑣 ∈ V, we utilize the feedback linearization control law: 
−1  2 (3.52) 𝑢𝑣 = 𝐿 𝑔 𝐿 𝑓 𝑦𝑣 −𝐿 𝑓 𝑦 𝑣 + 𝜇𝑣 , with 𝐿 the Lie derivative. Applying this control law yields the output dynamics 𝑦¥ 𝑣 = 𝜇𝑣 . Further, by picking 𝜇𝑣 as 2 1 𝜇𝑣 = − 𝑦¤ 𝑣 − 2 𝑦 𝑣 , 𝜀 𝜀 0 < 𝜀 < 1, (3.53) the virtual constraints will converge to zero exponentially at the rate 1/𝜀 > 0. Since the number of virtual constraints is less than the degrees of freedom of the robot, the uncontrolled states evolve according to the zero dynamics. In other words, we have a set of states defined by the vector: " # 𝑦𝑣 𝜂𝑣 , ∈ R2𝑘 𝑦¤ 𝑣 57 that are controllable, and the set of states defined by 𝑧 𝑣 , that are uncontrollable and normal to 𝜂𝑣 for each domain D𝑣 . We can then reformulate (3.46) to the following form: " # " # 0 𝑘×𝑘 1 𝑘×𝑘 0 𝑘×𝑘 𝜂¤𝑣 = 𝜂𝑣 + 𝜇𝑣 0 𝑘×𝑘 0 𝑘×𝑘 1 𝑘×𝑘 | {z } | {z } 𝐹 𝐺 𝑧¤𝑣 = 𝜔𝑣 (𝜂𝑣 , 𝑧 𝑣 ), (3.54) where 𝜔𝑣 is assumed Lipschitz continuous. The convergence of the outputs 𝜂𝑣 can be shown in terms of Lyapunov functions: 𝑉𝜀 (𝜂𝑣 ) = 𝜂𝑣> 𝑃𝜀 𝜂𝑣 , (3.55) where 𝑃𝜀 is the solution to the continuous time algebraic Riccati equation (CARE). See (A. Ames, Galloway, et al., 2014a, eq(23)). By choosing 𝜇𝑣 (𝜂) from (3.53), we have 𝛾 𝑉¤𝜀 ≤ − 𝑉𝜀 𝜀 with 𝛾 > 0 the constant obtained from the CARE. Given the control law (3.53), the controllable states 𝜂𝑣 are driven exponentially to zero. In other words, the control law (3.52) renders the zero dynamics surface exponentially stable and invariant over both continuous domains. However, due to the impact dynamics at the end of each domain, the zero dynamics invariance is not guaranteed. Therefore, the goal is to find a set of parameters 𝛼 = {𝛼s , 𝛼f }, which defines the desired outputs (3.48), to ensure there exists a periodic orbit and the zero dynamics surface: Z𝑣 = {(𝑞, 𝑞) ¤ ∈ D𝑣 | 𝑦 𝑣 (𝑞) = 0, 𝑦¤ 𝑣 (𝑞, 𝑞) ¤ = 0}, 𝑣 ∈ V, is invariant through impacts, i.e., hybrid invariant. Mathematically, hybrid invariance is represented as Δ(Zs ∩ Ss ) ⊂ Zf , Δ(Zf ∩ Sf ) ⊂ Zs . (3.56) Feedback Linearization for Time-Based Outputs By using the feedback control law given by (3.24) and (3.53), it can be shown that with sufficiently small 𝜀, the output dynamics are exponentially driven to 58 zero. In fact, (A. Ames, Galloway, et al., 2014a) shows that by picking a rapidly exponentially stable control Lyapunov function (RES-CLF), locally exponentially stable hybrid periodic orbits can be realized. However, in reality, due to the difficulty in estimating the phase variable (3.50) (which depends on the un-actuated degrees of freedom), a more robust controller is required that is less susceptible to the noisy state feedback. Motivated by the time-based implementation of the tracking controller in (Shishir Kolathaya, Ayonga Hereid, and Aaron D Ames, 2016; Jake Reher, Wen-Loong Ma, and Aaron D. Ames, 2019), the goal of this section is to construct a controller that uses a time-based instead of state-based desired trajectory for robotic running. For the ease of notations, we will omit the domain representations (the subscripts 𝑣) in this section. If the state-based desired relative degree two outputs are functions of 𝑞, 𝑦 𝑑 : Q → R 𝑘 , then the time-based desired outputs are functions of time 𝑦 𝑡,𝑑 : R+ → R 𝑘 . Following (3.51), we define the time-based output representation as follows: 𝑦(𝑡, 𝑞) = 𝑦 𝑎 (𝑞) − 𝑦 𝑡,𝑑 (𝜏(𝑡)) . (3.57) Taking the derivative of (3.57) twice, we have 𝑦¥ (𝑡, 𝑞) = 𝐿 2𝑓 𝑦 𝑎 + 𝐿 𝑔 𝐿 𝑓 𝑦 𝑎 𝑢 − 𝑦¥ 𝑡,𝑑 , (3.58) which is different from (3.23). Similar to the construction of state-based controller (3.24), we would drive 𝑦 → 0 exponentially. Therefore, the feedback controller that linearizes the time-based output is given as 
−1  2 𝑎 𝑢𝑡 = 𝐿 𝑔 𝐿 𝑓 𝑦 𝑎 −𝐿 𝑓 𝑦 + 𝑦¥ 𝑡,𝑑 + 𝜇𝑡 , (3.59) where 𝜇𝑡 is the linear feedback applied after the feedback linearization. We can either pick 𝜇𝑡 via a simple PD law: 1 2 𝜇𝑡 = − 𝑦¤ − 2 𝑦, (3.60) 𝜀 𝜀 for some 0 < 𝜀 < 1, or via an optimal control law through control Lyapunov functions (CLFs). Nevertheless, using the time-based feedback linearizing controller (3.59) reduces the nonlinear system 𝑥¤ = 𝑓 (𝑥) + 𝑔(𝑥)𝑢 to the normal form: " # " # 0 𝑘×𝑘 1 𝑘×𝑘 0 𝑘×𝑘 𝜂¤𝑡 = 𝜂𝑡 + 𝜇𝑡 0 𝑘×𝑘 0 𝑘×𝑘 1 𝑘×𝑘 | {z } | {z } 𝐹 𝑧¤𝑡 = 𝜔𝑡 (𝜂𝑡 , 𝑧𝑡 ), 𝐺 (3.61) 59 which is similar to (3.54), but with the use of time-based outputs: " # 𝑦 𝜂𝑡 , ∈ R2𝑘 . 𝑦¤ Note that the zero dynamics coordinates, 𝑧𝑡 , evolve-based on time due to the dependency on 𝜂𝑡 . Accordingly, if the time-based transverse dynamics 𝜂𝑡 are 0, we have the zero dynamics 𝑧¤𝑡 = 𝜔𝑡 (0, 𝑧𝑡 ). Convergence of the time-based outputs can be ensured by picking an appropriate time-based control law (3.60). But this controller does not necessarily ensure the convergence of the state-based outputs. We are interested in the stability of the state-based transverse dynamics 𝜂¤ = 𝐹𝜂𝑡 + 𝐺 𝜇𝑡 , given that the time-based control law is implemented on the robot. This implementation can be seen in Fig. 3.14, which is different from Fig. 3.2. Figure 3.14: A block diagram showing the time-based feedback control structure. State-Based vs. Time-Based Control Laws. Given the controller (3.60) that drives the time-based outputs 𝜂𝑡 → 0, we will study the evolution of the state-based outputs 𝜂 in (3.54). By the assumption of Theorem 1 in (A. Ames, Galloway, et al., 2014a), the controller yields an exponentially stable periodic orbit for hybrid dynamics. Therefore, we will obtain conditions for the stability of this hybrid periodic orbit when a time-based control law is applied. Picking the input (3.59) on the dynamics of state-based output 𝑦, we have 𝑦¥ =𝐿 2𝑓 𝑦 + 𝐿 𝑔 𝐿 𝑓 𝑦𝑢 𝑡 , 𝑦¥ = 𝐿 2𝑓 𝑦 + 𝐿 𝑔 𝐿 𝑓 𝑦𝑢 + 𝐿 𝑔 𝐿 𝑓 𝑦(𝑢 𝑡 − 𝑢) , {z } | {z } | ,𝑑 ,𝜇 h −1 2 𝑦¥ =𝜇 + 𝐿 𝑔 𝐿 𝑓 𝑦 (𝐿 𝑔 𝐿 𝑓 𝑦 𝑎 ) −1 (−𝐿 2𝑓 𝑦 𝑎 + 𝑦¥ > 𝑑 + 𝜇𝑡 ) − (𝐿 𝑔 𝐿 𝑓 𝑦) (−𝐿 𝑓 𝑦 + 𝜇) 𝑦¥ =𝜇 + 𝑑, i (3.62) 60 where we can see that 𝑑 = 𝐿 𝑔 𝐿 𝑓 𝑦(𝐿 𝑔 𝐿 𝑓 𝑦 𝑎 ) −1 (−𝐿 2𝑓 𝑦 𝑎 + 𝑦¥ 𝑡,𝑑 + 𝜇𝑡 ) − 𝜇 + 𝐿 2𝑓 𝑦, (3.63) is obtained by substituting for 𝑢 𝑡 , 𝑢 from (3.24) and (3.59). We can interpret (3.62) as that, the stabilizing control input 𝜇(𝜂) (which is state-based) should have been applied, but instead, the time-based input 𝜇 + 𝑑 was applied to the state-based output dynamics of 𝑦. Applying a time-based feedback control law completely eliminated the dependency on the noisy phase variable 𝜏(𝑞), but the consequence is the appearance of the disturbance input 𝑑. The expression for 𝑑 can be further simplified to 𝑑 (𝑡, 𝑞, 𝑞, ¤ 𝑞, ¥ 𝜇𝑡 , 𝜇) = (𝜇𝑡 − 𝜇) + ( 𝑦¥ 𝑡,𝑑 − 𝑦¥ 𝑑 ). (3.64) We know that, 𝑦 𝑑 = 𝑦 𝑑 (𝜏(𝑞)) (for bipedal robots), and it can be observed that 𝑑 becomes small by minimizing the error 𝑦¥ 𝑡,𝑑 (𝜏(𝑡)) − 𝑦¥ 𝑑 (𝜏(𝑞)). Therefore 𝑑 can be termed time-phase uncertainty, or just phase uncertainty. In the context of linear systems, it is important to have bounded state-based output dynamics if 𝑑 is bounded. Of course, the time-based outputs 𝜂𝑡 → 0. Denoting the supremum of the uncertainty over time as k𝑑 k ∞ , we can easily establish that a bounded 𝑑 results in bounded outputs 𝑦, 𝑦¤ (or just 𝜂), for the continuous dynamics. However, due to the impact dynamics that are not just nonlinear but also extremely destabilizing (the noisy impacts can be observed in the video (Wen-Loong Ma, 2016c)), output boundedness cannot be guaranteed for the hybrid dynamics. This motivates using the notion of input to state stability to establish boundedness on the state-based outputs for bipedal robotic running on DURUS-2D. Going back to (3.62), we can substitute this formulation in (3.54), which results in the following representation:    𝜂¤ = 𝐹𝜂 + 𝐺 𝜇 + 𝐺𝑑  . (3.65)   𝑧¤ = 𝜔(𝜂, 𝑧)  As mentioned before, we are free to pick 𝜇(𝜂) (say (3.53)), since the actual control input applied is time-based 𝜇𝑡 (𝜂𝑡 ) (from (3.60)) which is implicit in 𝑑. From the point of view of the state-based outputs 𝜂, we have the following representation dynamics of the Lyapunov function, which is given in (3.55): 𝑉¤𝜀 = 𝜂> (𝐹 > 𝑃𝜀 + 𝑃𝜀 𝐹)𝜂 + 2𝜂> 𝑃𝜀 𝐺 𝜇 + 2𝜂> 𝑃𝜀 𝐺𝑑. (3.66) 61 This is obtained by substituting (3.65) for 𝜂. ¤ Using the linear feedback law 𝜇(𝜂) from (3.53), the following is obtained: 𝛾 𝑉¤𝜀 ≤ − 𝑉𝜀 + 2𝜂> 𝑃𝜀 𝐺𝑑. 𝜀 (3.67) It should be noted that even though the time-based controller leads to convergence of time-based outputs 𝑦 → 0, (3.67) extends it to state-based outputs 𝑦 that are driven exponentially to an ultimate bound; and this ultimate exponential bound is explicitly derived from 𝑑, which is established via the notion of input to state stability (ISS), which is given below. Input-to-State Stability (ISS) We will first introduce the basic definitions and results related to ISS for a general nonlinear system and then focus on the running dynamics. See (Eduardo D Sontag, 2008) for a detailed survey on ISS. Assume we have a general nonlinear system, represented in the following fashion: 𝑥¤ = 𝑓 (𝑥, 𝑑), (3.68) with 𝑥 taking values in Euclidean space R𝑛 , the input 𝑑 ∈ R𝑚 for some positive integers 𝑛, 𝑚. The mapping 𝑓 : R𝑛 × R𝑚 → R𝑛 is considered Lipschitz continuous and 0 = 𝑓 (0, 0). It should be noted that in this system, we considered general dynamics 𝑓 and general inputs 𝑑. In the context of the robotic problem in this thesis, the construction is such that a stabilizing controller 𝑢(𝑥) has been applied (such as (3.24)), which results in a closed-loop system 𝑓 . Any deviation from this stabilizing controller can be viewed as 𝑢(𝑥) + 𝑑, with 𝑑 being a new disturbance input. In the example of the linearized system (3.65), a suitable stabilizing controller 𝜇(𝜂) is applied and the effect of the disturbance input 𝑑 is analyzed. We assume that 𝑑 takes values in the space of all Lebesgue measurable functions: k𝑑 k ∞ = ess. sup𝑡≥0 k𝑑 (𝑡)k < ∞, which can be denoted as 𝑑 ∈ L∞ . Class K∞ and KL functions. A class K∞ function is a function 𝛼 : R+ → R+ which is continuous, strictly increasing, unbounded, and satisfies 𝛼(0) = 0. And a class KL function is a function 𝛽 : R+ × R+ → R+ such that 𝛽(𝑟, .) ∈ K∞ for each 𝑡 and 𝛽(., 𝑡) → 0 as 𝑡 → ∞. 62 We can now define ISS for the system (3.68). Definition 8. The system (3.68) is input to state stable (ISS) if there exists 𝛽 ∈ KL, 𝜄 ∈ K∞ such that |𝑥(𝑡, 𝑥0 )| ≤ 𝛽(|𝑥 0 |, 𝑡) + 𝜄(k𝑑 k ∞ ), ∀𝑥 0 , ∀𝑡 ≥ 0, (3.69) and considered locally ISS, if this inequality is valid for an open ball of radius 𝑟, 𝑥 0 ∈ B𝑟 (0). Definition 9. The system (3.68) is exponentially input to state stable (e-ISS) if there exists 𝛽 ∈ KL, 𝜄 ∈ K∞ and a positive constant 𝜆 > 0 such that |𝑥(𝑡, 𝑥0 )| ≤ 𝛽(|𝑥 0 |, 𝑡)𝑒 −𝜆𝑡 + 𝜄(k𝑑 k ∞ ), ∀𝑥 0 , ∀𝑡 ≥ 0, (3.70) and considered locally e-ISS, if the inequality (3.70) is valid for an open ball of radius 𝑟, 𝑥 0 ∈ 𝐵𝑟 (0). Definition 10. The system is said to hold the asymptotic gain (AG) property if there exists 𝜄 ∈ K∞ such that lim𝑡→∞ |𝑥(𝑡, 𝑥0 )| ≤ 𝜄(k𝑑 k ∞ ), ∀𝑥 0 , 𝑑. (3.71) Definition 11. The system is said to be zero stable if there exists 𝛽 ∈ KL such that: |𝑥(𝑡, 𝑥0 )| ≤ 𝛽(|𝑥0 | , 𝑡), ∀𝑥 0 , ∀𝑡 ≥ 0. (3.72) ISS-Lyapunov functions. We can develop Lyapunov functions that satisfy the ISS conditions and achieve the stability property. Definition 12. A smooth function 𝑉 : R𝑛 → R+ is an ISS-Lyapunov function for ¯ 𝛼, 𝜄 ∈ K∞ such that (3.68) if there exist functions 𝛼, 𝛼, 𝛼(|𝑥|) ≤ 𝑉 (𝑥) ≤ 𝛼(|𝑥|) ¯ 𝑉¤ (𝑥, 𝑑) ≤ −𝛼(|𝑥|) for |𝑥| ≥ 𝜄(k𝑑 k ∞ ). (3.73) The following lemma establishes the relationship between the ISS-Lyapunov function and the ISS of (3.68). Lemma 1. The system (3.68) is ISS if and only if it admits a smooth ISS-Lyapunov function. 63 The proof of was given in (Eduardo D Sontag, 2008). In fact, the inequality can be made stricter by using the exponential estimate: 𝑉¤ (𝑥, 𝑑) ≤ −𝑐𝑉 (𝑥) + 𝜄(k𝑑 k ∞ ), ∀𝑥, 𝑑. (3.74) which is then called the e-ISS Lyapunov function. Phase Uncertainty to State Stability (PSS) Coming back to our discussion in (3.67), we now define the notion of phase to state stability (PSS). Without loss of generality, we denote (𝜂, 𝑧) = (𝜂𝑣 , 𝑧 𝑣 ), and the subscript 𝑣 will be specified when a specific domain (s or f) is considered. Definition 13. Assume a ball of radius 𝑟 centered at the origin 0. The system given by (3.65) is locally phase to 𝜂 stable, if there exists 𝛽 ∈ KL, 𝜄 ∈ K∞ such that |𝜂(𝑡)| ≤ 𝛽(|𝜂(0)| , 𝑡) + 𝜄(k𝑑 k ∞ ), ∀𝜂(0) ∈ B𝑟 (0), ∀𝑡 ≥ 0, (3.75) and it is locally PSS if |(𝜂(𝑡), 𝑧(𝑡))| ≤ 𝛽(|(𝜂(0), 𝑧(0))| , 𝑡) + 𝜄(k𝑑 k ∞ ), ∀𝜂(0) ∈ B𝑟 (0), ∀𝑡 ≥ 0. (3.76) Based on the asymptotic gain and zero stability property of the system (3.65) w.r.t. the phase uncertainty 𝑑, we have the following lemma. Lemma 2. Given the controller 𝜇(𝜂) in (3.53), the system (3.65) is phase to 𝜂 stable. Proof. Based on the constructions of the Lyapunov function 𝑉𝜀 in (3.66), we have the dynamics of the from (3.67): 𝛾 𝑉¤𝜀 ≤ − 𝑉𝜀 + 2𝜂> 𝑃𝜀 𝐺𝑑 𝜀 𝛾 ≤ − 𝑉𝜀 + 2 |𝜂| k𝑃𝜀 k 2 k𝑑 k ∞ 𝜀 𝛾 4𝑐 2 k𝑑 k ∞ , ≤ − 𝑉𝜀 for |𝜂| ≥ 2𝜀 𝛾𝑐 1 𝜀 which is thus an ISS-Lyapunov function, as defined in (3.73). (3.77)  Time+state-based Control Law. We can also realize exponentially ultimate boundedness of the entire dynamics by appending a state-based linear feedback law to the time-based feedback controller in (3.60): 𝑢𝑇 = 𝑢 𝑡 + 𝜇, ¯ (3.78) 64 which results in the following output dynamics in the place of (3.62): 𝑦¥ = 𝜇 + 𝑑 + 𝐿 𝑔 𝐿 𝑓 𝑦 𝜇. ¯ (3.79) ¯ where 𝐷 and 𝐵¯ are obtained 𝐿 𝑔 𝐿 𝑓 𝑦 can be explicitly computed as 𝐿 𝑔 𝐿 𝑓 𝑦 = J 𝐷 −1 𝐵, from the generalized manipulator dynamics in (2.18), and J , 𝜕 𝑦(𝑞) 𝜕𝑞 is the Jacobian of the outputs. Since 𝐷 is invertible, it can be easily shown that J 𝐷 −1 𝐵¯ is invertible. By applying the controller (3.78), the full-order system (3.65) will have an extra input 𝜇¯ that yields:    𝜂¤ = 𝐹𝜂 + 𝐺 𝜇 + 𝐺𝑑 + 𝐺 J 𝐷 −1 𝐵¯ 𝜇¯  ,   𝑧¤ = 𝜔(𝜂, 𝑧)  then (3.67) gets reformulated as 𝛾 𝑉¤𝜀 ≤ − 𝑉𝜀 + 2𝜂> 𝑃𝜀 𝐺𝑑 + 2𝜂> 𝑃𝜀 𝐺 J 𝐷 −1 𝐵¯ 𝜇. ¯ 𝜀 By picking a control law for the auxiliary input: 1 ¯ −1 𝐺 > 𝑃𝜀 𝜂, (J 𝐷 −1 𝐵) 2𝜀¯ we have the following simplification of (3.81): 𝜇¯ = − 𝛾 1 𝑉¤𝜀 ≤ − 𝑉𝜀 + 2𝜂> 𝑃𝜀 𝐺𝑑 − 𝜂> 𝑃𝜀 𝐺𝐺 > 𝑃𝜀 𝜂. 𝜀 𝜀¯ Therefore, by defining the positive semi-definite function (3.80) (3.81) (3.82) (3.83) 𝑉¯𝜀 (𝜂) = 𝜂> 𝑃𝜀 𝐺𝐺 > 𝑃𝜀 𝜂, we can pick 𝜀¯ small enough to cancel the effect of phase uncertainty on the dynamics. Lemma 2 can now be redefined to obtain exponential ultimate boundedness for the new control input (3.78). Lemma 3. Given the controllers 𝜇(𝜂) in (3.53), and 𝜇(𝜂) ¯ in (3.82), the system (3.80) is exponentially phase to 𝜂 stable w.r.t. the input disturbance 𝑑 ∈ L∞ . Proof. We again pick the derivative of the Lyapunov function 𝑉𝜀 resulting in 𝛾 1 𝑉¤𝜀 ≤ − 𝑉𝜀 − 𝜂> 𝑃𝜀 𝐺𝐺 > 𝑃𝜀 𝜂 + 2𝜂> 𝑃𝜀 𝐺𝑑 𝜀 𝜀¯ 𝛾 2𝜀𝑐 ¯ 2 ≤ − 𝑉𝜀 for |𝜂| ≥ 2 2 k𝑑 k ∞ , 𝜀 𝑐1 𝜀 which satisfies the exponential estimate given by (3.74). (3.84)  65 Periodic Orbit. Now Lemma 3 can be extended to include the uncontrolled states 𝑧 given that they are stable on the zero dynamics manifold. Let Y ⊂ R2𝑘 , Z ⊂ R2(𝑛−𝑘) , and 𝜙𝑡 (𝜂, 𝑧) be the flow of the dynamics in (3.80) with the initial condition: (𝜂, 𝑧) ∈ Y × Z; And let the flow 𝜙𝑡 be periodic with the period 𝑇∗ > 0, and a fixed point (𝜂∗ , 𝑧∗ ) if 𝜙𝑇∗ (𝜂∗ , 𝑧∗ ) = (𝜂∗ , 𝑧∗ ). Associated with the periodic flow is the periodic orbit defined as O , {𝜙𝑡 (𝜂∗ , 𝑧∗ ) ∈ Y × Z | 0 ≤ 𝑡 ≤ 𝑇∗ }. (3.85) Similarly, we denote the flow of the zero dynamics 𝑧¤ = 𝜔(0, 𝑧) (3.86) from (3.80) by 𝜙𝑡 | 𝑧 , and for a periodic flow we denote the corresponding periodic orbit by O𝑧 = O| 𝑧 . Due to the invariance of the zero dynamics (guaranteed by the HZD condition in Sec. 3.3), we have the mapping O = 𝜄0 (O𝑧 ), where 𝜄0 : Z ↩→ Y × Z is the canonical embedding. For any (𝜂, 𝑧), we can denote the distance from O as k(𝜂, 𝑧) k O . We now have the following theorem to establish phase-to-state stability of the periodic orbit O. Theorem 1. Assume that the periodic orbit O𝑧 ⊂ Z is exponentially stable in the zero dynamics. Given the controllers 𝜇(𝜂) in (3.53), 𝜇(𝜂) ¯ in (3.82) applied on (3.80), that render the outputs exponential phase to 𝜂 stable, then the periodic orbit O obtained from the canonical embedding is exponentially phase to state stable. Proof. By the converse Lyapunov theorems, we can construct a quadratic Lyapunov function for the zero dynamics, 𝑉𝑧 (𝑧) that satisfies the exponential inequality constraints: 𝑟 1 k𝑧k 2O𝑧 ≤ 𝑉𝑧 ≤ 𝑟 2 k𝑧k 2O𝑧 , 𝑉¤𝑧 ≤ −𝑟 3𝑉𝑧 , 𝜕𝑉𝑧 ≤ 𝑟 4 k𝑧k O𝑧 , 𝜕𝑧 (3.87) 66 where k𝑧k O𝑧 = k (0, 𝑧)k O . We consider the following Lyapunov candidate for the full-order dynamics 𝑉 (𝜂, 𝑧) = 𝜎𝑉𝑧 (𝑧) + 𝑉𝜀 (𝜂). (3.88) This Lyapunov function is quadratic and satisfies the boundedness properties. Denote 𝐿 𝑞 as the Lipschitz constant of 𝜔(·, 𝑧). We can take the derivative 𝜕𝑉𝑧 𝑉¤ = 𝜎 𝜔(0, 𝑧) + 𝑉¤𝜀 𝜕𝑧 𝜕𝑉𝑧 𝜕𝑉𝑧 (𝜔(𝜂, 𝑧) − 𝜔(0, 𝑧)) + 𝑉¤𝜀 =𝜎 𝜔(0, 𝑧) + 𝜎 𝜕𝑧 𝜕𝑧 2𝜀𝑐 ¯ 2 𝛾 ≤ −𝜎𝑟 3𝑉𝑧 + 𝜎𝑟 4 𝐿 𝑞 k𝑧k O𝑧 k𝜂k − 𝑉𝜀 , for |𝜂| ≥ 2 2 k𝑑 k ∞ . 𝜀 𝑐1 𝜀 (3.89) The rest of the derivation follows (A. Ames, Galloway, et al., 2014a, Appendix A.B), and the bounds on 𝜂 are obtained from (3.84). By picking a suitable 𝜎, we can render 𝑉¤𝜀 negative definite, which satisfies Lemma 1.  This theorem has powerful implications due to the elimination of the noisy phase variable estimation. This elimination affects tracking, which yields lower errors than that for the noisy phase-based modulation. The time-based phase modulation is a smooth and better candidate to replicate the unknown actual phase of the robot. This methodology can be easily extended to all kinds of additive uncertainties observed in hybrid systems in general. See (Shishir Kolathaya and D. A. Ames, 2016) for the analysis on parameter uncertainty. Fig. 3.15 depicts the periodic orbit O and the tube, which is defined by the bound 𝛿 𝑑 . This theorem means that by using a time+state-based RES-CLF, any trajectory starting close to the tube will ultimately enter the tube defined by 𝛿 𝑑 as long as k𝑑 k ∞ < 𝛿 𝑑 . This is also illustrated in Fig. 3.16. Simulation and Experimental Realization of Bipedal Running With the optimal running gait generated (introduced in Sec. 3.3) and time-dependent RES-CLF controller defined, we achieved sustainable robotic running. The goal of this section is to describe the experimental setup and the control methods adopted to realize stable running on DURUS-2D. DURUS-2D Hardware. A popular approach for robotic running is to utilize the spring-loaded inverted pendulum (SLIP) model (R. Blickhan, n.d.; Rezazadeh et al., 2015), where the presence of springs allows for storing energy during high-speed 67 ( a) (b) ( c) Figure 3.15: Limit cycles of (a) simulation where time-based IO + state-based PD controller was applied for 100 steps; (b) Simulation where white noise was added to 𝜏𝑣 (𝑞); (c) Experimental data. Note the solid lines are the designated gait from optimization. impacts, thereby improving energy efficiency and torque performance. The previous version of DURUS-2D (E. Cousineau and A. D. Ames, 2015), had rigid carbon fiber calves, unlike the current version, which has a linear spring at the end of each aluminum calf. The spring has a stiffness of 20000 N m and damping constant of 100 N s/m. In addition, an 11.5 kg torso is installed to resemble the human weight distribution. The positions and velocities of the torso, knee, and hip joints are measured by the attached incremental encoders. Further, the actuated joints, knees and hips, are powered by BLDC motors via cycloidal gear reduction, which provides a maximum continuous torque of 200 N·m. With the new legs, DURUS-2D weights 41.7 kg. EtherLAB software with MATLAB Simulink is built into DURUS-2D to guarantee a hard real-time environment. Other details about the electrical and software system can be found in (E. Cousineau and A. D. Ames, 2015). Figure 3.16: Bounded tube around the designed periodic orbit. 68 0 ms 70 ms 140 ms 210 ms 280 ms 350 ms 420 ms Figure 3.17: Running tiles of simulation (top) vs. experiment (bottom) for one step. Experimental Setup. As shown in Fig. 3.18, DURUS-2D is mounted on a carbon fiber boom structure which is attached to a cage frame via a fixed one-dimensional track. This setup is used to isolate the lateral motions, leaving DURUS-2D to move freely in the sagittal plane. Moreover, the treadmill speed is measured by an encoder wheel and fed to the robot as environment feedback. Figure 3.18: Experimental setup for DURUS-2D running: 1) control station computer, 2) emergency stop, 3) four Li-Po batteries, 4) tripping harness, 5) treadmill control panel, 6) encoder wheel to measure treadmill speed, 7) treadmill. Switching logic. Guard condition is used to switch the controller to the subsequent domain (stance or flight). In simulation, the guard condition is triggered when non-stance spring returns to the neutral position for stance domain, i.e., 𝑟 𝑠𝑝 = 0. 69 And when the nonstance foot lands on the ground, i.e., 𝑛𝑠 𝑓𝑧 = 0, the flight domain ends. However, due to a lack of an effective sensing mechanism, we developed a time+state-based switching logic for experiments. For a particular domain D𝑣 , the maximum value of time 𝑡 𝑣𝑚𝑎𝑥 and phase variable 𝜏𝑣𝑚𝑎𝑥 can be obtained from the gait design process. Then the guard condition is triggered when 𝑡 > 1.2𝑡 𝑣max . But if 𝑡 < 1.2𝑡 𝑣max , the guard will be triggered if 𝜏𝑣 (𝑞) > 𝜏𝑣max . This way, the controller can respond to the feedback similarly to simulation while allowing for sensing noise of the phase variable. Experimental Controller. Motivated by the results on ISS properties of PD controlled robotic systems in (Angeli, 1999), we can replace the time-based IO with a time-based PD control law, and claim that the resulting system still retains desirable stability properties. For a robot like DURUS-2D , the inertia of the motor (proportional to the square of the gear ratio) coupled with relatively light legs results in stronger ISS conditions for model-based uncertainty (see Shishir Kolathaya and D. A. Ames, 2016; Angeli, 1999). We therefore pick a time+state based PD control law as follows 𝑢 𝐸 = −𝐾 𝑝𝑡 𝑦 𝑡𝑣 − 𝐾 𝑑𝑡 𝑦¤ 𝑡𝑣 − 𝐾 𝑝 𝑦 𝑣 − 𝐾 𝑑 𝑦¤ 𝑣 , (3.90) where 𝐾 𝑝𝑡 , 𝐾 𝑑𝑡 , 𝐾 𝑝 , 𝐾 𝑑 are constant gain matrices with appropriately tuned values. Results. We first validate the proposed control law in simulation. As explained in Sec. 3.3, a HZD running gait was first generated that meets all physical limitations, which assumed a feedback linearization controller (3.24). Then we utilized the time-based feedback linearization + state-based PD control law given by (3.78) in simulation; stable trajectory tracking is achieved that is ultimately bounded to the periodic orbit (see Fig. 3.21b for the evolution of virtual constraints, i.e., output errors, for 100 steps, and Fig. 3.15a for phase portrait that is also bounded around the desired gait) when the phase uncertainty is bounded (Fig. 3.21a). However, in experiments, noisy sensing often occurs around impact dynamics. Therefore to simulate an unideal case, we added a noise signal with amplitude 0.1 to 𝜏s (𝑞) before and after impacts (see Fig. 3.21c). By applying the same controller, ultimate boundless was also achieved (see Fig. 3.21d and Fig. 3.15b) and a stable bipedal running is accomplished. The running tiles are shown in Fig. 3.17. These simulated results, as a proof of concept, aligned with Theorem 1. 70 Foot height(m) In reality, neither state-based phase measurements 𝜏𝑣 (𝑞), nor 0.1 time-based phase calculation 𝜏𝑣 (𝑡) is capable of producing suc0.05 cessful bipedal running (watch (Wen-Loong Ma, 2016c) for the failed running when the pure 0 17 18 19 time-based controller was used). time(s) lfz rfz However, by applying a variant of time + state-based feedback Figure 3.19: Left and right foot height (ground as shown by (3.90), a sustainable clearance). running on DURUS-2D is immediately shown in real-world experiments. Multiple views in (Wen-Loong Ma, 2016a) show that the running is repeatable for over 150 steps. The phase portrait for 30 steps is shown in Fig. 3.15c, and the output errors are shown in Fig. 3.21f, both of which have shown ultimate boundedness. Further, the time-based and state-based phase variables are shown in Fig. 3.21e. Experimental running tiles are compared to simulation at Fig. 3.17. The most distinguishable feature of running, foot clearance, is shown in Fig. 3.19, with the maximum clearance 13 cm, and the flight domain takes 60% of one step. The average running speed is 1.75 m/s, and the measured average mechanical cost of transport (MCOT) for 100 steps is 0.5287. 1 0.2 Angles (rad) Angles (rad) 0.8 0.6 0.4 0.2 0 15 16 17 18 time(s) 19 -0.2 -0.4 -0.6 15 -0.1 1.2 -0.2 1 -0.3 -0.4 -0.5 -0.6 15 16 a θsh d θsk Angles (rad) Angles (rad) a θsk 0 17 18 time(s) 19 d θsh 0.8 0.6 0.4 16 a θnsh 17 18 time(s) 19 d θnsh 0.2 15 16 a θnsk 17 18 time(s) 19 d θnsk Figure 3.20: The actual vs. desired joint angles from the experiment. 71 Virtual Constrinats(rad) 0.5 Phase Variable 1 0.8 0.6 0.4 0.2 0 0 -0.5 0 0.5 1 1.5 0 2 10 20 30 time(s) time(s) τ (q) τ (t) y(1) ( a) y(2) y(3) y(4) (b) Virtual Constrinats(rad) 0.5 Phase Variable 1 0.8 0.6 0.4 0.2 0 0 -0.5 0 0.5 1 0 1.5 10 τ (q) 20 30 time(s) time(s) τ (t) y(1) ( c) y(2) y(3) y(4) (d) 0.5 Virtual Constrinats(rad) Phase Variable 1 0.8 0.6 0.4 0.2 0 0 0.5 1 1.5 0 -0.5 10 15 τ (q) τ (t) ( e) 20 time(s) time(s) y(1) y(2) y(3) y(4) ( f) Figure 3.21: The left column is the phase variables which are used to calculate timeand state-based outputs: 𝑦 𝑑,𝑡 (𝑡) = 𝑦 𝑣𝑑 (𝜏𝑣 (𝑡)), 𝑦 𝑑 (𝑞) = 𝑦 𝑣𝑑 (𝜏𝑣 (𝑞)); the right column is output errors (virtual constraints) showing ultimate boundedness. (a, b, c, d) are from two simulations with controller given by (3.78). And (c, d) has a sinusoidal disturbance with 10% amplitude added to 𝜏𝑣 (𝑞); (e, f) are from experiments. 72 Chapter 4 COUPLED CONTROL SYSTEM AND QUADRUPEDAL APPLICATIONS This chapter studies the control problem of quadrupedal locomotion. We first state our problem from the aforementioned HZD methodology in Chapter 3 to quadruped, enabling versatile gait generation and experimental realization. However, when it comes to real-world implementation, this methodology lacks implementation robustness and is thus not practical for field testing and iterative engineering. We then develop a framework, coupled control system (CCS), to design gaits and controllers for quadrupedal locomotion. The result is that we are able to apply this framework to two different quadrupedal robots (see Fig. 4.1) walking on various rough and sloped terrains. Figure 4.1: (a) Vision 60 V3.2, 26 kg, 0.45m tall; (b) Vision 60 V3.9, 44 kg, 0.55m tall; (c) Configuration coordinates of both quadrupedal robots. Both robots were manufactured by Ghost Robotics. 4.1 HZD, from Bipedal to Quadrupedal Robots Quadrupedal locomotion has a long and rich history of outstanding agility and dynamic stability without formal analysis, thanks to the multi-support nature of such systems. Some famous quadrupedal examples include, but are not limited to, the BigDog (M. Raibert, Blankespoor, et al., 2008), Minitaur (De and Daniel E. Koditschek, 2018), ANYmal (M. Hutter et al., 2016), and Cheetah robot (Boussema et al., 2019a). State-of-the-art approaches for the controls and planning of quadrupeds mainly utilize model reduction to partly mitigate the computational complexity of the full-order techniques arising from nonlinearity and hybrid nature of models. For instance, massless legs, the linear inverted pendulum model (S. Kajita, Tani, and 73 Kobayashi, 1990; Hof, Gazendam, and Sinke, 2005) and planar motion planning are often utilized assumptions. While they do have many implementation advantages, one needs to design controllers that overcome the uncertainty induced by the difference between modeling and reality. For examples, the legs’ total mass accounts for 44% of the mass of the whole body for the robot studied here in Fig. 4.15. Further, formal guarantees on stability are rarely studied in quadrupedal research. On the other hand, the hybrid system approaches have become a powerful tool for modeling the dynamics of bipedal locomotion, in which, steady state locomotion corresponds to periodic solutions of these hybrid dynamical systems. One of the hybrid control approaches is the hybrid zero dynamics (HZD) framework (E. Westervelt et al., 2007). HZD is an extension of the notion of Byrnes-Isidori zero dynamics (Isidori, 1995) to hybrid models of locomotion for which the resultant zero dynamics manifolds are invariant under the continuous- and discrete-time dynamics. HZD has been successful for designing gaits for bipedal locomotion and provides experimentlevel controllers, see e.g., (J.W. Grizzle, G. Abba, and F. Plestan, 2001; A. Ames, Galloway, et al., 2014a; C. Chevallereau, J.W. Grizzle, and Shih, 2009; Sreenath, H.-W. Park, I. Poulakakis, and J. W. Grizzle, 2011; H.-W. Park, Ramezani, and J.W. Grizzle, 2013; Akbari Hamed, Buss, and J.W. Grizzle, 2016; R. Gregg and Righetti, 2013; A. E. Martin, Post, and Schmiedeler, 2014b; H. Dai and R. Tedrake, 2012; I. Poulakakis and J.W. Grizzle, 2009; Johnson, Burden, and D. E. Koditschek, 2016), but has not yet been applied to the control of quadrupeds. The challenge in computation and controls mainly arises from the increased degrees of freedom (DOF) and richer contact scenarios of quadrupeds over bipeds. In this section, instead of building a controller based on the aforementioned empirically simplified models, we follow the HZD approach to design gaits for the full-order rigid-body dynamics of quadrupedal robots. In particular, we model their dynamics as a hybrid system, optimize trajectories via a HZD optimization algorithm, analyze the dynamic stability via the Poincaré return map and then validate the theoretically stable controller with experiments on a quadrupedal behavior, ambling on the Vision 60 robot (Fig. 4.1 (a)). The Robot and Dynamics Following the abstract construction of Sec. 2.1, we give more details of the configuration coordinates here. As shown in Fig. 4.1 (c), we model the quadruped as a 13-link system: one body link and four legs, each of which has three children 74 links —the hip, upper, and lower links. The configuration variables of Vision 60 are denoted by 𝑞 ∈ Q ⊂ R𝑛 where 𝑛 = 18 is the total number of degrees of freedom (DOF) without considering any contact constraints. Utilizing the floating base convention (Jessy W. Grizzle et al., 2014), we can have 𝑞 > = (𝜉 > , 𝜃 > ), in which 𝜉 ∈ R3 × SO(3) represents the global Cartesian position and orientation of a frame attached to the body linkage, and the local coordinates 𝜃 ∈ R𝑚 , with 𝑚 = 12, denote the 12 joint variables: hip roll, hip pitch, and knee angles. These angles are denoted by 𝜃 ℎ𝑟 𝑗 , 𝜃 ℎ𝑝 𝑗 , 𝜃 𝑘 𝑗 for the 𝑗-th leg, all of which are actuated by Brushless DC electric (BLDC) motors. This yields the system’s total DOF (without considering any constraints, see Sec. 2.2) to be 18 and control inputs 𝑢 ∈ R12 . With different scenarios of foot contacts with the ground, we have a mixture of overactuated, fully-actuated, and underactuated domains (i.e., phases) for the dynamics. We can define the state space X = 𝑇 Q ⊆ R2𝑛 with the state vector 𝑥 > = (𝑞 > , 𝑞¤ > ). We consider the nonlinear model of quadrupedal locomotion as a hybrid dynamical system, which is an alternating sequence of continuous- and discrete-time dynamics. The order of the sequence is dictated by contact events. In comparison with bipedal walking, the increased number of contact points of quadrupeds increased the complexity of the hybrid model substantially. In this section, we introduce a unified model for quadrupedal behaviors including walking, ambling, and trotting, based on which we design full-model-based optimal controllers as well as simulation validation. The Continuous-Time Domain, Constrained Dynamics Given the floating base coordinates, we can derive the unconstrained dynamics, i.e. without any contact constraints, by the Euler-Lagrange equations as: 𝐷 (𝑞) 𝑞¥ + 𝐻 (𝑞, 𝑞) ¤ = 𝐵𝑢 where the notations follows Sec. 2.3. Note that here the actuation matrix 𝐵 ∈ R𝑛×𝑚 only contains 0, 1 as its entries. Because quadrupedal systems’ numerous contact scenarios, we consider then 𝑘 ∈ {1, 2, 3, 4} feet standing on the ground, which means 𝑘¯ = 4 − 𝑘 feet are swinging in the air. This creates a variety of contact situations, which defines different types of quadrupedal behaviors (see Fig. 4.2). For each contact situation, we associate a continuous domain: D𝑣 , {(𝑥, 𝑢) | ℎ𝑣 (𝑞) = ℎ¤ 𝑣 (𝑞, 𝑞) ¤ = 0, 𝑁𝑣  0, ℎs,𝑣 (𝑞)  0} (4.1) 75 where 𝑣 ∈ {1, 2, 3...} is the domain index and ℎ𝑣 (𝑞) ∈ R 𝑘 is the height of all standing feet with ground reaction force 𝑁𝑣 (𝑥, 𝑢) ∈ R 𝑘 . The height of the other feet, ¯ referred as the swing feet, is given by ℎs,𝑣 (𝑞) ∈ R 𝑘 . We then have the constrained dynamics for D𝑣 as:    𝐷 (𝑞) 𝑞¥ + 𝐻 (𝑞, 𝑞)  ¤ = 𝐵𝑣 𝑢 + 𝐽𝑣𝑇 (𝑞) 𝜆 𝑣   𝐽𝑣 (𝑞) 𝑞¥ + 𝐽¤𝑣 (𝑞, 𝑞) ¤ 𝑞¤ = 0  (4.2) where 𝐽𝑣 (𝑞) = 𝜕 𝑝 𝑣 (𝑞)/𝜕𝑞 represents the Jacobian matrix of the Cartesian position of the standing feet 𝑝 𝑣 (𝑞) ∈ R3𝑘 , with the corresponding constraint wrench 𝜆 𝑣 ∈ R3𝑘 . The first equation is considering ground forces as inputs and the second equation is treating the contacting as a holonomic constraint. Note that the actuation matrix 𝐵𝑣 is domain-dependent. This is because the double and triple support phases (Fig. 4.2) create closed-chain structures that induce redundancy in control and constraints (internal overactuation). As detailed in Sec. 2.4, this is an underdetermined problem which often appears in multi-contact locomotion that could yield nonunique control value 𝑢. Therefore, we manually turn off the rear standing leg’s hip pitch motor in the double support domain and turn off the diagonal standing legs’ hip pitch motors and the other standing leg’s hip roll motors for triple supporting phase. This implementation in return yields underactauted dynamics for the system. Now we can convert the constrained EOM (4.2) in domain 𝑣 into the following form: 𝑥¤ = 𝑓𝑣 (𝑥) + 𝑔𝑣 (𝑥)𝑢. (4.3) To track a given set of time-based trajectories B𝑣 (𝑡), which will be detailed in later, we deployed an input-output feedback linearization controller:
𝑢 io (𝑥, 𝑡) = A (𝑥) L (𝑥, 𝑡) − 2𝜀𝑦(𝑞, 𝑡) − 𝜀 2 𝑦¤ (𝑞, 𝑞, ¤ 𝑡) , (4.4) with the outputs 𝑦(𝑞, 𝑡) = 𝑦 𝑎 (𝑞) − B𝑣 (𝑡) and 𝜀 > 0. In this formulation, we chose the actual outputs 𝑦 𝑎 (𝑞) as all of the actuated joints. The notations follow directly from Sec. 3.2. As a result, (4.4) forces the system to converge to a desired gait exponentially, that is, 𝑦 𝑎 (𝑞) → B𝑣 (𝑡). The output dynamics become 𝑦¥ = −2𝜀 𝑦¤ − 𝜀 2 𝑦 (4.5) for which (𝑦, 𝑦¤ ) = (0, 0) is exponentially stable. Prior to introducing the edges and the discrete dynamics, some assumptions are necessary to construct a feasible model both for computation and experiment: 76 Figure 4.2: The cyclic directed graph for the multi-domain hybrid dynamics of walking, ambling, and trotting gaits. The dashed lines represent a relabeling map (as given in (3.32)) that flips the left and right legs’ contact attributes. We took these nomenclature from (Muybridge, n.d.). • There is no ground slippage. This is partially guaranteed by enforcing a friction cone condition. However, it is worthwhile to mention that slipping locomotion has been observed on quadrupedal animals for energy efficiency, see (Gamus and Yizhar Or, 2013; W. Ma, Y. Or, and A D. Ames, 2019). • The zoology studies (Muybridge, n.d.) have observed a pattern of 4 × 2domain1 locomotion on quadrupedal animals. We refer interested readers to (Akbari Hamed, Wen-Loong. Ma, and Aaron. D. Ames, 2019) for the corresponding model. In this chapter, we assume the stance leg transition domains—one leg strikes while another leg lifts—are instantaneous and passive for walking and ambling. Hence s1 and s2 in Fig. 4.2 become edges, and we can have a 2 × 2-domain behavior for walk and amble. But a 4 × 2-domain model is still used for trotting. The Discrete-Time Domain: Impact and Lift-Off On the edge of D𝑣 in (4.1), one of the conditions reaches its bound. Thus we have two switching mechanisms: 1 The term 𝑘 × 2 denotes a gait with 𝑘-domain. See Fig. 4.2 for the directed graph of the gaits with 𝑘 domains. The second half of the motion is directly a left-right mirror (i.e., symmetry) of the gait through the relabeling map. See Fig. 4.3 for the full motion with 𝑚 × 2 domains. 77 • Lift off : a standing foot of leg 𝑙∗ lifts off from the ground, meaning 𝑁𝑣𝑙∗ (𝑥, 𝑢) = 0. ∗ • Impact: a swing foot of leg 𝑙∗ impacts the ground, meaning ℎ𝑙𝑠,𝑣 (𝑞) = 0, 𝑙 ∗ ℎ¤ 𝑠,𝑣 (𝑞, 𝑞) ¤ < 0. + For lift off, an identity map 𝑥 𝑣+1 = 𝑥 𝑣− is sufficient to represent the transition from current to the next domain, where 𝑥 𝑣− is the state at the end of the domain D𝑣 and + is the state at the beginning of D 𝑥 𝑣+1 𝑣+1 . However, the impact shall cause a jump in the velocity terms, the dynamics are given by " #" # " # 𝑇 𝐷 −𝐽𝑣+1 𝑞¤ +𝑣+1 𝐷 𝑞¤ −𝑣 = (4.6) 𝐽𝑣+1 0 Λ 0 which follows Sec. 2.7. Figure 4.3: On the top left is the snapshot for a 2 × 2-domain walking gait; on the top right is a 2 × 2-domain ambling gait; and on the bottom is a 4 × 2-domain trotting gait. The symbol 0 is a left-right mirror of the contact attributes for domain . The orange-colored legs are the left-side legs whose contact points are highlighted by triangles; the blue-colored legs are the right-side legs with contact points highlighted by inverted triangles. Gait Design for Full-Order Dynamics An alternating sequence of the continuous dynamics (2.19) and the discrete dynamics (2.31) composites a hybrid control system. Since this nonlinear hybrid model has captured abundant details of the dynamics, its complexity challenges the controller design and motion planning. We hereby employ an optimization toolbox FROST (Ayonga Hereid, C. M. Hubicki, et al., 2018) that parses hybrid system control problems into a nonlinear programming (NLP) based on direct collocation. As introduced in (3.44), to generate a feasible 𝑁-domain motion such as walking, 78 ambling, and trotting as shown in Fig. 4.2, the NLP is formulated as: min Õ 𝛼 𝑣 ,𝑥 𝑖 ,𝑥¤𝑖 𝑢 𝑖 k𝑢𝑖 k 22 𝑖 = 1, 2, ... 𝑁 Õ 𝑖 𝑀𝑣 (4.7) 𝑣=1 s.t. C1. closed-loop dynamics C2. hybrid & periodic continuity C3. physical feasibility where 𝑀𝑣 is the number of collocation points and 𝛼𝑣 is the decision variable paramterizing the desired trajectory for domain D𝑣 . The cost function is to minimize the torque so that experiment implementation is achievable. The constraint C1 is from (2.19) and (4.5), and the constraint C2 is referring to the state continuity through each edge, which could be equipped with a discrete jump in states. C3 enforces conditions including k𝑢𝑖 k ∞ ≤ 50, (𝑞𝑖 , 𝑞¤𝑖 ) ∈ X, foot clearance and the friction pyramid conditions, so that the optimal solution is experimentally feasible. Remark 3. Due to many types of aleatoric uncertainty in the model, not every solution of the NLP can be robust enough to lead to experimental success. Furthermore, some constraint setup can be ill-posed that the NLP converges poorly. Thus in experiments, “fine tuning” on the constraint setup are often needed. Using machine learning to automate the tuning process is presented in (). To present a complete and scientific report, the following heuristics were used: 1. Efficiently producing closed-loop controllers is the core innovation of FROST, but it also boosts the complexity of the problem. Thus using an open-loop setup (optimization without a feedback controller) to warm start a closed-loop problem is effective. 2. Tuning constraints on the acceleration (force) terms are normally more effective than tuning positions (“effective” is both referring to the converging speed and experimental robustness.) It is indeed intuitive to tune the constraints about the positional terms to change the appearance of a behavior, but it can often be too restrictive for the optimization to converge fast enough. A natural-looking gait should be the result of forces, not that of human-expertise refinement. Optimal Gaits Under the umbrella of the HZD framework (see Sec. 3.3), the sole difference among these behaviors in Fig. 4.2 is nothing but the ordered sequence of contact events, 79 which can be predefined by specifying the stance foot clearance as 0 and swing foot clearance as nonzero values. Therefore, in the HZD optimization (4.7), by changing the foot clearance constraints in C3, multiple quadrupedal behaviors such as walk, amble, and trot can be produced efficiently. We show the simulated behaviors in Fig. 4.3 and their phase portraits in Fig. 4.4. With some initial guesses supplied, the computation time for the presented gaits are 262.13 s for walking, 42.69 s for ambling, and 116.05 s for trotting on a Ubuntu 16.04 machine with Intel Core i7-6820 HQ CPU @ 2.7 GHz with 16 GB RAM. In a loose comparison, the reinforcement learning methods take hours to find a feasible solution (Tan et al., 2018). 5 1 0 -1 HR0 HR2 -2 0 HP0 HP2 K0 K2 HR1 HR3 HP1 HP3 0.5 1 K1 K3 1 0 -1 -2 HR0 HR2 -3 1.5 position (rad) velocity (rad/s) 2 2 velocity (rad/s) velocity (rad/s) 3 0 HP0 HP2 K0 K2 HR1 HR3 HP1 HP3 0.5 1 K1 K3 1.5 0 -5 HR0 HR2 HP0 HP2 0 position (rad) K0 K2 HR1 HR3 HP1 HP3 0.5 1 K1 K3 1.5 position (rad) Figure 4.4: The periodic trajectories designed by NLP (4.7): walk (left), amble (middle), and trot (right). Simulation and Experiment With these stabilized quadrupedal dynamic gaits in simulation and optimization, we conduct an experiment with the ambling gait on Vision 60. The implemented controller is a PD approximation of the input-output linearizing controllers to track the time-based trajectories given by the optimization (4.7). That is, for a continuous domain D𝑣 , we have

𝑢(𝑞 𝑎 , 𝑞¤ 𝑎 , 𝑡) = −𝐾 𝑝 𝑞 𝑎 − B𝑣 (𝑡) − 𝐾 𝑑 𝑞¤ 𝑎 − B¤ 𝑣 (𝑡) (4.8) as the motor torque commands sent to each joint. This time-based PD implementation has been shown to have exceptional robustness for bipedal locomotion (S. Kolathaya, A. Hereid, and A. D. Ames, 2016). In addition, the domain switching method is also time-based with the event function given by the optimized trajectories. The result is that Vision 60 ambles stably with the desired speed of 0.3 m/s. See (Wen-Loong Ma, 2019b) for Vision 60 ambling in an outdoor tennis court, and the snapshots are shown in Fig. 4.5. We logged 20 seconds of data and compared it with the simulated ambling controller in Fig. 4.6. Additionally, the average torque inputs (absolute value) are 7.73 N·m on the hip roll joints, 9.46 N·m on the hip pitch joints, and 16.17 N·m on the knee joints. It is worth mentioning that the consistent drifting 80 Figure 4.5: Snapshots of the Vision 60 ambling in an outdoor environment, showing a full step of 2 × 2 domains of the amble gait. aside is expected, as there is no feedback information for the uneven terrain and that some manufacturing defects could cause asymmetric weight distribution. Some common solutions to avoid the drift is to use a joystick to manually offset the hip roll joints or Raibert-type regulators (see (Marc H. Raibert, Brown, and Chepponis, 1984)). The robot that keeps ambling without falling supports the feasibility of the full model-based HZD methods for quadrupedal locomotion. 0.4 0.6 0.00 0.05 0 -2 0.6 1.4 -0.05 0.8 position (rad) 1.0 0 -2 1.2 1.3 0.05 0.4 velocity (rad/s) 0.6 1.4 position (rad) 0 -1 -2 -0.05 0.00 position (rad) 0 -2 1.1 1.0 1.2 0.05 0 -2 0.6 1.4 K3 2 0.4 1.3 position (rad) HP3 1 -3 0.8 2 position (rad) HR3 2 1.1 0.00 0 -2 position (rad) K2 2 0.4 1.3 velocity (rad/s) -2 position (rad) 1.2 position (rad) velocity (rad/s) velocity (rad/s) velocity (rad/s) 0 -1 -0.05 1.1 1.0 HP2 HR2 1 -3 0.8 position (rad) velocity (rad/s) 0.05 position (rad) 0 -2 2 velocity (rad/s) 0 -2 1 0 -1 -2 -3 K1 HP1 HR1 2 velocity (rad/s) 2 velocity (rad/s) 0.00 velocity (rad/s) velocity (rad/s) velocity (rad/s) -0.05 K0 HP0 HR0 1 0 -1 -2 -3 0.8 position (rad) 1.0 2 0 -2 1.1 1.2 1.3 1.4 position (rad) Figure 4.6: The designed gaits (in red) from optimization/simulation vs. the experimental data (in cyan) in the form of phase portrait for amble. HR is short for hip roll, HP is for hip pitch, and k is for knee. Even though we can produce gaits and nominal controllers that lead to experimental success on hardware, the computational time around 2-4 minutes is not ideal for real-world implementation. The robustness against model uncertainty and rough terrain is also not practical compared to many simplified model-based approaches (Bledt et al., 2018; Boussema et al., 2019b). We next take the inspiration from Raibert’s virtual leg principle (M. Raibert and Tello, 1986) and combine it with the HZD methodology. That is, the coupled control system (CCS) framework, where instead of empirically assuming a quadrupedal robot is equivalent to a low-DOF 81 bipedal robot, we consider it as a “collaboration” between two bipedal subsystems. 4.2 Coupled Control System and Its Solutions To achieve dynamic walking on high-dimensional robotic systems, methods that assume simplified models have been applied, such as embedding the central pattern generators to multi-legged locomotion (Danner et al., 2017). Through another methodology, dimension reduction, hybrid zero dynamics (HZD) has proven to be an effective methodology. as a result of its ability to make theoretic guarantees (E. R. Westervelt, J. W. Grizzle, Chevallereau, et al., 2007) and yield walking for complex humanoids (Sreenath, 2011; Jacob Reher et al., 2016) without assuming model simplifications. The main idea behind this approach is that the full-order dynamics of the robot can be reduced to a lower-dimensional surface on which the system evolves. The system can then be studied via the low-dimensional dynamic representation and, importantly, guarantees made can be translated back to the full-order dynamics, i.e., periodic orbits (or walking gaits) in the low-dimensional system imply corresponding periodic orbits in the full-order system. The goal of this section is to capture this dimension reduction in a more general context—that of coupled control systems, which captures the ability to decompose a complex system into low-dimensional subsystems. Another means of dimension reduction for robotic systems comes from isolating subsystems and coupling these subsystems at the level of reaction forces, i.e., Lagrange multipliers that enforce holonomic constraints. This is the idea underlying the highly efficient method for calculating the dynamics of robotic systems: spatial vector algebra (R. Featherstone, 2008). For example, a double pendulum can be decomposed into two single pendula connected via a holonomic constraint at the pivot joint (Ganesh, Aaron D Ames, and Bajcsy, 2007). More generally, for a robotic system, one can consider two equivalent ways of expressing the dynamics of this system (R. M. Murray et al., 1994; R. Featherstone, 2008): ( 𝐷 𝑖 (𝑞𝑖 ) 𝑞¥𝑖 + 𝐻𝑖 (𝑞𝑖 , 𝑞¤𝑖 ) = 𝑢𝑖 + 𝐽ℎ>𝑖 𝜆 𝐷 (𝑞) 𝑞¥ + 𝐻 (𝑞, 𝑞) ¤ =𝑢 ⇔ | {z } s.t. ℎ(𝑞) ≡ 0 Full-Order Dynamics | {z } Reduced-Order Coupled Dynamics for 𝑖 = 1, 2, where here ℎ is a coupling (holonomic) constraint that is enforced via the Lagrange multiplier 𝜆 allowing for the higher-dimensional 𝑞 to be decomposed into lower-dimensional components 𝑞𝑖 , i.e., 𝑞 = (𝑞 1 , 𝑞 2 ). For example, a quadruped can be decomposed into two bipeds as in Fig. 4.7. Therefore, if one can make 82 guarantees on the reduced-order coupled systems, they can be translated into fullorder dynamics. This idea has also, again, proven successful in robotic walking when dealing with what is referred to as the “pinned” and “unpinned” models— this success has been especially prevalent in the context of constructing efficient nonlinear constrained optimization problems, especially for bipedal walking robots that utilize hybrid zero dynamics (A. Hereid et al., 2018). Figure 4.7: Conceptual illustration of the full-body dynamics decomposition, where the 3D quadruped — the Vision 60 — is decomposed into two constrained 3D bipedal robots. The study of coupled dynamical and control systems has a long and rich history from which the method presented in this thesis has taken inspiration. First, from the computational perspective, the highly efficient method for calculating the dynamics of robotic systems — spatial vector algebra (R. Featherstone, 2008) — uses a similar concept: Lagrange multipliers that enforce holonomic constraints. Second, focusing on the coupled dynamics, the interconnected systems (Antonelli, 2013) have studied the synchronization of coupled oscillators (Dörfler and Bullo, 2014; Fujisaka and Yamada, 1983). Further, the passivity-based control (Hatanaka et al., 2015) has been proposed to design coupled controllers for multi-agent systems. The inputto-state stability analysis (Jiang, Mareels, and Wang, 1995) studied the Lyapunov stability of decoupled control laws. Third, in the control community, the most relevant examples are the multi-agent networks (Mesbahi and Egerstedt, 2010), the consensus problem (Ren, Beard, and Atkins, 2005) and the cooperative control problem (Mellinger et al., 2013; Chung and Slotine, 2009). These methods have been successfully demonstrated on a wide range of robotic applications, especially on drones. However, the problems considered in these frameworks are often coupled 83 on the control level — shared feedback information — but not the dynamics level, such as the general formulation considered in (Sloth, 2016). This allows the designer to utilize the built-in stabilizing controller of each subsystem to achieve some add-on optimality. In other words, each subsystem’s stability does not critically rely on the other subsystems’. In related work, the coordination of multiple quadrupedal robots via reaction forces has recently been studied (Kaveh Akbari Hamed, Kamidi, et al., 2020). This section generalizes the aforementioned methods — zero dynamics and system decomposition through coupling constraints — and unifies them through a novel formulation: coupled control systems. We then utilize zero dynamics to reduce to a subsystem dependent on coupling constraints which are then eliminated via coupling relations to yield the final isolated subsystem. The main result of this paper is that solutions to the isolated subsystem are solutions to the full-order system. Thus periodic orbits on the subsystem yield periodic orbits on the full-order system. This result is leveraged to construct a nonlinear optimization problem utilizing collocation methods to generate these periodic solutions. Coupled Control Systems. To define coupled control systems, we will consider a graph on which each node represents a control system, and each edge represents a coupling condition between these control systems. First, we briefly establish the notation related to graphs. A bidirected graph (also termed a symmetric directed graph) is a tuple G = (N , E), with V = {1, 2, . . . ℓ} ⊂ Z a set of vertices (representing the indices of all subsystems) and E ⊂ {(𝑖, 𝑗) | (𝑖, 𝑗) ∈ N × V, 𝑖 ≠ 𝑗 } is a set of edges (these represent the connection relation between two subsystems, hence the bidirectionality—if a subsystem is connected to another subsystem, then the same holds in reverse). The bidirectionality of the graph implies that if 𝑒 = (𝑖, 𝑗) ∈ E then 𝑒 = ( 𝑗, 𝑖) ∈ E, i.e., every direct edge connects each vertex symmetrically. All of the edges of a sub-system 𝑖 form a set, G𝑖 = {𝑒 ∈ E | 𝑒 = (𝑖, 𝑗) ∈ E} with |G𝑖 | the number of neighbor systems. By bidirectionality, if 𝑒 ∈ G𝑖 then 𝑒 ∈ G 𝑗 . We can now define the main object of interest: Definition 14. A coupled control system (CCS) is defined to be a tuple: CC = (G, X, Z, U, Λ, 𝐹𝐺, 𝑃𝑄, 𝐶) consisting of the following components: (4.9) 84 • G = (N , E) is a bidirected graph, • X = {X𝑖 }𝑖∈N , where 𝑥𝑖 ∈ X𝑖 is a set of internal states, • Z = {Z𝑖 }𝑖∈N , where 𝑧𝑖 ∈ Z𝑖 is a set of coupled states, • U = {U𝑖 }𝑖∈N , where 𝑢𝑖 ∈ U𝑖 is a set of admissible inputs, • Λ = {𝜆 𝑒 } 𝑒∈E , is a set of coupling inputs that enforce dynamic coupling, • 𝐹𝐺 = {( 𝑓𝑖 , 𝑔𝑖 , 𝑔˘ 𝑒 )}𝑖∈N ,𝑒∈G𝑖 are the internal state dynamics where each systems’ internal states evolve according to: Õ 𝑥¤𝑖 = 𝑓𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑔𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 + 𝑔˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒 , (4.10) 𝑒∈G𝑖 • 𝑃𝑄 = {( 𝑝𝑖 , 𝑞𝑖 , 𝑞˘ 𝑒 )}𝑖∈N ,𝑒∈G𝑖 is the set of coupled state dynamics where the coupled states evolve according to: Õ 𝑧¤𝑖 = 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 + 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒 , (4.11) 𝑒∈G𝑖 • 𝐶 = {𝑐 𝑒 } 𝑒∈E is a set of coupling constraints:2 𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 ) ≡ 0, ∀ 𝑒 = (𝑖, 𝑗) ∈ E. (4.12) Note that it is often convenient to write a CCS in a more compact form. More concretely, with the objects defined above in mind, we can denote a CCS in the following form of semi-explicit differential-algebraic equation (DAEs): Õ  𝑥 ¤ = 𝑓 (𝑥 , 𝑧 ) + 𝑔 (𝑥 , 𝑧 )𝑢 + 𝑔˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒  𝑖 𝑖 𝑖 𝑖 𝑖 𝑖 𝑖 𝑖    𝑒∈G𝑖   Õ   CC , 𝑧¤𝑖 = 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 + (4.13) 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒   𝑒∈G  𝑖     s.t. 𝑐 (𝑧 , 𝑧 ) ≡ 0 ∀ 𝑒 = (𝑖, 𝑗) ∈ G𝑖 𝑒 𝑖 𝑗  wherein the systems in (4.13) are defined for each 𝑖 ∈ N , and implicit in the definition for the corresponding bidirected graph G = (N , E) and the fact that 𝑥𝑖 ∈ X𝑖 , 𝑧𝑖 ∈ Z𝑖 and 𝑢𝑖 ∈ U𝑖 as defined above. 2 As used in (2.8), the operator ≡ (“identical equality” of functions) is defined as, 𝑓 ≡ 𝑔 for functions 𝑓 , 𝑔 meaning 𝑓 (𝑥) = 𝑔(𝑥) for all 𝑥 defined. For example, 𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 ) ≡ 0 implies that 𝑐 𝑒 (𝑧 𝑖 (𝑡), 𝑧 𝑗 (𝑡)) = 0 for all 𝑡 ∈ R for which 𝑧𝑖 (𝑡) and 𝑧 𝑗 (𝑡) are defined. 85 Î For a CCS CC we will denote the total internal state by 𝑥 = {𝑥𝑖 }𝑖∈N ∈ 𝑖∈N X𝑖 Î and the total coupled state by 𝑧 = {𝑧𝑖 }𝑖∈N ∈ 𝑖∈N Z𝑖 . By slight abuse of notation, we may denote function evaluations on the total states such that the corresponding functions are evaluated system by system. For example, we can write 𝑐(𝑧) , {𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 )} 𝑒=(𝑖, 𝑗)∈E for the coupling constraint. Solutions. We can define solutions to coupled control systems by assuming the existence of a collection of feedback control laws: 𝑢(𝑥, 𝑧) , {𝑢𝑖 (𝑥𝑖 , 𝑧)}𝑖∈N . Applying these controllers to (4.13) yields a coupled dynamical system (CDS): Õ   𝑥 ¤ = 𝑓 (𝑥 , 𝑧 ) + 𝑔 (𝑥 , 𝑧 )𝑢 (𝑥 , 𝑧) + 𝑔˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒  𝑖 𝑖 𝑖 𝑖 𝑖 𝑖 𝑖 𝑖 𝑖   | {z }  𝑒∈G𝑖    𝑓𝑖cl (𝑥 𝑖 ,𝑧)   Õ   DC , 𝑧¤𝑖 = 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 (𝑥𝑖 , 𝑧) + (4.14) 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒  | {z } 𝑒∈G𝑖     𝑝 cl  𝑖 (𝑥 𝑖 ,𝑧)      s.t. 𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 ) ≡ 0 ∀ 𝑒 = (𝑖, 𝑗) ∈ G𝑖  again defined for all 𝑖 ∈ N . The solution to the coupled dynamic system, DC , is a set of solutions: {(𝑥𝑖 (𝑡), 𝑧𝑖 (𝑡), 𝜆 𝑒 (𝑡))}𝑖∈N ,𝑒∈G𝑖 s.t. (4.14) ∀𝑡 ∈ I ⊂ R with initial condition: {(𝑥𝑖 (0), 𝑧𝑖 (0), 𝜆 𝑒 (0))}𝑖∈N ,𝑒∈G𝑖 . Per the above notation, we will sometimes denote the solutions by (𝑥(𝑡), 𝑧(𝑡), 𝜆(𝑡)) with initial condition (𝑥(0), 𝑧(0), 𝜆(0)). Note that solutions must satisfy the coupling constraints (4.12) at all time. Therefore, 𝑐(𝑥, ¤ 𝑧) ≡ 0 ⇒ ⇒ 𝑐¤𝑒 (𝑥, 𝑧𝑖 , 𝑧 𝑗 , 𝑧¤𝑖 , 𝑧¤ 𝑗 ) ≡ 0 ∀ 𝑒 ∈ E 𝜕𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 ) 𝜕𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 ) 𝑧¤ 𝑗 ≡ 0. 𝑐¤𝑒 (𝑥, 𝑧, 𝑧¤) = 𝑧¤𝑖 + 𝜕𝑧𝑖 𝜕𝑧 𝑗 | {z } | {z } (𝑖, 𝑗) ,𝐽𝑒 (𝑧) ( 𝑗,𝑖) ,𝐽𝑒 (4.15) (𝑧) Since 𝑧¤𝑖 (𝑡) and 𝑧¤ 𝑗 (𝑡) both depend on 𝜆 𝑒 (𝑡), for 𝑒 = (𝑖, 𝑗), it is this derivative condition that 𝜆(𝑡) must satisfy for all systems and all coupling constraints in order to be a solution. Concretely, and utilizing (4.15), this implies that the following condition must hold: © (𝑖, 𝑗) 𝑐¤𝑒 (𝑥, 𝑧) = 𝐽𝑒 (𝑧) ­ 𝑝𝑖cl (𝑥𝑖 , 𝑧) + Õ ª 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑘 )𝜆 𝑒 ® 𝑒=(𝑖,𝑘)∈G𝑖 « ¬ Õ © ª ( 𝑗,𝑖) + 𝐽𝑒 (𝑧) ­ 𝑝 cl𝑗 (𝑥 𝑗 , 𝑧) + 𝑞˘ 𝑒 0 (𝑥 𝑗 , 𝑧 𝑗 , 𝑧 𝑙 )𝜆 𝑒 0 ® ≡ 0. 𝑒 0 =( 𝑗,𝑙)∈G 𝑗 « ¬ (4.16) 86 Therefore, to solve for coupling inputs 𝜆 𝑒 that satisfy the coupling constraints, it is necessary to solve an equation that depends on the states of all systems connected to the 𝑖 th system via G𝑖 . To address this, we present a method for isolating a subsystem via conditions on the controllers of all other systems. Isolating Control Subsystems The main idea in approaching the analysis and design of controllers for CCSs is to isolate subsystems that encode the behavior of the overall CCS. This section outlines the procedure for isolating these subsystems through a two-step approach: restricting systems to the zero dynamics surface and leveraging this to calculate the coupling conditions explicitly. These can be combined to reduce the full-order CCS to a subsystem that no longer depends on the internal states of the other subsystems. We establish the main result of the paper encapsulating these constructions: solutions of the subsystem yield solutions of the full-order dynamics. Zero Dynamics Manifolds. In addition to restricting the dynamics to the constraint manifold, we are also interested in understanding on controllers that restrict the system to invariant surfaces can be utilized to further isolate subsystems of the CCS in a way amenable to analysis. Consider a CCS CC . The zero dynamics manifold is given for each subsystem, i.e., for each 𝑖 ∈ N , by: Z𝑖 , {(𝑥, 𝑧) ∈ X × Z | 𝑥𝑖 ≡ 0}. (4.17) Î where, by abuse of notation (although clear from context), X , 𝑖∈N X𝑖 and Î Z , 𝑖∈N Z𝑖 . Thus, the zero dynamics manifold consists of internal states that are zero, i.e., the system evolves only according to the coupled states. Definition 15. Consider a coupled control system CC . For 𝑖 ∈ N , a controller for the subsystem, 𝑢𝑖Z,𝜆 (𝑥𝑖 , 𝑧), renders the zero dynamics manifold invariant if it satisfies the following algebraic condition: Õ 0 ≡ 𝑓𝑖 (0, 𝑧𝑖 ) + 𝑔𝑖 (0, 𝑧𝑖 )𝑢𝑖Z,𝜆 (0, 𝑧) + 𝑔˘ 𝑒 (0, 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒 (4.18) 𝑒∈G𝑖 where 𝑢𝑖Z,𝜆 implicitly depends on 𝜆 𝑒 for 𝑒 ∈ G𝑖 . We call 𝑢𝑖Z,𝜆 the nominal controller for the 𝑖 th subsystem. Zero Dynamics. Consider the controller 𝑢𝑖Z that renders the zero dynamics manifold Z𝑖 invariant. By applying this controller, the coupled dynamics evolve according to 87 the zero dynamics: 𝑧¤𝑖 = 𝑝𝑖 (0, 𝑧) + 𝑞𝑖 (0, 𝑧)𝑢𝑖Z,𝜆 (0, 𝑧) + Õ 𝑞˘ 𝑒 (0, 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒 𝑒∈G𝑖 Z,𝜆 , 𝑝𝑖,𝑒 (𝑧). (4.19) Z,𝜆 Note that 𝑝𝑖,𝑒 depends on 𝜆 𝑒 for 𝑒 ∈ G𝑖 . In many cases, one can explicitly solve for a controller that satisfies this condition (via a relative degree type condition)—often through the proper choice of internal states. This will be the case for mechanical systems, and especially those representing quadrupedal robots. Specifically, if for all 𝑖 ∈ N , the matrix 𝑔𝑖 (0, 𝑧) is invertible for all 𝑧 ∈ Z then we can explicitly solve for the condition on 𝑢𝑖Z that renders the zero dynamics invariant: ! Õ 𝑢𝑖Z,𝜆 (0, 𝑧) = −𝑔𝑖 (0, 𝑧𝑖 ) −1 𝑓𝑖 (0, 𝑧𝑖 ) + 𝑔˘ 𝑒 (0, 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒 (4.20) 𝑒∈G𝑖 which now explicitly depends on 𝜆 𝑒 for 𝑒 ∈ G𝑖 . 𝜆-Coupled Subsystems. The key idea underlying the analysis of CCSs is to reduce the entire coupled system into the behavior of a single subsystem. This is achieved through the above constructions related to the zero dynamics. For a CCS CC , assume that there exist 𝑢 Z𝑗 that render the zero dynamics manifold Z 𝑗 invariant for all 𝑗 ∈ N \ {𝑖} with 𝑖 ∈ N . Consider the following 𝜆-coupled control subsystem (𝜆-CCSub): Õ   𝑥 ¤ = 𝑓 (𝑥 , 𝑧 ) + 𝑔 (𝑥 , 𝑧 )𝑢 + 𝑔˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒  𝑖 𝑖 𝑖 𝑖 𝑖 𝑖 𝑖 𝑖    𝑒∈G𝑖   Õ   Z,𝜆 C𝑖 , 𝑧¤𝑖 = 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 + , (4.21) 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒   𝑒∈G  𝑖    Z,𝜆   𝑧¤ 𝑗 = 𝑝 𝑗,𝑒 0 (𝑧) ∀ 𝑗 ∈ N \ {𝑖}s.t. 𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 ) = 0 ∀𝑒 ∈E  Thus, the subsystem evolves according to the dynamics of the system 𝑖 and the zero dynamics of all remaining systems—all of which are coupled via the coupling inputs 𝜆 = {𝜆 𝑒 } 𝑒∈E . When a feedback controller 𝑢𝑖 (𝑥𝑖 , 𝑧) is applied to C𝑖Z,𝜆 , the result is a dynamical subsystem, denoted by: D𝑖Z,𝜆 . Solutions of this system are denoted by (𝑥𝑖 (𝑡), 𝑧(𝑡), 𝜆(𝑡)). The fact that the 𝑖 th subsystem yields behavior of the entire system is summarized in the following lemma. Before stating this result, we need some notation. Let 88 (𝑥𝑖 , 𝑧) ∈ X𝑖 × Z, and consider the canonical embedding 𝜄 : X𝑖 × Z ↩→ X × Z given by 𝜄(𝑥𝑖 , 𝑧) = (𝑥, 𝑧) where 𝑥 = {𝑥 𝑗 } 𝑗 ∈N with 𝑥 𝑗 = 0 if 𝑗 ∈ N \ {𝑖} and 𝑥 𝑗 = 𝑥𝑖 if 𝑗 = 𝑖. Lemma 4. Let CC be a CCS and C𝑖Z,𝜆 the corresponding 𝜆-CCSub for some 𝑖 ∈ N obtained by applying 𝑢 Z,𝜆 𝑗 , for 𝑗 ∈ N \ {𝑖}, that render the zero dynamics manifold Z 𝑗 invariant. Let 𝑢𝑖 (𝑥𝑖 , 𝑧) be a feedback controller applied to C𝑖Z,𝜆 resulting in the corresponding dynamical subsystem D𝑖Z,𝜆 with solution (𝑥𝑖 (𝑡), 𝑧(𝑡), 𝜆(𝑡)). Then (𝜄(𝑥𝑖 (𝑡), 𝑧(𝑡)), 𝜆(𝑡)) is a solution to DC —the CDS obtained by applying these controllers. Proof. The proof follows trivially from the fact that the zero dynamics are invariant for all 𝑗 ∈ N \ {𝑖}, i.e., Ù 𝜄(𝑥𝑖 (𝑡), 𝑧(𝑡)) ∈ Z𝑗 ∀ 𝑡 ∈ 𝐼 ⊂ R 𝑗 ∈N \{𝑖} for 𝐼 the interval of existence of (𝑥𝑖 (𝑡), 𝑧(𝑡)).  Explicit Coupling Conditions The coupling between the control systems (4.13) is enforced via 𝜆 𝑒 and the coupling constraints of the form (4.16). Similarly, even in the reduction to a subsystem (4.21), the coupling was still achieved through 𝜆 𝑒 . We wish to generalize this to remove the coupling, i.e., isolate subsystems, while still preserving the overall behavior of the full system. Coupling Relations. Consider (4.15) but now for the case of a 𝜆-CCSub, as in (4.21), yielding: ! Õ (𝑖, 𝑗) 𝑐¤Z,𝜆 (𝑧) 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 + 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑘 )𝜆 𝑒 + 𝑒 (𝑥𝑖 , 𝑧, 𝑢𝑖 ) = 𝐽𝑒 𝑒∈G𝑖 Õ © ª (𝑧) ­ 𝑝 𝑗 (0, 𝑧 𝑗 ) + 𝑞 𝑗 (0, 𝑧 𝑗 )𝑢 Z,𝜆 (0, 𝑧) + 𝑞˘ 𝑒 0 (0, 𝑧 𝑗 , 𝑧 𝑙 )𝜆 𝑒 0 ® 𝑗 𝑒 0 ∈G 𝑗 « ¬ ≡0 (4.22) ( 𝑗,𝑖) 𝐽𝑒 Thus 𝜆 𝑒 with 𝑒 = (𝑖, 𝑗) ∈ G𝑖 , implicitly depends on 𝑢𝑖 —while the use of the controllers 𝑢 Z,𝜆 𝑗 eliminates the dependence on the controllers and internal states of the other systems. This motivates the following formulation: 89 Definition 16. For a 𝜆-coupled control subsystem C𝑖Z,𝜆 (𝑖 ∈ N ), a coupling relation is a functional relationship on the coupling inputs: 𝜆Z𝑒 (𝑥𝑖 , 𝑧; 𝑢𝑖 ) = 𝐴Z𝑒 (𝑥𝑖 , 𝑧)𝑢𝑖 + 𝑏 Z𝑒 (𝑥𝑖 , 𝑧), (4.23) for all 𝑒 = (𝑖, 𝑗) ∈ E, that satisfies the coupling constraint (4.22). Note that 𝜆 𝑒 only depends on 𝑢𝑖 and 𝑥𝑖 for all 𝑒 ∈ E. Specifically, given another edge 𝑒0 = ( 𝑗, 𝑙) ∈ E we have 𝜆Z𝑒 0 (𝑥𝑖 , 𝑧; 𝑢𝑖 ) = 𝐴Z𝑒 0 (𝑥𝑖 , 𝑧)𝑢𝑖 + 𝑏 Z𝑒 0 (𝑥𝑖 , 𝑧). In this case, it may happen that 𝐴Z𝑒 0 = 0, meaning that the relation is simply a constant relation for that edge. Leveraging Coupling Relations. Given a coupling relation 𝜆Z we can remove the dependence on 𝜆 throughout C𝑖Z,𝜆 and leave only a dependence on 𝑢𝑖 . Zero dynamics controllers: Recall that the controller 𝑢 Z,𝜆 that renders the zero dynamics surface invariant implicitly depends on 𝜆 𝑒 via (4.18). Given a coupling relation, the dependence of 𝜆 𝑒 is removed, and as a result we say that 𝑢 Z renders the zero dynamics manifold Z 𝑗 invariant if:   Z Z Z 0 ≡ 𝑓 𝑗 (0, 𝑧) + 𝑔 𝑗 (0, 𝑧)𝑢𝑖 + 𝑔 𝑗 (0, 𝑧 𝑗 ) 𝑢 𝑗 (0, 𝑧; 𝑢𝑖 ) − 𝑢𝑖 (4.24) where 𝑢 Z𝑗 is now a function of 𝑢𝑖 and Õ 𝑓 𝑗Z (𝑥 𝑗 , 𝑧) , 𝑓 𝑗 (𝑥 𝑗 , 𝑧 𝑗 ) + 𝑒 0 ∈G (4.25) 𝑔˘ 𝑒 0 (𝑥 𝑗 , 𝑧 𝑗 , 𝑧 𝑙 ) 𝐴Z𝑒 0 (𝑥𝑖 , 𝑧). (4.26) 𝑗 Õ 𝑔 Z𝑗 (𝑥 𝑗 , 𝑧) , 𝑔 𝑗 (𝑥 𝑗 , 𝑧 𝑗 ) + 𝑔˘ 𝑒 0 (𝑥 𝑗 , 𝑧 𝑗 , 𝑧 𝑙 )𝑏 Z𝑒 0 (𝑥𝑖 , 𝑧), 𝑒 0 ∈G 𝑗 Coupling conditions: Returning to (4.22), given a coupling relation we can rewrite this coupling constraint as:   (𝑖, 𝑗) Z Z 𝑐¤𝑒 (𝑥𝑖 , 𝑧, 𝑧¤) = 𝐽𝑒 (𝑧) 𝑝𝑖 (𝑥𝑖 , 𝑧) + 𝑞𝑖 (𝑥𝑖 , 𝑧)𝑢𝑖   ( 𝑗,𝑖) + 𝐽𝑒 (𝑧) 𝑝 Z𝑗 (𝑥𝑖 , 𝑧) + 𝑞 Z𝑗 (𝑥𝑖 , 𝑧)𝑢𝑖 ≡ 0 (4.27) where for 𝑖 ∈ N the 𝑖 associated with the subsystem C𝑖Z,𝜆 : Õ 𝑝𝑖Z (𝑥𝑖 , 𝑧) , 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝑏 Z𝑒 (𝑥𝑖 , 𝑧) (4.28) 𝑒∈G𝑖 𝑞𝑖Z (𝑥𝑖 , 𝑧) , 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 ) + Õ 𝑒∈G𝑖 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 ) 𝐴Z𝑒 (𝑥𝑖 , 𝑧) (4.29) 90 and for 𝑗 ∈ N \ {𝑖}, we have: 𝑝 Z𝑗 (𝑥𝑖 , 𝑧) , 𝑝 𝑗 (0, 𝑧 𝑗 ) + 𝑞 𝑗 (0, 𝑧 𝑗 )𝑢 Z𝑗 (0, 𝑧) + Õ 𝑞˘ 𝑒 0 (0, 𝑧 𝑗 , 𝑧 𝑙 )𝑏 Z𝑒 0 (𝑥𝑖 , 𝑧) (4.30) 𝑒 0 ∈G 𝑗 𝑞 Z𝑗 (𝑥𝑖 , 𝑧) , Õ 𝑒 0 ∈G 𝑞˘ 𝑒 0 (0, 𝑧 𝑗 , 𝑧 𝑙 ) 𝐴Z𝑒 0 (𝑥𝑖 , 𝑧). (4.31) 𝑗 Isolating Subsystems We now arrive at the key concept for which all of the previous constructions have built: reducing a CCS to a single subsystem that can be used to give guarantees about the entire CCS. This is achieved using the above coupling relation. Definition 17. For a CCS CC , for 𝑖 ∈ N , assume a coupling relation 𝜆Z that there exist 𝑢 Z𝑗 that render the zero dynamics manifold Z 𝑗 invariant for 𝑗 ∈ N \ {𝑖}. Then the 𝑖 th control subsystem (CSub) associated with the CCS CC is given by:   𝑥¤𝑖 = 𝑓𝑖Z (𝑥𝑖 , 𝑧) + 𝑔𝑖Z (𝑥𝑖 , 𝑧)𝑢𝑖     C𝑖Z , 𝑧¤𝑖 = 𝑝𝑖Z (𝑥𝑖 , 𝑧) + 𝑞𝑖Z (𝑥𝑖 , 𝑧)𝑢𝑖     𝑧¤ 𝑗 = 𝑝 Z (𝑥𝑖 , 𝑧) + 𝑞 Z (𝑥𝑖 , 𝑧)𝑢𝑖 𝑗 𝑗  (4.32) ∀ 𝑗 ∈ N \ {𝑖} where 𝑓𝑖Z and 𝑔𝑖Z are given in (4.25) and (4.26) and 𝑝𝑖Z , 𝑞𝑖Z , 𝑝 Z𝑗 , and 𝑞 Z𝑗 are given in (4.28)—(4.31). For the sake of definiteness, we note that: Õ 𝑓𝑖Z (𝑥𝑖 , 𝑧) , 𝑓𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑔˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝑏 Z𝑒 (𝑥𝑖 , 𝑧), (4.33) 𝑒∈G𝑖 𝑔𝑖Z (𝑥𝑖 , 𝑧) , 𝑔𝑖 (𝑥𝑖 , 𝑧𝑖 ) + Õ 𝑔˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 ) 𝐴Z𝑒 (𝑥𝑖 , 𝑧). (4.34) 𝑒∈G𝑖 Given a feedback controller 𝑢𝑖 (𝑥𝑖 , 𝑧) the corresponding dynamical subsystem is denoted by D𝑖Z . Constrained Dynamics. Note that the coupling constraint (4.27) was not explicitly stated in the CSub C𝑖Z . This was because it was solved for via the coupling relation 𝜆Z . That is, the system naturally evolves on the constraint manifold: C , {(𝑥, 𝑧) ∈ X × Z : 𝑐 𝑒 (𝑧) ≡ 0, ∀ 𝑒 ∈ E}. (4.35) This is made formal in the following result which is the ultimate result of all the constructions presented in this paper. Additionally, it will be seen that solutions of the 𝑖 th subsystem can be used to construct solutions of the full-order CCS. In this context, recall that in Lemma 4 we utilized the embedded 𝜄 : X𝑖 × Z → X × Z. 91 Theorem 2. Let CC be a CCS, and for 𝑖 ∈ N assume that there exist 𝑢 Z𝑗 , 𝑗 ∈ N \ {𝑖} that render the zero dynamics manifold Z 𝑗 invariant. Let C𝑖Z be the corresponding 𝑖 th subsystem. Given a feedback controller 𝑢𝑖 (𝑥𝑖 , 𝑧) for the CSub with corresponding dynamical subsystem D𝑖Z with solution (𝑥𝑖 (𝑡), 𝑧(𝑡)) with 𝑡 ∈ 𝐼 ⊂ R. If 𝜄(𝑥𝑖 (0), 𝑧(0)) ∈ C ⇒ 𝜄(𝑥𝑖 (𝑡), 𝑧(𝑡)) ∈ C ∀ 𝑡 ∈ 𝐼 ⊂ R and (𝜄(𝑥𝑖 (𝑡), 𝑧(𝑡)), 𝜆Z (𝑡)), with 𝜆Z (𝑡) = {𝜆Z𝑒 (𝑥𝑖 (𝑡), 𝑧(𝑡); 𝑢𝑖 (𝑥𝑖 (𝑡), 𝑧(𝑡)))} 𝑒∈E , is a solution to DC , the CDS obtained by applying 𝑢𝑖 and 𝑢 Z𝑗 , 𝑗 ∈ N \ {𝑖}. Proof. The condition that (𝑥(0), 𝑧(0)) ∈ C is equivalent to 𝑐 𝑒 (𝑧(0)) = 0. Concretely, 𝑐 𝑒 (𝑧𝑖 (0), 𝑧 𝑗 (0)) = 0. Since 𝜆Z𝑒 is a coupling relation it satisfies (4.22) and more explicitly (4.27); therefore, and being explicit about the arguments, 𝑐¤𝑒 (𝑥(𝑡), 𝑧(𝑡)) = 0 for all 𝑡 ∈ 𝐼 and all 𝑒 ∈ E. It follows that 𝑐 𝑒 (𝑧(𝑡)) = 0 for all 𝑡 ∈ 𝐼 and 𝑒 ∈ E. The fact that (𝜄(𝑥𝑖 (𝑡), 𝑧(𝑡)), 𝜆Z (𝑡)) is a solution to DC assuming that (𝑥𝑖 (𝑡), 𝑧(𝑡)) is a solution to D𝑖Z follows Lemma 4 together with the construction of D𝑖Z .  Periodic Orbits. In the context of quadrupeds, we will be interested in generating periodic motions, i.e., walking. A solution to a CDS DC is periodic of period 𝑇 > 0 if for some initial condition (𝑥(0), 𝑧(0), 𝜆(0)): (𝑥(𝑡 + 𝑇), 𝑧(𝑡 + 𝑇), 𝜆(𝑡 + 𝑇)) = (𝑥(𝑡), 𝑧(𝑡), 𝜆(𝑡)) with the resulting periodic orbit: O = {(𝑥(𝑡), 𝑧(𝑡)) ∈ X × Z | 0 ≤ 𝑡 ≤ 𝑇 }. As a result of Theorem 2, periodic orbits in a subsystem correspond to periodic orbits in the full-order dynamics. Corollary 1. Under the conditions of Theorem 2, assume that (𝑥𝑖 (𝑡), 𝑧(𝑡)) is a periodic solution of D𝑖Z with period 𝑇 > 0 and corresponding orbit O𝑖 = {(𝑥𝑖 (𝑡), 𝑧(𝑡)) ∈ X𝑖 × Z | 0 ≤ 𝑡 ≤ 𝑇 }then (𝜄(𝑥(𝑡), 𝑧(𝑡), 𝜆Z (𝑡)) is a periodic solution of the CDS with period 𝑇 > 0 and corresponding periodic orbit O = 𝜄(O𝑖 ). 92 4.3 CCS Optimization and Application to Quadrupedal Gaits With the construction of coupled control systems in the graph in the previous section, we present a general optimization framework to solve for the solution to the 𝑖 th control subsystem (CSub) associated with the CCS CC , while synthesising the corresponding nominal controllers that render forward invariance of the zero dynamics manifolds. The application of these ideas to periodic orbit generation will be discussed. The approach we will take is a locally direct collocation (Rao, 2009) based optimization method, which has been detailed in Sec. 2.9 and Sec. 3.3. In this section, we build on this method to solve for the solution and controllers to coupled control systems, while optimizing some metrics. The target is that through the proposed control decoupling framework, the computation of numerical solutions (including control synthesis) to a coupled control system can be achieved efficiently and robustly. Coupled System Optimization We first pose the previous formulations as a series of equality constraints to represent the controlled dynamics of C𝑖Z for system 𝑖 ∈ V. In particular, assume a coupling relation as given in Def. 16 of the form: 𝜆Z𝑒 (𝑥𝑖 , 𝑧; 𝑢𝑖 ) = 𝐴Z𝑒 (𝑥𝑖 , 𝑧)𝑢𝑖 + 𝑏 Z𝑒 (𝑥𝑖 , 𝑧), as in (4.23). To define the decision variables, we first discretized time evenly 𝑡 ∈ [0, 𝑇], with grid index 𝜅 = 0, 1, ...K, i.e., 𝑡 𝜅 = K𝜅 𝑇. Consequently, we have the initial time 𝑡 0 = 0, final time 𝑡 𝑀 = 𝑇 and step size Δ𝑡 = 𝑇/K. To formulate direct collocation constraints, we referred the even nodes (grid index that is an even number) as cardinal nodes and the other grids as interior nodes. In addition, we abbreviated the dependency on time 𝑡 as 𝜅 , (𝑡 𝜅 ) for notation simplicity. Decision Variables. To introduce the other constructions, decision variable for optimization is denoted as  X , 𝜗 𝜅 𝜅=0,1,...𝑀 , with 𝜗 𝜅 , {𝑥𝑖𝜅 , 𝑥¤𝑖𝜅 , 𝑧𝑖𝜅 , 𝑧¤𝑖𝜅 , 𝑧 𝜅𝑗 , 𝑧¤𝜅𝑗 , 𝑢𝑖𝜅 , 𝑢 Z,𝜅 𝑗 } 𝑗 ∈N \{𝑖} . Thus we have decision variables associated with the 𝑖 th control subsystem C𝑖Z . Note that we will use 𝑧 𝜅 = {𝑧𝑖𝜅 , 𝑧 𝜅𝑗 } 𝑗 ∈N \{𝑖} to denote all of the coupled coordinates. Zero Dynamics Constraints. Recall that given a coupling relation, we had associated zero dynamics invariance conditions given by (4.58). We will enforce these conditions in the optimization to ensure that 𝑢 Z,𝜅 𝑗 renders the zero dynamics Z 𝑗 invariant. 93 Specifically, we have the constraint:   𝜅 𝐹zero (𝜗 𝜅 ) , 𝑓 𝑗Z (0, 𝑧 𝜅 ) + 𝑔 Z𝑗 (0, 𝑧 𝜅 )𝑢𝑖𝜅 + 𝑔 𝑗 (0, 𝑧 𝜅𝑗 ) 𝑢 Z,𝜅 − 𝑢 𝑖 , ∀ 𝑗 ∈ N \ {𝑖} (C.1) 𝑗 where 𝑓 𝑗Z and 𝑔 Z𝑗 are given as in (4.25) and (4.26), respectively. Subsystem Dynamics Constraints. Following from the constructions in Sec. 4.2, we wish to define constraints corresponding to dynamics of the 𝑖 th control subsystem C𝑖Z (as obtained from the coupling relation). By denoting 𝜒 𝜅 = (𝑥𝑖𝜅 , 𝑧𝑖𝜅 , 𝑧 𝜅𝑗 ), and   𝑓𝑖Z (𝑥𝑖𝜅 , 𝑧 𝜅 ) + 𝑔𝑖Z (𝑥𝑖𝜅 , 𝑧 𝜅 )𝑢𝑖𝜅     𝐹 ( 𝜒 𝜅 ) , 𝑝𝑖Z (𝑥𝑖𝜅 , 𝑧 𝜅 ) + 𝑞𝑖Z (𝑥𝑖𝜅 , 𝑧 𝜅 )𝑢𝑖𝜅 ;     𝑝 Z (𝑥 𝜅 , 𝑧 𝜅 ) + 𝑞 Z (𝑥𝑖 , 𝑧)𝑢 𝜅 ∀ 𝑗 ∈ N \ {𝑖} 𝑗 𝑖  𝑗 𝑖 we hence obtain the dynamic constraints as: 𝐹dyn (𝜗 𝜅 ) , 𝜒¤ 𝜅 − 𝐹 ( 𝜒 𝜅 ) = 0. (C.2) Here these dynamics are given as in Def. 17, i.e., 𝑓𝑖Z and 𝑔𝑖Z are given in (4.25) and (4.26) and 𝑝𝑖Z , 𝑞𝑖Z , 𝑝 Z𝑗 , and 𝑞 Z𝑗 are given in (4.28)-(4.31). The end result is an equality constraint imposed on the cardinal nodes to enforce all of the states and controllers satisfy the dynamic system (17). Direct Collocation. To enforce that those solutions on cardinal nodes stay on the same vector flow, i.e., belong to one unique solution to the dynamic system (17), we employ an implicit stage-3 Runge-Kutta method for formulating this condition as equality constraints. We first use a piecewise Hermite interpolation (cubic) polynomial to compute the interpolated value of 𝜒𝑐 and its slope 𝜒¤ 𝑐 at the center of the subinterval [𝑡 𝜅 , 𝑡 𝜅+1 ], i.e., the collocation point, as: 𝜒¯ 𝑐 = ( 𝜒 𝜅 + 𝜒 𝜅+1 )/2 + 𝛿𝑡 ( 𝜒¤ 𝜅 − 𝜒¤ 𝜅+1 )/8 𝜒¤̄ 𝑐 = −3( 𝜒 𝜅 − 𝜒 𝜅+1 )/2𝛿𝑡 − ( 𝜒¤ 𝜅 + 𝜒¤ 𝜅+1 )/4. Then the collocation constraints is formed as: 𝑑 (X) , 𝜒¤̄ 𝑐 − 𝐹 ( 𝜒¯ 𝑐 ) = 0. (C.3) Physical Constraints. Additionally, the path constraints: 𝑝(·) ≥ 0, which are a set of nonequality constraints, are always used to enforce conditions along the time horizon. Examples include obstacle avoidance constraints for a robot arm 94 manipulation task, or some feasibility conditions on the solution to the dynamic equation, for which some real-world physics is represented. A set of equality constraints 𝑏(·) = 0 that are imposed on the decision variables at 𝑡 = 0, 𝑇 are often utilized to connect the initial and final condition: " # 0 − 𝜒K 𝜒 𝑏(𝜗 𝜅 ) , 0 =0 (4.36) 𝜒¤ − 𝜒¤ K to force the optimization to only search for a solution to the dynamic system that is a periodic orbit. Optimization problem. To find the solution to dynamical system in (4.32), we parse the decoupled control problem of the isolating subsystem 𝑖 ∈ V as a mathematical programming: argmin 𝐽 (X) (CCS-OPT) X s.t. 𝐹zero (𝜗 𝜅 ) = 0 𝜅 = 0, 1, . . . , K (C.1) 𝐹dyn (𝜗 𝜅 ) = 0 𝜅 = 0, 2, 4, . . . (C.2) 𝑑 (X) = 0 (C.3) 𝜗𝜅 ∈ X × Z × U 𝜅 = 0, 1, . . . , K (C.4) 𝑝(𝜗 𝜅 ) ≥ 0 𝜅 = 0, 1, . . . , K (C.5) 𝑏(𝜗 𝜅 ) = 0 𝜅 = 0, 1, . . . , K (C.6) where 𝐽 (·) ∈ R is the cost function, often picked as the total energy consumption. Here (C.4) defines the upper and lower bounds of the decision variables, i.e., that they live in the admissible space of values. The other constraints are as stated above. Solutions. As a result, the optimization (CCS-OPT) can simultaneously output trajectories (solutions) of the states {𝑥𝑖 (𝑡), 𝑧(𝑡)}, 𝑢 Z𝑗 (𝑡) that renders the zero dynamics manifold Z 𝑗 invariant and the open-loop controller 𝑢𝑖 (𝑡), with 𝑡 = [0, 𝑇] for which these solutions are defined. Note that one can also enforce the dynamics 𝑥¤𝑖𝜅 +𝜀𝑥𝑖𝜅 = 0 with 𝜀 > 0 to guarantee the converging attribute of the isolating subsystem 𝑖, in which case the controller 𝑢𝑖 (𝑥𝑖 , 𝑧) is equivalently an input-output feedback linearization controller. Per Theorem 2, given 𝑢 Z𝑗 that renders invariant Z 𝑗 and the feedback controller 𝑢𝑖 (𝑥𝑖 , 𝑧), we can compute 𝜆Z (𝑡) using (4.23), hence (𝜄(𝑥𝑖 (𝑡), 𝑧(𝑡)), 𝜆Z (𝑡)) is then a solution to the original CDS. If we further impose a periodic condition on the solution’s boundary such as (4.36), the optimization shall produce a periodic solution of period 𝑇 to the CCS. Therefore, 95 according to Corollary 1, (𝜄(𝑥𝑖 (𝑡), 𝑧(𝑡), 𝜆Z (𝑡)) is a periodic solution to the CDS with period 𝑇. Quadrupedal Locomotion as Symmetric Coupled Systems A special case of CCSs that is of particular interest to our study of quadrupedal robots is CCSs that are symmetric, i.e., Symmetric Coupled Control Systems. In this case, we consider a CCS with only two systems that have identical (symmetric) coupling constraints—hence the use of the term “symmetric.” We will demonstrate this with an example consisting of a quadrupedal robot as two connected bipedal systems. Definition 18. A symmetric coupled control system (SCCS) is given by a graph G = (N = {1, 2}, E = {𝑒 = (1, 2), 𝑒 = (2, 1)}) , (4.37) together with a tuple represented by the following conditional expression:   𝑥¤𝑖 = 𝑓𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑔𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 + 𝑔˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒       𝑧¤𝑖 = 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 + 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧𝑖 , 𝑧 𝑗 )𝜆 𝑒  + CC ,  s.t. 𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 ) = 𝑧𝑖 − 𝑧 𝑗 ≡ 0       𝜆 𝑒 = −𝜆 𝑒  (4.38) for 𝑖, 𝑗 = 1, 2 with 𝑖 ≠ 𝑗. Note that 𝑥𝑖 ∈ X𝑖 and 𝑧𝑖 ∈ Z𝑖 and we let X = X1 × X2 and Z = Z1 × Z2 , i.e., 𝑥 = (𝑥1 , 𝑥2 ) and 𝑧 = (𝑧1 , 𝑧2 ). Solutions. We define solutions to coupled control systems by assuming the existence of feedback control laws: 𝑢(𝑥, 𝑧) , {𝑢 1 (𝑥1 , 𝑧), 𝑢 2 (𝑥2 , 𝑧)}. Applying these controllers to (4.38) yields a symmetric coupled dynamical system (+DC ):    𝑥¤𝑖 = 𝑓𝑖cl (𝑥𝑖 , 𝑧) + 𝑔˘ 𝑒 (𝑥𝑖 , 𝑧)𝜆 𝑒     + DC , 𝑧¤𝑖 = 𝑝𝑖cl (𝑥𝑖 , 𝑧) + 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧)𝜆 𝑒      s.t. 𝑐 𝑒 (𝑧) ≡ 0, 𝜆 𝑒 = −𝜆 𝑒¯  where, 𝑓𝑖cl , 𝑓𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑔𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 (𝑥𝑖 , 𝑧) 𝑝𝑖cl , 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 (𝑥𝑖 , 𝑧). Then the solution of the coupled dynamic system, +DC , is a set of solutions: n

o 𝑥 1 (𝑡), 𝑧1 (𝑡), 𝜆 𝑒 (𝑡) , 𝑥 2 (𝑡), 𝑧2 (𝑡), 𝜆 𝑒¯ (𝑡) s.t. (4.39) ∀𝑡 ∈ I ⊂ R (4.39) 96 Figure 4.8: A conceptual illustration of the full-body dynamics decomposition, where the 3D quadruped — the Vision 60 — is decomposed into two constrained 3D bipedal robots. with initial condition: {(𝑥 1 (0), 𝑧1 (0), 𝜆 𝑒 (0)), (𝑥 2 (0), 𝑧2 (0), 𝜆 𝑒¯ (0))}, and I ⊂ R is the time interval of their existence. Per the above notation, we will sometimes denote the solutions by (𝑥(𝑡), 𝑧(𝑡), 𝜆(𝑡)) with initial condition (𝑥(0), 𝑧(0), 𝜆(0)). Coupling constraints. Importantly, the solutions must satisfy the coupling constraints at all time. Therefore, 𝑐 𝑒 (𝑧) ≡ 0 ⇒ 𝑐¤𝑒 (𝑧, 𝑧¤) ≡ 0 𝜕𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 ) 𝜕𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 ) 𝑧¤𝑖 + ⇒ 𝑧¤ 𝑗 ≡ 0 𝜕𝑧𝑖 𝜕𝑧 𝑗 | {z } | {z } (𝑖, 𝑗) ,𝐽𝑒 ( 𝑗,𝑖) (𝑧) ,𝐽𝑒 (𝑧)  (𝑧) 𝑝𝑖cl (𝑥𝑖 , 𝑧) + 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧)𝜆 𝑒   ( 𝑗,𝑖) cl + 𝐽𝑒 (𝑧) 𝑝 𝑗 (𝑥 𝑗 , 𝑧) + 𝑞˘ 𝑒¯ (𝑥 𝑗 , 𝑧)𝜆 𝑒¯ ≡ 0. (𝑖, 𝑗) ⇒ 𝑐¤𝑒 (𝑥, 𝑧) = 𝐽𝑒 (4.40)  (4.41) Hence, to solve for the coupling inputs 𝜆 𝑒 that satisfy the coupling constraints, it is necessary to solve an equation that depends on the states of both subsystems. To address this, we present a method for isolating a subsystem via conditions on the controllers of the other systems in the next section. Before doing this, we utilize the following example to illustrate the concepts of coupled control systems. Decompose a Quadruped into Bipeds The motivating application considered here is to compute periodic solutions to the quadrupedal dynamics. As Fig. 4.1 shows, we decompose this quadruped into two bipeds, whose dynamics are on a CCS graph (according to Def. 18):
G , N = {f, r}, E = {𝑒 = (f, r), 𝑒¯ = (r, f)} , where f, r label the front and rear bipedal systems, correspondingly. We pick the coordinates for these two subsystems 97 as 𝑞 f = (𝜉f> , 𝜃 L>2 , 𝜃 L>0 ) > , 𝑞 r = (𝜉r> , 𝜃 L>1 , 𝜃 L>3 ) > with 𝜉𝑖 ∈ R3 × SO(3) and the leg joints 𝜃 𝐿 ∗ ∈ R3 . Since all leg joints are actuated, the inputs are 𝑢𝑖 ∈ U ⊂ R6 . The continuous-time dynamics in Fig. 4.2, when toe1 and toe2 are on the ground, are modeled as constrained dynamics:    𝐷 (𝑞) 𝑞¥ + 𝐻 (𝑞, 𝑞) ¤ = 𝐵𝑢 + 𝐽f> (𝑞)𝐹1 + 𝐽r> (𝑞) 𝐹2     𝐽f (𝑞) 𝑞¥ + 𝐽¤f (𝑞, 𝑞) ¤ 𝑞¤ = 0      𝐽r (𝑞) 𝑞¥ + 𝐽¤r (𝑞, 𝑞) ¤ 𝑞¤ = 0  , (4.42) with the domain D , {𝑥 ∈ X : ℎ¤ 1 (𝑞, 𝑞) ¤ = ℎ¤ 2 (𝑞, 𝑞) ¤ = 0, ℎ 𝑧1 (𝑞) = ℎ 𝑧2 (𝑞) = 0}. In this formulation, we utilize the following notation: 𝐷 (𝑞) ∈ R𝑛×𝑛 is the inertia-mass matrix; 𝐻 (𝑞, 𝑞) ¤ ∈ R𝑛 contains Coriolis forces and gravity terms; ℎ1 (𝑞), ℎ2 (𝑞) ∈ R3 are the Cartesian positions of toe1 and toe2, their Jacobians are 𝐽∗ = 𝜕ℎ∗ /𝜕𝑞; ℎ 𝑧1 (𝑞), ℎ 𝑧2 (𝑞) are these toes’ height; 𝐹f , 𝐹r ∈ R3 are the ground reaction forces on toe1 and toe2; 𝐵 ∈ R𝑛×𝑚 is the actuation matrix. Essentially, we use a set of differential algebra equations (DAEs) to describe the dynamics of the quadrupedal robot that is subject to two holonomic constraints on toe1 and toe2. We now decompose the quadrupedal full-body dynamics into two bipedal robots. First, as shown in Fig. 4.8, the open-loop dynamics can be equivalently written as:  𝐷 f 𝑞¥f + 𝐻f = 𝐽f> 𝐹f + 𝐵f 𝑢 f − 𝐽𝑒>𝜆 𝑒       𝐽f 𝑞¥f + 𝐽¤f 𝑞¤ f = 0     OL-Dyn , 𝐷 r 𝑞¥r + 𝐻r = 𝐽r> 𝐹r + 𝐵r 𝑢 r + 𝐽𝑒>𝜆 𝑒     𝐽r 𝑞¥r + 𝐽¤r 𝑞¤ r = 0     ¥ ¥  𝜉f − 𝜉r = 0 (4.43) (4.44) (4.45) (4.46) (4.47) wherein we utilized the following notation: 𝐷 f (𝑞 f ), 𝐷 r (𝑞 r ) ∈ R12×12 are the inertiamass matrices of the front and rear bipedal robots; the Jacobians 𝐽f = 𝜕ℎf /𝜕𝑞 f , 𝐽r1 = 𝜕ℎr /𝜕𝑞 r with the Cartesian positions of toe2 as ℎf (𝑞 f ) and toe1 as ℎr (𝑞 r ); the Jacobian matrix for the connection constraint (4.47) is 𝐽𝑒 = 𝜕 (𝜉f − 𝜉f )/𝜕𝑞 f ; 𝑢 > f = > > > > > (𝑢 0 , 𝑢 2 ) and 𝑢 r = (𝑢 1 , 𝑢 3 ). Note that the Cartesian position of toe2 only depends on 𝑞 f , which is due to the floating base coordinate convention. Proposition 1. The dynamical system (OL-Dyn) is equivalent to the system (4.42). 98 Proof. We can write (4.43) and (4.45) as:  𝐷 𝑏 𝐷 𝑏 𝐷 𝑏   𝜉¥f  𝐻𝑏  0 2    f  f  >  ¥    0  𝜃 0  +  𝐻0  = 𝐵f 𝑢 f + 𝐽f> 𝐹f − 𝐽𝑐>𝜆 𝑒  𝐷 𝑏0 𝐷 0      𝐷 > 𝐷 2  𝜃¥2   𝐻2   𝑏2 0  𝐷 𝑏 𝐷 𝑏 𝐷 𝑏   𝜉¥r  𝐻𝑏  1 3   r   r  >  ¥    𝐷 𝐷 0  𝑏1  𝜃 1  +  𝐻1  = 𝐵r 𝑢 r + 𝐽r> 𝐹r + 𝐽𝑒>𝜆 𝑒 1      𝐷 >  𝜃¥3   𝐻3  0 𝐷 3  𝑏3     where each entry has a proper dimension to make the equations consistent. Expanding them yields: 𝐷𝑏 𝐷𝑏 0  f  >  𝐷 𝑏0 𝐷 0   0 0   >  𝐷 𝑏2 0   0 0  𝐷𝑏  r   0  𝐷 >  𝑏1   0  𝐷 >  𝑏3 0 𝐷 𝑏2 0  𝜉¥f  𝐻𝑏f  −𝜆 𝑒        0 0 0 𝜃¥0   𝐻0   𝑢 0        0 0 0 𝜃¥1  +  0  =  0  + 𝐽r> 𝐹r ,       0 𝐷 2 0 𝜃¥2   𝐻2   𝑢 2        0 0 0 𝜃¥3   0   0  0 𝐷 𝑏1 0 𝐷 𝑏3   𝜉¥r  𝐻𝑏r  𝜆 𝑒        0 0 0 0  𝜃¥0   0   0        0 𝐷 1 0 0  𝜃¥1  +  𝐻1  = 𝑢 1  + 𝐽f> 𝐹f .       0 0 0 0  𝜃¥2   0   0        0 0 0 𝐷 3  𝜃¥3   𝐻3  𝑢 3  Combining these two equations, and using the fact that 𝜉f − 𝜉r ≡ 0 (holonomic constraint) yields the dynamics given in (2.7). It is worthwhile to note that all the terms that appear in these equations can be verified using traditional rigid body dynamics, and the corresponding details of the structure and necessary properties of the inertia-mass matrices can be found from the branch-induced sparsity, see (R. Featherstone, 2008).  Quadruped as CCS. Return to the symmetric CCS, we further abstract (OL-Dyn) into a set of Differential Algebraic Equations (DAEs): ℛ𝑄 ,  𝐷 𝑖 𝑞¥𝑖 + 𝐻𝑖 = 𝐽𝑖> 𝐹𝑖 + 𝐵𝑖 𝑢𝑖 + 𝐽𝑒>𝜆 𝑒       𝐽 𝑞¥ + 𝐽¤ 𝑞¤ = 0  (4.48)   s.t. 𝑐 𝑒 (𝜉𝑖 , 𝜉 𝑗 ) = 𝜉𝑖 − 𝜉 𝑗 ≡ 0     𝜆 𝑒 = −𝜆 𝑒¯  (4.50) 𝑖 𝑖 𝑖 𝑖 (4.49) (4.51) 99 𝑛×𝑛 𝑛 with 𝐷 h 𝑖 (𝑞𝑖 ) ∈ Ri the mass-inertia matrix, 𝐻𝑖 (𝑞𝑖 , 𝑞¤𝑖 ) ∈ R the drift vector, and 𝐵𝑖 = 06×6 𝐼6×6 the actuation matrix. The contact (holonomic) constraint ℎ𝑖 (𝑞𝑖 ) ≡ 0 is enforced via ground reaction forces 𝐹𝑖 ∈ R3 , whose second derivative is given in (4.49). More details of these notations can be found in (Wen-Loong Ma and Aaron D. Ames, 2020). Note that 𝐹𝑖 can be eliminated by the solving (4.48)-(4.49) to have a shorter form: 𝐷 𝑖 𝑞¥𝑖 + 𝐻¯ 𝑖 = 𝐵¯ 𝑖 𝑢𝑖 + 𝐽¯𝑒>𝜆 𝑒 . The derivation is similar to (2.17) hence omitted. To obtain a CCS as in Def. 18, we pick “normal form” type coordinates (see Sastry, 1999), with the “output” that we wish to zero, given by 𝑦𝑖 (𝑞𝑖 , 𝛼𝑖 ) = 𝑦 𝑎 (𝑞𝑖 ) − 𝑦 𝑑 (𝜉𝑖 , 𝛼𝑖 ), (4.52) where 𝑦 𝑎 , 𝑦 𝑑 are the actual and desired outputs, 𝜉𝑖 represents a parameterization of time and 𝛼𝑖 ∈ R6×6 are the coefficients for six fifth-order Beźier polynomials that are designed by the optimization algorithm in (CCS-OPT). Since our goal is to find a symmetric ambling gait for quadrupeds, we chose 𝛼r = M𝛼f , with the matrix M representing a mirroring relation. We can then construct our internal states 𝑥𝑖 = (𝑦𝑖> , 𝑦¤ 𝑖> ) > , leaving the coupled states as 𝑧𝑖 = (𝜉𝑖> , 𝜉¤𝑖> ) > . The end result is a CCS of the form given in (4.38) for this mechanical system: " # " # " # 𝑦¤ 𝑖 0 0 + 𝑢𝑖 + 𝜆𝑒 𝑥¤𝑖 = 𝐽¤𝑦𝑖 𝑞¤𝑖 − 𝐽𝑦𝑖 𝐷 𝑖−1 𝐻¯ 𝑖 𝐽𝑦𝑖 𝐷 𝑖−1 𝐵¯ 𝑖 𝐽𝑦𝑖 𝐷 𝑖−1 𝐽¯𝑒> | {z } | {z } | {z } 𝑓𝑖 (𝑥 𝑖 ,𝑧𝑖 ) " # 𝑔𝑖 (𝑥 𝑖 ,𝑧𝑖 ) " # 𝑔˘ 𝑒 (𝑥𝑖 ,𝑧𝑖 ,𝑧 𝑗 ) " # 𝜉¤𝑖 0 0 𝑢𝑖 + 𝜆𝑒 𝑧¤𝑖 = + −𝐽𝜉 𝐷 𝑖−1 𝐻¯ 𝑖 𝐽𝜉 𝐷 𝑖−1 𝐵¯ 𝑖 𝐽𝜉 𝐷 𝑖−1 𝐽¯𝑒> | {z } | {z } | {z } 𝑝 𝑖 (𝑥 𝑖 ,𝑧 𝑖 ) 𝑞 𝑖 (𝑥𝑖 ,𝑧𝑖 ) 𝑞˘ 𝑒 (𝑥 𝑖 ,𝑧𝑖 ,𝑧 𝑗 ) 𝜆 𝑒 = −𝜆 𝑒¯ , h i where 𝐽𝑦𝑖 = 𝜕𝑦𝑖 (𝑞𝑖 )/𝜕𝑞𝑖 , 𝐽𝜉 = 𝜕𝜉/𝜕𝑞 = 𝐼6×6 06×6 , and we suppressed the dependency on 𝑥𝑖 , 𝑧𝑖 for all entries. s.t. 𝑐 𝑒 (𝑧𝑖 , 𝑧 𝑗 ) = 𝑧𝑖 − 𝑧 𝑗 ≡ 0, 𝜆-Coupled Subsystem. Given a CCS CC , we define the zero dynamics manifold for each subsystem 𝑖 ∈ N as: Z𝑖 , {(𝑥, 𝑧) ∈ X × Z | 𝑥𝑖 ≡ 0}. (4.53) Thus, the zero dynamics manifold for 𝑖 th subsystem consists of the internal states, 𝑥𝑖 , being zero, i.e., the system evolves only according to the coupled states 𝑧. 100 The key idea underlying the analysis of CCSs is to reduce the entire coupled system into the behavior of a single subsystem. This is achieved through the above constructions related to the zero dynamics. We start by designing controllers for the overall CCS on the zero dynamics of subsystem 𝑗 ∈ N . A controller 𝑢 Z,𝜆 𝑗 (𝑥 𝑗 , 𝑧) is said to render the zero dynamics manifold Z 𝑗 invariant if it satisfies: 0 ≡ 𝑓 𝑗 (0, 𝑧 𝑗 ) + 𝑔 𝑗 (0, 𝑧 𝑗 )𝑢 Z,𝜆 𝑗 (0, 𝑧) + 𝑔˘ 𝑒¯ (0, 𝑧)𝜆 𝑒¯ (4.54) Z,𝜆 where 𝑢 Z,𝜆 𝑗 implicitly depends on 𝜆 𝑒¯ for 𝑒¯ = ( 𝑗, 𝑖) ∈ E. By applying 𝑢 𝑗 , we obtain a 𝜆-coupled control subsystem (𝜆-CCSub) for the 𝑖 th subsystem:    𝑥¤𝑖 = 𝑓𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑔𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 + 𝑔˘ 𝑒 (𝑥𝑖 , 𝑧)𝜆 𝑒       𝑧¤𝑖 = 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 )𝑢𝑖 + 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧)𝜆 𝑒  + Z,𝜆 C𝑖 ,   𝑧¤ 𝑗 = 𝑝 𝑗 (0, 𝑧 𝑗 ) + 𝑞 𝑗 (0, 𝑧 𝑗 )𝑢 Z,𝜆  𝑗 (0, 𝑧) + 𝑞˘ 𝑒¯ (0, 𝑧)𝜆 𝑒¯      s.t. 𝑐 𝑒 (𝑧) ≡ 0, 𝜆 𝑒 = −𝜆 𝑒¯  . (4.55) Thus, the 𝑖 th subsystem evolves according to its own dynamics and the zero dynamics of all remaining systems—all of which are coupled via the coupling inputs 𝜆 = {𝜆 𝑒 , 𝜆 𝑒¯ }. Explicit coupling conditions. The coupling between the control systems (4.38) is enforced via 𝜆 and the coupling constraints of the form (4.16). Similarly, even in the reduction to a subsystem (4.55), the coupling is still achieved through 𝜆. We wish to generalize this so as to remove the coupling, i.e., isolate subsystems, while still preserving the overall behavior of the full system. We first define the coupling relation that allows the use of the controllers 𝑢 Z,𝜆 to eliminate the dependence on 𝑗 the controllers and internal states of the other subsystem. Definition 19. For a 𝜆-CCSub C𝑖Z,𝜆 and 𝑖 ∈ N , a coupling relation is a functional relationship on the coupling inputs 𝜆Z𝑒 (𝑥𝑖 , 𝑧; 𝑢𝑖 ) = 𝐴Z𝑒 (𝑥𝑖 , 𝑧)𝑢𝑖 + 𝑏 Z𝑒 (𝑥𝑖 , 𝑧), that satisfies the coupling constraint (4.15) for all 𝑒 = (𝑖, 𝑗) ∈ E. The coupling relation is then summarized in the following: Lemma 5. For a CCS CC , if we have " # 𝑔 (0, 𝑧 ) − 𝑔 ˘ (0, 𝑧) 𝑗 𝑗 𝑒 ¯ ˘ 𝑖 , 𝑧) , 𝑄(𝑥 ( 𝑗,𝑖) (𝑖, 𝑗) ( 𝑗,𝑖) 𝐽𝑐 𝑞 𝑗 (0, 𝑧 𝑗 ) 𝐽𝑐 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧) − 𝐽𝑐 𝑞˘ 𝑒¯ (0, 𝑧) (4.56) 101 invertible, there exists a controller 𝑢 Z𝑗 that renders Z 𝑗 invariant and a coupling relation in (4.23), given by: # " # "  𝑢 Z𝑗 (0, 𝑧; 𝑢𝑖 ) 0 𝑢𝑖 + (4.57) = 𝑄˘ −1 (𝑖, 𝑗) Z −𝐽𝑐 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 ) 𝜆 𝑒 (𝑥𝑖 , 𝑧; 𝑢𝑖 ) " #  − 𝑓 𝑗 (0, 𝑧 𝑗 ) . (𝑖, 𝑗) ( 𝑗,𝑖) −𝐽𝑐 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) − 𝐽𝑐 𝑝 𝑗 (0, 𝑧 𝑗 ) Proof. Evaluating (4.15) along the zero dynamics manifold Z 𝑗 , i.e., 𝑥 𝑗 ≡ 0, yields: (𝑖, 𝑗) 𝐽𝑒 (𝑧)( 𝑝𝑖 (𝑥𝑖 , 𝑧) + 𝑞𝑖 (𝑥𝑖 , 𝑧)𝑢𝑖 + 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧)𝜆 𝑒 )+ ( 𝑗,𝑖) 𝐽𝑒¯ (𝑧)( 𝑝 𝑗 (0, 𝑧) + 𝑞 𝑗 (0, 𝑧)𝑢𝑖 + 𝑞˘ 𝑒¯ (0, 𝑧)𝜆 𝑒 ) = 0. Combining this with (4.18) and simultaneously solving for 𝑢 Z𝑗 and 𝜆Z𝑒 yields the desired result.  Recall that the controller 𝑢 Z,𝜆 that renders the zero dynamics surface invariant 𝑗 implicitly depends on 𝜆 𝑒¯ via (4.18). Now with a coupling relation, the dependence of 𝜆 𝑒¯ is removed, and as a result we say that 𝑢 Z𝑗 renders the zero dynamics manifold Z 𝑗 invariant if:   0 ≡ 𝑓 𝑗Z (0, 𝑧) + 𝑔 Z𝑗 (0, 𝑧)𝑢𝑖 + 𝑔 𝑗 (0, 𝑧 𝑗 ) 𝑢 Z𝑗 (0, 𝑧; 𝑢𝑖 ) − 𝑢𝑖 (4.58) where 𝑢 Z𝑗 is now a function of 𝑢𝑖 and    𝑓 𝑗Z (𝑥 𝑗 , 𝑧)  , 𝑓 𝑗 (𝑥 𝑗 , 𝑧 𝑗 ) − 𝑔˘ 𝑒¯ (𝑥 𝑗 , 𝑧)𝑏 Z𝑒 (𝑥𝑖 , 𝑧),   𝑔 Z𝑗 (𝑥 𝑗 , 𝑧)  , 𝑔 𝑗 (𝑥 𝑗 , 𝑧 𝑗 ) − 𝑔˘ 𝑒¯ (𝑥 𝑗 , 𝑧) 𝐴Z𝑒 (𝑥𝑖 , 𝑧). (4.59) Returning to (4.16), given a coupling relation we can rewrite this coupling constraint as:     (𝑖, 𝑗) ( 𝑗,𝑖) 𝑐¤𝑒 (𝑥𝑖 , 𝑧) = 𝐽𝑒 (𝑧) 𝑝𝑖Z (𝑥𝑖 , 𝑧) + 𝑞𝑖Z (𝑥𝑖 , 𝑧)𝑢𝑖 + 𝐽𝑒 (𝑧) 𝑝 Z𝑗 (𝑥𝑖 , 𝑧) + 𝑞 Z𝑗 (𝑥𝑖 , 𝑧)𝑢𝑖 = 0, (4.60) where for the subsystem C𝑖Z,𝜆 we have    𝑝𝑖Z (𝑥𝑖 , 𝑧) , 𝑝𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧)𝑏 Z𝑒 (𝑥𝑖 , 𝑧)       𝑞𝑖Z (𝑥𝑖 , 𝑧) , 𝑞𝑖 (𝑥𝑖 , 𝑧𝑖 ) + 𝑞˘ 𝑒 (𝑥𝑖 , 𝑧) 𝐴Z𝑒 (𝑥𝑖 , 𝑧)    𝑝 Z𝑗 (𝑥𝑖 , 𝑧) , 𝑝 𝑗 (0, 𝑧 𝑗 ) + 𝑞 𝑗 (0, 𝑧 𝑗 )𝑢 Z𝑗 (0, 𝑧) − 𝑞˘ 𝑒¯ (0, 𝑧)𝑏 Z𝑒 (𝑥𝑖 , 𝑧)       𝑞 Z𝑗 (𝑥𝑖 , 𝑧) , −𝑞˘ 𝑒¯ (0, 𝑧) 𝐴Z𝑒 (𝑥𝑖 , 𝑧)  . (4.61) 102 Isolating subsystems. We now arrive at the key concept for which all of the previous constructions have built: reducing a CCS to a single subsystem that can be used to give guarantees about the entire CCS. This is based on the following definition. Definition 20. For a CCS CC , and 𝑖 ≠ 𝑗 ∈ N , assume a coupling relation 𝜆Z𝑒 such that there exist 𝑢 Z𝑗 rendering the zero dynamics manifold Z 𝑗 invariant. Then the 𝑖 th control subsystem (CSub) associated with the CCS CC is given by:   𝑥¤𝑖 = 𝑓𝑖Z (𝑥𝑖 , 𝑧) + 𝑔𝑖Z (𝑥𝑖 , 𝑧)𝑢𝑖     C𝑖Z , 𝑧¤𝑖 = 𝑝𝑖Z (𝑥𝑖 , 𝑧) + 𝑞𝑖Z (𝑥𝑖 , 𝑧)𝑢𝑖     𝑧¤ 𝑗 = 𝑝 Z (𝑥𝑖 , 𝑧) + 𝑞 Z (𝑥𝑖 , 𝑧)𝑢𝑖 𝑗 𝑗  (4.62) where 𝑓𝑖Z (𝑥𝑖 , 𝑧) , 𝑓𝑖 (𝑥𝑖 , 𝑧𝑖 )+𝑔˘ 𝑒 (𝑥𝑖 , 𝑧)𝑏 Z𝑒 (𝑥𝑖 , 𝑧), 𝑔𝑖Z (𝑥𝑖 , 𝑧) , 𝑔𝑖 (𝑥𝑖 , 𝑧𝑖 )+𝑔˘ 𝑒 (𝑥𝑖 , 𝑧) 𝐴Z𝑒 (𝑥𝑖 , 𝑧), and 𝑝𝑖Z , 𝑞𝑖Z , 𝑝 Z𝑗 , 𝑞 Z𝑗 are given in (4.61). Furthermore, when a feedback controller 𝑢𝑖 (𝑥𝑖 , 𝑧) is applied to C𝑖Z , the result is a dynamical system, denoted by D𝑖Z . Apply to Quadrupedal Walking For the quadrupedal dynamics ℛ𝑄 , since the output (4.52) has (vector) relative degree 2 with respect to 𝑢𝑖 (see (E. R. Westervelt, J. W. Grizzle, Chevallereau, et al., 2007)), we can explicitly design the controller 𝑢 Z,𝜆 𝑗 that renders Z 𝑗 invariant:
−1 ¯ −1 −1 ¯ ¤𝑦 𝑗 𝑞¤ 𝑗 − 𝐽𝑦𝑖 𝐷 −1 ¯𝑒>𝜆 𝑒 , 𝑢 Z,𝜆 = (𝐽 𝐷 𝐵 ) 𝐽 𝐷 𝐻 − 𝐽 𝐽 𝑦 𝑗 𝑦 𝑗 𝑖 𝑗 𝑗 𝑗 𝑗 𝑗 as given by Lemma 5. Hence, this controller satisfies (4.18) and renders a 𝜆-coupled CSub, as in (4.55). For robotic systems, we take these ideas one step further to obtain “bipeds” that are the isolated subsystems associated with quadrupeds, and include slack variables that are beneficial for gait generation. Operating on the invariant zero dynamics manifold Z 𝑗 yields 𝑦 𝑗 (𝑞 𝑗 , 𝛼 𝑗 ) ≡ 0, hence ⇒ 𝜃 𝑎 ≡ 𝐻𝑎−1 𝑦 𝑑 (𝜉 𝑗 , 𝛼 𝑗 ), ⇒ 𝑞 Z𝑗 (𝜉 𝑗 ) ≡ 𝜉 >𝑗 , (𝐻𝑎−1 𝑦 𝑑 (𝜉 𝑗 , 𝛼 𝑗 )) >
> ⇒ 𝑞¥Z𝑗 (𝜉 𝑗 , 𝜉¤ 𝑗 , 𝜉¥𝑗 ) = 𝐽z (𝜉 𝑗 ) 𝜉¥𝑗 + 𝐽¤z (𝜉 𝑗 , 𝜉¤ 𝑗 ) 𝜉¤ 𝑗 , where 𝐽z = 𝜕𝑞 Z𝑗 (𝜉 𝑗 )/𝜕𝜉 𝑗 . In other words, if 𝑢 Z,𝜆 exists and is applied to 𝑗 th sub𝑗 system, the 𝑗 th bipedal dynamics given by in (4.48)-(4.49) are equivalent to: ( 𝐷 𝑗 𝑞¥Z𝑗 (𝜉 𝑗 , 𝜉¤ 𝑗 , 𝜉¥𝑗 ) + 𝐻 𝑗 = 𝐽 >𝑗 𝐹 𝑗 + 𝐵 𝑗 𝑢 Z𝑗 + 𝐽𝑒>𝜆 𝑒 (4.63) 𝐽 𝑗 𝑞¥Z𝑗 (𝜉 𝑗 , 𝜉¤ 𝑗 , 𝜉¥𝑗 ) + 𝐽¤𝑗 𝑞¤ Z𝑗 (𝜉 𝑗 , 𝜉¤ 𝑗 ) = 0 (4.64) 103 where for simplicity we have suppressed the dependencies of 𝐷 𝑗 (𝑞 𝑗 (𝜉 𝑗 )), 𝐽 𝑗 (𝑞 𝑗 (𝜉 𝑗 )) and 𝐻 𝑗 (𝑞 𝑗 (𝜉 𝑗 ), 𝑞¤ 𝑗 (𝜉 𝑗 , 𝜉¤ 𝑗 )). We then leverage a specific structure of rigid-body dy> > namics when using the floating base convention: 𝐵 𝑗 𝑢 𝑗 +𝐽𝑒>¯ 𝜆 𝑒 = (𝜆> 𝑒¯ , 𝑢 𝑗 ) . Utilizing this, (4.64) and the first 6 rows of (4.63) yield the following “bipedal” dynamics:    𝐷 Z𝑗 𝜉¥𝑗 + 𝐻 Z𝑗 = 𝐽ˆ>𝑗 𝐹 𝑗 + 𝜆 𝑒  Z𝑗 ℛ𝐵 ,   𝐽 Z𝑗 𝜉¥𝑗 + 𝑤 Z𝑗 = 0  (4.65) with 𝐷 Z𝑗 = 𝐷ˆ 𝑗 𝐽z , 𝐻 Z𝑗 = 𝐷ˆ 𝑗 𝐽¤z 𝜉¤ 𝑗 + 𝐻ˆ 𝑗 , 𝐽 Z𝑗 = 𝐽 𝑗 𝐽z , and 𝑤 Z𝑗 = 𝐽 𝑗 𝐽¤z 𝜉¤ 𝑗 + 𝐽¤𝑗 𝐽¤z 𝜉¤𝑖 . Here, Z we denote ˆ as the first 6 rows (block) of the a variable. Hence, ℛ𝐵 𝑗 represents the dynamics of a subsystem 𝑗 on Z 𝑗 , i.e., (4.65) evolves according to (4.18) where 𝐹 𝑗 can be uniquely determined. Coupling Relations for Robotic Systems. Following the previous construction, we can also have the explicit form of the controller 𝑢 Z𝑗 that renders zero dynamics manifold Z 𝑗 invariant and the coupling condition in (4.23) as " 𝑢 Z𝑗 (𝑥𝑖 , 𝑧; 𝑢𝑖 ; 𝑡) # " ¯ 𝐽𝑦 𝑗 𝐷 −1 𝑗 𝐵𝑗 ¯> −𝐽𝑦 𝑗 𝐷 −1 𝑗 𝐽𝑒¯ = −1 ¯> −1 ¯> ¯ 𝐽𝜉 𝑗 𝐷 −1 𝜆Z𝑒 (𝑥𝑖 , 𝑧; 𝑢𝑖 ; 𝑡) 𝑗 𝐵 𝑗 −𝐽𝜉 𝑗 𝐷 𝑗 𝐽𝑒¯ − 𝐽𝜉𝑖 𝐷 𝑖 𝐽𝑒 " # " #  𝑦¥ 𝑑 − 𝐽¤𝑦 𝑞¤ 𝑗 − 𝐽𝑦 𝐷 −1 𝐻¯ 𝑗  0 𝑗 𝑗 𝑖 + 𝑢𝑖 𝐽𝜉𝑖 𝐷 −1 𝐵¯ 𝑖 −𝐽𝜉 𝐷 −1 𝐻¯ 𝑖 + 𝐽𝜉 𝐷 −1 𝐻¯ 𝑗 𝑖 𝑖 𝑗 𝑗 # −1 (4.66) 𝑖 where 𝐽𝑦𝑖 = 𝜕𝑦𝑖 /𝜕𝑞𝑖 , 𝐽𝜉𝑖 = 𝜕𝑞𝑖 /𝜕𝜉𝑖 . Note that we suppressed the the dependence on 𝑥𝑖 , 𝑧, 𝑡 for notation simplicity. From this, we can obtain the CSub C𝑖Z , as in (4.62). Gait Generation for flat-ground ambling. Finally, we can apply the optimization in (CCS-OPT) to simultaneously produce trajectories (solutions) of the states {𝑥𝑖 (𝑡), 𝑧(𝑡)}, 𝑢 Z𝑗 (𝑡) that renders the zero dynamics manifold Z 𝑗 invariant and the open-loop controller 𝑢𝑖 (𝑡), ∀𝑡 ∈ [0, 𝑇] for which these solutions are defined. Note that one can also enforce the dynamics 𝑥¤𝑖𝜅 + 𝜀𝑥𝑖𝜅 = 0 with 𝜀 > 0 to guarantee the convergence attribute of the 𝑖 th isolating subsystem, in which case the controller 𝑢𝑖 (𝑥𝑖 , 𝑧) is equivalently an input-output feedback linearization controller. When posing the control problem of quadrupeds, we leverage the subsystems representing the front and rear bipeds: ℛZ𝐵 f and ℛZ𝐵 r , as given in (4.65). Note that these subsystems are still coupled through 𝜆—while this could be explicitly solved for via Lemma 5, we keep it implicit due to the complexity of inverting the mass-inertia matrix for this particular robotic application. The 𝑖 th subsystem yield (C.1), (C.2), and (C.3) for (CCS-OPT). Specifically for all of the grid indices 𝜅 = 0, 1, ...5, we 104 -0.20 -0.10 position (rad) 0.00 -4 0.6 0.7 0.8 position (rad) 0 -5 1 1.2 position (rad) 1.4 5 2 0 -2 0.00 0.10 position (rad) 0.20 knee0 hipPitch0 5 0 -5 0.6 0.8 velocity (rad/s) 0 -2 hipRoll0 4 velocity (rad/s) -4 knee2 5 2 velocity (rad/s) 0 -2 hipPitch2 velocity (rad/s) 4 velocity (rad/s) velocity (rad/s) hipRoll2 2 position (rad) 0 -5 1 1.2 1.4 position (rad) Figure 4.9: Top: Snapshots showing a full step of the ambling gait on an outdoor lawn. Bottom: The periodic trajectory produced by optimization (CCS-OPT) (in red) vs. the experimental tracking data (in cyan) vs. RaiSim simulation data (in green) in the form of phase portrait (limit cycle) using 18 seconds’ data. have the decision variables: 𝜗 𝜅 = {𝑞 f𝜅 , 𝑞¤ f𝜅 , 𝜉r𝜅 , 𝜉¤r𝜅 , 𝑢 f𝜅 , 𝐹f𝜅 , 𝐹r𝜅 , 𝛼f , 𝜆 𝑒𝜅 }. Finally, the optimization converges to a periodic solution to the isolated bipedal system, which can then be composed to obtain the ambling motion of the quadruped (shown in Fig. 4.9) according to Theorem 1. We report that the optimization took 17.6s and 295 iterations of searching, which is over 58% faster than the full-model based approach in Sec. 4.1 (see (W. Ma, K. A. Hamed, and A. D. Ames, 2019). The computational complexity is mitigated mainly due to the dimension reduction of the state space which is enabled by the representation of the quadrupedal dynamics as bipedal subsystems. For validation purposes, both simulations in the physics engine RaiSim and hardware experiments were conducted with a unified, time-based PD approximation of input-output linearizing controllers to track the the desired outputs (represented by 𝛼f , 𝛼r = M𝛼f ):

𝑢𝑖 (𝑞𝑖 , 𝑞¤𝑖 , 𝑡) = −𝑘 𝑝 𝑦¤ 𝑎 (𝑞𝑖 ) − 𝑦 𝑡𝑑 (𝑡, 𝛼𝑖 ) − 𝑘 𝑑 𝑦 𝑎 (𝑞𝑖 ) − 𝑦¤ 𝑡𝑑 (𝑡, 𝛼𝑖 ) (4.67) with 𝑘 𝑝 , 𝑘 𝑑 the PD gains. In addition, the switching detection and the event functions are also given by the optimized trajectories, meaning the walking controller will switch to next step when 𝑡 = 𝑇. The result is successful ambling in simulation, and experimentally walking on flat and outdoor uneven terrains (see the video (WenLoong Ma, 2020c)). See Fig. 4.9 for walking tiles and the tracking performance. Remark that the averaged absolute torque inputs are 11.16 N·m, which are well within the hardware limits. Time-based stepping in place. To benchmark the computational performance, we also generate stepping-in-place motion for the quadruped and demonstrate them on hardware. For this behavior, our desired output 𝑦 𝑑 will be a function of time instead 105 of states. We express the algorithm in a different way from (CCS-OPT) as: min 2𝑁+1 Õ X 2 𝜉¤f 2 (4.68) 𝑗=1 Z s.t. C1. isolating bipedal dynamics ℛ𝐵 𝑗 in (4.65) 𝑗 = 1, 3, ...2𝑁 + 1 C2. collocation constraints 𝑗 = 2, 4, ...2𝑁 C3. impact dynamics in(4.6) 𝑗 = 2𝑁 + 1 C4. periodic continuity 𝑗 = 1, 2𝑁 + 1 C5. physical feasibility 𝑗 = 1, 2, ...2𝑁 + 1 with the following notation: 2𝑁 + 1 = 11 is the total number of collocation grids; 𝛼 ∈ R36 are the coefficients for the Beźier polynomial that defines the desired trajectory 𝑦 𝑑 (𝑡); and  𝑗 is the corresponding quantities at time 𝑡 𝑗 with 𝑡 2𝑁+1 = 𝑇. The cost function is to minimize the body’s vibration rate to achieve a more static torso movement. The constraints C1-C3 solve the hybrid dynamics of bipedal robots subject to external forces. Details regarding the numerical optimization can be found in Sec. 2.9. Here, the periodic continuity constraint C4 enforces state continuity through an edge, i.e., the post-impact states 𝑞 + , 𝑞¤ + , are equivalent to the initial states 𝑞 1 , 𝑞¤ 1 . Therefore, the resultant trajectory is a periodic solution to the bipedal dynamics. C5 imposed some feasibility conditions on the dynamics, including torque limits k𝑢𝑖 k ∞ ≤ 50, joint feasible space (𝑞𝑖 , 𝑞¤𝑖 ) ∈ X, foot clearance, and the friction pyramid conditions. Note that we posed these constraints conservatively to reduce the difficulties implementing the optimized trajectories in experiments. To validate that the CCS-based optimization’s solutions are same as those of the full-order quadrupedal dynamics, we simulated an ambling step of the quadrupedal dynamics using the composed bipedal gaits. As shown in Fig. 4.10, we have the joint angles and constraint wrench (ground reaction force) on toe1 𝐹1,𝑧 , and toe2 𝐹2,𝑧 of the quadruped matched with those corresponding external force to the bipedal dynamics. The result of the methods presented is the ability to generate quadrupedal gaits rapidly. We benchmark the performance by considering computing speed for each of the quadrupedal locomotion patterns generated, as is shown in Table 4.1. In summary, with the objective tolerance and equality constraint tolerance configured as 10−8 and 10−5 respectively, we have the average computation time as 3.96 second, and time per iteration averages 0.039 second. For simulation, we first validated the dynamic stability of the gaits produced by the decomposition-based optimization 106 q HR2 b HR2 q HP0 b HP0 q HP2 b HP2 1.4 0.1 0.0 0.0 0.2 time (s) 0.4 position (rad) position (rad) position (rad) 0.2 0.8 0.6 0.4 0.0 0.2 q K0 0.4 b K0 q K2 b K2 170 quad 2,z 170 biped 2,z 160 1.2 1.0 0.0 0.2 time (s) 150 140 130 120 0.4 quad 1,z biped 1,z 160 force (Nm) b HR0 force (Nm) q HR0 0 0.2 time (s) 0.4 150 140 130 120 0 time (s) 0.2 0.4 time (s) Figure 4.10: A comparison between the solution of bipedal walking dynamics obtained from the decomposition-based optimization and a simulated step of the full-order quadrupedal dynamics using the composed bipedal gaits; here MATLAB ODE45 was used. problem using a third party physics engine, MuJoCo. These gaits include a diagonally symmetric ambling and four stepping-in-place behaviors. Then we conducted experiments, walking on a a outdoor tennis court, using the same control law as that in simulation in outdoor environments. We report that for all given optimal gaits, the PD gains are picked as 𝑘 𝑝 = 230, 230, 300, 𝑘 𝑑 = 5 for the hip roll, hip pitch, and knee joints, respectively. The averaged absolute joint torque inputs are logged in Table 4.2, all of which are well within the hardware limitations. See (Wen-Loong Ma, 2020a) for the video of Vision 60 stepping in place in both simulation and experiments. Table 4.1: Computing performance of gait generation. This is performed on a Linux machine with an i7-6820HQ CPU @2.70 GHz and 16 GB RAM. Behaviors frequency (Hz) clearance (cm) # of iterations time of IPOPT (s) time of evaluation (s) NLP time(s) gait1 2.5 11 96 1.60 1.94 3.54 gait2 2.3 12 122 2.10 3.24 5.34 gait3 2.2 15 98 1.62 2.10 3.72 gait4 2.6 13 46 0.81 0.94 1.75 amble 2.83 13 147 2.59 2.86 5.45 Table 4.2: Average torque inputs in experiments and simulations. Experiments 𝑢¯ HR (N·m) 𝑢¯ HP (N·m) 𝑢¯ K (N·m) MuJoCo 𝑢¯ HR (N·m) 𝑢¯ HP (N·m) 𝑢¯ K (N·m) gait1 5.04, 3.65 16.45 gait2 4.83 5.24 16.50 gait3 4.16 5.26 16.86 gait4 5.14 3.77 16.95 amble 7.11 6.28 18.36 7.80 6.78 18.49 9.23 9.14 18.38 10.27 10.71 18.45 8.68 6.64 18.61 8.06 7.27 19.03 107 Figure 4.11: Comparison between MuJoCo simulation (animated) and experiments. The upper two are for stepping in place, gait4; the lower two are logged for a full step of the ambling gait. Walking on Sloped Terrains We now make a few changes to the dynamics and the optimization algorithm to generate walking motion on slopped terrains. Guard condition and discrete dynamics on slopes. For the symmetric ambling motion, the diagonal toes of the quadrupedal robot stay on the ground while the other two toes are swing in the air. This means each bipedal robot has one toe-foot in contact with the ground. Concretely, the Cartesian position of the stance toe remains zero, i.e. ℎ 𝑠 (𝑞𝑖 ) ≡ 0. This contact constraint is enforced by a ground reaction force 𝐹𝑖 ∈ R3 , where 𝑖 ∈ {f, r}. We additionally denote the Cartesian position of the 𝑧 (𝑞 ) ∈ R. The nonstance foot’s nonstance toe along 𝑥, 𝑧 directions as ℎ𝑥𝑛𝑠 (𝑞 f ), ℎ𝑛𝑠 f height on the slope is then given by 𝑧 ℎˆ 𝑛𝑠 (𝑞𝑖 ) = ℎ𝑛𝑠 (𝑞𝑖 ) sin 𝜙 − ℎ𝑥𝑛𝑠 (𝑞𝑖 ) cos 𝜙. Hence we can define the single-support domain for bipedal dynamics (4.65) as: D𝑖 , {(𝑞𝑖 , 𝑞¤𝑖 ) ∈ Q𝑖 ×𝑇 Q𝑖 | ℎ 𝑠 (𝑞𝑖 ) = 0, ℎˆ 𝑛𝑠 (𝑞𝑖 ) ≥ 0}. The guard is then defined on the edge of the domain D𝑖 as S𝑖 , {(𝑞𝑖 , 𝑞¤𝑖 ) ∈ Q𝑖 ×𝑇 Q𝑖 | ℎ 𝑠 (𝑞𝑖 ) = ℎˆ 𝑛𝑠 (𝑞𝑖 ) = 0, ℎ¤̂ 𝑛𝑠 (𝑞𝑖 , 𝑞¤𝑖 ) < 0}, on which we define the discrete dynamics that represents plastic impacts (see equation (20) in (Wen-Loong Ma and Aaron D. Ames, 2020)) at the event that nonstance toe touches down on the slopped terrain. 108 Figure 4.12: (a) The cyclic directed graph for the multi-domain hybrid dynamics on sloped terrain; (b) friction cone (pyramid) condition on some sloped terrain. The friction cone condition on slopes. The feasibility conditions A𝑖 (𝐹𝑖 ) ≥ 0 are enforced to guarantee the solutions to the dynamics is physically realizable. In particular, we have the friction pyramid condition on sloped terrains as:    𝐹 𝑧 cos 𝜙 − 𝐹𝑖𝑥 sin 𝜙    𝑖  A𝑖 (𝐹𝑖 ) , 𝜇(𝐹𝑖𝑧 sin 𝜙 + 𝐹𝑖𝑥 cos 𝜙) − |𝐹𝑖𝑧 cos 𝜙 − 𝐹𝑖𝑥 sin 𝜙|      𝜇(𝐹𝑖𝑧 sin 𝜙 + 𝐹𝑖𝑥 cos 𝜙) − |𝐹𝑖𝑦 |  (4.69) where 𝜇 is the coefficient of dynamic friction of the ground. It is worth noting the feasibility formulation (4.69) is more restrictive for walking on sloped terrains than walking on stairs or level ground. Optimization constraints. In practice, path constraints (inequality constraints) are often used to “fine-tune” the optimal results according to human intuition and physical limitations. Evaluating the optimality of an optimization solution based on experimental performance is rather empirical, and intensive constraint tuning is often needed for field testing. The ultimate target of this paper is to present a method that can be seamlessly used to produce periodic gaits for hardware experiments on sloped terrains. Hence, we explicitly list our path constraints as follows: • Joint angles do not exceed physical limits; • Absolute joint velocities below 4 rad/s; • Absolute acceleration less than 120 rad/s2 ; • Absolute joint torque less than 50 N·m; • Stepping period 𝑡 K ∈ [0.29, 0.37]; 109 Figure 4.13: Phase portraits of the designed gaits (solid lines) vs. experimental data (transparent overlay) for quadrupedal walking on 13◦ (red) and 20◦ ∼ 25◦ (blue) slopes. HR, HP, K are short for hip roll, hip pitch, and knee, respectively. • Nonstance toe’s height ℎˆ 𝑖 (𝑞𝑖𝜅 ) ≥ 0.1 at 𝜅 = K/2; • Nonstance toe’s absolute velocities (parallel to the sloped ground) slower than 1.3 m/s; • Ground impact velocity ℎ¤̂ 𝑛𝑠 (𝑞𝑖 , 𝑞¤𝑖 ) ∈ [−2, −0.2]. Remark that these constraints setup are not modified throughout the optimization for all of the gaits that are experimentally tested. Results. By only changing the slope angles to 𝜙 = 0◦ , 13◦ , 15◦ , 20◦ , 25◦ in the optimization problem (4.68), we were able to generated periodic solutions efficiently. These solutions are further used to obtain quadrupedal gaits for experiments. We hereby report the average computational time for all five gaits are 9.7 seconds and 271 searching iterations on a Ubuntu 16.04 machine with Intel Core i7-6820 HQ CPU @ 2.7 GHz with 16 GB RAM. The phase portrait of the gaits on the 13◦ and 20◦ ∼ 25◦ sloped terrains are shown in solid lines in Fig. 4.13. Before directly enabling these sloped walking gaits optimized by the CCS optimization framework on the actual hardware — a 44 kg, 56 cm-wide robot (Fig. 1.1) — we first validate their physical feasibility and dynamic stability under a feedback control law in a physics engine, RaiSim (Hwangbo, J. Lee, and Marco Hutter, 2018). The quadrupedal robot was first tested on a consistently graded 13◦ grassy slope with minimal surface variation, which was replicated in RaiSim by creating a plane of constant incline. In order to truly test this methodology, the quadruped was next asked to traverse a grassy slope just after it had rained with inclination ranging from 20◦ to 25◦ . In order to emulate 110 Figure 4.14: Full steps of gait tiles for the Vision 60 ambling in outdoor grasslands. The top two is comparison between simulation and experiments for a 13◦ slope. The bottom two are for walking on a ramp with varying slopes of 20◦ ∼ 25◦ . the varying slopes in RaiSim, a terrain was created with a sinusoidally oscillating height varying between 20◦ and 25◦ with a frequency that approximated that of the outdoor environment. As a baseline, both a level-ground gait-based controller and the stock controller were tested on the 20◦ ∼ 25◦ slope. We report that neither were able to navigate on the sloped terrain successfully. As shown in the gait tiles in Fig. 4.14, our proposed method allows the robot to successfully amble across both the 13◦ slope and the 20◦ and 25◦ slope despite the unmodeled variation in slope and lowered friction effects. See Wen-Loong Ma, 2020b for a video demonstration of the experimental validation of the quadruped, in which we also showed all five gaits walking on slopes of 0◦ , 13◦ , 15◦ , 20◦ , 25◦ in RaiSim. This result demonstrates the necessity of designing optimal trajectories and controllers based on the specific terrain types using full-body dynamics. Additionally, we remark that the PD gains and ground friction coefficient are the same across all simulation and experimental implementations. In addition, we logged 20 seconds of experimental data and compared them with the desired ambling gait designed by the optimization, as seen in Fig. 4.13 with phase portraits. Note the difference in the desired behavior for the two terrains. This diversity of behavior further motivates the use of unique gaits conditioned on 111 the terrain to ensure the stability of the robotic system. An important metric to quantify control performance for locomotion is the mechanical cost of transport (MCOT), which was calculated using equation (16) of (W. Ma, Y. Or, and A D. Ames, 2019). Unlike the traditional formulation where the distance term only accounts for the horizontal displacement (Collins et al., 2005a), we used the averaged three-dimensional velocity on the sloped terrain, which also considers the vertical displacement. We report that in simulation, the MCOT for 13◦ and 20◦ ∼ 25◦ sloped ambling are 2.01 and 2.86, accordingly. 4.4 Coupled Control Lyapunov Functions After introducing the coupled control system-based optimization for generating physically feasible gaits for the high-dimensional quadrupedal dynamics, we still need to formally define the control law for driving the local subsystem to the designated stable behaviors. In this section, we will accomplish two targets. First, through the formulation of coupled control Lyapunov functions, we formally define the stability criteria of each subsystem while they are dynamically coupled with the rest of the system due to the shared zero dynamics. We can then utilize these Lyapunov functions to synthesize local optimal control laws for each subsystem to guarantee the overall coupled control system’s stability, hence the full-order dynamics. Second, when applying to rigid-body dynamics, we incorporate quadratic programming formulations with two types of Lyapunov functions for controller design. First, feedback linearizationbased Control Lyapunov Functions (CLFs) are synthesized and demonstrated on a cart-pole example showing stability. Second, PD-inspired Lyapunov functions are used to synthesize model-free CLFs for experimental robustness. These CLFs are applied to stabilizing the continuous dynamics of quadrupedal locomotion. This stabilization is demonstrated in simulation with regard to hopping and running. Finally, we present this framework on hardware, specifically the Vision 60 robot v3.9 (see Fig. 4.15). We empirically show that it is able to walk stably and robustly in outdoor environments. Recall some CLF definitions from Sec. 3.4, Definition 21. A smooth function 𝑉 : R𝑛 → R+ is an exponentially stabilizing control Lyapunov function (ES-CLF) for 𝑥¤ = 𝑓 (𝑥) + 𝑔(𝑥)𝑢, 112 if there exists constants 𝑐 1 , 𝑐 2 , 𝑐 3 > 0 such that 𝑐 1 |𝑥| 2 ≤ 𝑉 (𝑥) ≤ 𝑐 2 |𝑥| 2 (4.70)
inf 𝐿 𝑓 𝑉 (𝑥) + 𝐿 𝑔𝑉 (𝑥)𝑢 + 𝑐 3𝑉 (𝑥) ≤ 0 𝑢 for all 𝑥 ∈ X. If there further exists a constant 𝜀 ∈ (0, 1) such that 𝑐2 𝑐 1 |𝑥| 2 ≤ 𝑉 (𝑥) ≤ 2 |𝑥| 2 𝜀   𝑐3 inf 𝐿 𝑓 𝑉 (𝑥) + 𝐿 𝑔𝑉 (𝑥)𝑢 + 𝑉 (𝑥) ≤ 0 𝑢 𝜀 (4.71) for all 𝑥 ∈ X, then 𝑉 (𝑥) is a rapidly exponentially stabilizing control Lyapunov function (RES-CLF). Definition 22. A smooth function 𝑉 : R𝑛 → R+ is an input-to-state stabilizing control Lyapunov function (ISS-CLF) for 𝑥¤ = 𝑓 (𝑥) + 𝑔(𝑥)(𝑢 + 𝑑), if there exist functions 𝛼, 𝛼, 𝛼, 𝜄 ∈ K∞ as such that ∀ 𝑥, 𝑑 𝛼(|𝑥|) ≤ 𝑉 (𝑥) ≤ 𝛼(|𝑥|) 𝑉¤ (𝑥, 𝑑) ≤ −𝛼(|𝑥|) + 𝜄(k𝑑 k ∞ ), and the exponential estimate is: 𝑉¤ (𝑥, 𝑑) ≤ −𝑐𝑉 (𝑥) + 𝜄(k𝑑 k ∞ ). CLF from the Viewpoint of ISS We first consider an affine control system in the form of C,    𝑥¤1 = 𝑓1 (𝑥 1 , 𝑥2 ) + 𝑔1 (𝑥 1 , 𝑥2 )(𝑢 1 + 𝑑1 )  (4.72)   𝑥¤2 = 𝑓2 (𝑥 1 , 𝑥2 ) + 𝑔2 (𝑥 1 , 𝑥2 )(𝑢 2 + 𝑑2 )  " # " # 𝑓1 (𝑥) 𝑔1 (𝑥) 0 ⇔ 𝑥¤ = + (𝑢 + 𝑑) 𝑓2 (𝑥) 0 𝑓2 (𝑥) | {z } | {z } 𝑓 (𝑥) 𝑔(𝑥) > > 𝑚 > > > 𝑚 where 𝑢 = (𝑢 > 1 , 𝑢 2 ) ∈ U ⊂ R are the inputs, and 𝑑 = (𝑑 1 , 𝑑 2 ) ∈ R are termed as the disturbance inputs with 𝑑𝑖 the subsystem disturbance. Equivalently, we can write the 𝑖 th subsystem C𝑖 as 𝑥¤𝑖 = 𝑓𝑖 (𝑥) + 𝑔𝑖 (𝑥)(𝑢𝑖 + 𝑑𝑖 ), ∀𝑖 ∈ N (4.73) 113 for a set of nodes N = {1, 2}, where 𝑥 , (𝑥 1> , 𝑥2> ) > , 𝑥𝑖 ∈ X𝑖 ⊂ R𝑛𝑖 , 𝑢𝑖 ∈ U𝑖 ⊂ R𝑚𝑖 . Using the same notations from Sec. 4.2, we first present the following definitions that are the foundation of the main theorem in this section. Definition 23. A smooth function 𝑉 : R𝑛𝑖 → R+ is a rapidly exponential input-tostate stabilizing control Lyapunov function (Re-ISS-CLF) for 𝑥¤𝑖 = 𝑓 (𝑥, 𝑧) + 𝑔(𝑥, 𝑧)(𝑢𝑖 + 𝑑 𝑒 ) with 𝑥𝑖 ∈ R𝑛𝑖 , if there exists constants 𝑐 1 , 𝑐 2 , 𝑐 3 > 0, 𝜀 ∈ (0, 1), 𝜀¯ > 0 such that ∀ 𝑥, 𝑧, 𝑑, 𝑐2 𝑐 1 |𝑥𝑖 | 2 ≤ 𝑉𝑖 (𝑥𝑖 ) ≤ 2 |𝑥𝑖 | 2 𝜀   𝑐3 1 2 inf 𝐿 𝑓 𝑉𝑖 (𝑥, 𝑧) + 𝐿 𝑔𝑉𝑖 (𝑥, 𝑧)𝑢𝑖 + 𝑉𝑖 (𝑥𝑖 ) + 𝐿 𝑔𝑉𝑖 ≤ 0. 𝑢 𝑖 ∈U𝑖 𝜀 𝜀¯ (4.74) The construction of Def. 23 is motivated by the rapidly exponentially stabilizing control Lyapunov function (RES-CLF) from (A. Ames, Galloway, et al., 2014b). Based on Def. 23, we can form a class of control laws directly:  𝑐 3,𝑖 1 2 𝑉𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 ≤ 0 , 𝐾𝑖 (𝑥, 𝑧) , 𝑢𝑖 ∈ U𝑖 : 𝐿 𝑓𝑖 𝑉𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 𝑢𝑖 + 𝜀𝑖 𝜀¯𝑖 (4.75) which yields the set of control values which satisfy the desired convergence property for each subsystem 𝑖 ∈ N . The (constant) parameters 𝑐 1,𝑖 , 𝑐 2,𝑖 , 𝑐 3,𝑖 , 𝜀𝑖 , 𝜀¯𝑖 are associated with each subsystem with 𝑖 ∈ N . We then prove that the combination of these stabilizing controllers also yields stability for the full-order system in the following theorem. Theorem 3. Assume there exists a ISS-stabilizing controllers 𝑢𝑖 (𝑥) ∈ 𝐾𝑖 (𝑥) for each subsystem 𝑖 ∈ N given by (4.73). Given a collection of such controllers {𝑢𝑖 (𝑥)}𝑖∈N , the solution to the full-order dynamics in (4.72) is e-ISSable. Proof. We consider an Re-ISS-CLF candidate for the full-order system: 𝑉 (𝑥) = 𝑉1 (𝑥1 ) + 𝑉2 (𝑥 2 ). It can be seen that min (𝑐 1,𝑖 ) |𝑥| 2 = min (𝑐 1,𝑖 )(|𝑥 1 | 2 + |𝑥 2 | 2 ) ≤ 𝑉 (𝑥) 𝑖 𝑖 𝑐 2,𝑖 𝑐 2,𝑖 𝑉 (𝑥) ≤ max ( 2 )(|𝑥 1 | 2 + |𝑥 2 | 2 ) = max ( 2 ) |𝑥| 2 . 𝑖 𝑖 𝜀𝑖 𝜀𝑖 (4.76) 114 Note that we use the notation k𝑑𝑖 k ∞ , sup𝑡≥0 {|𝑑𝑖 |}, which yields some properties that will be used later: |𝑑| ≥ max (|𝑑𝑖 |) ≥ |𝑑𝑖 | 𝑖 k𝑑 k ∞ = sup{|𝑑|} ≥ sup{|𝑑𝑖 |} = k𝑑𝑖 k ∞ . 𝑡≥0 𝑡≥0 Then, 𝑉¤ (𝑥, 𝑢, 𝑑) = 𝑉¤1 ( ) + 𝑉¤2 ( ) Õ = 𝐿 𝑓𝑖 𝑉𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 𝑢𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 𝑑𝑖 Õ 𝑐 3,𝑖 1 ≤ − 𝑉𝑖 − 𝐿 𝑔𝑖 𝑉𝑖 𝐿 𝑔𝑖 𝑉𝑖> + 𝐿 𝑔𝑖 𝑉𝑖 𝑑𝑖 𝜀𝑖 𝜀¯𝑖 Õ 𝑐 3,𝑖 1 ≤ − 𝑉𝑖 − 𝐿 𝑔𝑖 𝑉𝑖 𝐿 𝑔𝑖 𝑉𝑖> + 𝐿 𝑔𝑖 𝑉𝑖 |𝑑𝑖 | 𝜀𝑖 𝜀¯𝑖 Õ 𝑐 3,𝑖 1 2 ≤ − 𝑉𝑖 − 𝐿 𝑔𝑖 𝑉𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 |𝑑| 𝜀𝑖 𝜀¯𝑖  1 Õ 𝑐 3,𝑖 √ k𝑑 k ∞  2 k𝑑 k 2∞ + 𝜀¯𝑖 ≤ − 𝑉𝑖 − √ 𝐿 𝑔𝑖 𝑉𝑖 − 𝜀¯𝑖 𝜀𝑖 2 4 𝜀¯𝑖 2 Õ 𝑐 3,𝑖 k𝑑 k ∞ ≤ − 𝑉𝑖 (𝑥𝑖 ) + 𝜀¯𝑖 𝜀𝑖 4 𝑐 3,𝑖 max𝑖 ( 𝜀¯𝑖 ) k𝑑 k 2∞ , ≤ −2 min ( )𝑉 (𝑥) + 𝑖 𝜀𝑖 2 which satisfies Def. 22, in which case 𝜄(k𝑑 k ∞ ) = max2𝑖 ( 𝜀¯𝑖 ) k𝑑 k 2∞ .  Remark that an effective way to reduce the effect of overall disturbance k𝑑 k ∞ is to decrease all 𝜀¯𝑖 . Exponential stability. Note that the Lyapunov function given by Def. 23 is also a RES-CLF for the system: 𝑥¤𝑖 = 𝑓𝑖 (𝑥) + 𝑔𝑖 (𝑥)𝑢𝑖 ∀𝑖 ∈ N with 𝑓𝑖 , 𝑔𝑖 the same as (4.72). Concretely, there exists constants 𝑐 1 , 𝑐 2 , 𝑐 3 > 0 and 0 < 𝜀 < 1 such that 𝑐 2,𝑖 𝑐 1,𝑖 |𝑥𝑖 | 2 ≤ 𝑉𝑖 (𝑥𝑖 ) ≤ 2 |𝑥𝑖 | 2 𝜀𝑖   𝑐 3,𝑖 inf 𝐿 𝑓𝑖 𝑉𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 𝑢𝑖 + 𝑉𝑖 ≤ 0. 𝑢 𝜀𝑖 Under certain conditions, we can establish exponential stability for system (4.72), which are summarized as follows. 115 Corollary 2. In addition to the conditions given by Theorem 3, if we further have |𝑑 (𝑡)| ≤ 𝑐 4 |𝑥(𝑡)| and 𝐿 𝑔𝑉 (𝑥) ≤ 𝑐 5 |𝑥| ∀ 𝑥. (4.77) The solution to the full-order dynamics in (4.72) is exponentially stable provided 𝑐 that 2 min𝑖 ( 𝜀3,𝑖𝑖 ) > 𝑐 4 𝑐 5 min𝑖 (𝑐 1,𝑖 ). Proof. We first have 𝑐 1 , 𝑐 2 given by the proof of Thm. 3. Then Õ Õ 𝑉¤ = 𝑉¤𝑖 = 𝐿 𝑓𝑖 𝑉𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 (𝑢𝑖 + 𝑑𝑖 ) Õ 𝑐 3,𝑖 ≤ − 𝑉𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 𝑑𝑖 𝜀𝑖 Õ 𝑐 3,𝑖 )𝑉 + 𝐿 𝑔𝑖 𝑉𝑖 · 𝑑𝑖 ≤ −2 min ( 𝑖 𝜀𝑖 𝑐 3,𝑖 )𝑉 + 𝐿 𝑔𝑉 · 𝑑 ≤ −2 min ( 𝑖 𝜀𝑖 𝑐 3,𝑖 )𝑉 + 𝐿 𝑔𝑉 |𝑑| = −2 min ( 𝑖 𝜀𝑖 𝑐 3,𝑖 ≤ −2 min ( )𝑉 + 𝑐 4 𝑐 5 |𝑥| 2 𝑖 𝜀𝑖 𝑐 3,𝑖 ≤ −2 min ( )𝑉 + 𝑐 4 𝑐 5 min (𝑐 1,𝑖 )𝑉 . 𝑖 𝑖 𝜀𝑖 𝑐 Therefore, if 2 min𝑖 ( 3,𝑖 ) > 𝑐 4 𝑐 5 min𝑖 (𝑐 1,𝑖 ), 𝑉¤ is negative definite which implies 𝜀𝑖 exponential stability of the full-order system, according to Def. 21.  Note that for a general situation when the disturbance does not completely vanish on the zero dynamics surface, i.e., |𝑑 (𝑡)| ≤ 𝑐 4 |𝜂| + 𝑐 6 |𝑧| + 𝑐 7 , the system exponentially converges to a ultimate bound for robotic dynamics. Shared zero dynamics. For interconnected systems, we often need to consider the shared dynamics among subsystems. We consider such systems in normal form as:    𝑥¤1 = 𝑓1 (𝑥, 𝑧) + 𝑔1 (𝑥, 𝑧)(𝑢 1 + 𝑑1 )     C 𝑧 , 𝑥¤2 = 𝑓2 (𝑥, 𝑧) + 𝑔2 (𝑥, 𝑧)(𝑢 2 + 𝑑2 ) (4.78)      𝑧¤ = 𝜔(𝑥, 𝑧)  where, the 𝑧-dynamics 𝑧 = 𝜔(𝑥, 𝑧) are regarded as the passive dynamics, and we call 𝑧¤ = 𝜔(0, 𝑧) the zero dynamics. We assume 𝜔(𝑥, 𝑧) is locally Lipschitz. Built on Theorem 3, we have the following theorem to characterize the full-order system’s stability. We now present the main theorem of this section that guarantees the stability of the full-order system by taking values from 𝐾𝑖 (𝑥, 𝑧), ∀ 𝑖 ∈ N . 116 Theorem 4. For a dynamical system given by (4.78), let O𝑧 be an exponentially stable periodic orbit of the zero dynamics 𝑧¤ = 𝜔(0, 𝑧). If there exists an ISS-CLF 𝑉𝑖 (𝑥𝑖 ) for each subsystem 𝑖 ∈ N , then for all locally Lipschitz continuous feedbacks 𝑢𝑖 (𝑥) ∈ 𝐾𝑖 (𝑥) given by (4.75), the full-order periodic orbit O , 𝜄(O𝑧 ) is ultimately bounded, with the bounds tending to zero as |𝑑1 | , |𝑑2 | → 0. Proof. First, we use the converse Lyapunov theorem from (Hauser and Chung Choo Chung, 1994) to construct the following Lyapunov function for the zero dynamics. Given O𝑧 is an exponentially stable periodic orbit of Z, there exists a Lyapunov function 𝑉𝑧 : Z → R+ such that in a neighborhood 𝐵𝛿 (O𝑧 ) of O𝑧 , 𝑟 1 k𝑧k 2O𝑧 ≤ 𝑉𝑧 (𝑧) ≤ 𝑟 2 k𝑧k 2O𝑧 , 𝑉¤𝑧 (𝑧) ≤ −𝑟 3 k𝑧k 2O𝑧 , 𝜕𝑉𝑧 ≤ 𝑟 4 k𝑧k O𝑧 . 𝜕𝑧 Next we have the following Lyapunov function candidate for the full-order system: Õ 𝑉 (𝑥, 𝑧) = 𝑉𝑖 (𝑥𝑖 ) + 𝜎𝑉𝑧 (𝑧). 𝑖 It is clear that 𝑉 (𝑥, 𝑧) satisfies the first inequality in Def. 23 (the definition of Lyapunov functions for an invariant set, such as periodic orbits, can be found in (E. D. Sontag and Yuan Wang, 1996).) We first take the derivative of the subsystems’ Lyapunov functions to get: Õ Õ 𝑉¤𝑖 = 𝐿 𝑓𝑖 𝑉𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 𝑢𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 𝑑𝑖 𝑖 𝑖 Õ 𝑐 3,𝑖 1 2 ≤ − 𝑉𝑖 − 𝐿 𝑔𝑖 𝑉𝑖 + 𝐿 𝑔𝑖 𝑉𝑖 |𝑑𝑖 | 𝜀𝑖 𝜀¯𝑖 𝑖  1 Õ 𝑐 3,𝑖 √ k𝑑 k ∞  2 k𝑑 k 2∞ ≤ − 𝑉𝑖 − √ 𝐿 𝑔𝑖 𝑉𝑖 − 𝜀¯𝑖 + 𝜀¯𝑖 𝜀𝑖 2 4 𝜀¯𝑖 𝑖 ≤ − min ( 𝑖 𝑐 3,𝑖 max𝑖 ( 𝜀¯𝑖 ) k𝑑 k 2∞ . 𝑐 1,𝑖 ) |𝑥| 2 + 𝜀𝑖 2 117 Then the total derivative of the Lyapunov function 𝑉 (𝑥, 𝑧) becomes:
Õ 𝜕𝑉𝑧 𝜕𝑉𝑧 𝑉¤ = 𝜎 𝑤(0, 𝑧) + 𝜎 𝑤(𝑥, 𝑧) − 𝑤(0, 𝑧) + 𝑉¤𝑖 𝜕𝑧 𝜕𝑧 𝑖 Õ 2 ≤ −𝜎𝑟 3 k𝑧k O𝑧 + 𝜎𝑟 4 k𝑧k O𝑧 |𝑤(𝑥, 𝑧) − 𝑤(0, 𝑧)| + 𝑉¤𝑖 𝑖 ≤ −𝜎𝑟 3 k𝑧k 2O𝑧 + 𝜎𝑟 4 k𝑧k O𝑧 𝐿 𝑧 |𝑥| + Õ 𝑉¤𝑖 𝑖 𝑐 3,𝑖 max𝑖 ( 𝜀¯𝑖 ) k𝑑 k 2∞ , 𝑐 1,𝑖 ) |𝑥| 2 + ≤ −𝜎𝑟 3 k𝑧k 2O𝑧 + 𝜎𝑟 4 𝐿 𝑧 k𝑧k O𝑧 |𝑥| − min ( 𝑖 𝜀𝑖 2 | {z } 𝑟6 h i = − k𝑧k O𝑧 |𝑥| 𝐴 " # k𝑧k O𝑧 max𝑖 ( 𝜀¯𝑖 ) k𝑑 k 2∞ , + 2 |𝑥| with 𝐿 𝑧 the Lipschitz constant for 𝜔(𝑥, 𝑧) and " # 𝜎𝑟 3 − 12 𝜎𝑟 4 𝐿 𝑧 𝐴= 1 , − 2 𝜎𝑟 4 𝐿 𝑧 𝑟6 which satisfies Def. 22. We then can pick 𝜎 such that 𝐴 is positive definite, i.e., 𝑉 is a Lyapunov function for the periodic orbit O = 𝜄(O𝑧 ).  The proof is inspired by the construction of (A. Ames, Galloway, et al., 2014b, Appx.B). We note that an effective way to reduce the effect of the disturbance is to decrease 𝜀¯𝑖 . Further, since ISS-CLF is one robust type of CLFs, we will continue to use the terminology CLFs for clarity. Note that for the class of robotic systems of interest (such as the quadruped in Fig. 4.18), it can be shown that any CLF qualifies as an ISS-CLF (see (Angeli, 1999)). In other words, the set given by (4.75) needs not have 𝐿 𝑔𝑖 𝑉𝑖 . Stability condition. As the theorem suggested, we can thus construct the local control Lyapunov functions for a coupled control system. Following the construction of (4.75), we have a class of controllers using a linear constraint of the input: 𝜌𝑖 (𝜂, 𝑧) + 𝜓𝑖 (𝜂, 𝑧)𝑢𝑖 + 1 |𝜓𝑖 (𝜂, 𝑧)| 2 ≤ 0 𝜀¯𝑖 (4.79) where 𝜌𝑖 (𝜂, 𝑧) = 𝐿 𝑓𝑖 𝑉𝑖 (𝜂, 𝑧) + 𝜀𝛾𝑖 𝑉𝑖 (𝜂𝑖 ) and 𝜓𝑖 (𝜂, 𝑧) = 𝐿 𝑔𝑖 𝑉𝑖 . A practical control law that satisfies (4.79) is a minimum-norm in 𝐾𝑖 (𝜂, 𝑧), given by 𝑚𝑖 (𝜂, 𝑧) = argmin{ |𝑢𝑖 | 2 : 𝑢𝑖 ∈ 𝐾𝑖 (𝜂, 𝑧)}, which can be solved by some quadratic programming (QP) algorithm. (4.80) 118 Figure 4.15: The quadrupedal robot, Vision 60 v3.9. 4.5 Coupled Mechanical Systems In this section, we apply the coupled control Lyapunov function to rigid-order dynamics. Given a system that is composed of multiple interconnected rigidbodies, the equations of motion (EOMs) of the full-order dynamics (also referred to as full-order dynamics) can be obtained through Euler–Lagrange equations: 𝐷 (𝑞) 𝑞¥ + 𝐻 (𝑞, 𝑞) ¤ = 𝐵(𝑞)𝑢, (4.81) The notation is consistent with Sec. 2.3. In this section, we are interested in the dynamical systems that can be considered as a collection of two subsystems with index 𝑖 ∈ {1, 2} , N . We first define subsystem configurations as 𝑞𝑖 ∈ Q𝑖 ⊂ R𝑛𝑖 such that ∪𝑖∈N 𝜄𝑖 (Q𝑖 ) = Q with 𝜄𝑖 : Q𝑖 → R𝑛 as a canonical embedding. Since the goal is to control each subsystem individually, the subsystem inputs are defined > 𝑚 𝑖 and as components of the full-system inputs 𝑢 > = (𝑢 > 1 , 𝑢 2 ) with 𝑢𝑖 ∈ U𝑖 ∈ R Í 𝑖∈N 𝑚 𝑖 = 𝑚. We also define a set of edges E , {(1, 2), (2, 1)} representing the subsystems’ connection. For a dynamical system that is composed of two subsystems (coupled via constraints), such as the coupled mechanical systems considered in Fig. 4.16, we have    𝐷 1 (𝑞 1 ) 𝑞¥1 + 𝐻1 (𝑞 1 , 𝑞¤ 1 ) = 𝐵1 (𝑞 1 )𝑢 1 + 𝐽𝑒> (𝑞 1 , 𝑞 2 )𝜆 𝑒     ℛ𝑄 , 𝐷 2 (𝑞 2 ) 𝑞¥2 + 𝐻2 (𝑞 2 , 𝑞¤ 2 ) = 𝐵2 (𝑞 2 )𝑢 2 + 𝐽𝑒>¯ (𝑞 1 , 𝑞 2 )𝜆 𝑒¯      s.t. 𝑐 𝑒,𝑞 (𝑞 1 , 𝑞 2 ) ≡ 0, 𝜆 𝑒 + 𝜆 𝑒¯ = 0  (4.82) 119 where 𝜆 𝑒 , 𝜆 𝑒¯ ∈ Λ𝑖 ∈ R𝑙𝑖 are the coupling forces and 𝑐 𝑒,𝑞 is the coupling constraint. We can solve the connection force explicitly to reach the form in (4.81) using  −1
−1 𝜆 𝑒 = −𝜆 𝑒¯ = 𝐽𝑒 𝐷 −1 𝐽𝑒 𝐷 −1 1 𝐽𝑒 − 𝐽𝑒¯ 𝐷 2 𝐽𝑒¯ 1 (𝐻1 − 𝐵1 𝑢 1 )  ¤ ¤ + 𝐽𝑒¯ 𝐷 −1 (4.83) 2 (𝐻2 − 𝐵2 𝑢 2 ) − 𝐽𝑒 𝑞¤ 1 − 𝐽𝑒¯ 𝑞¤ 2 where 𝐽𝑒 (𝑞 1 , 𝑞 2 ) = 𝜕𝑐 𝑒,𝑞 /𝜕𝑞𝑖 and 𝑒 , (𝑖, 𝑗), 𝑒¯ , ( 𝑗, 𝑖) ∈ E. Figure 4.16: A cart-pole system with two inverted pendula, each is directly actuated by a motor. The mass of the cart and pendula are 2𝑀 and 𝑚, correspondingly. The length of both pendulum is 𝑙. We have the configuration coordinates as (𝜉, 𝜃 1 , 𝜃 2 ) > . Subsystem Dynamics After defining the subsystem with an index set N , we can pick the outputs (the features that we are interested in controlling) of each 𝑖 th subsystem as 𝑦𝑖 (𝑞𝑖 ) = 𝑦𝑖𝑎 (𝑞𝑖 ) − 𝑦𝑖𝑑 (𝑞𝑖 ) (4.84) where 𝑦𝑖𝑑 , 𝑦𝑖𝑎 ∈ R𝑚𝑖 are the desired outputs and the actual outputs, respectively. Since 𝑦𝑖 is a function of the “positional states” 𝑞𝑖 , it has a relative degree two with respect to the control inputs. We then have the 𝑖 th subsystem dynamics in output coordinates as 𝑦¥𝑖 = L𝑖 (𝑞, 𝑞) ¤ + A𝑖 (𝑞)𝑢𝑖 + A 𝑗𝑖 (𝑞)𝑢 𝑗 (4.85) for all 𝑖 ∈ N . Note that A 𝑗𝑖 (𝑞) ∈ R𝑚𝑖 ×𝑚 𝑗 maps 𝑢 𝑗 with 𝑗 ≠ 𝑖 to the configuration space of the 𝑖 th subsystem. Depending on the given EOMs, there are different ways to obtain the expressions efficiently in (4.85). One direct method from (4.81) is given as h L (𝑞, 𝑞) ¤ i 1 L2 (𝑞, 𝑞) ¤ = 𝐽¤𝑦 𝑞¤ 2 − 𝐽𝑦 𝐷 −1 𝐻 , L (𝑞, 𝑞) ¤ h A (𝑞) 1 A21 (𝑞) i A12 (𝑞) A2 (𝑞) = 𝐽𝑦 𝐷 −1 𝐵 , A (𝑞), (4.86) 120 > > where 𝐽𝑦 = 𝜕𝑦/𝜕𝑞 with the full-system outputs are denoted as 𝑦 = (𝑦 > 1 , 𝑦2 ) . For underactuated systems where 𝑚 < 𝑛, zero dynamics will show up in the transition to output coordinates (see (Isidori, 2011)). As discussed in Sec. 3.2, there exists a change of coordinates via a diffeomorphism: 𝑞   1 𝜂 1      𝑞¤ 1    ↦→ 𝜂2  𝑞     2 𝑧     𝑞¤ 2    which yields a set of dynamic equations representing the coupled control system:    𝑦¥ 1 = L1 (𝜂, 𝑧) + A1 (𝜂, 𝑧)𝑢 1 + A21 (𝜂, 𝑧)𝑢 2     CC , 𝑦¥ 2 = L2 (𝜂, 𝑧) + A2 (𝜂, 𝑧)𝑢 2 + A12 (𝜂, 𝑧)𝑢 1      𝑧¤ = 𝜔(𝜂, 𝑧)  (4.87) where 𝜂 = (𝜂1> , 𝜂2> ) > ∈ X are the “controlled states,” and 𝜂𝑖 = (𝑦𝑖> , 𝑦¤ 𝑖> ) > . Note that both L and A now depend on the new coordinates 𝜂, 𝑧. The 𝑧-dynamics, 𝑧¤ = 𝜔(𝜂, 𝑧), are regarded as the internal dynamics with 𝑧 ∈ Z, and we call 𝑧¤ = 𝜔(0, 𝑧) the zero dynamics, i.e., the dynamics on the zero dynamics manifold: Z = {(𝜂, 𝑧) ∈ X × Z : 𝜂𝑖 = 0, ∀ 𝑖 ∈ N }. (4.88) We assume 𝜔(𝜂, 𝑧) is locally Lipschitz in 𝜂. Note that we can also convert the formulation given by W. Ma, Csomay-Shanklin, and A. D. Ames, 2021, Eq.1 into the form of (4.87) again by using (4.83). In this form, not only are the dynamics of each subsystem coupled through the shared zero dynamics coordinates, but the inputs are also coupled, i.e., 𝑢 𝑗 (𝑖 ≠ 𝑗) appears in the 𝑖 th subsystem dynamics. Disturbed Subsystem. To design local controllers for the 𝑖 th subsystem that are independent of the “disturbance” caused by the other subsystems’ inputs, we first introduce the nominal inputs that are built using the zero dynamics. We then introduce the main result of this paper, a theorem that leads to the synthesis of the networked control architecture. It is this controller that later enables us to control each sub-bipedal system individually with stability guarantees. Before designing the control law 𝑢(𝑥) , {𝑢𝑖 (𝑥)}𝑖∈N , we first give the concept of a nominal control input in the following definition. 121 Definition 24. The control input that renders the zero dynamics surface Z , {(𝑞, 𝑞) ¤ : 𝑦𝑖 = 𝑦¤ 𝑖 = 0, ∀ 𝑖 ∈ N } forward-invariant is the nominal input for a coupled control system (4.87), i.e., 0 = L𝑖 (0, 𝑧) + A𝑖 (0, 𝑧)𝑢𝑖Z + A 𝑗𝑖 (0, 𝑧)𝑢 Z𝑗 (4.89) for all 𝑖 ∈ N . We further define 𝑢 Z (𝑧) , {𝑢𝑖Z (𝑧)}𝑖∈N . For the rigid-body dynamics of interest, the decoupling matrix A (0, 𝑧) is assumed to be invertible. Hence, the unique controller that satisfies (4.89) would be as follows: " # 𝑢Z 𝑢 Z (𝑧) = −A −1 (0, 𝑧)L (0, 𝑧) , Z1 . (4.90) 𝑢2 By considering the nominal control input 𝑢 Z𝑗 of the 𝑗 th subsystem ( 𝑗 ≠ 𝑖), we can reformulate the subsystem dynamics (4.85) to remove the dependence on the other interconnected (coupled) subsystem’s control input. Concretely, we have 𝑦¥𝑖 = L𝑖 (𝜂, 𝑧) + A𝑖 (𝜂, 𝑧)𝑢𝑖 + A 𝑗𝑖 (𝜂, 𝑧)(𝑢 𝑗 + 𝑢 Z𝑗 (𝑧) − 𝑢 Z𝑗 (𝑧)) = L𝑖 (𝜂, 𝑧) + A 𝑗𝑖 (𝜂, 𝑧)𝑢 Z𝑗 (𝑧) + A𝑖 (𝜂, 𝑧)𝑢𝑖 + A 𝑗𝑖 (𝜂, 𝑧)(𝑢 𝑗 − 𝑢 Z𝑗 (𝑧)) | {z } (4.91) , 𝑑𝑒 where we denote 𝑑 𝑒 (𝜂, 𝑧, 𝑢 𝑗 ) , A 𝑗𝑖 (𝜂, 𝑧)(𝑢 𝑗 − 𝑢 Z𝑗 (𝑧)), 𝑒 , ( 𝑗, 𝑖) (4.92) as the disturbance induced by the 𝑗 th subsystem’s inputs to the 𝑖 th subsystem. Having established the disturbed subsystem dynamics as in (4.91), the coupled control system in (4.87) becomes a disturbed coupled control system, as:    𝑦¥ 1 = L1 (𝜂, 𝑧) + A21 (𝜂, 𝑧)𝑢 Z2 (𝑧) + A1 (𝜂, 𝑧)𝑢 1 + 𝑑 𝑒     CC𝑑 , 𝑦¥ 2 = L2 (𝜂, 𝑧) + A12 (𝜂, 𝑧)𝑢 Z1 (𝑧) + A2 (𝜂, 𝑧)𝑢 2 + 𝑑 𝑒¯ (4.93)      𝑧¤ = 𝜔(𝜂, 𝑧)  where each subsystem is only subject to local controller and a disturbance term. This is where we can utilize input-to-state stabilizing control Lyapunov functions to reject the disturbance while stabilizing each subsystem. Remark 4. Note that the overall disturbance vanishes on the invariant zero dynamics manifold Z, i.e., 𝑑 (0, 𝑧, 𝑢𝑖Z ) = {𝑑 𝑒 (0, 𝑧, 𝑢𝑖Z )}∀𝑒∈E = 0. This can be seen by plugging 𝑢𝑖 (0, 𝑧) = 𝑢𝑖Z from (4.89) into (4.91), whereby 0 = L𝑖 (0, 𝑧) + A 𝑗𝑖 (0, 𝑧)𝑢 Z𝑗 + A𝑖 (0, 𝑧)𝑢𝑖Z + 𝑑 𝑒 = 0 + 𝑑 𝑒 , which yields 𝑑 𝑒 = 0 and further 𝑑 = 0. 122 After posing the dynamics of regular rigid-body dynamics (4.81) into the coupled mechanical system form in (4.93), we are ready to design local controllers to stabilize each subsystem, further stabilizing the full-order system. To do so, we explain the controller design procedures with two concrete examples: a cart-pole system with two invert pendula and a quadrupedal robot that is decoupled into two bipedal systems. 4.6 Example 1. Feedback Linearization-Based CLFs For the cart-pole system shown in Fig. 4.16. We use Lagrangian method to get: 1 1 𝐿 = 𝑀 𝜉¤2 + 𝑚𝑣 21 + 𝑚𝑣 22 − 𝑚𝑔𝑙 cos 𝜃 1 − 𝑚𝑔𝑙 cos 𝜃 2 2 2 2 2 𝑣 𝑖 = ( 𝜉¤ − 𝑙 cos 𝜃 𝑖 𝜃¤𝑖 ) + (𝑙 sin 𝜃 𝑖 𝜃¤𝑖 ) 2  2𝑀 + 2𝑚 −𝑚𝑙 cos 𝜃 1 −𝑚𝑙 cos 𝜃 2      ⇒ 𝐷 = −𝑚𝑙 cos 𝜃 1 𝑚𝑙 2 0 ,   2 −𝑚𝑙 cos 𝜃 2  0 𝑚𝑙   𝑚𝑙 𝜃¤1 sin 𝜃 1 + 𝑚𝑙 𝜃¤2 sin 𝜃 2  −𝐹 (𝜉, 𝜉) ¤         𝐻= + −𝑚𝑔𝑙 sin 𝜃 1 0   .         −𝑚𝑔𝑙 sin 𝜃 0 2     ¤ can be one the following: The forcing function 𝐹 (𝜉, 𝜉) 2(𝑚 + 𝑀)(−𝜉 + 𝜉¤ (1 − 𝜉 2 − 𝜉¤2 )) (F1) 2(𝑚 + 𝑀)(−𝜉 + 𝜉¤ (1 − 𝜉 2 )) (F2) ¤ 2(𝑚 + 𝑀)(−100𝜉 − 10𝜉) (F3) which will yield (a variation of) the von der Pol oscillation, or an exponentially stable equilibrium point for the zero dynamics. (Powell and Aaron D Ames, 2014) has used a similar setup. We will study the case with a globally stable orbit on zero dynamics, which is given by (F1), denoted by O𝑧 . Separated by the dashed line as Fig. 4.16, we can view each pendulum-cart system as a subsystem with index 𝑖 ∈ N , {1, 2}. Let the subsystem configuration be 𝑞𝑖 = (𝜉, 𝜃 𝑖 ). With a target to control the outputs (a.k.a. the virtual constraint (Jessy W. Grizzle et al., 2014)), we define the subsystem output (virtual constraint) as 𝑦𝑖 (𝑞𝑖 ) = 𝜃 𝑖 for the 𝑖 th subsystem, i.e. the goal is to drive both pendula upright as 𝑡 → ∞. With the output Jacobian obtained by 123 " 𝑦(𝑞) = 𝑦 1 (𝑞 1 ) 𝑦 2 (𝑞 2 ) # " # 0 1 0 𝜕𝑦 ⇒ 𝐽𝑦 , = , 𝜕𝑞 0 0 1 we can use (4.86) to obtain the dynamics in the form of coupled control systems, as in (4.87). Note that the zero dynamics 𝑧¤ = 𝜔(0, 𝑧) — when 𝜃 𝑖 = 𝜃¤𝑖 = 0, ∀𝑖 ∈ N — ¤ become a Van der Pol oscillator due to the force function: 𝜉¥ = 𝐹 (𝜉, 𝜉)/(2𝑚 + 2𝑀). For rigid-body dynamics with invertible decoupling matrices A , we can apply an input-output feedback-linearization:
𝑢𝑖 (𝜂, 𝑧) = A𝑖−1 (𝜂, 𝑧) − L𝑖 (𝜂, 𝑧) − A 𝑗𝑖 (𝜂, 𝑧)𝑢 Z𝑗 (𝑧) + 𝜇𝑖 (4.94) with 𝜇𝑖 the auxiliary input for each subsystem 𝑖 ∈ N . The nominal control input 𝑢 Z𝑗 with 𝑗 ≠ 𝑖 is then given by (4.90). The subsystem output dynamics now become: 𝑦¥𝑖 = 𝜇𝑖 + 𝑑 𝑒 . (4.95) If we define 𝜂𝑖 = (𝑦𝑖> , 𝑦¤ 𝑖> ) > , we can obtain the linearized subsystem dynamics as " # " # 0 𝐼 0 (𝜇𝑖 + 𝑑 𝑒 ), (4.96) 𝜂¤𝑖 = 𝜂𝑖 + 0 0 𝐼 | {z } |{z} 𝐹 𝐺 which is in the form of (4.78). Therefore, we can define the coupled control Lyapunov functions according to Theorem 4. Concretely, for each subsystem 𝑖, we have " # " # 1 1 𝐼 0 𝐼 0 𝑃 𝜀𝑖 . (4.97) 𝑉𝑖 (𝜂𝑖 ) = 𝜂𝑖> 𝑃𝑖 𝜂𝑖 , with 𝑃𝑖 , 𝜀𝑖 0 𝐼 0 𝐼 with 𝜀𝑖 ∈ (0, 1) a constant and 𝑃 ∈ R2×2 the solution to the continuous time algebraic Riccati equation (CARE). More details can be found in A. Ames, Galloway, et al., 2014b, Sec.3. Remark 5. Based on the CLF chosen, an appropriate control can be constructed that yields control robustness. For example, using the feedback linearization of the form (4.94) we can choose the control law as: 𝑢𝑖 = A𝑖−1 − L𝑖 − A 𝑗𝑖 𝑢 Z𝑗 − 1 1 1 >
𝐾 𝑦 − 𝐾 𝑦 ¤ − 𝐿 𝑉 𝑝 𝑖 𝑑 𝑖 𝐺 𝑖 𝜀𝑖 𝜀¯𝑖 𝜀𝑖2 (4.98) where 𝐾 𝑝 , 𝐾 𝑑  0. This controller, inspired by (Eduardo D. Sontag, 1999), is a specific example that belongs to the set 𝐾𝑖 (𝜂, 𝑧). 124 CLF-QP. We now present the QP formulation that calculates control values using the chosen CLFs. Note that 𝜇𝑖 is only an auxiliary input instead of the actual system-level input. We will replace it with 𝑢𝑖 for better numerical conditioning for the optimal control problem. Based on (4.94) we have 𝜇𝑖 as a function of 𝑢𝑖 : 𝜇𝑖 = L𝑖 + A 𝑗𝑖 𝑢 Z𝑗 + A𝑖 𝑢𝑖 . Then the stability condition (4.79) can be re-written as: 𝜌𝑖 + 𝜓𝑖 (A𝑖 𝑢𝑖 + L𝑖 + A 𝑗𝑖 𝑢 Z𝑗 ) + with 𝜌𝑖 = 𝐿 𝐹 𝑉𝑖 + 1 |𝜓𝑖 | 2 ≤ 0, 𝜀¯𝑖 (4.99) 𝛾 𝛾 𝑉𝑖 = 𝜂𝑖> (𝐹 > 𝑃𝑖 + 𝑃𝑖 𝐹)𝜂𝑖 + 𝑉𝑖 , 𝜀𝑖 𝜀𝑖 𝜓𝑖 = 𝐿 𝐺 𝑉𝑖 = 2𝜂𝑖> 𝑃𝑖 𝐺. Finally, we have the following QP formulation that encodes the CLF for subsystem 𝑖 ∈ N: 𝑢𝑖∗ = argmin L𝑖 + A 𝑗𝑖 𝑢 Z𝑗 + A𝑖 𝑢𝑖 2 (4.100) 𝑢 𝑖 ∈U𝑖 s.t. (C1) 𝜌𝑖 + 𝜓𝑖 (A𝑖 𝑢𝑖 + L𝑖 + A 𝑗𝑖 𝑢 Z𝑗 ) + 1 |𝜓𝑖 | 2 ≤ 0 𝜀¯𝑖 (C2) − 𝑢 max  𝑢𝑖  𝑢 max where, (C1) is the stability constraint, and (C2) is added according to the actual torque bounds from the physical actuators to guarantee the realizability. We regarded (4.100) the CLF-QP for coupled mechanical systems. Remark 6. With both subsystems taking values from 𝐾𝑖 (𝜂, 𝑧), we have the disturbance as 𝑑 𝑒 (𝜂, 𝑧) = A 𝑗𝑖 (𝜂, 𝑧)(𝑢 𝑗 (𝜂, 𝑧) − 𝑢 Z𝑗 (𝑧)). Assuming 𝑑 (𝜂, 𝑧) to be locally Lipschitz in 𝜂 yields |𝑑 (𝜂, 𝑧) − 𝑑 (0, 𝑧)| ≤ 𝑐 4 |𝜂 − 0| ⇒ |𝑑| ≤ 𝑐 4 |𝜂| with 𝑐 4 the Lipschitz constant. An effective way to reduce 𝑐 4 is to form an optimization problem inside a tube around the given desired trajectory O, i.e. minmax (𝜂,𝑧)∈tube near O |𝑑 (𝜂, 𝑧)| /|𝜂| . Additionally, we have Õ Õ Õ k𝑃𝑖 𝐺 k 2 . 𝐿 𝐺 𝑉𝑖 = 2 𝜂𝑖> 𝑃𝑖 𝐺 ≤ 2 |𝜂| 125 Í Hence, if we pick 𝑐 5 = 2 k𝑃𝑖 𝐺 k 2 , we can obtain exponential stability for the periodic solution to the full-order system according to Corollary 2. This is also reflected in the simulation data that will be shown next. Simulation. We present two simulation results (see Fig. 4.17 and video (WenLoong Ma, 2021)) to demonstrate this stability result. As shown in Fig. 4.16, we pick the model as 𝑙 = 0.5, 𝑀 = 15, 𝑚 = 5. Given an initial condition 𝑥(0) = (0, 0.1, −0.1, 0.1, 0, 0) > , we first simulate the specific control law given by (4.98) with 𝐾 𝑝 = 5, 𝐾 𝑑 = 0.1, 𝜀𝑖 = 0.5, 𝜀¯𝑖 = 0.5. Then we simulate the decentralized optimal controller given by (4.100) with 𝜀𝑖 = 0.5, 𝜀¯𝑖 = 0.5 The data is shown in Fig. 4.17. As Corollary 2 suggests, both simulations show exponentially stability, and the disturbance vanishes on the zero dynamics surface. Figure 4.17: The cart-pole simulation. data labeled FL (red) used the feedback linearization controller (4.98); data labeled CLF-QP (blue) used the controller (4.100); dark and light variants of the colors are used to distinguish between the first and second pendula. Both simulations show stability. 4.7 Example 2. Quadrupedal Walking with Model-Free CLFs We can also use local CLFs to stabilize the overall system for more complicated robots, such as quadrupedal locomotion. In this section, we will apply the local control laws to an 18-DOF quadrupedal robot (see Fig. 4.15) by viewing it as two connected bipedal robots (see Fig. 4.18). The advantage is that we simultaneously consider each subsystem’s stability through the local CLFs and the feasibility conditions such as the motor torque saturation. Note that for the gaits of interest, walking, running, and hopping, we have a diagonal double-support phase, and a flight phase. In the flight phase, where none of the toes touch the ground, the holonomic constraints are not required. The detailed model of multi-domain behaviors for quadrupedal robots is discussed in Sec. 4.1, and we omitted domain index for the ease of notations in this section. Note that in the hybrid system setting, it was previously shown that RES-CLFs provably stabilize 126 Figure 4.18: (a) The decomposition of a quadrupedal robot into two bipedal systems with toe indices labeled and (b) the configuration of the quadruped; each leg has a point contact toe and three rotational joints with motors. the continuous dynamics in such a way that the hybrid dynamics are also stabilized under the assumption of HZD (A. Ames, Galloway, et al., 2014b, Thm. 2). This result has been extended to the ISS-CLFs in (S. Kolathaya, J. Reher, et al., 2018). Recall the quadruped dynamics ℛ𝑄 that were posed as coupled bipedal dynamics in (4.48)—(4.51). We then write the coordinates for these two subsystems as > , 𝜃 > ) > ∈ Q , where the subscript 𝑞𝑖 = (𝜉 > , 𝜃𝑖,st 𝑖 st marks the joints of the leg that 𝑖,nst are in contact with the ground, and nst for the swing legs. We then define outputs for each subsystem, the bipeds, as 𝑦𝑖 (𝑡, 𝑞𝑖 ) = 𝑦𝑖𝑎 (𝑡) − 𝑦𝑖𝑑 (𝑞𝑖 ) 𝑖∈N (4.101) where the desired outputs (trajectory) 𝑦𝑖𝑑 (𝑡) ∈ R6 are given by a set of Bézier polynomials generated by the CCS optimization in Sec. 4.3. The actual outputs are picked as   𝜃 st,hr    𝜃 st,k   𝜃 st, hp − 2      𝜃   st,k 𝑎 𝑦𝑖 (𝑞𝑖 ) = Y𝑖 𝑞𝑖 =  ,   𝜃 nst,hr   𝜃 nst,k  𝜃  nst, hp − 2      𝜃 nst,k   (4.102) where the subscript ℎ𝑟, ℎ𝑝, 𝑘 are short for the hip-roll, hip-pitch, and knee joints. This output structure represents the roll angle, pitch angle, and leg length of the virtual leg, which is the virtual linkage connecting the hip and the toe. Note that if the quadrupedal robot has nonidentical legs for the front and rear subsystems, we will have a different output structure, i.e., Yf ≠ Yr . This will be an interesting future 127 direction for understanding how to cooperate asymmetric quadrupeds. Next, given the full-system output Jacobian 𝐽𝑦 = 𝜕𝑦/𝜕𝑞, we can use (4.86) to obtain the CCS dynamics as in (4.85). On the zero dynamics surface where both subsystems’ output coordinates remain zero, we have the configuration coordinates and their velocity terms satisfying: (𝑞 Z , 𝑞¤ Z ) = {(𝑞, 𝑞) ¤ | 𝑦𝑖 (𝑞) = 𝑦¤ 𝑖 (𝑞, 𝑞) ¤ = 0, ∀ 𝑖 ∈ N }. The nominal inputs that satisfy the invariant condition (4.89) can therefore be obtained by " # 𝑢Z 𝑢 Z (𝑞 Z , 𝑞¤ Z , 𝑡) = −A (𝑞 Z ) −1 L (𝑞 Z , 𝑞¤ Z ) , Z1 . (4.103) 𝑢2 We then can have the disturbed subsystem dynamics as given in (4.93), after which we can control each bipedal system using the coupled control Lyapunov functions. Model-free CLF-QP. For the quadruped chosen in this study, any CLF qualifies as an ISS-CLF (Angeli, 1999). Hence, we can choose a specific form of the CLF, which is motivated by the Proportional-Derivative control law-inspired Lyapunov function S. Kolathaya and Veer, 2019, eq(24), and use it for the underactuated bipedal systems. The advantage with this class of CLFs is that the corresponding stability constraint can be expressed in a model-free fashion. Therefore, an improved experimental robustness against model uncertainty can be obtained. Formally, we have the following model-free stability constraint: (𝛼𝑖 (𝑦𝑖 )𝑦𝑖> + 𝑦¤ 𝑖> )(𝐾 𝑝 𝑦𝑖 + 𝐾 𝑑 𝑦¤ 𝑖 ) + (𝛼𝑖 (𝑦𝑖 )𝑦𝑖> + 𝑦¤ 𝑖> )𝐽𝑦−>A 𝑢𝑖 ≤ 0 (4.104) 𝑖 𝑘0 with a constant 𝑘 0 > 0. Concretely, in for subsystem 𝑖 ∈ N , where, 𝛼𝑖 (𝑦𝑖 ) = 1+|𝑦 𝑖| > > −1 comparison with (4.75), (𝛼𝑖 (𝑦𝑖 )𝑦𝑖 + 𝑦¤ 𝑖 )𝐽𝑦𝑖 is in the place of 𝐿 𝑔𝑖 𝑉𝑖 terms, and the remaining terms are in place of 𝐿 𝑓𝑖 𝑉𝑖 terms. 𝐾 𝑝 , 𝐾 𝑑  0 are the diagonal matrices that form the PD gains. The Jacobian matrix of the actual output with respect to the actuated joints for the 𝑖 th subsystem is given by 𝐽𝑦𝑖 A = 𝜕𝑦𝑖 /𝜕𝑞𝑖A , where 𝑞𝑖A are the actuated joints of the 𝑖 th bipedal system. We then have the QP formulation utilizing the model-free CLFs as: argmin 2 𝑢𝑖 − 𝑢𝑖ref + 1000𝛿2 (4.105) 𝑢 𝑖 ∈R6 ,𝛿∈R s.t. 𝛼𝑖 (𝑦𝑖 )𝑦𝑖> + 𝑦¤ 𝑖>
h i (𝐾 𝑝 𝑦𝑖 + 𝐾 𝑑 𝑦¤ 𝑖 ) + 𝐽𝑦−>A 𝑢𝑖 ≤ 𝛿 − 𝑢 max  𝑢𝑖  𝑢 max 𝑖 128 for the 𝑖 th subsystem, where 𝛿 ≥ 0 is a relaxation for better numerical stability given a high penalty weight of 1000. To formulate a model-free QP problem, we also modify the nominal inputs 𝑢𝑖Z to a output-feedback PD control law, 𝑢𝑖ref = −𝐽𝑦> A (𝐾 𝑝 𝑦𝑖 + 𝐾 𝑑 𝑦¤ 𝑖 ). 𝑖 Simulation. Before enabling the proposed method on hardware, we first validated the CCLF-QP in simulation using the physics engine RaiSim (Hwangbo, J. Lee, and Marco Hutter, 2018). In particular, we wish to control this quadrupedal robot as two connected bipedal robots performing quadrupedal behaviors such as walking, hopping, and running. These behaviors can be generated as single-domain or multidomain periodic solutions to the coupled control system using the optimization method introduced in Sec. 4.2. The specific controller we put to the test to achieve stable tracking of the desinated periodic gaits is given in (4.105). The PD gains 𝐾 𝑝 , 𝐾 𝑑  0 are diagonal matrices and are picked as the same value across all three simulation tests. As a result, the local controller utilizing CCLFs renders stabilization of the given periodic gaits for walking, hopping, and running on Vision 60 in RaiSim. An animation is provided in (Wen-Loong Ma, 2021). We show the gait tiles in Fig. 4.19, and the phase portraits of the simulation data in Fig. 4.20. Figure 4.19: Top: Snapshots showing a two full steps of the walking gait on an outdoor lawn. Bottom: Running gait in the RaiSim simulation environment. Hardware. The robot we studied in this paper is the Vision 60 v3.9 quadrupedal robot from Ghost Robotics. As show in Fig. 4.15, this robot is 44 kg, 54 cm wide and 50∼60 cm tall. It uses a hierarchical computation structure to perform various tasks. In our experiments, we implement the optimal controller with a QP solver OSQP on the onboard Jetson AGX Xavier computer from NVIDIA. Furthermore, a 1kHz hard real-time operating system enforces the communication between the mainboard and motor drivers to realize the torque commands from the control algorithm (4.105). Experiments and Data Analysis. As a first step towards controlling complex systems such as quadrupedal robots to achieve various dynamical behaviors using the 129 Figure 4.20: Left three: Experimental (dashed transparent) and simulated (solid transparent) phase portraits for walking plotted against the desired values (solid). Right three: Simulated (transparent) vs. desired (solid) phase portraits for walking (red), hopping (green), and running (blue) behaviors. local control laws, we conducted some walking experiments on the Vision 60 robot. To avoid robustness challenges caused by model uncertainties, especially unpredictable uncertainties introduced by the terrain dynamics, we applied the model-free QPs in (4.105). As the supplementary video (Wen-Loong Ma, 2021) shows, we are able to achieve robust walking with the Vision 60 on rough outdoor terrains with moderate slope variation and surface roots. We show the gait tiles of the walking experiments in Fig. 4.19. We also provide a comparison between experimental data, simulation data, and the desired trajectory in Fig. 4.20. We note that the tracking is ultimately bounded by a tube around the desired trajectory in the continuous domains, which provides empirical evidence for future works formally establishing hybrid stability. 130 Chapter 5 CONCLUSION AND FUTURE WORK A complete procedure of modeling, optimizing, controller design, and experiments of complex legged robots was shown in this work. The methodology was generalized around the concept of Lyapunov stability. To conclude, we will summarize the main contributions and discuss some interesting future directions. 5.1 Summary Trajectory optimization for complex locomotion. Trajectory optimization was originally widely used in computational fluid dynamics and often was used with inverse kinematics and searching algorithms when it was first introduced to robotics. From a dynamics perspective, off-line trajectory optimization is used to find solutions to dynamical systems. In this dissertation, we first formally define the solution to a hybrid system, which represents the dynamics of the legged systems ranging from bipedal to quadrupedal systems. We then formulated the solution-finding problem into open-loop and closed-loop trajectory optimization problems. In various applications, these methods have been scaled from simple motions to multiple dynamic locomotions on over ten robotic systems, including monopedal, bipedal, and quadrupedal systems, and we have shown high-performance behaviors such as bipedal running, rough terrain locomotion, and high-energy efficiency walking. The applicability has been successfully validated. Robust high-performance locomotion. In a perfect world, the off-line planning with a predefined feedback controller should drive the system to the desired behavior since we possess the principal information of the dynamics. But this is not true. Even with a slight difference in the measurement or the modeling, a carefully crafted controller could fail. While there are many types of uncertainty, and each has a different worst-case scenario to destroy stability, we used experimental research to find the most critical uncertainty, phase-time uncertainty, to bipedal running. Building on the top of the hybrid zero dynamics framework, we used the concept of input-to-state stability to describe the phase uncertainty to state stability for bipedal locomotion. This new characterization of stability has provided some slack space for parameter tuning while guaranteeing dynamic stability formally. Eventually, the synthesized 131 control laws yield bipedal running on a human-size robot, and it is repeatable and robust. Coupled Control System. Because of bipedal systems’ highly dynamic nature and instability from underactuation, understanding and controlling the full-order dynamics appears necessary. When we carry this philosophy from bipedal systems to quadrupedal robots, the computational cost has increased due to the more complicated contact scenario and hybrid system structure. We considered a quadruped as two connected bipedal systems, and design motion and controller using those methods applied to the bipedal system to address this problem. This intuitive approach was then formally characterized as the framework of the coupled control system. In this methodology, we can design trajectory and controller for each subsystem while also formally guaranteeing the original system’s stability and preserving its solutions. This new way of controlling robotic systems drove two quadrupedal robots to walk on multiple outdoor terrains. 5.2 Future Work The development of this thesis originates from the methodology: 𝐴1 offline prediction/planning/optimization + 𝐴2 online/real-time control/tracking, where 𝐴1 searches for a plan that is globally or locally optimal, and 𝐴2 executes the tracking problem in implementation and rejects unplanned problems. For example, we put all of the details (motor, compliance) of the running robot, DURUS, into our model, which has made the dynamical system high-dimensional, stiff, and hard to solve. But because of this, we can confidently derive a plan from it that can represent “most” of the reality. The HZD framework and CCS methods both are to reduce the stability analysis for this complex problem so that it is feasible for field implementation. Then we can use the control Lyapunov functions based QP to reject small perturbation with the smallest necessary control effort. An important future direction is to merge these two components. This can be from a model predictive control (MPC) perspective. We can use more advanced computational tools to accelerate trajectory optimization in both algorithmic software and hardware. The optimal control component can “look further” into the future with a more precise model. We can also rigorously synthesize a computationally cheap model for a well-defined task (the SLIP model for lightweight legs on flat ground is a good example). Besides, many theorems in nonlinear control assume a given Lyapunov function. While this is a restrictive assumption, many unexplored Lyapunov 132 candidates in existing robotic literature and computational Lyapunov functions are also compelling. Understanding these methods of control Lyapunov functions for locomotion will be an important future direction. Additionally, the coupled control system framework looks at the distributed/decentralized/parallel/networked computation/control problem from a dynamics perspective. New directions include expanding these methods to more physically connected agents collaboration and incorporating more biped-related control methods in subsystem design. Lastly, collaboratively carrying a payload that is more than each subsystem’s capacity is becoming necessary for disaster response and extreme environment autonomy. Further development of CCS can bridge this gap between high-performance single robot maneuver and multi-agent collaboration. 133 BIBLIOGRAPHY Aëšström, Karl J. and P.R. Kumar (2014). “Control: A perspective”. In: Automatica 50.1, pp. 3–43. issn: 0005-1098. doi: https : / / doi . org / 10 . 1016 / j . automatica . 2013 . 10 . 012. url: http : / / www . sciencedirect . com / science/article/pii/S0005109813005037. Ahmadi, Mojtaba and Martin Buehler (2006). “Controlled Passive Dynamic Running Experiments With the ARL-Monopod II”. In: IEEE Trans. on Robotics 22.5, pp. 974–986. Akbari Hamed, K., B.G. Buss, and J.W. Grizzle (2016). “Exponentially stabilizing continuous-time controllers for periodic orbits of hybrid systems: Application to bipedal locomotion with ground height variations”. In: The International Journal of Robotics Research 35.8, pp. 977–999. Akbari Hamed, Kaveh., Wen-Loong. Ma, and Aaron. D. Ames (July 2019). “Dynamically Stable 3D Quadrupedal Walking with Multi-Domain Hybrid System Models and Virtual Constraint Controllers”. In: 2019 American Control Conference (ACC), pp. 4588–4595. doi: 10.23919/ACC.2019.8815085. Ambrose, E., W. Ma, and A. D. Ames (2021). “Towards the Unification of System Design and Motion Synthesis for High-Performance Hopping Robots”. In: International Conference on Robotics and Automation (ICRA). Ambrose, E., W. Ma, C. Hubicki, et al. (2017). “Toward benchmarking locomotion economy across design configurations on the modular robot: AMBER-3M”. In: 2017 IEEE Conference on Control Technology and Applications (CCTA), pp. 1270–1276. doi: 10.1109/CCTA.2017.8062633. Ames, A. D. (May 2014). “Human-Inspired Control of Bipedal Walking Robots”. In: IEEE Transactions on Automatic Control 59.5, pp. 1115–1130. issn: 0018-9286. doi: 10.1109/TAC.2014.2299342. – (May 2014). “Human-Inspired Control of Bipedal Walking Robots”. In: Automatic Control, IEEE Transactions on 59.5, pp. 1115–1130. issn: 0018-9286. doi: 10.1109/TAC.2014.2299342. Ames, A.D., K. Galloway, et al. (Apr. 2014a). “Rapidly Exponentially Stabilizing Control Lyapunov Functions and Hybrid Zero Dynamics”. In: Automatic Control, IEEE Transactions on. issn: 0018-9286. doi: 10.1109/TAC.2014.2299335. – (2014b). “Rapidly Exponentially Stabilizing Control Lyapunov Functions and Hybrid Zero Dynamics”. In: Automatic Control, IEEE Transactions on 59.4, pp. 876–891. issn: 0018-9286. doi: 10.1109/TAC.2014.2299335. 134 Ames, Aaron D. et al. (2017). “First steps toward formal controller synthesis for bipedal robots with experimental implementation”. In: Nonlinear Analysis: Hybrid Systems 25, pp. 155–173. issn: 1751-570X. doi: https://doi.org/10. 1016 / j . nahs . 2017 . 01 . 002. url: http : / / www . sciencedirect . com / science/article/pii/S1751570X1730002X. Angeli, David (1999). “Input-to-state stability of PD-controlled robotic systems”. In: Automatica 35.7, pp. 1285–1290. issn: 0005-1098. doi: http://dx.doi. org/10.1016/S0005-1098(99)00037-0. Antonelli, Gianluca (2013). “Interconnected dynamic systems: An overview on distributed control”. In: IEEE Control Systems Magazine 33.1, pp. 76–88. Arnold, V.I. (1989). Mathematical Methods of Classical Mechanics. 2nd ed. 60. Springer-Verlag New York. Ascher, Uri M. and Linda R. Petzold (1998). Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations. 1st. USA: Society for Industrial and Applied Mathematics. isbn: 0898714125. Barton, Paul I. and Cha Kun Lee (Oct. 2002). “Modeling, Simulation, Sensitivity Analysis, and Optimization of Hybrid Systems”. In: ACM Trans. Model. Comput. Simul. 12.4, pp. 256–289. issn: 1049-3301. doi: 10.1145/643120.643122. url: http://doi.acm.org/10.1145/643120.643122. Becerra, V. M. (2010). “Solving complex optimal control problems at no cost with PSOPT”. In: 2010 IEEE International Symposium on Computer-Aided Control System Design, pp. 1391–1396. doi: 10.1109/CACSD.2010.5612676. Betts, John T. (1998). “Survey of Numerical Methods for Trajectory Optimization”. In: Journal of Guidance, Control, and Dynamics 21.2, pp. 193–207. doi: 10. 2514/2.4231. eprint: https://doi.org/10.2514/2.4231. url: https: //doi.org/10.2514/2.4231. Bhounsule, Pranav A. et al. (2014). “Low-bandwidth reflex-based control for lower power walking: 65 km on a single battery charge”. In: The International Journal of Robotics Research 33.10, pp. 1305–1321. doi: 10.1177/0278364914527485. url: http://ijr.sagepub.com/content/33/10/1305.short. Big Dog Walking (2010). Boston Dynamics. url: https://youtu.be/cNZPRsrwumQ. Bledt, G. et al. (Oct. 2018). “MIT Cheetah 3: Design and Control of a Robust, Dynamic Quadruped Robot”. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2245–2252. doi: 10.1109/IROS. 2018.8593885. Blickhan, R. (n.d.). “The spring–mass model for running and hopping”. In: Journal of Biomechanics 22.11 (), pp. 1217–1227. Blickhan, Reinhard and RJ Full (1993). “Similarity in multilegged locomotion: bouncing like a monopode”. In: Journal of Comparative Physiology A 173.5, pp. 509–517. 135 Boussema, C. et al. (Apr. 2019a). “Online Gait Transitions and Disturbance Recovery for Legged Robots via the Feasible Impulse Set”. In: IEEE Robotics and Automation Letters 4.2, pp. 1611–1618. issn: 2377-3766. doi: 10.1109/LRA. 2019.2896723. – (Apr. 2019b). “Online Gait Transitions and Disturbance Recovery for Legged Robots via the Feasible Impulse Set”. In: IEEE Robotics and Automation Letters 4.2, pp. 1611–1618. issn: 2377-3766. doi: 10.1109/LRA.2019.2896723. Box, George E. P. (1976). “Science and Statistics”. In: Journal of the American Statistical Association 71.356, pp. 791–799. doi: 10.1080/01621459.1976. 10480949. eprint: https://www.tandfonline.com/doi/pdf/10.1080/ 01621459.1976.10480949. url: https://www.tandfonline.com/doi/ abs/10.1080/01621459.1976.10480949. Champneys, Alan R and Péter L Várkonyi (2016). “The Painlevé paradox in contact mechanics”. In: IMA Journal of Applied Mathematics 81.3, pp. 538–588. Chang, A. H. et al. (2017). “Learning to jump in granular media: Unifying optimal control synthesis with Gaussian process-based regression”. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2154–2160. doi: 10.1109/ICRA.2017.7989248. Chang, Tai-Heng and Yildirim Hurmuzlu (1993). “Sliding control without reaching phase and its application to bipedal locomotion”. In: Journal of Dynamic Systems, Measurement, and Control 115.3, pp. 447–455. Chevallereau, C., J.W. Grizzle, and C.-L. Shih (Feb. 2009). “Asymptotically Stable Walking of a Five-Link Underactuated 3-D Bipedal Robot”. In: Robotics, IEEE Transactions on 25.1, pp. 37–50. issn: 1552-3098. doi: 10.1109/TRO.2008. 2010366. Chung, Soon-Jo and Jean-Jacques E Slotine (2009). “Cooperative robot control and concurrent synchronization of Lagrangian systems”. In: IEEE transactions on Robotics 25.3, pp. 686–700. Clark, Andrew J and Timothy E Higham (2011). “Slipping, sliding and stability: locomotor strategies for overcoming low-friction surfaces”. In: Journal of Experimental Biology 214.8, pp. 1369–1378. Collins, S. et al. (2005a). “Efficient Bipedal Robots Based on Passive-Dynamic Walkers”. In: Science 307.5712, pp. 1082–1085. doi: 10 . 1126 / science . 1107799. eprint: http : / / www . sciencemag . org / content / 307 / 5712 / 1082.full.pdf. – (2005b). “Efficient bipedal robots based on passive-dynamic walkers”. In: Science 307.5712, pp. 1082–1085. 136 Cousineau, E. and A. D. Ames (May 2015). “Realizing underactuated bipedal walking with torque controllers via the ideal model resolved motion method”. In: Robotics and Automation (ICRA), IEEE International Conference on, pp. 5747– 5753. doi: 10.1109/ICRA.2015.7140004. Da, X. et al. (2016). “From 2D Design of Underactuated Bipedal Gaits to 3D Implementation: Walking With Speed Tracking”. In: IEEE Access 4, pp. 3469– 3478. issn: 2169-3536. doi: 10.1109/ACCESS.2016.2582731. Dai, H. and R. Tedrake (Dec. 2012). “Optimizing robust limit cycles for legged locomotion on unknown terrain”. In: Decision and Control, IEEE 51st Annual Conference on, pp. 1207–1213. doi: 10.1109/CDC.2012.6425971. Dai, Hongkai, Andres Valenzuela, and Russ Tedrake (2014). “Whole-body Motion Planning with Simple Dynamics and Full Kinematics”. In: 2014 14th IEEE-RAS International Conference on Humanoid Robots (Humanoids). isbn: 9781479971732. doi: 10.1109/HUMANOIDS.2014.7041375. Dalibard, Sébastien et al. (2013). “Dynamic walking and whole-body motion planning for humanoid robots: an integrated approach”. In: The International Journal of Robotics Research 32.9-10, pp. 1089–1103. doi: 10.1177/0278364913481250. eprint: https : / / doi . org / 10 . 1177 / 0278364913481250. url: https : //doi.org/10.1177/0278364913481250. Danner, Simon M et al. (Nov. 2017). “Computational modeling of spinal circuits controlling limb coordination and gaits in quadrupeds”. In: eLife 6. Ed. by Ronald L Calabrese, e31050. issn: 2050-084X. doi: 10 . 7554 / eLife . 31050. url: https://doi.org/10.7554/eLife.31050. De, Avik and Daniel E. Koditschek (2018). “Vertical hopper compositions for preflexive and feedback-stabilized quadrupedal bounding, pacing, pronking, and trotting”. In: The International Journal of Robotics Research 37.7, pp. 743–778. doi: 10.1177/0278364918779874. eprint: https://doi.org/10.1177/ 0278364918779874. url: https://doi.org/10.1177/0278364918779874. Di Carlo, J. et al. (Oct. 2018). “Dynamic Locomotion in the MIT Cheetah 3 Through Convex Model-Predictive Control”. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1–9. doi: 10.1109/IROS.2018. 8594448. Dörfler, Florian and Francesco Bullo (2014). “Synchronization in complex networks of phase oscillators: A survey”. In: Automatica 50.6, pp. 1539–1564. issn: 0005-1098. doi: https://doi.org/10.1016/j.automatica.2014.04. 012. url: http : / / www . sciencedirect . com / science / article / pii / S0005109814001423. Featherstone, R. (2008). Rigid Body Dynamics Algorithms. Kluwer international series in engineering and computer science: Robotics. Springer. isbn: 9780387743158. url: http://books.google.com/books?id=UjWbvqWaf6gC. 137 Featherstone, Roy (2010). “Exploiting Sparsity in Operational-space Dynamics”. In: The International Journal of Robotics Research 29.10, pp. 1353–1368. doi: 10 . 1177 / 0278364909357644. eprint: https : / / doi . org / 10 . 1177 / 0278364909357644. url: https://doi.org/10.1177/0278364909357644. Fujisaka, Hirokazu and Tomoji Yamada (1983). “Stability theory of synchronized motion in coupled-oscillator systems”. In: Progress of theoretical physics 69.1, pp. 32–47. Gamus, Benny and Yizhar Or (2013). “Analysis of dynamic bipedal robot walking with stick-slip transitions.” In: ICRA, pp. 3348–3355. – (2015). “Dynamic bipedal walking under stick-slip transitions”. In: SIAM Journal on Applied Dynamical Systems 14.2, pp. 609–642. Ganesh, Sumitra, Aaron D Ames, and Ruzena Bajcsy (2007). “Composition of dynamical systems for estimation of human body dynamics”. In: International Workshop on Hybrid Systems: Computation and Control. Springer, pp. 702–705. Geijtenbeek, Thomas, Michiel van de Panne, and a. Frank van der Stappen (Nov. 2013). “Flexible muscle-based locomotion for bipedal creatures”. In: ACM Transactions on Graphics 32.6, pp. 1–11. issn: 07300301. doi: 10.1145/2508363. 2508399. Gill, Philip E., Walter Murray, and Michael A. Saunders (2005). “SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization”. In: SIAM Review 47.1, pp. 99–131. doi: 10.1137/S0036144504446096. eprint: https://doi. org/10.1137/S0036144504446096. url: https://doi.org/10.1137/ S0036144504446096. Gregg, R.D. and L. Righetti (Oct. 2013). “Controlled Reduction With Unactuated Cyclic Variables: Application to 3D Bipedal Walking With Passive Yaw Rotation”. In: Automatic Control, IEEE Transactions on 58.10, pp. 2679–2685. issn: 00189286. doi: 10.1109/TAC.2013.2256011. Griewank, Andreas, David Juedes, and Jean Utke (June 1996). “Algorithm 755: ADOL-C: A Package for the Automatic Differentiation of Algorithms Written in C/C++”. In: ACM Trans. Math. Softw. 22.2, pp. 131–167. issn: 0098-3500. doi: 10 . 1145 / 229473 . 229474. url: https : / / doi . org / 10 . 1145 / 229473 . 229474. Grizzle, J W, G Abba, and F Plestan (Jan. 2001). “Asymptotically Stable Walking for Biped Robots: Analysis via Systems with Impulse Effects”. In: IEEE Trans. on Automatic Control 46.1, pp. 51–64. – (Jan. 2001). “Asymptotically stable walking for biped robots: Analysis via systems with impulse effects”. In: Automatic Control, IEEE Transactions on 46.1, pp. 51– 64. issn: 0018-9286. doi: 10.1109/9.898695. 138 Grizzle, Jessy W. et al. (2014). “Models, feedback control, and open problems of 3D bipedal robotic walking”. In: Automatica 50.8, pp. 1955–1988. issn: 00051098. doi: https : / / doi . org / 10 . 1016 / j . automatica . 2014 . 04 . 021. url: http : / / www . sciencedirect . com / science / article / pii / S0005109814001654. Gruhn, Matthias, Lyuba Zehl, and Ansgar Büschges (2009). “Straight walking and turning on a slippery surface”. In: Journal of Experimental Biology 212.2, pp. 194–209. Hager, William W. et al. (Sept. 2019). “Convergence Rate for a Radau Hp Collocation Method Applied to Constrained Optimal Control”. In: Comput. Optim. Appl. 74.1, pp. 275–314. issn: 0926-6003. doi: 10.1007/s10589- 019- 00100- 1. url: https://doi.org/10.1007/s10589-019-00100-1. Hamed, Kaveh Akbari and Jessy W. Grizzle (June 2013). “Robust event-based stabilization of periodic orbits for hybrid systems: Application to an underactuated 3D bipedal robot”. In: American Control Conference (ACC), 2013, pp. 6206– 6212. doi: 10.1109/ACC.2013.6580811. Hamed, Kaveh Akbari, Vinay R. Kamidi, et al. (2020). “Distributed Feedback Controllers for Stable Cooperative Locomotion of Quadrupedal Robots: A Virtual Constraint Approach”. In: American Control Conference. arXiv: 1902.03690 [math.OC]. Hatanaka, Takeshi et al. (2015). Passivity-Based Control and Estimation in Networked Robotics. Springer International Publishing. Hauser, John and Chung Choo Chung (1994). “Converse Lyapunov functions for exponentially stable periodic orbits”. In: Systems & Control Letters 23.1, pp. 27– 34. issn: 0167-6911. doi: https : / / doi . org / 10 . 1016 / 0167 - 6911(94 ) 90078-7. Hereid, A. et al. (2018). “Dynamic Humanoid Locomotion: A Scalable Formulation for HZD Gait Optimization”. In: IEEE Transactions on Robotics, pp. 1–18. issn: 1552-3098. doi: 10.1109/TRO.2017.2783371. Hereid, Ayonga and Aaron D. Ames (Sept. 2017). “FROST: Fast Robot Optimization and Simulation Toolkit”. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE/RSJ. Vancouver, BC, Canada. Hereid, Ayonga, Eric. Cousineau, et al. (2016). “3D Dynamic Walking with Underactuated Humanoid Robots: A Direct Collocation Framework for Optimizing Hybrid Zero Dynamics”. In: IEEE International Conference on Robotics and Automation. Hereid, Ayonga, Christian M Hubicki, et al. (2018). “Dynamic Humanoid Locomotion: A Scalable Formulation for HZD Gait Optimization”. In: IEEE Transactions on Robotics. 139 Hereid, Ayonga, Shishir Kolathaya, et al. (2014). “Dynamic Multi-domain Bipedal Walking with Atrias Through SLIP Based Human-inspired Control”. In: Proceedings of the 17th International Conference on Hybrid Systems: Computation and Control. HSCC ’14. Berlin, Germany: ACM, pp. 263–272. isbn: 978-1-45032732-9. doi: 10.1145/2562059.2562143. url: http://doi.acm.org/10. 1145/2562059.2562143. Hof, A.L., M.G.J. Gazendam, and W.E. Sinke (2005). “The condition for dynamic stability”. In: Journal of Biomechanics 38.1, pp. 1–8. issn: 0021-9290. doi: https://doi.org/10.1016/j.jbiomech.2004.03.025. url: http:// www.sciencedirect.com/science/article/pii/S0021929004001642. Hutter, M. et al. (Oct. 2016). “ANYmal - a highly mobile and dynamic quadrupedal robot”. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). doi: 10.1109/IROS.2016.7758092. Hwangbo, Jemin, Joonho Lee, and Marco Hutter (2018). “Per-contact iteration method for solving contact dynamics”. In: IEEE Robotics and Automation Letters 3.2, pp. 895–902. Isidori, A. (1995). Nonlinear Control Systems. Springer; 3rd edition. – (2011). “The zero dynamics of a nonlinear system: From The Origin To the latest progresses of a long successful story”. In: Proceedings of the 30th Chinese Control Conference, pp. 18–25. Jiang, Zhong-Ping, Iven Mareels, and Yuan Wang (1995). “A Lyapunov Formulation of Nonlinear Small Gain Theorem for Interconnected Systems”. In: IFAC Proceedings Volumes 28.14. 3rd IFAC Symposium on Nonlinear Control Systems Design 1995, Tahoe City, CA, USA, 25-28 June 1995, pp. 625–630. issn: 1474-6670. doi: https://doi.org/10.1016/S1474-6670(17)46898-4. url: http:// www.sciencedirect.com/science/article/pii/S1474667017468984. Johansen, Tor A. and Thor I. Fossen (2013). “Control allocation—A survey”. In: Automatica 49.5, pp. 1087–1103. issn: 0005-1098. doi: https://doi.org/ 10.1016/j.automatica.2013.01.035. Johnson, A. M., S. A. Burden, and D. E. Koditschek (2016). “A hybrid systems model for simple manipulation and self-manipulation systems”. In: The International Journal of Robotics Research 35.11, pp. 1354–1392. doi: 10.1177/ 0278364916639380. eprint: https://doi.org/10.1177/0278364916639380. url: https://doi.org/10.1177/0278364916639380. Kajita, S., F. Kanehiro, et al. (2001). “The 3D linear inverted pendulum mode: a simple modeling for a biped walking pattern generation”. In: Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180). Vol. 1, 239–246 vol.1. 140 Kajita, S., K. Tani, and A. Kobayashi (July 1990). “Dynamic walk control of a biped robot along the potential energy conserving orbit”. In: IEEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications, 789–794 vol.2. doi: 10.1109/IROS.1990.262497. Kajita, Shuuji et al. (2004). “Biped walking on a low friction floor”. In: Intelligent Robots and Systems, 2004.(IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on. Vol. 4. IEEE, pp. 3546–3552. Kaneko, Kenji et al. (2005). “Slip observer for walking on a low friction floor”. In: Intelligent Robots and Systems, 2005.(IROS 2005). 2005 IEEE/RSJ International Conference on. IEEE, pp. 634–640. Kelly, M. (2017). “An Introduction to Trajectory Optimization: How to Do Your Own Direct Collocation”. In: SIAM Review 59.4, pp. 849–904. doi: 10.1137/ 16M1062569. eprint: https://doi.org/10.1137/16M1062569. url: https: //doi.org/10.1137/16M1062569. Khalil, Hassan K (2002). Nonlinear systems. Vol. 3. Prentice hall Upper Saddle River. Kiehl, M. (1994). “Parallel multiple shooting for the solution of initial value problems”. In: Parallel Computing 20.3, pp. 275–295. issn: 0167-8191. doi: https: / / doi . org / 10 . 1016 / S0167 - 8191(06 ) 80013 - X. url: https : / / www . sciencedirect.com/science/article/pii/S016781910680013X. Kolathaya, S., A. Hereid, and A. D. Ames (July 2016). “Time dependent control Lyapunov functions and hybrid zero dynamics for stable robotic locomotion”. In: 2016 American Control Conference (ACC), pp. 3916–3921. doi: 10.1109/ACC. 2016.7525524. Kolathaya, S., J. Reher, et al. (June 2018). “Input to State Stabilizing Control Lyapunov Functions for Robust Bipedal Robotic Locomotion”. In: 2018 Annual American Control Conference (ACC), pp. 2224–2230. doi: 10 . 23919 / ACC . 2018.8430946. Kolathaya, S. and S. Veer (2019). “PD based Robust Quadratic Programs for Robotic Systems”. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6834–6841. Kolathaya, Shishir and D Aaron Ames (2016). “Parameter to State Stability of Control Lyapunov Functions for Hybrid System Models of Robots”. In: Nonlinear Analysis Hybrid Systems. Elsevier. Kolathaya, Shishir, Ayonga Hereid, and Aaron D Ames (2016). “Time Dependent Control Lyapunov Functions and Hybrid Zero Dynamics for Stable Robotic Locomotion”. In: American Control Conference (ACC). IEEE. Kolathaya, Shishir, Wen-Loong Ma, and Aaron D Ames (2015). “Composing Dynamical Systems to Realize Dynamic Robotic Dancing”. In: Algorithmic Foundations of Robotics XI. Springer, Cham, pp. 425–442. 141 Koolen, T. et al. (2012). “Capturability-based analysis and control of legged locomotion, Part 1: Theory and application to three simple gait models”. In: The International Journal of Robotics Research. doi: 10.1177/0278364912452673. eprint: http : / / ijr . sagepub . com / content / early / 2012 / 06 / 29 / 0278364912452673.full.pdf+html. url: http://ijr.sagepub.com/ content/early/2012/06/29/0278364912452673.abstract. Lanczos, C. (1986). The Variational Principles of Mechanics. Dover Books On Physics. Dover Publications. isbn: 9780486650678. Li, Chen, Tingnan Zhang, and Daniel I. Goldman (2013). “A Terradynamics of Legged Locomotion on Granular Media”. In: Science 339.6126, pp. 1408–1412. issn: 0036-8075. doi: 10.1126/science.1229163. eprint: https://science. sciencemag . org / content / 339 / 6126 / 1408 . full . pdf. url: https : //science.sciencemag.org/content/339/6126/1408. Ma, W., N. Csomay-Shanklin, and A. D. Ames (2021). “Coupled Control Systems: Periodic Orbit Generation With Application to Quadrupedal Locomotion”. In: IEEE Control Systems Letters 5.3, pp. 935–940. doi: 10.1109/LCSYS.2020. 3006963. Ma, W., K. A. Hamed, and A. D. Ames (2019). “First Steps Towards Full Model Based Motion Planning and Control of Quadrupeds: A Hybrid Zero Dynamics Approach”. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5498–5503. doi: 10.1109/IROS40897.2019.8968189. Ma, W., Y. Or, and A D. Ames (2019). “Dynamic Walking on Slippery Surfaces : Demonstrating Stable Bipedal Gaits with Planned Ground Slippage*”. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 3705–3711. doi: 10.1109/ICRA.2019.8793761. Ma, W. -L., N. Csomay-Shanklin, and A. D. Ames (2020). “Quadrupedal Robotic Walking on Sloped Terrains via Exact Decomposition into Coupled Bipedal Robots”. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4006–4011. doi: 10.1109/IROS45743.2020.9341181. Ma, W. -L., N. Csomay-Shanklin, S. Kolathaya, et al. (2021). “Coupled Control Lyapunov Functions for Interconnected Systems, With Application to Quadrupedal Locomotion”. In: IEEE Robotics and Automation Letters 6.2, pp. 3761–3768. doi: 10.1109/LRA.2021.3065174. Ma, Wen-Loong (2016a). Bipedal robotic running of DURUS-2D. Caltech. url: https://youtu.be/6k9_tf4ctk4. – (2016b). DURUS running. Caltech. url: https://youtu.be/7uQOY3reavo. – (2016c). Failed time-based running. Caltech. url: https : / / youtu . be / RrqeVDjB7ec. – (2019a). Bipedal walking on slippery surfaces. Caltech. url: https://youtu. be/G9dhcgyvcyI. 142 Ma, Wen-Loong (2019b). Quadrupedal Robot Ambling Experiment. Caltech. url: https://youtu.be/MQCPOKCop8Q. – (2020a). Experimental video of Vision 60 using the composed stepping gaits. Caltech. url: https://youtu.be/dn2kl03VXlY. – (2020b). Experimental video of Vision 60 walking on slopes. Caltech. url: https://youtu.be/uJedboyzDjc. – (2020c). Generating gaits using CCS-OPT. Caltech. url: https://youtu.be/ GlpgSXMinoU. – (2021). CCLF-QP controlled studies. Caltech. url: https : / / youtu . be / D0LWc7VFxlM. Ma, Wen-Loong and Aaron D. Ames (2020). “From Bipedal Walking to Quadrupedal Locomotion: Full-Body Dynamics Decomposition for Rapid Gait Generation”. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 4491–4497. doi: 10.1109/ICRA40945.2020.9196841. Ma, Wen-Loong, Ayonga Hereid, et al. (2016). “Efficient HZD gait generation for three-dimensional underactuated humanoid running”. In: Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, pp. 5819– 5825. Ma, Wen-Loong, Shishir Kolathaya, et al. (2017). “Bipedal Robotic Running with DURUS-2D: Bridging the Gap Between Theory and Experiment”. In: Proceedings of the 20th International Conference on Hybrid Systems: Computation and Control. HSCC ’17. Pittsburgh, Pennsylvania, USA: ACM, pp. 265–274. isbn: 978-1-4503-4590-3. doi: 10.1145/3049797.3049823. url: http://doi. acm.org/10.1145/3049797.3049823. Manchester, I. R. et al. (Jan. 2011). “Stable dynamic walking over uneven terrain”. In: The International Journal of Robotics Research 30.3, pp. 265–279. issn: 02783649. doi: 10.1177/0278364910395339. url: http://ijr.sagepub.com/ cgi/doi/10.1177/0278364910395339. Martin, A. E., D. C. Post, and J. P. Schmiedeler (2014a). “Design and experimental implementation of a hybrid zero dynamics-based controller for planar bipeds with curved feet”. In: The International Journal of Robotics Research 33.7, pp. 988– 1005. issn: 0278-3649. doi: 10.1177/0278364914522141. url: http://ijr. sagepub.com/cgi/doi/10.1177/0278364914522141. – (2014b). “The effects of foot geometric properties on the gait of planar bipeds walking under HZD-based control”. In: The International Journal of Robotics Research 33.12, pp. 1530–1543. doi: 10.1177/0278364914532391. Martin, William C, Albert Wu, and Hartmut Geyer (2015). “Robust Spring Mass Model Running for a Physical Bipedal Robot”. In: International Conference on Robotics and Automation, pp. 6307–6312. 143 Mellinger, Daniel et al. (2013). “Cooperative Grasping and Transport Using Multiple Quadrotors”. In: Distributed Autonomous Robotic Systems: The 10th International Symposium. Ed. by Alcherio Martinoli et al. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 545–558. isbn: 978-3-642-32723-0. doi: 10 . 1007 / 978 - 3 642-32723-0_39. Mesbahi, Mehran and Magnus Egerstedt (2010). Graph theoretic methods in multiagent networks. Vol. 33. Princeton University Press. Mombaur, Katja (2009). “Using optimization to create self-stable human-like running”. In: Robotica 27.03, pp. 321–330. Morris, B. and J. W. Grizzle (Dec. 2005). “A Restricted Poincaré Map for Determining Exponentially Stable Periodic Orbits in Systems with Impulse Effects: Application to Bipedal Robots”. In: Decision and Control, 2005 and 2005 European Control Conference. CDC-ECC ’05. 44th IEEE Conference on, pp. 4199– 4206. doi: 10.1109/CDC.2005.1582821. Moyer, BE et al. (2006). “Gait parameters as predictors of slip severity in younger and older adults”. In: Ergonomics 49.4, pp. 329–343. Murray, Richard M et al. (1994). A mathematical introduction to robotic manipulation. CRC press. Muybridge, Eadweard (n.d.). Animals in Motion. London:Chapman and Hall, LD. url: https://ecommons.cornell.edu/handle/1813/58718. Or, Yizhar (2014). “Painlevé’s paradox and dynamic jamming in simple models of passive dynamic walking”. In: Regular and Chaotic Dynamics 19.1, pp. 64–80. Or, Yizhar and Moti Moravia (2016). “Analysis of foot slippage effects on an actuated spring-mass model of dynamic legged locomotion”. In: International Journal of Advanced Robotic Systems 13.2, p. 69. Orin, David E., Ambarish Goswami, and Sung-Hee Lee (2013). Centroidal dynamics of a humanoid robot. Vol. 35. Springer, pp. 161–176. Park, H.-W., A. Ramezani, and J.W. Grizzle (Apr. 2013). “A Finite-State Machine for Accommodating Unexpected Large Ground-Height Variations in Bipedal Robot Walking”. In: Robotics, IEEE Transactions on 29.2, pp. 331–345. issn: 15523098. doi: 10.1109/TRO.2012.2230992. Park, HW et al. (May 2012). “Switching control design for accommodating large step-down disturbances in bipedal robot walking”. In: IEEE/RSJ International Conference on Robotics and Automation (ICRA). Ieee, pp. 45–50. isbn: 978-14673-1405-3. doi: 10.1109/ICRA.2012.6225056. url: http://ieeexplore. ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6225056%20http: //ieeexplore.ieee.org/xpls/abs%5C_all.jsp?arnumber=6225056. 144 Poulakakis, I. and J.W. Grizzle (Aug. 2009). “The Spring Loaded Inverted Pendulum as the Hybrid Zero Dynamics of an Asymmetric Hopper”. In: Automatic Control, IEEE Transactions on 54.8, pp. 1779–1793. issn: 0018-9286. doi: 10.1109/ TAC.2009.2024565. Poulakakis, Ioannis and Jessy W Grizzle (2009). “The spring loaded inverted pendulum as the hybrid zero dynamics of an asymmetric hopper”. In: Automatic Control, IEEE Transactions on 54.8, pp. 1779–1793. Poulakakis, Ioannis and JW Grizzle (2007). “Formal embedding of the spring loaded inverted pendulum in an asymmetric hopper”. In: Proc. of the European Control Conference, pp. 1–8. Powell, Matthew J and Aaron D Ames (2014). “Hierarchical control of series elastic actuators through control Lyapunov functions”. In: Decision and Control (CDC), 2014 IEEE 53rd Annual Conference on. IEEE, pp. 2986–2992. Raibert, M., K. Blankespoor, et al. (2008). “BigDog, the Rough-Terrain Quadruped Robot”. In: IFAC Proceedings Volumes 41.2. 17th IFAC World Congress, pp. 10822– 10825. issn: 1474-6670. doi: https://doi.org/10.3182/20080706-5-KR1001.01833. url: http://www.sciencedirect.com/science/article/ pii/S1474667016407020. Raibert, M. and E. R. Tello (Nov. 1986). “Legged Robots That Balance”. In: IEEE Expert 1.4, pp. 89–89. issn: 0885-9000. doi: 10.1109/MEX.1986.4307016. Raibert, M. H. et al. (n.d.). “Dynamically Stable Legged Locomotion (September 1985-Septembers1989)”. In: (), pp. 49–77. Raibert, Marc H., H. Benjamin Brown, and Michael Chepponis (1984). “Experiments in Balance with a 3D One-Legged Hopping Machine”. In: The International Journal of Robotics Research 3.2, pp. 75–92. doi: 10.1177/027836498400300207. eprint: http://ijr.sagepub.com/content/3/2/75.full.pdf+html. url: http://ijr.sagepub.com/content/3/2/75.abstract. Rao, A. V. (2009). “A survey of numerical methods for optimal control”. In: Advances in the Astronautical Sciences. Reher, J. et al. (May 2016). “Realizing dynamic and efficient bipedal locomotion on the humanoid robot DURUS”. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 1794–1801. doi: 10.1109/ICRA.2016. 7487325. Reher, J. P. et al. (2016). “Algorithmic foundations of realizing multi-contact locomotion on the humanoid robot DURUS”. In: Twelfth International Workshop on Algorithmic Foundations on Robotics. Reher, Jacob et al. (2016). “Realizing Dynamic and Efficient Bipedal Locomotion on the Humanoid Robot DURUS”. In: IEEE International Conference on Robotics and Automation (ICRA). 145 Reher, Jake, Wen-Loong Ma, and Aaron D. Ames (2019). “Dynamic Walking with Compliance on a Cassie Bipedal Robot”. In: 2019 18th European Control Conference (ECC), pp. 2589–2595. doi: 10.23919/ECC.2019.8796090. Ren, Wei, Randal W Beard, and Ella M Atkins (2005). “A survey of consensus problems in multi-agent coordination”. In: Proceedings of the 2005, American Control Conference, 2005. IEEE, pp. 1859–1864. Rezazadeh, Siavash et al. (2015). “Spring-mass Walking with ATRIAS in 3D: Robust Gait Control Spanning Zero to 4.3 KPH on a Heavily Underactuated Bipedal Robot”. In: Proceedings of the ASME 2015 Dynamic Systems and Control Conference. Sakagami, Yoshiaki et al. (2002). “The intelligent ASIMO: System overview and integration”. In: Intelligent Robots and Systems, 2002. IEEE/RSJ International Conference on. Vol. 3. IEEE, pp. 2478–2483. Sastry, Shankar (1999). Nonlinear systems: analysis, stability, and control. Vol. 10. Springer New York. Sloth, C. (2016). “On the computation of Lyapunov functions for interconnected systems”. In: 2016 IEEE Conference on Computer Aided Control System Design (CACSD), pp. 850–855. doi: 10.1109/CACSD.2016.7602545. Sontag, E. D. and Yuan Wang (1996). “New characterizations of input-to-state stability”. In: IEEE Transactions on Automatic Control 41.9, pp. 1283–1294. Sontag, Eduardo D (2008). “Input to state stability: Basic concepts and results”. In: Nonlinear and optimal control theory. Springer, pp. 163–220. – (1999). “Control-Lyapunov functions”. In: Open Problems in Mathematical Systems and Control Theory. Ed. by Vincent Blondel et al. London: Springer London, pp. 211–216. isbn: 978-1-4471-0807-8. doi: 10.1007/978- 1- 4471- 08078_40. url: https://doi.org/10.1007/978-1-4471-0807-8_40. Spence, A. et al. (2007). “Effects of substrate properties on equine locomotion”. In: Comparative Biochemistry and Physiology a-Molecular & Integrative Physiology 146.4. Suppl. S, S109–S109. Spong, Mark (1998). “Underactuated mechanical systems”. In: Control Problems in Robotics and Automation. Ed. by Bruno Siciliano and Kimon Valavanis. Vol. 230. Lecture Notes in Control and Information Sciences. Springer Berlin / Heidelberg, pp. 135–150. url: http://dx.doi.org/10.1007/BFb0015081. Spong, Mark W. (1989). Robot Dynamics and Control. 1st. USA: John Wiley & amp; Sons, Inc. isbn: 047161243X. Spotmini Walking (2016). Boston Dynamics. url: https://youtu.be/tf7IEVTDjng. Sreenath, K. (Aug. 2011). “A Compliant Hybrid Zero Dynamics Controller for Stable, Efficient and Fast Bipedal Walking on MABEL”. In: The International Journal of Robotics Research 30.9, pp. 1170–1193. doi: 10.1177/0278364910379882. 146 Sreenath, K., H.-W. Park, I. Poulakakis, and J. Grizzle (Mar. 2013). “Embedding active force control within the compliant hybrid zero dynamics to achieve stable, fast running on MABEL”. In: The International Journal of Robotics Research 32.3, pp. 324–345. issn: 0278-3649. doi: 10.1177/0278364912473344. url: http://ijr.sagepub.com/cgi/doi/10.1177/0278364912473344. Sreenath, K., H.-W. Park, I. Poulakakis, and J. W. Grizzle (Aug. 2011). “Compliant Hybrid Zero Dynamics Controller for achieving Stable, Efficient and Fast Bipedal Walking on MABEL”. In: The International Journal of Robotics Research 30.9, pp. 1170–1193. – (2013). “Embedding active force control within the compliant hybrid zero dynamics to achieve stable, fast running on MABEL”. In: The International Journal of Robotics Research 32.3, pp. 324–345. doi: 10.1177/0278364912473344. Srinivasan, Manoj and Andy Ruina (2006). “Computer optimization of a minimal biped model discovers walking and running”. In: Nature 439.7072, pp. 72–75. Strandberg, Lennart and Håkan Lanshammar (1981). “The dynamics of slipping accidents”. In: Journal of occupational accidents 3.3, pp. 153–162. Tabuada, P. et al. (2017). “Data-driven control for feedback linearizable singleinput systems”. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp. 6265–6270. doi: 10.1109/CDC.2017.8264603. Tamada, T. et al. (Nov. 2014). “High-speed bipedal robot running using high-speed visual feedback”. In: 2014 IEEE-RAS International Conference on Humanoid Robots, pp. 140–145. doi: 10.1109/HUMANOIDS.2014.7041350. Tan, Jie et al. (2018). “Sim-to-Real: Learning Agile Locomotion For Quadruped Robots”. In: CoRR abs/1804.10332. arXiv: 1804.10332. url: http://arxiv. org/abs/1804.10332. Tedrake, R. et al. (Nov. 2015). “A closed-form solution for real-time ZMP gait generation and feedback stabilization”. In: 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pp. 936–940. doi: 10.1109/ HUMANOIDS.2015.7363473. Tinetti, Mary E and Christianna S Williams (1997). “Falls, injuries due to falls, and the risk of admission to a nursing home”. In: New England journal of medicine 337.18, pp. 1279–1284. Tsai, Lung-Wen (1999). Robot Analysis and Design: The Mechanics of Serial and Parallel Manipulators. 1st. USA: John Wiley & Sons, Inc. isbn: 0471325937. Vázquez, JA and M Velasco-Villa (2013). “Experimental estimation of slipping in the supporting point of a biped robot”. In: Journal of applied research and technology 11.3, pp. 348–359. Vejdani, H. R. et al. (May 2015). “Touch-down angle control for spring-mass walking”. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 5101–5106. doi: 10.1109/ICRA.2015.7139909. 147 Villarreal, Dario J and Robert D Gregg (2014). “A survey of phase variable candidates of human locomotion”. In: Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE. IEEE, pp. 4017–4021. Vukobratovic, Miomir and Branislav Borovac (2004). “Zero-moment point? thirty five years of its life”. In: International Journal of Humanoid Robotics 1.1, pp. 157– 173. Vulobratocić, M. and B. Borovac (2004). “ZERO-MOMENT POINT — THIRTY FIVE YEARS OF ITS LIFE”. In: International Journal of Humanoid Robotics 01.01, pp. 157–173. doi: 10.1142/S0219843604000083. eprint: http://www. worldscientific . com / doi / pdf / 10 . 1142 / S0219843604000083. url: http://www.worldscientific.com/doi/abs/10.1142/S0219843604000083. Wächter, A. and L. T. Biegler (2006). “On the Implementation of a Primal-Dual Interior Point Filter Line Search Algorithm for Large-Scale Nonlinear Programming”. In: Mathematical Programming 106.1, pp. 25–57. Wensing, Patrick M and David Orin (2014). “3D-SLIP steering for high-speed humanoid turns”. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, pp. 4008–4013. Westervelt, E. R., J. W. Grizzle, C Chevallereau, et al. (June 2007). Feedback Control of Dynamic Bipedal Robot Locomotion. Control and Automation. Boca Raton: CRC Press. – (2007). Feedback control of dynamic bipedal robot locomotion. CRC press Boca Raton. – (2007). Feedback Control of Dynamic Bipedal Robot Locomotion. Taylor & Francis/CRC. Xi, Weitao, Yevgeniy Yesilevskiy, and C David Remy (2015). “Selecting gaits for economical locomotion of legged robots”. In: The International Journal of Robotics Research, pp. 0278–3649. doi: 10.1177/0278364915612572. url: http://ijr.sagepub.com/cgi/doi/10.1177/0278364915612572. Zhao, Hui-Hua et al. (2014). “Human-inspired multi-contact locomotion with AMBER2”. In: Cyber-Physical Systems (ICCPS), 2014 ACM/IEEE International Conference on. Best Paper Award Finalist of ICCPS 2014. IEEE, pp. 199–210. doi: 10.1109/ICCPS.2014.6843723. Zhao, Huihua et al. (Feb. 2016). “Multi-contact bipedal robotic locomotion”. In: Robotica FirstView, pp. 1–35. issn: 1469-8668. doi: 10.1017/S0263574715000995. url: http://journals.cambridge.org/article_S0263574715000995. 148