Robust Safety-Critical Control: A Lyapunov and Barrier Approach Citation Taylor, Andrew James (2023) Robust Safety-Critical Control: A Lyapunov and Barrier Approach. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/bpht-by81. https://resolver.caltech.edu/CaltechTHESIS:06022023-032907616 Abstract Accompanying the technological advances of the past decade has been the promise for widespread growth of autonomous systems into nearly all domains of human society, including manufacturing, transportation, and healthcare. At the same time, there have been several tragic failures that reveal potential risks with the expansion of autonomous systems into everyday life, and indicate that it is vital for safety to be accounted for in the design of control systems. This thesis seeks to develop a theory of robust safety-critical control for autonomous systems. This theory will be built upon the foundational tools of Control Lyapunov Functions (CLFs) and Control Barrier Functions (CBFs), which provide a powerful paradigm for the design of model-based safety-critical controllers. The dependence of CLF and CBF-based controllers on a system model makes them susceptible to modeling inaccuracies, potentially resulting in unsafe behavior when deploying these controllers on real-world systems. In this thesis I present methods for resolving four classes of model inaccuracies referred to as model error, disturbances, measurement error, and input sampling, which are commonly faced challenges when designing controllers for robotic systems. The proposed methods are unified by their shared use of CLFs and CBFs to produce controllers possessing rigorous and robust safety guarantees that can be demonstrated in simulation or experimentally. A hallmark of these methods is a focus on enabling control synthesis through convex optimization, which ensures that controllers can be efficiently computed on real-world robotic hardware platforms. In addressing model error, I consider both data-driven learning approaches and adaptive control approaches. I present three episodic learning frameworks that iteratively augment existing CLF and CBF-based controllers specified via convex optimization problems to improve the stability and safety properties of a system, which I demonstrate in simulation and experimentally. I also establish a relationship between the degradation of stability and safety properties with the magnitude of residual learning error through the perspective of Input-to-State Stability (ISS) and Input-to-State Safety (ISSf). Lastly, I develop an adaptive safety-critical control framework for systems with parametric model error through the notion of adaptive CBFs. In addressing disturbances, I resolve challenges in balancing performance and robustness with ISSf-based controllers through the notion of Tunable Input-to-State Safety (TISSf), which permits prioritizing robustness to disturbances only when safety requirements are close to being violated. I demonstrate the capabilities of TISSf-based control design experimentally on an autonomous semi-trailer truck system that is subject to input disturbances due to complex unmodeled actuator dynamics. Lastly, I develop a framework for achieving ISSf-like finite-time safety guarantees for discrete-time systems subject to stochastic disturbances through the use of CBFs and convex optimization. In addressing measurement error, I develop the notion of Measurement-Robust CBFs (MR-CBFs), which permit control synthesis through convex optimization in the presence of imperfect measurements. I demonstrate the capability of MR-CBFs on an experimental Segway system using a vision-based measurement system, validating the tractability of using controllers specified through increasingly complex classes of convex optimization problems on real-world systems. Lastly, I present an application of Preference Based Learning (PBL) in tuning the robustness parameters of a CBF-based controller, demonstrating the first use of PBL with CBFs and providing a tool for tuning the safety and performance of the robust controllers proposed in this thesis. In addressing input sampling, I consider both sampled-data and event-triggered paradigms for modeling input sampling. I provide a method for synthesizing CLF-based controllers for sampled-data systems by integrating feedback linearization with approximate discrete-time models, leading to a significant improvement over continuous-time CLF-based controllers implemented with input sampling. I then develop a framework for achieving safety of sampled-data systems through approximate discrete-time models through the notion of practical safety and Sampled-Data CBFs (SD-CBFs), which I demonstrate with convex-optimization based controllers in simulation. Lastly, I develop a method for event-triggered safety-critical control that uses ISSf to achieve safety while satisfying the requirement of a minimum interevent time. Collectively, these contributions constitute a significant advance in the theory of robust safety-critical control by establishing a framework, unified by the use of CLFs and CBFs in conjunction with convex optimization, that addresses a wide class of challenges faced in the design of safety-critical control systems. Item Type: Thesis (Dissertation (Ph.D.)) Subject Keywords: control; robotics; machine learning; optimization; robust; safety-critical; Lyapunov; barrier Degree Grantor: California Institute of Technology Division: Engineering and Applied Science Major Option: Control and Dynamical Systems Awards: Thomas A. Tisch Prize for Graduate Teaching in Computing and Mathematical Sciences, 2020. Thesis Availability: Public (worldwide access) Research Advisor(s): Ames, Aaron D. Thesis Committee: Yue, Yisong (chair) Murray, Richard M. Burdick, Joel Wakeman Ames, Aaron D. Defense Date: 22 May 2023 Funders: Funding Agency Grant Number Google UNSPECIFIED Defense Advanced Research Projects Agency (DARPA) HR00111890035 NSF 1932091 Record Number: CaltechTHESIS:06022023-032907616 Persistent URL: https://resolver.caltech.edu/CaltechTHESIS:06022023-032907616 DOI: 10.7907/bpht-by81 Related URLs: URL URL Type Description https://doi.org/10.1109/IROS40897.2019.8967820 DOI Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems, presented in Section 3.3 https://doi.org/10.1109/IROS40897.2019.896782010.1109/CDC40024.2019.9029226 DOI A Control Lyapunov Perspective on Episodic Learning via Projection to State Stability, presented in Section 3.4 https://doi.org/10.1109/IROS40897.2019.896782010.23919/ACC45564.2020.9147463 DOI Adaptive Safety with Control Barrier Functions, presented in Section 3.9 https://proceedings.mlr.press/v120/taylor20a.html Publisher Learning for Safety-Critical Control with Control Barrier Functions, presented in Section 3.5 https://doi.org/10.1109/IROS40897.2019.896782010.1109/LCSYS.2020.300510 DOI Safety-Critical Event Triggered Control via Input-to-State Safe Barrier Functions, presented in Section 6.5 https://doi.org/10.1109/IROS40897.2019.896782010.1109/LCSYS.2020.3009082 DOI A Control Barrier Perspective on Episodic Learning via Projection-to-State Safety, presented in Section 3.6 https://proceedings.mlr.press/v155/dean21a/dean21a.pdf Publisher Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functions, presented in Section 5.2 http://proceedings.mlr.press/v144/csomay-shanklin21a/csomay-shanklin21a.pdf Publisher Episodic Learning for Safe Bipedal Locomotion with Control Barrier Functions and Projection-to-State Safety, presented in Section 3.6 https://doi.org/10.1109/IROS40897.2019.896782010.1109/IROS51168.2021.9636584 DOI Measurement-robust control barrier functions: Certainty in safety with uncertainty in state, presented in Section 5.3 https://doi.org/10.1109/IROS40897.2019.896782010.1109/LCSYS.2021.3087443 DOI Safe Controller Synthesis With Tunable Input-to-State Safe Control Barrier Functions, presented in Section 4.3 https://doi.org/10.1109/IROS40897.2019.896782010.1109/LCSYS.2021.3085172 DOI Sampled-Data Stabilization With Control Lyapunov Functions via Quadratically Constrained Quadratic Programs, presented in Section 6.3 https://doi.org/10.1109/IROS40897.2019.896782010.1109/CDC45484.2021.9683511 DOI Towards robust data-driven control synthesis for nonlinear systems with actuation uncertainty, presented in Section 3.8 https://proceedings.mlr.press/v168/cosner22a.html Publisher Safety-Aware Preference-Based Learning for Safety-Critical Control, presented in Section 5.4 https://doi.org/10.1109/IROS40897.2019.896782010.1109/CDC51059.2022.9993226 DOI Safety of Sampled-Data Systems with Control Barrier Functions via Approximate Discrete Time Models, presented in Section 6.4 https://arxiv.org/abs/2206.03568 arXiv Control Barrier Functions and Input-to-State Safety with Application to Automated Vehicles, presented in Section 4.3 and 4.4 https://arxiv.org/abs/2302.07469 arXiv Robust Safety under Stochastic Uncertainty with Discrete-Time Control Barrier Functions, presented in Section 4.5 ORCID: Author ORCID Taylor, Andrew James 0000-0002-5990-590X Default Usage Policy: No commercial reproduction, distribution, display or performance rights in this work are provided. ID Code: 15275 Collection: CaltechTHESIS Deposited By: Andrew Taylor Deposited On: 02 Jun 2023 23:32 Last Modified: 08 Nov 2023 00:10 Thesis Files PDF - Final Version See Usage Policy. 31MB Repository Staff Only: item control page

Robust Safety-Critical Control: A Lyapunov and Barrier Approach Thesis by Andrew James Taylor In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy, Control and Dynamical Systems CALIFORNIA INSTITUTE OF TECHNOLOGY Pasadena, California 2023 Defended May 22nd, 2023 ii © 2023 Andrew James Taylor ORCID: 0000-0002-5990-590X All rights reserved iii ACKNOWLEDGEMENTS When I show my father my travel pictures, he often responds that “your pictures need people in them”. He will say that thirty years from now, the picture of that beautiful landscape I took will not matter very much to me, but the ones capturing memories of the people I love will be ones I cherish forever. I have come to believe that a similar sentiment is true about graduate school. In thirty years, I likely will not remember every detail of every proof or every small mathematical technicality that has seemed so important at one point or another. But rather, I will remember the people that were there with me sharing in the discovery process. Those there with me checking proofs, running last-minute experiments on robots, and pushing to get our ideas down on paper for an impending deadline. I will also remember those people who were there when research was not “clicking” and gave me the support to just keep pushing. Without them, this thesis would not have been possible. My thesis advisor, Professor Aaron Ames, has played an integral role in my development as a control theorist, researcher, and engineer. Under his guidance, I have developed in my capacity to identify interesting problems, propose insightful solutions, and perhaps most importantly, communicate why the problems and solutions matter. My writing skills have improved tremendously because of his teaching, and for this, I am very grateful. Professor Yisong Yue has provided invaluable advice to me throughout my PhD. I came in to my PhD knowing nearly nothing about the field of machine learning, and through his mentoring, have come to understand many of the important challenges that must be solved in building truly autonomous systems. Professors Richard Murray and Joel Burdick have also provided me with much wisdom on the relationship between control theory research and control systems practice. This has been fundamental in forming my perspectives on which problems are worth solving. Professor James Forbes, my undergraduate advisor, introduced me to control systems with his course on attitude dynamics. He was instrumental in my decision to pursue graduate studies at Caltech, and has continually encouraged my research efforts. There have been several colleagues of mine who have been a fundamental part of my research experience. I met Victor Dorobantu at the very first wel- iv come event for our program (something I often tease him for for forgetting). Throughout the entirety of our PhDs, we have spent countless hours working on homework assignments, whiteboarding ideas, pair-programming, and arguing over notation as we tried to write our papers as precisely as possible. His attention to detail and mathematical rigor has vastly impacted my abilities as a researcher, and I am so glad to call him a colleague and friend. I was exceptionally fortunate to have had the opportunity to serve as a research mentor for Noel Csomay-Shanklin and Ryan Cosner. Their boundless enthusiasm and constant excitement for solving the next big problem in robotics has been a great source of inspiration for me throughout my PhD, and for their friendships I am deeply grateful. There are many additional colleagues I have had the chance to collaborate with throughout my PhD, including Andrew Singletary, Ruben Grandia, Anil Alan, Tamás Molnar, Pio Ong, Jason Choi, Kim Wabersich, Preston Culbertson, Maegan Tucker, Amy (Kejun) Li, Wyatt Ubellacker, Min Dai, Ugo Rosolia, Hoang Le, Chaozhe He, Manuel Galliker, Farbod Farshidian, Mohamed Maghenem, and Meera Krishnamoorthy, as well as Professors Sarah Dean, Gábor Orosz, Marco Hutter, Benjamin Recht, Paulo Tabuada, Jorge Cortés, Melanie Zeilinger, Claire Tomlin, Koushil Sreenath, Katherine Bouman, and Ricardo Sanfelice. Every one of them has in some way contributed in building my perspective on research. Likewise, there are other colleagues who have provided me with fruitful discussions on various research and engineering problems, including Riley Murray, Apurva Badithela, Fernando Castenada GarciaRozas, Bike Zhang, Thomas Youmans, James Croughan, and Mrdjan Janković. I am also grateful for the other past and present members of the AMBER Lab, including Jenna Reher, Rachel Gehlhar, Thomas Gurriet, Ivan Jimenez Rodriguez, Albert Li, Gilbert Bahati, Sergio Esteban, and Adrian Ghansah. My ability to succeed in research has depended on my happiness when not doing research. There have been many people who have helped me achieve that during my time at Caltech. From our Dungeons & Dragons group to our trips out in the wilderness, my time spent with my friends Bijan Mazaheri, Aidan Fenwick, Kristjan Eldjarn, Ellande Tang, Prithvi Akella, and Rebecca Gallivan has brought much joy. Another huge facet of my life outside of research has been music. My friends Connor McMahan and Christian Leefmans in the Caltech Classical Guitar Ensemble and all of the individuals I have played with v at St. Philip the Apostle have helped recharged my research efforts through our shared musical experiences. Some of my favorite memories from graduate school were with James Holper, Varun Wadia, Cam Gray, David Needell, and Janis Hesse in our band Sunshine & The Deep Jams. I loved every night we spent practicing, every post-practice stop for a beer, and every show we played. When I had the brilliant idea to leap from the Gene Pool and break my leg during a show, they were incredibly good friends supporting my recovery. Beyond our music, they have been some of my best friends through my hardest moments in graduate school. My guitar teacher, Matthew Elgart has taught me the importance of having an idea, of nothing being done unconsidered. This is perhaps one of the most important lessons I have learned with respect to my research, as it now underlies every word I write. For everything he has taught me, about music, and life, I am so immensely grateful. Throughout my PhD, I have been carried by the overwhelming support and love of my family. Roya Saadat and Farhad Razavian have been my California parents, opening their home and lives to me in the heart of the pandemic and supporting me throughout many difficult times during my studies. My Nannie and Dzia, Terenia and Andrew Zawisza, have given so much to build a life for their grandchildren, including me. They have nurtured my curiosity and always expressed their belief in my ability to achieve anything I set out to. My grandparents, Glen and Christine Taylor, lived a life that has been deeply inspirational for me. In the face of many challenges, they created a loving environment for their children and grandchildren that has shaped who I have grown into. My aunts, Suzette Dalrymple and Lisa Byerley, have been there throughout my entire life, and have never failed to support me whenever I have needed it. Jason Eickmann has been an uncle, friend, and mentor to me, and has never hesitated to support me in any way possible. He taught me to be honest with myself, which has been an invaluable guiding principle whenever I make difficult decisions. The unconditional love and limitless support of my parents, James and Rachelle Taylor, has made all of my achievements possible. They have spent countless hours encouraging me when I have been frustrated, shared in the joy of my successes, and offered me profound wisdom on how to find happiness in my life. I can not fathom navigating many of the situations I have faced without the guidance of my father’s experience. My mother’s endless compassion has vi always brought me back to the fundamental truth that “it will be okay”. The completion of my education is as much their achievement as it is mine. My sister, Isabel Taylor, is the best sibling that a brother could ask for. She has never faltered in showing me support, never failed to brighten my mood, and never allowed me to take myself too seriously. I am so glad that I have got to share so many life experiences with her, and I am excited to share so many more. I love all three of you. To my partner, Noosha Razavian, you and I met right before the start of the pandemic, and had to choose to likely never see each other again, or go for it. We chose the latter, and it has been one of the greatest decisions of my life. You, more than anybody, have been in the trenches with me through the thick and thin of graduate school. You have always recognized and celebrated my highest points, and shown me the deepest of compassion when I have been at my lowest points. You have demonstrated immense patience when I had to commit myself fully to research, reminded me the virtue of modesty, and rallied the people who care about me when I have needed it the most. We have shared so many exhilarating adventures together, and I am thrilled for the ones that lie ahead of us. I love you. Above all, I thank God. You have been with me at every step of this journey, and when I was not strong enough to walk it, you carried me. Each individual in this acknowledgement is a blessing you have bestowed upon me, for which I am eternally grateful. Without you, I am nothing. vii ABSTRACT Accompanying the technological advances of the past decade has been the promise for widespread growth of autonomous systems into nearly all domains of human society, including manufacturing, transportation, and healthcare. At the same time, there have been several tragic failures that reveal potential risks with the expansion of autonomous systems into everyday life, and indicate that it is vital for safety to be accounted for in the design of control systems. This thesis seeks to develop a theory of robust safety-critical control for autonomous systems. This theory will be built upon the foundational tools of Control Lyapunov Functions (CLFs) and Control Barrier Functions (CBFs), which provide a powerful paradigm for the design of model-based safety-critical controllers. The dependence of CLF and CBF-based controllers on a system model makes them susceptible to modeling inaccuracies, potentially resulting in unsafe behavior when deploying these controllers on real-world systems. In this thesis I present methods for resolving four classes of model inaccuracies referred to as model error, disturbances, measurement error, and input sampling, which are commonly faced challenges when designing controllers for robotic systems. The proposed methods are unified by their shared use of CLFs and CBFs to produce controllers possessing rigorous and robust safety guarantees that can be demonstrated in simulation or experimentally. A hallmark of these methods is a focus on enabling control synthesis through convex optimization, which ensures that controllers can be efficiently computed on real-world robotic hardware platforms. In addressing model error, I consider both data-driven learning approaches and adaptive control approaches. I present three episodic learning frameworks that iteratively augment existing CLF and CBF-based controllers specified via convex optimization problems to improve the stability and safety properties of a system, which I demonstrate in simulation and experimentally. I also establish a relationship between the degradation of stability and safety properties with the magnitude of residual learning error through the perspective of Inputto-State Stability (ISS) and Input-to-State Safety (ISSf). Lastly, I develop an adaptive safety-critical control framework for systems with parametric model error through the notion of adaptive CBFs. viii In addressing disturbances, I resolve challenges in balancing performance and robustness with ISSf-based controllers through the notion of Tunable Inputto-State Safety (TISSf), which permits prioritizing robustness to disturbances only when safety requirements are close to being violated. I demonstrate the capabilities of TISSf-based control design experimentally on an autonomous semi-trailer truck system that is subject to input disturbances due to complex unmodeled actuator dynamics. Lastly, I develop a framework for achieving ISSf-like finite-time safety guarantees for discrete-time systems subject to stochastic disturbances through the use of CBFs and convex optimization. In addressing measurement error, I develop the notion of Measurement-Robust CBFs (MR-CBFs), which permit control synthesis through convex optimization in the presence of imperfect measurements. I demonstrate the capability of MR-CBFs on an experimental Segway system using a vision-based measurement system, validating the tractability of using controllers specified through increasingly complex classes of convex optimization problems on real-world systems. Lastly, I present an application of Preference-Based Learning (PBL) in tuning the robustness parameters of a CBF-based controller, demonstrating the first use of PBL with CBFs and providing a tool for tuning the safety and performance of the robust controllers proposed in this thesis. In addressing input sampling, I consider both sampled-data and event-triggered paradigms for modeling input sampling. I provide a method for synthesizing CLF-based controllers for sampled-data systems by integrating feedback linearization with approximate discrete-time models, leading to a significant improvement over continuous-time CLF-based controllers implemented with input sampling. I then develop a framework for achieving safety of sampleddata systems through approximate discrete-time models through the notion of practical safety and Sampled-Data CBFs (SD-CBFs), which I demonstrate with convex-optimization based controllers in simulation. Lastly, I develop a method for event-triggered safety-critical control that uses ISSf to achieve safety while satisfying the requirement of a minimum interevent time. Collectively, these contributions constitute a significant advance in the theory of robust safety-critical control by establishing a framework, unified by the use of CLFs and CBFs in conjunction with convex optimization, that addresses a wide class of challenges faced in the design of safety-critical control systems. ix PUBLISHED CONTENT AND CONTRIBUTIONS [1] R. K. Cosner, P. Culbertson, A. J. Taylor, and A. D. Ames, “Robust safety under stochastic uncertainty with discrete-time control barrier functions,” accepted in Robotics: Science and Sys. (RSS) XIX, 2023. [Online]. Available: https://arxiv.org/abs/2302.07469, A. J. Taylor participated in the conception of the project, theoretical analysis, and writing of the article. The contents are presented in Chapter 4. [2] A. Alan, A. J. Taylor, C. R. He, A. D. Ames, and G. Orosz, “Control barrier functions and input-to-state safety with application to automated vehicles,” conditionally accepted as a Full Paper in IEEE Trans. on Control Sys. Tech., 2022. [Online]. Available: https://arxiv.org/ abs/2206.03568, A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. The contents are presented in Chapter 4. [3] A. Alan, A. J. Taylor, C. R. He, G. Orosz, and A. D. Ames, “Safe controller synthesis with tunable input-to-state safe control barrier functions,” IEEE Control Sys. Let., vol. 6, pp. 908–913, 2022. doi: 10 . 1109/LCSYS.2021.3087443, A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, and writing of the article. The contents are presented in Chapter 4. [4] R. K. Cosner, M. Tucker, A. J. Taylor, et al., “Safety-aware preferencebased learning for safety-critical control,” in Proc. 4th Learning for Dynamics and Control (L4DC), vol. 168, Stanford, CA, USA, 2022, pp. 1020–1033. [Online]. Available: https://proceedings.mlr.press/ v168/cosner22a.html, A. J. Taylor participated in the conception of the project, theoretical analysis, and writing of the article. The contents are presented in Chapter 5. [5] A. J. Taylor, V. D. Dorobantu, R. K. Cosner, Y. Yue, and A. D. Ames, “Safety of sampled-data systems with control barrier functions via approximate discrete time models,” in Proc. IEEE 61st Conf. on Decision and Control (CDC), Cancún, Mexico, 2022, pp. 7127–7134. doi: 10.1109/CDC51059.2022.9993226, A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. The contents are presented in Chapter 6. x [6] A. J. Taylor, V. D. Dorobantu, Y. Yue, P. Tabuada, and A. D. Ames, “Sampled-data stabilization with control lyapunov functions via quadratically constrained quadratic programs,” IEEE Control Sys. Let., vol. 6, pp. 680–685, 2022. doi: 10.1109/LCSYS.2021.3085172, A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. The contents are presented in Chapter 6. [7] A. J. Taylor, P. Ong, T. G. Molnar, and A. D. Ames, “Safe backstepping with control barrier functions,” in Proc. IEEE 61st Conf. on Decision and Control (CDC), Cancún, Mexico, 2022, pp. 5775–5782. doi: 10. 1109/CDC51059.2022.9992763, A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, and writing of the article. [8] R. K. Cosner, A. W. Singletary, A. J. Taylor, T. G. Molnar, K. L. Bouman, and A. D. Ames, “Measurement-robust control barrier functions: Certainty in safety with uncertainty in state,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Prague, Czech Republic, 2021, pp. 6286–6291. doi: 10.1109/IROS51168.2021.9636584, A. J. Taylor participated in the conception of the project, algorithm design, and writing of the article. The contents are presented in Chapter 5. [9] N. Csomay-Shanklin, R. K. Cosner, M. Dai, A. J. Taylor, and A. D. Ames, “Episodic learning for safe bipedal locomotion with control barrier functions and projection-to-state safety,” in Proc. 3rd Learning for Dynamics and Control (L4DC), vol. 144, Zürich, Switzerland, 2021, pp. 1041–1053. [Online]. Available: http://proceedings.mlr.press/ v144/csomay-shanklin21a/csomay-shanklin21a.pdf, A. J. Taylor participated in the conception of the project, algorithm design, and writing of the article. The contents are presented in Chapter 3. [10] A. J. Taylor, V. D. Dorobantu, S. Dean, B. Recht, Y. Yue, and A. D. Ames, “Towards robust data-driven control synthesis for nonlinear systems with actuation uncertainty,” in Proc. IEEE 60th Conf. on Decision and Control (CDC), Austin, TX, USA, 2021, pp. 6469–6476. doi: 10.1109/CDC45484.2021.9683511, A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. The contents are presented in Chapter 3. [11] A. J. Taylor, P. Ong, J. Cortés, and A. D. Ames, “Safety-critical event triggered control via input-to-state safe barrier functions,” IEEE Control Sys. Let., vol. 5, no. 3, pp. 749–754, 2021. doi: 10.1109/LCSYS. 2020.3005101, A. J. Taylor participated in the conception of the project, algorithm xi design and theoretical analysis, simulation code implementation, and writing of the article. The contents are presented in Chapter 6. [12] A. J. Taylor, A. Singletary, Y. Yue, and A. D. Ames, “A control barrier perspective on episodic learning via projection-to-state safety,” IEEE Control Sys. Let., vol. 5, no. 3, pp. 1019–1024, 2021. doi: 10.1109/ LCSYS.2020.3009082, A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation and experimental code implementation, conduction of experiments, and writing of the article. The contents are presented in Chapter 3. [13] S. Dean, A. J. Taylor, R. K. Cosner, B. Recht, and A. D. Ames, “Guaranteeing safety of learned perception modules via measurementrobust control barrier functions,” in Proc. Conf. on Robotics Learning (CoRL), Cambridge, MA, USA, 2020. [Online]. Available: https: //proceedings.mlr.press/v155/dean21a/dean21a.pdf, A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, and writing of the article. The contents are presented in Chapter 5. [14] A. J. Taylor and A. D. Ames, “Adaptive safety with control barrier functions,” in Proc. IEEE American Control Conf. (ACC), Denver, CO, USA, 2020, pp. 1399–1405. doi: 10.23919/ACC45564.2020.9147463, A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. The contents are presented in Chapter 3. [15] A. J. Taylor, A. Singletary, Y. Yue, and A. D. Ames, “Learning for safety-critical control with control barrier functions,” in Proc. 2nd Learning for Dynamics and Control (L4DC), vol. 120, Berkeley, CA, USA, 2020, pp. 708–717. [Online]. Available: https://proceedings.mlr. press/v120/taylor20a.html, A. J. Taylor participated in the conception of the project, algorithm design, simulation and experimental code implementation, conduction of experiments, and writing of the article. The contents are presented in Chapter 3. [16] A. J. Taylor, V. D. Dorobantu, M. Krishnamoorthy, H. M. Le, Y. Yue, and A. D. Ames, “A control lyapunov perspective on episodic learning via projection to state stability,” in Proc. IEEE 58th Conf. on Decision and Control (CDC), Nice, France, 2019, pp. 1448–1455. doi: 10.1109/ CDC40024.2019.9029226, A. J. Taylor participated in the conception of the project, theoretical analysis, simulation code implementation, and writing of the article. The contents are presented in Chapter 3. xii [17] A. J. Taylor, V. D. Dorobantu, H. M. Le, Y. Yue, and A. D. Ames, “Episodic learning with control lyapunov functions for uncertain robotic systems,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Macau, China, 2019, pp. 6878–6884. doi: 10.1109/IROS40897. 2019.8967820, A. J. Taylor participated in the conception of the project, algorithm design, simulation code implementation, and writing of the article. The contents are presented in Chapter 3. xiii TABLE OF CONTENTS Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Published Content and Contributions . . . . . . . . . . . . . . . . . . . Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . List of Illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nomenclature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter I: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 General Mathematical Outlook . . . . . . . . . . . . . . . . . . Chapter II: Background . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Nonlinear Dynamics . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Stability & Safety . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Control Lyapunov Functions . . . . . . . . . . . . . . . . . . . 2.4 Feedback Linearization . . . . . . . . . . . . . . . . . . . . . . 2.5 Control Barrier Functions . . . . . . . . . . . . . . . . . . . . . 2.6 Nonlinear Dynamics with Disturbances . . . . . . . . . . . . . 2.7 Input-to-State Stability & Input-to-State Safety . . . . . . . . 2.8 Input-to-State Stable Control Lyapunov Functions . . . . . . . 2.9 Input-to-State Safe Control Barrier Functions . . . . . . . . . . Chapter III: Learning & Adaptive Control for Nonlinear Systems . . . 3.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Model Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Learning with Control Lyapunov Functions . . . . . . . . . . . 3.4 Projection-to-State Stability . . . . . . . . . . . . . . . . . . . 3.5 Learning with Control Barrier Functions . . . . . . . . . . . . . 3.6 Projection-to-State Safety . . . . . . . . . . . . . . . . . . . . . 3.7 Projected Disturbance Learning with Control Barrier Functions 3.8 Data-Driven Nonlinear Control . . . . . . . . . . . . . . . . . . 3.9 Adaptive Safety via Control Barrier Functions . . . . . . . . . 3.10 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . Chapter IV: Disturbance-Robust Safety-Critical Control . . . . . . . . 4.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Limitations of Input-to-State Safety . . . . . . . . . . . . . . . 4.3 Tunable Input-to-State Safety . . . . . . . . . . . . . . . . . . 4.4 Tunable Input-to-State Safety on Automated Truck . . . . . . 4.5 Safety with Stochastic Disturbances . . . . . . . . . . . . . . . 4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter V: Measurement-Robust Safety-Critical Control . . . . . . . . 5.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . iii vii ix xii xv xxvi xxvii 1 12 30 30 31 33 39 46 51 53 55 60 64 65 70 72 83 97 105 112 125 144 163 165 167 171 174 188 202 221 224 226 xiv 5.2 Measurement-Robust Control Barrier Functions . . . . . . . . 5.3 Integration with Backup-Set Methods . . . . . . . . . . . . . . 5.4 Preference-Based Learning for Robust Controller Tuning . . . . 5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter VI: Sampled-Data & Event-Triggered Control . . . . . . . . . 6.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Sampling Paradigms . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Sampled-Data Stabilization . . . . . . . . . . . . . . . . . . . . 6.4 Sampled-Data Safety . . . . . . . . . . . . . . . . . . . . . . . 6.5 Event-Triggered Safety . . . . . . . . . . . . . . . . . . . . . . 6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter VII: Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Infeasible Optimization Problems . . . . . . . . . . . . . . . . 7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Appendix A: Unmatched Tunable Input-to-State Safety Proof . . . . . 228 240 249 262 264 266 271 279 297 316 329 333 336 341 343 376 xv LIST OF ILLUSTRATIONS Number Page 1.1 Ideal control system diagram. . . . . . . . . . . . . . . . . . . . 2 1.2 Practical control system diagram featuring model error, disturbances, measurement modules, and input sampling. . . . . . . . 3 1.3 Spectrum of robustness properties of results presented in this thesis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.1 (Left) Nominal model-based (CLF-QP) controller fails to stabilize the system to the trajectory. (Right) Improvement in angle tracking of system with learning-informed (CLF-QP) controller over nominal PD controller. (Bottom) Corresponding visualizations of state data. Video of this simulation is found at https://youtu.be/cB5MY_8vCrQ. . . . . . . . . . . . . . . . . 81 3.2 Augmented controllers consistently improve stabilization to trajectory across episodes. Ten instances of the algorithm are executed with the shaded region formed from minimum and maximum angles for each time step within an episode. The corresponding average angle trajectories are also displayed. . . . . . 82 3.3 Comparison of tracking performance for PD controller and final learning-informed (CLF-QP) controller in inverted pendulum example. The final learning-informed (CLF-QP) controller tracks the desired angle trajectory significantly more accurately. . . . 95 xvi 3.4 Comparison of worst-case residual learning errors (3.80) with nominal model-based (CLF-QP) controller (Left) and (3.81) with final learning-informed (CLF-QP) controller (Right). Closedloop trajectories using the two controllers are displayed with dashed lines. The learning-informed (CLF-QP) controller keeps the system in regions with lower residual learning error, while the system leaves the region around the training data under the nominal model-based (CLF-QP) controller. The heat maps were generated by sampling states randomly about training data points and evaluating the upper bound for each sampled state. The results were then discretized for ease of visualization. Each bin is colored by the maximum disturbance observed in the bin. . . 96 3.5 Simulation results using the nominal model-based (CBF-QP) controller (green) and learning-informed (CBF-QP) controller (3.88) (blue). Episodic data appears as red traces. (Left) Segway simulation CAD model. (Middle) The phase portrait showing the evolution of the state of the system leaves the safe set (black ellipse) with the nominal model-based (CBF-QP) controller, while it remains within using the learning-informed (CBF-QP) controller. (Right) The value of h drops below zero (twice) using the nominal model-based (CBF-QP) controller, while it remains above zero using the learning-informed (CBF-QP) controller. Video can be found at https://youtu.be/tD8zH4GF_8U. 103 3.6 Experimental results using the nominal model-based (CBF-QP) controller (green) and learning-informed (CBF-QP) controller (3.88) (blue). (Left) Custom robotic Ninebot Segway platform. (Middle) The phase portrait showing the evolution of the state of the system leaves the safe set (black strip) with the nominal model-based (CBF-QP) controller, while it remains within using the learning-informed (CBF-QP) controller. (Right) The value of h drops below zero (at t = 3) using the nominal modelbased (CBF-QP) controller, while it remains above zero using the learning-informed (CBF-QP) controller. Video can be found at https://youtu.be/YCesvD_Ae00 . . . . . . . . . . . . . . . 104 xvii 3.7 Simulation results with Segway platform demonstrating improvement in PSSf behavior. (Left) Absolute value of the projected disturbance along the trajectory without learning models ((3.115), red) and with learning models ((3.116), blue), with learning reducing the worse case projected disturbance (δ/k). (Right) The value of the barrier satisfies the corresponding worst case lower bound with and without learning being used to compute δ. The worst case lower bound is raised with learning (the blue dashed line lies above the red dashed line). . . . . . . . . . . . . . . . 111 3.8 Experimental results with Segway platform demonstrating improvement in PSSf behavior. (Left) Absolute value of the projected disturbance along the trajectory without learning models ((3.115), red) and with learning models ((3.116), blue), with learning reducing the worse case projected disturbance (δ/k). (Right) The value of the barrier satisfies the corresponding worst case lower bound with and without learning being used to compute δ. The worst case lower bound is raised with learning (the blue dashed line lies above the red dashed line). . . . . . . . . 112 3.9 Schematic for challenges in learning CBF time derivatives with structured models. . . . . . . . . . . . . . . . . . . . . . . . . . 114 3.10 Schematic for introducing additional data for resolving challenges in learning CBF time derivatives with structured models. . . . 115 3.11 (Left) Schematic diagram of the AMBER-3M robot with position coordinates. (Center) Schematic of the foot placement in the stepping-stone problem. The boundaries of virtual steppingstones are captured via the blue and orange vertical lines. (Right) Virtual stepping stone width as function of the phase variable τ (q).121 xviii 3.12 Simulation (S) and Hardware (H) data where model mismatch causes violations. (Far Left) Simulation where the barrier functions h1 (solid blue) and h2 (solid orange) are enforced via a nominal model-based (CBF-QP) controller. A conservative controller using a worst-case upper bound on the projected disturbance is also shown (dashed blue, dashed orange), which results in more conservative behavior over many steps. (Mid Left) After three episodes of learning using the (SS-QP) controller in simulation, the maximum barrier violation decreases from 2.0 to 0.3 [cm]. (Mid Right) Hardware where the barrier functions h1 (blue) and h2 (orange) are enforced via a nominal model-based (CBF-QP) controller. (Far Right) After two episodes of learning on hardware, the maximum barrier violation decreases from 9.2 to 1.9 [cm] using the (SS-QP) controller. . . . . . . . . . . . . . 123 3.13 Gait tiles for Episode 2 of learning showing the AMBER-3M robot safely traversing a set of stepping stones. Notice the change in step width and added lean of the torso induced by the barrier functions. Video can be found at https://vimeo.com/481809664124 3.14 Schematic of model error set arising from multiple data points. 127 3.15 Projected view of model error set used in proving feasibility of the (DR-CCF-OP). The green region corresponds to Π(UC (x)) while the orange region highlights the set U C (x) lying above the hyperplane H. The line Hβ represents the strictly separating hyperplane with normal s and offset β, while H0 is the hyperplane shifted to pass through the origin. . . . . . . . . . . . . . . . . 139 3.16 The value of the certificate function C(x) (Top) and input u (Bottom) over time for the stability (Top Pair) and safety (Bottom Pair) experiments. I compare the behavior of a true modelbased (CCF-QP) controller (oracle, black square), a nominalmodel based (CCF-QP) controller (baseline, red diamond), and the proposed (DR-CCF-SOCP) controller using dense input data (R-dense, blue triangle) and sparse input data (R-Sparse, green circle). For stability, the desired behavior for C(x) is to converge towards 0, while for safety it is to remain non-positive. . . . . . 143 3.17 Schematic of aCBF counterexample. . . . . . . . . . . . . . . . 157 xix 3.18 Simulation trajectory of aCBF counterexample. The system state leaves the prescribed safe set as it converges to the limit cycle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 3.19 Comparison of different adaptive and non-adaptive control methodologies. (Top, Left) An aCBF controller is able to enforce safety of the nominal controller knom . (Top, Right) An aCLF-aCBF controller is able to track the desired velocity with zero steady state error (Top, Right). (Bottom) Both aCBF controllers are able to keep the vehicle within the safe region for all time. . . . 161 4.1 Simple example demonstrating limits of ISSf in achieving performance and meaningful theoretical safety guarantees. (a) Safe controller with no disturbances. (b) Impact of harmonic disturbance signal without modifications to controller design. (c) ISSf based controller designs for different values of the parameter ϵ0 . Large values of ϵ0 lead to weak theoretical guarantees, while small values lead to conservative behavior. . . . . . . . . . . . . 173 4.2 Examples of the function ϵ that satisfy the condition in (4.14). 177 4.3 Simple example using TISSf to achieve both meaningful theoretical safety guarantees and non-conservative behavior. . . . . . . 180 4.4 Simulation results for the inverted pendulum system. The gold ellipse is the set C as defined in (4.37). The black line is the set where Lg h(x) = 0 as defined in (4.38). The dashed blue line is the trajectory of the system evolving under knom as defined in (4.43), which leaves the set C. The green and purple regions indicate where the controller knom meets and fails to meet the CBF condition, respectively. The dashed red line is the trajectory of the system evolving under kCBF which remains inside the set C. 183 4.5 Disturbance signal for the inverted pendulum system example as defined in (4.45). . . . . . . . . . . . . . . . . . . . . . . . . . . 185 4.6 (Right) Simulation results for the inverted pendulum system with disturbances. The gold ellipse is the safe set C defined in (4.37). The blue line is the trajectory of the system evolving under kCBF defined in (CBF-QP), which is not robust to disturbances and leaves the safe set. The black, green, and dashed red lines are trajectories of the system evolving under kTISSf defined in (TISSf-CBF-QP) with different values for ϵ0 and λ. . . . . . 186 xx 4.7 (Left) Curves corresponding to the value of h∗ solving (4.49) across the (ϵ0 , λ) parameter space for the inverted pendulum example. (Right) The boundary of the set Cd,T rendered forward invariant for different choices of the parameters ϵ0 and λ for the inverted pendulum example. Note that the Cd,T contains the set C for each parameter choice. . . . . . . . . . . . . . . . . . . . 187 4.8 Experimental configuration for heavy-duty semi-trailer truck problem. (Top) Controller design without robustifying element yields safety violation. (Bottom) Robust safety-critical controller ensures semi-trailer truck brakes early and aggressively enough to maintain safe distance. . . . . . . . . . . . . . . . . . . . . . . . 189 4.9 A connected automated truck following a connected vehicle. . . 190 4.10 (Left) The value of the function ρ defined in (4.55), which defines the minimum safe following distance as a function of the leader’s velocity vL and truck velocity v. (Right) The value of Lf h(x) + α(h(x)) as defined in (4.58) when Lg h(x) = 0 for various distances. As the function is strictly positive over the domain of interest, h is a CBF on C for (4.52). . . . . . . . . . . . . . . 192 4.11 (Top) Range policy V defined in (4.60). (Bottom) Speed policy W defined in (4.61). . . . . . . . . . . . . . . . . . . . . . . . . 192 4.12 Example profile for acceleration aL of lead vehicle used in simulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 4.13 (Left) Velocity vL of lead vehicle (black) and velocity of the truck using the nominal controller (4.59) (blue) and (CBF-QP) controller (red). (Right) Following distance D and value of CBF h using the nominal controller (4.59) and (CBF-QP) controller. . 194 4.14 (a) Vehicles used in experiments. (b) Image from the dashboard of the truck during an experimental run. (c,d,e) Final configurations of separate experiments. See https://youtu.be/ 9dJtC1TCBbA for a video. . . . . . . . . . . . . . . . . . . . . . 196 4.15 Mean value (lines) and standard deviations (fills) of the distance D and the CBF h when using the nominal controller (4.59) (blue) and the (CBF-QP) controller (red) in the truck experiment. The experiments with these controllers are highly consistent. . . . 197 xxi 4.16 (Left) Discrepancy between acceleration commanded by safetycritical controller (black) and actual acceleration of the automated truck (red). (Right) Disturbance signal in input seen by the truck used to define the model (4.62). . . . . . . . . . . . 4.17 Following distance D and value of h using the nominal controller (4.59) (blue), (CBF-QP) controller (red), and (TISSf-CBF-QP) controller (green) in the disturbed simulation. . . . . . . . . . . 4.18 Mean value (lines) and standard deviations (fills) of the following distance D and h using the nominal controller (4.59) (blue), (CBF-QP) controller (red), and (TISSf-CBF-QP) controller (green) in experiment. . . . . . . . . . . . . . . . . . . . 4.19 (Left) Parameter values for ϵ0 and λ used in the truck experiments, with contours showing theoretical values of h∗ . Green markers denote parameter sets which achieve the original safety goal (h(φd (t)) ≥ 0), while red markers denote parameter sets for which the original safety goal is violated. (Right) Theoretical values of h∗ and the shift in steady-state tracking distance D̃ss , for the parameter sets used in the truck experiments. The blue markers denote parameter sets with λ > 0, while the black markers denote parameter sets with λ = 0. . . . . . . . . . . . . . . 4.20 Experimental results for parameter pairs (B), (C), and (D) in Table 4.3. Case (B) is conservative as indicated by the large steady-state tracking distance. Cases (C) and (D) display nearly identical behavior, but case (C) possesses a stronger theoretical guarantee. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.21 The dashed lines represent the theoretical probability bounds for the system as in Theorem 34. The solid lines represent the Monte Carlo (MC) estimated Pu across 500 experiments. . . . 4.22 (Top Left) System diagram of the inverted pendulum. (Top Right) 500 one second long example trajectories starting at x0 = 0. (Bottom Left) Monte Carlo estimates of Pu for γ = 0 using 500 one second long trials for each initial conditions represented by a black dot. (Bottom Right) The (conservative) theoretical bounds on Pu from Theorem 34. . . . . . . . . . . . . . . . . . 198 199 199 200 202 218 219 xxii 4.23 Safety of a simulated quadrupedal robot locomoting on a narrow path for a variety of controllers. (Top Left) The safe region that the quadruped is allowed to traverse. (Bottom Left) A system diagram depicting the reduced-order model states of the h i⊤ quadruped x y θ . (Top Right) 50 trajectories for 3 controllers: one without any knowledge of safety (knom ), one with the standard (DT-CBF-OP) controller, and finally the proposed method which accounts for stochasticity (JED). (Bottom Right) Plots of h(x), a scalar value representing safety. The system is safe (i.e., in the green safe region) if h(x) ≥ 0. . . . . . . . . . b). The set 5.1 Visualization of the point-wise sets Xb(x) and X (y, x b X (x) consists of all possible estimates that may correspond to b) consists of all the possible states the state x, and the set X (y, x b). . . . . . . . for a given measurement-state estimate pair (y, x 5.2 (Left) The Segway model used in simulation from the perspective of the fixed virtual camera used to estimate its state. (Right) Simulation results for worst-case measurement model uncertainty of ϵ = 0.2 subtracted from the true pitch angle θy when measured. A state trajectory generated using the the (CBF-QP) controller (red) and the (MR-CBF-SOCP) controller (blue) are shown as projections onto their pitch angle and pitch rate components. The set C is plotted in green. Given the same initial condition, the (MR-CBF-SOCP) controller ensured safety of the trajectory whereas the (CBF-QP) controller did not. . . . . . . 5.3 Simulation results for a measurement model of x b = x − 0.4 [m] and constant desired velocity of 1 [m/s]. (Left) Trajectories generated using the (BS-QP) controller. Solid line represents the true state, dashed line shows the estimated state, and green region indicates the safe set C. The true trajectory fails to be safe and exits the safe set at t = 3 [s]. (Right) Trajectories generated using the (MR-BS-SOCP) controller. An additional robustness region is plotted in blue to indicate the set of of true states which the control input renders safe. Both the true and measured trajectories are safe demonstrating the robustness of the (MR-BS-SOCP) controller when compared to the (BS-QP) controller. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220 230 236 246 xxiii 5.4 Experimental results using SLAM from the onboard Intel RealSense T265 and constant desired velocity of 1 m/s. The notation and color schemes are the same as in Figure (5.3). (Left) Trajectories generated using the (BS-QP) controller. The true trajectory exits the safe set at t = 6.7 [s]. The measurement error is plotted in blue. (Right) Trajectories generated using the (MR-BS-SOCP) controller. Both the true and measured trajectories are safe demonstrating the robustness of the (MR-BS-SOCP) controller compared to the (BS-QP) controller. . . . . . . . . . 248 5.5 Images from the experiment using the (MR-BS-SOCP) controller. The Segway is piloted towards a wall of yellow boxes and the controller ensures that it remains safe, i.e. that it does not crash into the boxes. (Top) Time lapse of the Segway trajectory. (Bottom) Camera images taken from the perspective of the Segway throughout the experiment. The images are displayed in chronological order from left to right. A video can be found at https://vimeo.com/520247516. . . . . . . . . . . . . . . . . . 248 5.6 An overview of the Safety-Aware Preference-Based Learning design paradigm. Safety-Aware LineCoSpar is used to generate actions which are rolled out in experiments as parameters of the CBF-based safety filter to obtain user preferences and safety ordinal labels which are then used to update the user’s estimated utility and generate new actions. . . . . . . . . . . . . . . . . . 250 5.7 A comparison of Safety-Aware LineCoSpar and standard LineCoSpar on a synthetic utility function (drawn from the Gaussian prior) averaged over 50 runs with standard error shown by the shaded region. The safety-aware criteria reduces the number of sampled unsafe actions with a minimal effect on the prediction error, defined as |â∗i − a∗ | with â∗i ≜ argmaxa r̂Si and a∗ ≜ argmaxa r(a). 254 xxiv 5.8 Actions sampled during simulation in 30 iterations with 3 new actions in each iteration. The preferred action, â30 = (3, 0.6, 0.5, 0.015), is shown in black and white. A conservative action, a = (2, 0.5, 0.0651, 0.485), is indicated by the black circle, where a and b were determined by estimating the Lipschitz coefficients present in the proof of Theorem 40. The conservative action fails to progress whereas Safety-Aware LineCoSpar provides an action which successfully navigates between obstacles. . . . . . . . . . . . . . . . . . . . 260 5.9 Seven additional iterations of 3 actions executed indoors. The preferred action, â∗37 = (4, 0.6, 0.4, 0), successfully traverses between the obstacles. . . . . . . . . . . . . . . . . . . . . . . . . 261 5.10 The preferred action, â∗40 = (5, 0.1, 0.4, 0.02), after simulation, indoor experiments, and 3 additional iterations of 3 actions in an outdoor environment is shown alongside views from the onboard camera. Video can be found at https://youtu.be/QEuwRDTG7TE 262 6.1 Results using sample-and-hold implementation of (CLF-QP) controller on inverted pendulum at various sample rates. . . . . . . 272 6.2 Double Inverted Pendulum Schematic . . . . . . . . . . . . . . 273 6.3 Results using sample-and-hold implementation of (CLF-QP) controller on double inverted pendulum at various sample rates. . . 274 6.4 Planar Quadrotor Schematic . . . . . . . . . . . . . . . . . . . 275 6.5 Results using sample-and-hold implementation of (CLF-QP) controller on planar quadrotor at various sample rates. . . . . . . . 276 6.6 With inputs applied via a zero-order hold, the (CLF-QP) controller does not stabilize the system (6.84) (Top), while the (CLF-QCQP) does (Bottom). Simulation code listed at https://bit.ly/CLFQCQP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 6.7 Simulation results for the inverted pendulum system using the (CLF-QCQP) controller. . . . . . . . . . . . . . . . . . . . . . 297 6.8 Simulation results for the double inverted pendulum system using the (CLF-QCQP) controller. . . . . . . . . . . . . . . . . . 298 6.9 Simulation results for the planar quadrotor system using the (CLF-QCQP) controller. . . . . . . . . . . . . . . . . . . . . . 299 xxv 6.10 Visualizing the properties (6.106)-(6.109) of SD-BF candidates. The dark green region represents the lower bound T α(hT (x4 )) and hT (x6 ) cannot be in the red region due to the Lipschitz bound. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.11 A visual representation of the main sets and three cases discussed in the proof of Theorem 44. . . . . . . . . . . . . . . . . . . . . 6.12 The single (left) and double (right) inverted pendulums and their sets C. The inverted pendulum sets C are, from left to right, a configuration ellipsoid, a halfspace, and a Lyapunov sublevel set. The far right image shows the set C for the double inverted pendulum where (θ1 , θ2 ) ∈ R2 are constrained to an ellipse and no constraint is placed on (θ̇1 , θ̇2 ) ∈ R2 (represented by vertical fibers). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.13 The maximum distance from the safe set C (lower is better) achieved during trials vs. the the sampling frequency. The simulations and animations be found at https://bit.ly/CBF-OP and https://vimeo.com/690803272. (Top) The inverted pendulum for 3 different safe sets. (Bottom) The double inverted pendulum. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.14 Simulation results for the system (6.167) using the trigger law (6.169). Even as the boundary of the safe set is approached (h(φET (t)) → 0), the growth rate of the error does not diminish, leading to arbitrarily small interevent times. . . . . . . . . . . . 6.15 Simulation results demonstrating safety achieved with an eventtriggered controller. (Top, Left) State trajectories for both triggers remain within the safe set for the length of the simulation. (Top, Right) The value of the ISSf-BF h remains above zero for the length of the simulation, corresponding to the system remaining safe. (Right) The interevent times of the two trigger laws. The interevent times of the trigger law (6.169) decreases towards 0 as predicted by Lemma 11 while the trigger law (6.189) satisfies the theoretical bound. . . . . . . . . . . . . . . . . . . 307 309 315 316 325 329 xxvi LIST OF TABLES Number Page 1.1 General stability and safety properties. . . . . . . . . . . . . . . 13 1.2 General unstable and unsafe properties. . . . . . . . . . . . . . 15 1.3 General partially robust stability and safety properties. . . . . 16 4.1 Parameters sets used in (4.47) and (4.49) for the inverted pendulum example. . . . . . . . . . . . . . . . . . . . . . . . . . . 186 4.2 Parameter values used in controller design. . . . . . . . . . . . 193 4.3 Sets of parameter values used for the exponential function (4.63) in the automated truck experiments with theoretical safety guarantee h∗ , minimum experimental value of the TISSf-CBF hmin , and shift in the steady-state tracking distance D̃ss in (4.65). . . 201 5.1 The safety-aware hyperparameters, and action space bounds (minimum and maximum) with discretizations ∆. . . . . . . . . . . 259 xxvii NOMENCLATURE aBF. Adaptive Barrier Function. aCBF. Adaptive Control Barrier Function. ACC. Adaptive Cruise Control. aCLF. Adaptive Control Lyapunov Function. aLF. Adaptive Lyapunov Function. BF. Barrier Function. BO. Bayesian Optimization. BS. Backup Set. CAN. Controller Area Network. CBF. Control Barrier Function. CCC. Connected Cruise Controller. CCF. Control Certificate Function. CCP. Continuous Control Property. CLF. Control Lyapunov Function. CTLE. Continuous Time Lyapunov Equation. DAgger. Dataset Aggregation. DT. Discrete-Time. DT-BF. Discrete-Time Barrier Function. DT-CBF. Discrete-Time Control Barrier Function. ECU. Engine Control Unit. ERM. Empirical Risk Minimization. GPR. Gaussian Process Regression. GPS. Global Positioning System. i.i.d. Independently and Identically Distributed. ID. Inverse Dynamics. xxviii IMU. Inertial Measurement Unit. ISS. Input-to-State Stability. ISS-CCP. Input-to-State Stable Continuous Control Property. ISS-CLF. Input-to-State Stable Control Lyapunov Function. ISS-LF. Input-to-State Stable Lyapunov Function. ISS-SCP. Input-to-State Stable Small Control Property. ISSf. Input-to-State Safety. ISSf-BF. Input-to-State Safe Barrier Function. ISSf-CBF. Input-to-State Safe Control Barrier Function. LF. Lyapunov Function. LQR. Linear Quadratic Regulator. MAP. Maximum A Posteriori. MIET. Minimum Interevent Time. MPC. Model Predictive Control. MR-BF. Measurement-Robust Barrier Function. MR-CBF. Measurement-Robust Control Barrier Function. MRISS. Measurement-Robust Implicit Safe Set. NMPC. Nonlinear Model Predictive Control. OBU. Onboard Unit. PBL. Preference-Based Learning. PD. Proportional-Derivative. PSS. Projection-to-State Stability. PSSf. Projection-to-State Safety. QCQP. Quadratically Constrained Quadratic Program. QP. Quadratic Program. ReLu. Rectified Linear Unit. RL. Reinforcement Learning. xxix ROS. Robot Operating System. RTOS. Real-Time Operating Systems. SCP. Small Control Property. SGD. Stochastic Gradient Descent. SLAM. Simultaneous Localization And Mapping. SOCP. Second-Order Cone Program. SOS. Sums-of-Squares. TISSf. Tunable Input-to-State Safety. TISSf-BF. Tunable Input-to-State Safe Barrier Function. TISSf-CBF. Tunable Input-to-State State Safe Control Barrier Function. V2X. Vehicle-to-Everything. VIO. Visual Inertial Odometry. 1 Chapter 1 INTRODUCTION The goal of this thesis is to make progress towards a complete theory of robust safety-critical control for autonomous systems. The technological developments of the past decade have set the stage for the rapid proliferation of autonomous systems that will fundamentally change the landscape of human activity, including labor, transportation, and healthcare. It is estimated that 87% of hours spent on production activities performed by workers in manufacturing roles can be automated [1], while it is predicted that by 2030, 12% of new passenger cars sold will feature L3+ autonomous driving technologies, with that number increasing to 37% by 2035 [2]. At the same time, there have been a number of high profile incidents of failures in autonomous driving systems [3], [4], including some that have produced fatal accidents for both passengers in the vehicle [5] and others outside the vehicle [6], [7]. These tragic failures reveal the potential risks with the expansion of autonomous systems into everyday life, and indicate that it is more important than ever for safety to be accounted for in the design of control systems. I will build this theory upon Control Lyapunov Functions (CLFs) [8], [9] and Control Barrier Functions (CBFs) [10], [11], which are a foundation of modelbased safety-critical control theory, but which can break down when deployed on real-world systems. The contributions of this thesis will extend the capabilities of CLFs and CBFs to provide a coherent approach for synthesizing robust safety-critical controllers that explicitly address the gap between system models and real-world systems. This direction is motivated by the fact that CLFs and CBFs have been shown to be a promising tool for the constructive synthesis of safety-critical controllers through a wealth of results demonstrating their capabilities in experimental applications including mobile robots [12]– [14], robotic swarms [15], autonomous aerial vehicles [16], robotic arms [17], [18], robotic manipulators [19], brachiating robots [20], automotive systems [21], quadrupedal robots [22], [23], and bipedal robots [24], [25]. 2 Controller k(x) Control Input u Dynamics State x ẋ = f (x) + g(x)u State Feedback Figure 1.1. Ideal control system diagram. The starting point of control design with CLFs and CBFs is often described by the ideal control system seen in Figure 1.1. The system that is to be controlled is captured by a nonlinear differential equation: ẋ = f (x) + g(x)u, (1.1) with state x ∈ Rn , input u ∈ Rm , and dynamics functions f : Rn → Rn and g : Rn → Rn×m . The goal of control design is to produce a state-feedback controller k : Rn → Rm that yields a closed-loop system: ẋ = f (x) + g(x)k(x), (1.2) displaying desired stability and safety properties. In practice, real systems are subject to a number of discrepancies from this ideal model, and are more accurately described by the control system seen in Figure 1.2. This diagram highlights four challenges faced when designing safety-critical controllers for real-world systems that I will address in this thesis. The first challenge, highlighted in red, is referred to as model error, and refers to the fact that access to the true dynamics in (1.1) is not available when designing a controller. Rather, a nominal model for the dynamics is utilized: ḃ = b b(x)k(x), x f (x) + g (1.3) b : Rn → Rn×m are not necessarily equal where the functions b f : Rn → Rn and g to their true counterparts f and g. The second challenge, highlighted in green, is referred to as disturbances, and refers to both internal perturbations in the 3 Disturbances Nominal Model ˆ = f̂ (x) + ĝ(x)u ẋ Controller k(x̂) Sample-Hold u(t) = u(tk ) t ∈ [tk , tk+1) State Estimator x̂ = q(y) d1(t) d2(t) + True Dynamics + State x ẋ = f (x) + g(x)u Measurement y Sensor + y = p(x) + Noise n Figure 1.2. Practical control system diagram featuring model error, disturbances, measurement modules, and input sampling. actuators of a system and external perturbations that are generally difficult to model, leading to a nonlinear system with disturbances: ẋ = f (x) + g(x)(k(x) + d1 (t)) + d2 (t), (1.4) where d1 : R≥0 → Rm and d2 : R≥0 → Rn capture disturbance signals. The third challenge, highlighted in purple, is referred to as measurement error, and refers to the fact the state x is almost never perfectly measured and provided to b is formed through potentially a controller. Rather, an estimate of the state x imperfect measurement models and state estimators and used in a controller: ẋ = f (x) + g(x)k(b x). (1.5) Notably, the system dynamics f and g depend on the true state of the system, b ̸= x, the controller may not be and thus if the estimate is imperfect, such that x using an accurate representation of the system dynamics. The last challenge, highlighted in blue, is referred to as input sampling, and refers to the fact that for cyber-physical systems such as robots, the underlying dynamics of the system evolve according to a differential equation that is continuous in time, while controllers are implemented on computing resources that are inherently discrete-time in nature, leading to a mixture of continuous and discrete-time 4 dynamics where inputs are held constant between sampling times tk and tk+1 : ẋ(t) = f (x(t)) + g(x(t))k(x(tk )), t ∈ [tk , tk+1 ). (1.6) As I will show in the respective chapters of this thesis, neglecting these challenges in control design can lead to a failure to obtain the stability and safety properties that make the paradigm of CLF and CBF-based control so powerful. Thus, addressing these challenges will make substantial progress in developing a theory of robust safety-critical control. A key focus in my development of this theory will be on obtaining robust safety-critical controllers that can be specified as convex optimization problems, which is a critical feature of the CLF and CBF-based control paradigm [11], [26], [27]. The importance of this focus is best captured by the insightful remark that “the great watershed in optimization isn’t between linearity and nonlinearity, but convexity and nonconvexity” [28]. Convex optimization [29], [30] provides a theoretically rigorous framework for minimizing a cost function subject to a collection of problem-specific constraints, both of which must possess the important property of convexity. By satisfying these requirements, a convex optimization problem enjoys two significant benefits. First, there exists strong statements about the ability to solve the problem numerically to an arbitrary degree of accuracy with a complexity that is polynomial in the problem dimensionality [31], [32]. This is a vital property for safety-critical control systems as decisions must often be made both accurately and quickly. Second, there exists significant software infrastructure and commercial solvers including tools such as CVXPY [33], YALMIP [34], ECOS [35], MOSEK [36], and Gurobi [37], enabling not only rapid-prototyping of control systems, but robust real-time implementations needed for real-world safety-critical systems. These benefits suggest that extending the CLF and CBF-based control paradigm to achieve robust theoretical guarantees and improved performance while preserving convexity of the control synthesis process is a valuable endeavor. Summary of Contributions The contributions of this thesis are a collection of methods for addressing the challenges of model error, disturbances, measurement error, and input sampling in safety-critical control. Utilizing the design primitives of CLFs and CBFs in conjunction with convex optimization, these methods constitute a framework for robust safety-critical control that efficiently addresses a wide set of challenges seen with real-world systems. 5 Below I provide a statement of related works pertaining to each contribution (identified by a bullet point). Note that a detailed discussion of related work on each challenge is presented at the beginning of each chapter. Chapter III: Learning & Adaptive Control for Nonlinear Systems There has been a recent wealth of results addressing model error in safetycritical control through data-driven learning techniques in conjunction with CLFs [17], [38]–[41] and CBFs [16], [41]–[45]. I will focus on three directions that have been unexplored in this work. The first is studying what properties can ensure that CLFs and CBFs constructively synthesized for a nominal model (such as with feedback linearization) can still be utilized in well-posed control design for the true system, allowing learning to be used to augment a nominal model-based controller that partially realizes a desired stability or safety task encoded by a CLF or CBF. The second is utilizing the fact that only model error that impacts the time derivative of a CLF or a CBF is relevant for achieving stability or safety, respectively, and that learning model error in the full-system dynamics is not required to achieve these objectives. The third is analyzing the impacts of residual learning error in order to formalize a relationship between learning accuracy and stability and safety. • My first contribution is a collection of three episodic learning methods that capture model error directly as it impacts the evolution of CLFs and CBFs. I consider structural assumptions on the true system that enable transferring CLFs and CBFs from a nominal model to the true system. This allows me to utilize structured learning models that augment CLF and CBF-based controllers specified via convex optimization problems while preserving convexity and yielding significant improvements in stability and safety (Sections 3.3, 3.5, and 3.7). My second contribution is a characterization of the impact of residual learning errors on stability and safety built through the lens of Input-to-State Stability (ISS) and Input-to-State Safety (ISSf) (Sections 3.4 and 3.6). I demonstrate the episodic learning methods and analyze the effects of residual learning error experimentally on a Segway and bipedal robotic system (Sections 3.5, 3.6, and 3.7). Establishing the preceding contributions reveals that the frequent inability to densely sample the control input space at a given state leads to potentially 6 large worst-case residual learning errors associated with input directions not seen in the data set. Conversely, residual learning error associated with input directions seen in the data set tends to be significantly smaller, motivating studying how to capitalize on these input directions during control synthesis. The only work studying this question with CLFs and CBFs developed in parallel to my contributions and used Gaussian Process Regression (GPR) with affine kernels and CLFs [46]. • My contribution is a data-driven control method that utilizes CLFs and CBFs with robust convex optimization to ensure the stability and safety of a system with only a partial characterization of model errors from data. I integrate a characterization of possible model errors that are permitted by a data set with CLF and CBF conditions to yield a controller specified by a convex second-order cone program (SOCP) that I prove feasibility results for and demonstrate in simulation (Section 3.8). A key development in stabilizing adaptive control for nonlinear systems in the presence of parametric model error was the concept of an adaptive CLF introduced in [47] and further developed in [20], [48], [49]. Absent from this work has been a focus on adaptively enforcing the forward invariance of a particular set in the state space in the presence of parametric model error. • My contribution is a method for safety-critical control of systems with parametric model error. Inspired by [47], I achieve this by proposing adaptive Control Barrier Functions as a tool for constructively synthesizing adaptive controllers that render a particular set in the state space forward invariant. I highlight challenges in adaptively ensuring forward invariance that are not seen in adaptive stabilization through a counterexample, and demonstrate the method in simulation (Section 3.9). Chapter IV: Disturbance-Robust Safety Critical Control The notion of CBFs that are robust to disturbances have primarily focused on incorporating bounds on the worst-case disturbance in the controller to enforce forward invariance of a particular set [50]–[55], which can often lead to unnecessarily conservative behavior if the bounds are not tight. Building upon the notion of Input-to-State Stability (ISS) [56], [57], an alternative notion of robustness that describes the expansion of a forward invariant set in 7 the presence of disturbances was captured by Input-to-State Safety (ISSf) and ISSf-CBFs in [58]. A challenge with robust control design through ISSf-CBFs is that it can still be difficult to balance meaningful safety guarantees (a small expansion of the forward invariant set) with conservativeness (restricting the system to deep within a forward invariant set), as I explore in Section 4.2. • My contribution is an extension of ISSf-CBFs in the form of Tunable Input-to-State Safe CBFs (TISSf-CBFs), which permit robust yet performant control synthesis in the presence of disturbances (Section 4.3). I provide a thorough analysis and design of a TISSf-CBF-based controller that is experimentally demonstrated on a semi-trailer truck, the first demonstration of a CBF-based controller on such a system (Section 4.4). Early work studied the impact of stochastic disturbances on stability guarantees through the perspective of Lyapunov functions [59], which was extended to the setting of barrier functions and set invariance in [60], [61]. The advent of CBFs in [11], [62] has led to a wide set of results addressing the safety of systems with stochastic disturbances, both in continuous-time [43], [63]–[71] and discrete-time systems [69], [72]. While the work in [69] considers using CBFs to provide finite-time safety guarantees for discrete-time systems with stochastic disturbances using tools from [59], it did not address how to synthesize control policies given a CBF for a system with stochastic disturbances. • My contribution is an approach for synthesizing control policies given a CBF for a discrete-time system with stochastic disturbances. Inspired by both [69] and [59], I propose a method for establishing ISSf-like finitetime safety guarantees for discrete-time systems subject to stochastic disturbances. I explore how convexity properties of a CBF can be used to permit convex optimization-based controllers that provide such safety guarantees, which are demonstrated in simulation (Section 4.5). Chapter V: Measurement-Robust Safety Critical Control The work in [50] was the first to consider the impact of measurement errors on CBF-based controllers by incorporating the covariance of an unscented Kalman filter into the standard CBF inequality as an additive robustness term. Subsequently, additional work has considered measurement noise using stochastic differential equations [66], required a restricted sub-tangentiality 8 condition on the boundary of a set that is to be kept forward invariant [73], or studied measurement error using interval arithmetic and incremental stability [74]. Absent from these works was a consideration of the interaction between measurement errors and convexity as it impacts the control synthesis process. • My contribution is the definition of Measurement-Robust CBFs (MRCBFs), which permit safe control synthesis through convex optimization in the presence of measurement errors (Section 5.2), and the experimental demonstration of an MR-CBF-based controller specified via a convex SOCP on a Segway platform using a vision-based measurement system (Section 5.3). This experimental demonstration validates that the controllers proposed throughout this thesis that are specified via increasingly complex classes of convex optimization problems can be solved efficiently enough to be deployed on real-world hardware platforms. Preference-Based Learning (PBL) provides an approach for searching complex parameter spaces via subjective feedback, without an explicitly defined reward function [75]. In the context of control systems, there has recently been significant effort in developing PBL approaches for tuning existing tools built from control theory [76]–[80]. Notably, the use of PBL to tune parameters of CBF controllers to balance the tradeoff between conservativeness in safety and performance has yet to be considered. • I will present an application of PBL to tune robustness parameters of a CBF-based controller to achieve both safe and performant behavior on an experimental quadrupedal robotic system. My particular contributions are the conceptualization of the idea to use PBL to tune robustness parameters in a CBF-based controller, and an analysis of safety guarantees using reduced-order models in the presence of disturbances and measurement error (Section 5.4). This work highlights a tool for tuning the robust safety-critical control methods presented in this thesis. Chapter VI: Sampled-Data & Event-Triggered Control A key development in the design of stabilizing controllers for sampled-data systems came with the framework using approximate discrete-time models proposed in [81], [82], which led to a wide range of results focused on synthesizing controllers for sampled-data systems using approximate discrete-time 9 models [83]–[88]. This work has not addressed the design of controllers for sampled-data systems with approximate discrete-time dynamics using CLFs and convex optimization, which I highlight the importance of by demonstrating the fragility of continuous-time CLF-based controllers specified through convex optimization to input sampling in Section 6.2. • My contribution is an extension of the work in [81] that uses approximate discrete-time models in conjunction with CLFs to synthesize stabilizing convex optimization-based controllers that significantly outperforms continuous-time CLF-based controllers. I address how a CLF synthesized for a system’s continuous-time dynamics through feedback linearization possesses the necessary properties of a CLF for a set of approximate discrete-time dynamics, and show how stability of zero-dynamics is preserved with input sampling, providing a full practical stability result for a partially feedback linearizable system (Section 6.3). There has been a number of recent works exploring safety-critical control for sampled-data systems from the perspective of CBFs, [19], [73], [74], [89]–[93] much of which has taken an emulation approach, where an additive correction term is factored in to the standard continuous-time CBF inequality. This correction term often relies on Lipschitz constants of the system dynamics and bounds on the control inputs, which can be difficult to produce, and over-approximations will often lead to conservative behavior that can only be resolved with exceptionally high sample rates [90]. None of this work has considered using approximate discrete-time models along the lines of the work in [81], which often works exceptionally well even at low sample rates [94]. • My contribution is a method for studying safety of sampled-data systems using approximate discrete-time models. I propose a notion of practical safety and Sampled-Data CBFs that can be utilized with approximate discrete-time models to enable safety-critical control synthesis through convex optimization (Section 6.4), which I demonstrate in simulation. Event-triggered control is a paradigm for effectively using actuation resources when it is costly to switch the inputs to the system [95]. The work developed in [96] utilized Input-to-State Stable Lyapunov functions (ISS-LFs) to capitalize on inherent robustness in a continuous-time system to produce a 10 stabilizing event-triggered control implementation while maintaining a minimum interevent time (MIET), which ensures that events (and input switching) do not occur arbitrarily close together [97]. Notably, event-triggered work using CBFs has primarily focused on how they can be used to improve stability [89], [98], rather than trying to enforce the forward invariance of a specific set. • My contribution is a method for safety-critical control of event-triggered systems. Drawing inspiration from [96], I utilize the concept of ISSf barrier functions to provide guarantees of both safety and a minimum interevent time, which I show in simulation (Section 6.5). Outline of Thesis Chapter II: Background presents a review of background material on core tools from nonlinear control theory that will be utilized throughout this thesis, including nonlinear dynamics, stability, safety, CLFs, feedback linearization, CBFs, nonlinear dynamics with disturbances, ISS and ISSf, and their corresponding variants of CLFs and CBFs. While this section is not a contribution of the thesis, it provides pedagogical examples highlighting some of the finer points of these concepts. Chapter III: Learning & Adaptive Control for Nonlinear Systems presents my first set of contributions on the topic of addressing model error as it impacts safety-critical control. I begin by exploring related work addressing model error in the context of CLF and CBF-based controllers as well safety-critical adaptive control. Next, I present a description of general and parametric model error. In Section 3.3 I present an episodic learning method for mitigating model error in the time derivative of a CLF, and establish theoretical results on the robustness of stability to residual learning error in Section 3.4. I extend this episodic learning method to the setting of CBFs in Section 3.5 with an experimental demonstration on a Segway platform, and provide theoretical safety guarantees in the presence of residual learning error in Section 3.6. In Section 3.7 I study some of the challenges with the preceding learning method for high-dimensional systems, and propose an alternative episodic learning method that overcomes these challenges and is demonstrated experimentally on a bipedal robotic platform. Building off of these previous results, in Section 3.8 I present a data-driven control method that meets stability or safety requirements in the presence of a partial characterization of 11 model error from data using robust convex optimization. Lastly, in Section 3.9, I provide a method for safety-critical control in the presence of parametric model error through the notion of adaptive CBFs (aCBFs). I conclude with thoughts on directions for future work. Chapter IV: Disturbance-Robust Safety-Critical Control presents my second set of contributions on the topic of addressing robustness to disturbances in safety-critical control. I begin by exploring related work on disturbancerobust safety critical control and safety-critical control in the presence of stochastic disturbances. In Section 4.2 I present some of the limitations of ISSf and ISSf-CBFs as posed in [58] regarding robustness and performance. Drawing inspiration from this, in Section 4.3 I present an extension of ISSf in the form of Tunable Input-to-State Safety (TISSf), which prioritizes robustness to disturbances near the boundary of a set that is to be kept forward invariant as opposed to equally throughout the set. After demonstrating TISSf on a simple inverted pendulum example, in Section 4.4 I present a detailed deployment of TISSf-based controllers on an experimental semi-trailer truck, once again showing that TISSf can achieve safety in the presence of disturbances without significantly conceding performance. Lastly, in Section 4.5 I present a method for safety-critical control of discrete-time systems in the presence of stochastic disturbances. I conclude with directions for future work. Chapter V: Measurement-Robust Safety-Critical Control presents my third set of contributions on the topic of addressing robustness to measurement errors in safety-critical control. I begin by exploring related work in the areas of measurement-robust safety critical control and the tuning of controllers using PBL. In Section 5.2 I present a definition of Measurement-Robust Control Barrier Functions (MR-CBFs) that enable robust control synthesis through convex optimization for systems with measurement errors. Following this, in Section 5.3 I present the integration of MR-CBFs with backup set CBF methods and an experimental demonstration of a MR-CBF controller specified via a SOCP on an experimental Segway system. Lastly, in Section 5.4 I present results on using PBL to tune disturbance and measurement robustness parameters in a CBF-based controller to achieve both safety and performance. Chapter VI: Sampled-Data & Event-Triggered Control presents my fourth set of contributions on the topic of addressing robustness to sampling in safety-critical control. I will begin exploring related work in the areas 12 of sampled-data control through approximate discrete-time models, sampleddata control and feedback linearization, and event-triggered control. In Section 6.2 I highlight the fragility of continuous-time CLF-based controllers specified through convex optimization to sampling with simulation results and define the sampled-data and event-triggered sampling paradigms. Following this, in Section 6.3 I present the design of stabilizing controllers using approximate discrete-time models and CLFs through convex optimization, with a focus on how feedback linearizability of a continuous-time system can produce a CLF for an approximate discrete-time model. I revisit the simulation settings in Section 6.2, showing that the proposed control approach greatly outperforms the continuous-time approach. In Section 6.4 I present a framework for safety-critical sampled-data control using approximate discrete-time models and CBFs. I develop notions of practical safety, Sampled-Data barrier functions, and Sampled-Data CBFs, which I use to produce controllers through convex optimization that I demonstrate in simulation. Lastly, in Section 6.5, I present a framework for event-triggered safety-critical control. I review stabilizing event-triggered control as presented in [96], and show how naïvely transferring the methods of this paper to the safety-critical setting fail to produce a minimum interevent time with a counterexample. I then propose modifications that yield both safety and a minimum interevent time, which I demonstrate in simulation. I conclude with directions for future work. Chapter VII: Conclusion provides my final conclusions, as well as a discussion on future work that extends beyond the results presented in the thesis. 1.1 General Mathematical Outlook This section briefly presents the mathematical methodology of robust control that will be utilized throughout this thesis. There are two primary goals of this presentation. The first is to highlight the two fundamental perspectives on robustness that underlie the collection of results in this thesis. It is my hope that readers will discover that many challenges in robust safety-critical control are connected by the types of outcomes that can be produced through robust control synthesis. The second is to preface the technical contributions of each section in an effort to prepare a reader with an idea of the destination that mathematical developments are building towards. This math in this section is presented at an informal level, and should be interpreted to serve merely as a collection of guide posts to the more rigorous theoretical descriptions 13 provided in the subsequent chapters. A reader well-versed in nonlinear systems, Control Lyapunov Functions (CLFs), Control Barrier Functions (CBFs), Input-to-State Stability (ISS), and Input-to-State Safety (ISSf) will find the details of this section establish a conceptual understanding of what types of technical contributions will be made in this thesis, while readers newer to these concepts will be best served by reading the background material in Chapter 2 before returning to this section. Nonlinear Dynamics Consider a nonlinear control-affine system: ḃ = b b(x)u, x f (x) + g (1.7) b : Rn → with state x ∈ Rn , input u ∈ Rm , and functions b f : Rn → Rn and g Rn×m . Given a controller k : Rn → Rm , we may define a closed-loop system: ḃ = b b(x)k(x). x f (x) + g (1.8) Assume that for any initial condition in x0 ∈ Rn , there exists a unique continuously differentiable solution φ : R≥0 → Rn solving this closed-loop system. Stability & Safety Stability and safety properties for these types of system will be encoded through a scalar function C : Rn → R that is continuously differentiable on Rn . More specifically, stability and safety properties will require that: Stability Safety limt→∞ |C(φ(t))| = 0 C(φ(t)) ≤ 0, ∀t ∈ R≥0 Table 1.1. General stability and safety properties. Achieving this goal through control will rely on the fact that the time derivative of C can be expressed as:   ∂C ḃ b(x)u , (x) b f (x) + g (1.9) C(x, u) = ∂x noting that the input u appears. Consequently, we will seek to design a controller k : Rn → Rm that satisfies: ḃ C(x, k(x)) ≤ −αC(x), (1.10) 14 for some α ∈ R>0 , as such a controller will lead to the stability and safety properties defined above. The ability to find a controller satisfying this inequality will be the special property that defines the function C as a CLF or CBF in the settings of stability and safety, respectively. An optimization-based controller structure that will serve as a starting point for more complex robust control formulations that enforce this derivative inequality will be given by: (1.11) k(x) = argmin ∥u − knom (x)∥2 u∈Rm s.t. ḃ C(x, u) ≤ −αC(x), (1.12) where knom : Rn → Rm is some nominal controller. Importantly, this controller is specified by a convex quadratic program, ensuring that it can be efficiently implemented on real-world hardware platforms. Modeling Inaccuracies The modeling inaccuracies of model error, disturbances, measurement error, and input sampling will lead to a deviation from the nominal model in (1.7) in various ways. Instead, let us consider a nonlinear control-affine system: b(x)u + ∆(x, u, t), ẋ = b f (x) + g (1.13) with a function ∆ : Rn × Rm × R≥0 → Rn capturing a general perturbation to the nominal model in (1.7). Note that this general perturbation not only can depend on the state of the system, but it may depend on the input and time. Given a controller k : Rn → Rm , denote the closed-loop system: b(x)k(x) + ∆(x, k(x), t), ẋ = b f (x) + g (1.14) and assume that for any initial condition x0 ∈ Rn , there exists a unique continuously differentiable solution φ : R≥0 → Rn to this closed-loop system. The time derivative of C is given by:   ∂C ∂C b b(x)u + Ċ(x, u, t) = (x) f (x) + g (x)∆(x, u, t) . |∂x {z } |∂x {z } ḃ C(x,u) (1.15) δ(x,u,t) Observe that the perturbation ∆ affects the time derivative of C in the form of δ and will play a role in the ability to achieve stability and safety properties. 15 No Robustness The idea of robustness can be seen as living on a spectrum. At one end, we can forgo any effort in ensuring our control design is robust to ∆ by designing a controller that simply satisfies (1.10). Consequently, we have that: Ċ(x, k(x), t) ≤ −αC(x) + δ(x, k(x), t). (1.16) The presence of the term δ(x, k(x), t) without further assumptions will typically lead to a failure in rigorous stability and safety guarantees, such that one may have an unstable or unsafe system: Unstable Unsafe limt→∞ |C(φ(t))| = ∞ ∃ t ∈ R≥0 s.t. C(φ(t)) > 0 Table 1.2. General unstable and unsafe properties. Absolute Robustness At the opposite end of the spectrum is an absolute form of robustness, in which we will seek to design a controller k : Rn → Rm such that: Ċ(x, k(x), t) ≤ −αC(x), (1.17) and therefore we will recover the desired stability and safety properties by effectively eliminating the affect of δ on the time derivative of C. This perspective on robustness is akin to binary notion of robustness in linear systems theory [99], in which a system is either stable, or unstable. In this case, the nonlinear system is either stable (safe), or unstable (unsafe). As we will see in this thesis, this perspective on robustness typically leads to more complex controllers that account for the value of δ(x, k(x), t) when specifying k. Partial Robustness In between these two ends of the robustness spectrum is an approach that seeks to manage the degradation of stability or safety properties. In particular, control design is done to produce controllers that achieve: Ċ(x, k(x), t) ≤ −αC(x) + δ(x, k(x), t) ≤ −αC(x) + δ, (1.18) for some δ ∈ R≥0 . By achieving such a bound, one may establish the following partially robust stability and safety properties for a function R : R≥0 → R≥0 : 16 Partially Robust Stability Partially Robust Safety ∃ t∗ ∈ R≥0 s.t. |C(φ(t))| ≤ R(δ) ∀t ≥ t∗ C(φ(t)) ≤ R(δ) ∀t ∈ R≥0 Table 1.3. General partially robust stability and safety properties. An important property of the function R is that limδ→0+ R(δ) = 0, such that for small perturbation bounds δ, the degradation in stability and safety properties is small. This concept of robustness was first proposed in [56] in the form of Input-to-State Stability (ISS) and later as Input-to-State Safety (ISSf) in [58]. Instead of trying to completely eliminate δ(x, k(x), t), one tries to manage it to some value (δ) that is tolerable in terms of its impact on stability and safety properties. While I will explicitly use the technical definitions of ISS and ISSf throughout this thesis when addressing disturbances (for which they were originally posed), I will also make use of this idea of relaxing stability and safety guarantees in a amount proportional to a perturbation at a conceptual level throughout this thesis for settings outside of disturbances (such as residual learning error and input sampling effects). Section 2.7: Input-to-State Stability & Safety Sections 3.3-3.7: Learning CLFs & CBFs Sections 4.3-4.4: Tunable Input-to-State Safety Section 4.5: Stochastic Safety Section 5.4: Section 3.8: Preference-Based Learning Data-Driven Control Section 6.3: Section 3.9: Sampled-Data Stability Adaptive CBFs Section 6.5: Event-Triggered Safety No Robustness Section 6.4: Sampled-Data Safety Sections 5.2-5.3: Measurement-Robust CBFs Partial Robustness Absolute Robustness Figure 1.3. Spectrum of robustness properties of results presented in this thesis. Chapter 3: Model Error In Chapter 3 I will explore the problem of model error. In Sections 3.3-3.8 I will consider a general form of model error yielding the perturbation: b(x)) u, ∆(x, u) = f (x) − b f (x) + (g(x) − g | {z } | {z } e f (x) e(x) g (1.19) 17 with unknown functions f : Rn → Rn and g : Rn → Rn×m which capture the true dynamics of the system. These unknown functions lead to the model e : Rn → Rn×m . Importantly, the time derivative error terms e f : Rn → Rn and g of the function C will be given by: δ(x,u) z }| { ∂C ∂C (x)e f (x) + (x)e g(x) u . Ċ(x, u) = ḃ C(x, u) + ∂x | {z } |∂x {z } (1.20) a(x)⊤ b(x) As previously noted, neglecting the function δ can lead to unstable and unsafe behavior. Sections 3.3-3.6: Learning CLFs and CBFs The focus in Sections 3.3 (CLFs and stability) and 3.5 (CBFs and safety) will be attempting to learn the functions b and a using data-driven estimators, such that an estimator for δ can be constructed as follows: b u) = bb(x) + b δ(x, a(x)⊤ u ≈ b(x) + a(x)⊤ u, (1.21) with δb : Rn × Rm → R, bb : Rn → R, and b a : Rn → Rm . The control-affine form of this estimator is explicitly enforced in its construction to ensure that it can be integrated into a convex optimization-based controller: (1.22) k(x) = argmin ∥u − knom (x)∥2 u∈Rm C(x, u) + bb(x) + b a(x)⊤ u ≤ −αC(x). s.t. ḃ Challenges that arise due to non-independently and identically distributed training data will motivate constructing the estimator δb in an episodic manner in which data is collected, learning models are trained, a new controller is synthesized using the learning models, and the new controller is deployed to produce additional data that is aggregated with previous data. The learninginformed controllers will enforce that: δ ′ (x,k(x)) }| { z ⊤ b a(x)) k(x) . Ċ(x, k(x)) ≤ −αC(x) + b(x) − b(x) + (a(x) − b | {z } | {z } e b(x) (1.23) e(x)⊤ a Notably, residual learning error leads to the function δ ′ appearing, which will still degrade the properties of stability and safety. In Sections 3.4 (Projectionto-State Stability) and 3.6 (Projection-to-State Safety), I will establish theoretical stability and safety properties of the form: 18 Proposition 1 (Informal). For a sufficiently large (and problem dependent) set D ⊆ Rn , we have that:    ∃ t∗ ∈ R≥0 s.t.  Section 3.4 , |δ ′ (x, k(x))| ≤ δ ∀x ∈ D =⇒ |C(φ(t))| ≤ R(δ) ∀t ≥ t∗    Section 3.6 C(φ(t)) ≤ R(δ) ∀t ∈ R . ≥0 This informal result captures the fact that when using CLF and CBF-based controllers, residual learning error present after identifying the functions b and a with corresponding estimators leads to degradation in stability and safety properties that is proportional to the magnitude of residual learning error. Importantly, small residual learning errors do not lead to catastrophic failures in stability and safety. This is aligned with the notion of partial robustness described above. Section 3.7: Projected Disturbance Learning In Section 3.7 I will explore some of the challenges related to data richness that arise using the control-affine learning models in Sections 3.3-3.6. In particular, given a data point (xi , ui , Ċi ) ∈ Rn × Rm × R, there are infinitely many values of the estimators bb(xi ) and b a(xi ) that satisfy: C(xi , ui ) + bb(xi ) + b a(xi )⊤ ui , Ċi = ḃ (1.24) as the problem can be viewed as an underdetermined linear system. In the absence of a variety of inputs in the data set (which is typical of high-dimensional systems), this challenge can lead to poor estimators and a large residual learning error bounds δ that significantly degrade stability and safety properties. Instead, given a controller k : Rn → Rm , a purely state-based estimator can be constructed: b δ(x) ≈ b(x) + a(x)⊤ k(x). (1.25) This estimator is built to capture the difference between Ċ and ḃ C using the particular controller k. Importantly, it does not have the same underdetermined problem that appeared with the control-affine estimator and its accuracy can be evaluated when it is built. The estimator can subsequently be integrated into an optimization-based controller as follows: k′ (x) = argmin ∥u − knom (x)∥2 u∈Rm b s.t. ḃ C(x, u) + δ(x) ≤ −αC(x). (1.26) 19 A consequence of this state based approach is that we then have that: b + δ(x, k′ (x)), Ċ(x, k′ (x)) ≤ −αC(x) − δ(x) | {z } (1.27) δ ′ (x) such that a new perturbation δ ′ : Rn → R appears due to using the new controller k′ , while the estimator δb was built based on information from using the previous controller k. Despite this difference, the structure of the optimization-based controller k′ will enable the following result: Proposition 2 (Informal). If the estimator δb is an accurate estimator of the function δ(·, k(·)), then: ∥a(x)∥ ≤ ∂C (x)b g(x) ∂x b and δ(x) ≤ 0 =⇒ Ċ(x, k′ (x)) ≤ Ċ(x, k(x)). (1.28) The key part of this result is the first part of the precedent, which states that the model error affecting the input to the system is small compared to how the nominal model says input will affect the system. If this true, and the estimator δb built using the previous controller k indicates the system is not meeting the necessary inequality on Ċ at the state x (the second term in the precedent), the new controller k′ will improve upon the stability and safety of the system at the state x. Section 3.8: Data-Driven Control In Section 3.8 I will explore an alternative approach to resolving the issues with control-affine estimators by using robust convex optimization. I will construct a convex set U(x) ⊂ Rm+1 of all possible values of the model errors (b(x), a(x)) that are permissible with a data set: (b(x), a(x)) ∈ U(x). (1.29) These sets can be integrated into an optimization-based controller as: k(x) = argmin ∥u − knom (x)∥2 u∈Rm (1.30) s.t. ḃ C(x, u) + q + p⊤ u ≤ −αC(x), ∀(q, p) ∈ U(x). This controller has an infinite number of constraints (one for each (q, p) ∈ U(x)). Addressing this leads to the following result: 20 Proposition 3 (Informal). The controller k can be represented as a finitedimensional convex second-order cone program (SOCP) and under Lipschitz e, leads to: continuity assumptions of e f and g Ċ(x, k(x)) ≤ −αC(x). (1.31) This result state that is possible to tractably implement the robust controller, and by doing so, ensure that stability and safety requirements are satisfied for the true values of the model error (b(x), a(x)). I note that this aligns with the form of absolute robustness described above, where the original stability or safety requirement is still met in the presence of a perturbation. Section 3.9: Adaptive CBFs In Section 3.9 I consider a special form of parametric model error of the form: ∆(x) = F(x)θ ⋆ , (1.32) where F : Rn → Rn×p is a known collection of basis functions and θ ⋆ is an unknown vector of parameters. I note that this setting does not consider model error impacting how input affects the system. Motivated by the work studying adaptive CLFs in [47], I will consider an adaptive control approach for ensuring safety. In particular, I will explore controllers that maintain an estimate of b ∈ Rp , and update them according to an adaptation law: the parameters, θ b˙ = τ (x, θ), b θ (1.33) where τ : Rn × Rp → Rp is the adaptation law. This will result in the definition of adaptive Control Barrier Functions, which lead to optimizationbased controllers of the form: k(x) = argmin ∥u − knom (x)∥2 u∈Rm s.t. ḃ C(x, u) + (1.34) ∂C b ≤ 0, (x)F(x)θ ∂x b and a (stronger in this case) bound on that utilize the parameter estimate θ the time derivative of C. This type of adaptation law and controller lead to the following result: b0 ∈ Rp , there Proposition 4 (Informal). For any initial parameter estimate θ exists an update law (with potentially high adaptive gains) such that the controller k enforces: C(φ(t)) ≤ 0, ∀t ∈ R≥0 . (1.35) 21 This result states that it is possible to produce an adaptation law and controller that ensures a system remains safe in the presence of potentially large parametric model errors. Furthermore, this is an instance of absolute robustness, where a safety requirement is met despite the perturbation ∆. Chapter 4: Disturbances In Chapter 4 I will explore the problem of disturbances. In Sections 4.2-4.4 I will consider a perturbation of the form: ∆(x, t) = g(x)d1 (t) + d2 (t), (1.36) with matched disturbance signal d1 : R≥0 → Rm and unmatched disturbance signal d2 : R≥0 → Rn . The difference between matched and unmatched disturbances will matter in transferring CLFs and CBFs for systems without disturbances to systems with disturbances. Both types of disturbance signal can capture a very large class of unstructured perturbations to a system, and are generally assumed not to have any known underlying structure that might be identifiable with data. The time derivative of the function C is given by: δ(x,t) }| { z ∂C (x) (g(x)d1 (t) + d2 (t)) . Ċ(x, u, t) = ḃ C(x, u) + ∂x (1.37) Once again, ignoring δ can lead to unstable and unsafe behavior. Section 4.2 Limits of Input-to-State Safety In Section 4.2 I will explore some of the limitations of the partial robustness notion of Input-to-State Safety (ISSf). In particular, ISSf-based design focuses on producing a controller k : Rn × R>0 → Rm such that: Ċ(x, k(x, ϵ), t) ≤ −αC(x) + δ(ϵ), (1.38) where ϵ ∈ R>0 is a robustness parameter that determines the size of the perturbation δ(ϵ) related to disturbances in the time derivative of Ċ. As ISSf is a notion of partial robustness, this leads to the outcome that: C(φ(t)) ≤ R(δ(ϵ)), ∀t ∈ R≥0 . (1.39) In this way, it is possible to change the degradation of safety guarantees through the parameter ϵ. 22 In practice, this design approach struggles to balance performance and theoretical safety guarantees. In particular, for small values of ϵ, one achieves a strong theoretical statement about safety, such that R(δ(ϵ)) ≈ 0, but the closed-loop system displays extremely conservative behavior, such that C(φ(t)) << 0 for all t ∈ R≥0 . For large values of ϵ, one does not have that C(φ(t)) << 0, such that performance is recovered, but instead vacuous safety guarantees occur as the value of R(δ(ϵ)) can be quite large. Sections 4.3 and 4.4: Tunable Input-to-State Safety In Sections 4.3 and 4.4 the limits of ISSf will be approached by viewing the robustness parameter ϵ as a function of the value of C(x). This will allow prioritizing robustness when the system is close to unsafe (ϵ(C(x)) is small when C(x) ≈ 0) and relaxing robustness when the system is very safe (ϵ(C(x)) is large when C(x) << 0). In particular, a bound of the from: Ċ(x, k(x, ϵ(C(x))), t) ≤ −αC(x) + δ(ϵ(C(x))), (1.40) will be achieved. Theoretical safety properties with this modification will be encoded by Tunable Input-to-State Safety, leading to a result of the form: Proposition 5 (Informal). For a monotonically decreasing function ϵ : R → R>0 , the controller k : Rn ×R>0 → Rm using a parameter function ϵ : R → R>0 leads to a safety result of the form: C(φ(t)) ≤ R(δ(ϵ(C(φ(t))))) ≈ 0, (1.41) without leading to C(φ(t)) << 0 for all t ∈ R≥0 . This result states that Tunable Input-to-State Safety allows for tuning controllers to achieve strong theoretical safety properties (the right-hand side of the preceding equation is approximately 0) while not requiring that the system display excessive degrees of conservativeness in maintaining safety. This makes Tunable Input-to-State Safety appealing for designing controllers that enforce partial robustness as it also allows for strong performance. Section 4.5: Stochastic Disturbances In Section 4.5 I will consider safety of discrete-time systems of the form: xk+1 = F(xk , uk ) + dk , k ∈ Z≥0 , (1.42) 23 where F : Rn × Rm → Rn and dk is a stochastic disturbance signal that is distributed according to some distribution D, such that dk ∼ D. Given a controller k : Rn → Rm , one may consider a closed-loop system: xk+1 = F(xk , k(xk )) + dk , k ∈ Z≥0 , (1.43) which is assumed to have a solution φ : Z≥0 → Rn for any initial condition x0 ∈ Rn and sufficiently well-behaved sequence of random disturbances. The potential for unbounded disturbance signals (such as disturbances drawn from a Gaussian distribution) will motivate the need to consider a notion of finitetime safety, in which an inequality on C(φ(k)) will only need to be met for some finite values of k with some probability. Pursuing this will lead to a result of the following form: Proposition 6 (Informal). If a controller k : Rn → Rm leads to a closed-loop system satisfying: k ∈ Z≥0 , (1.44) P[C(φ(k)) ≤ R(δ(D)) for all k ≤ K] ≥ 1 − ϵ(D, K). (1.45) E[C(xk+1 )] − C(xk ) ≤ −αC(xk ) + δ(D), then we have that for any K ∈ Z≥0 : This result states that if a bound on the expected increment of the value of C can be achieved, the probability of meeting a given safety requirement (R(δ(D))) for a finite number (K) of time-steps can be lower-bounded. This is a form of partial robustness, as it allows the value of C to increase above 0, and only holds over a finite-time horizon. This work will further explore how such a controller k can be designed using convex optimization, and what additional modifications must occur to enforce the desired expectation condition. Chapter 5: Measurement Error In Chapter 5 I will explore the problem of measurement error. In Sections 5.2-5.4 I will consider a perturbation of the form: ∆(x, u) = f (x + e(x)) − f (x) + (g(x + e(x)) − g(x))u, (1.46) where e : Rn → Rn is an unknown function capturing error in the estimate of the state used by a controller. This perturbation can be seen as replacing the 24 dynamics of the system with the dynamics of the system at the state x + e(x), but the controller only has access to the state x. Sections 5.2 and 5.3: Measurement-Robust CBFs The focus in Sections 5.2 and 5.3 is on designing safety-critical controllers in the presence of measurement error. To accomplish this, this setting will be based on access to a state-based measurement y = p(x) ∈ Rp . This measurement may be low-dimensional and directly include parts of the state, or it may be high-dimensional, such as an image, and need to be converted into a low-dimensional representation of the system state. Consequently, it will be assumed that given a measurement y, an estimate of the state x can be produced (though the state dynamics still use x + e(x)). It will be assumed that accompanying the measurement y is an error set E(y), such that it is known that: e(x) ∈ E(y). (1.47) This assumption states that given a measurement y, we can form an estimate of the state x and know that the error e(x) is restricted to the set E(y). It will additionally be assumed that the error set satisfies the following property: sup ∥e∥ ≤ ϵ(y), (1.48) e∈E(y) for a known function ϵ : Rp → R≥0 , such that it is known that ∥e(x)∥ ≤ ϵ(y). This construction will lead to the following result: Proposition 7 (Informal). There exists minimum values of the parameters a, b ∈ R≥0 such that for all larger values, the convex second-order cone program-based controller: k(x, y) = argmin ∥u − knom (x)∥2 u∈Rm (1.49) s.t. ḃ C(x, u) + (b + a∥u∥)ϵ(y) ≤ −αC(x), enforces: C(φ(t)) ≤ 0, t ∈ R≥0 . (1.50) This results states that by incorporating the bound ϵ(y) on the measurement error into the controller design, it is possible to meet the safety requirement without needing to access e(x). This is a form of absolute robustness, as the 25 original safety requirement is met despite the perturbation. Furthermore, this controller is parameterized (in terms of a and b), and there is minimal values of these parameters beyond which robust guarantees are achieved. Section 5.4: Preference-Based Learning The disturbance and measurement-robust controllers explored in Chapters 4 and 5 are often defined using a collection of robustness parameters (ϵ for disturbance robustness and a, b for measurement robustness). As multiple forms of robustness are built into a controller, it often leads to excessively conservative behavior as the system is robust to more than it needs to be at any given point in time. Tuning the robustness parameters is often a tedious task, and control designers often have a limited understanding of how the parameters interact to balance performance and robustness. Motivated by this challenge, Section 5.4 explores how feedback on performance and robust safety from a human control designer can be used to automate the process of tuning control gains using preference-based learning. In particular, I will consider controllers of the form: k(x) = argmin ∥u − knom (x)∥2 u∈Rm (1.51) s.t. ḃ C(x, u) + ϕ(x, u, θ) ≤ −αC(x), with a robustness function ϕ : Rn × Rm × Rp → R≥0 capturing multiple forms of robustness (disturbance and measurement error) and robustness parameters θ ∈ Rp . The robustness function ϕ will be constructed using the formulations for disturbance and measurement robustness to ensure this controller is defined by a convex optimization problem. This controller construction will lead to the following observation: Proposition 8 (Informal). Utilizing human feedback and preferences about closed-loop performance and safety, preference-based learning can be successfully used to iteratively tune the parameters θ to achieve safety of the form: C(φ(t)) ≤ R(δ(θ)) ≈ 0, (1.52) without requiring that C(φ(t)) << 0. This result states that it is possible to automate the process of tuning robustness parameters using feedback about a control designer’s qualitative observations about performance and safety properties to achieve strong safety 26 results without requiring that the system display excessive degrees of conservativeness. This result, while more algorithmic and experimental in nature, is promising as it may be deployed with many of the robust control approaches in this thesis which may be parameterized. Chapter 6: Input Sampling In Chapter 6 I will explore the problem of input sampling. In Sections 6.3-6.5 I will consider a perturbation of the form: ∆(x, u, t) = −g(x)u + g(x)u(tk ), ∀t ∈ [tk , tk+1 ), (1.53) where tk , tk+1 ∈ R≥0 are sequential sample times with tk < tk+1 , and u(tk ) is the input applied to the system at time tk . I note that this type of perturbation requires more care in modeling in this continuous-time formulation as multiple points in time must be considered, but the approach taken in this thesis will actually not begin its investigation by considering input-sampling as a perturbation to a continuous-time system. Sections 6.3 and 6.4: Approximate Discrete-Time Model Design Instead of viewing input sampling from a continuous-time perspective, I will view it from a discrete-time perspective. Given a sample period T ∈ R>0 separating sample times (tk+1 − tk = T ), the evolution of a sample-and-hold system can be described by: Z tk+1 e f (φ(τ )) + g(φ(τ ))uk dτ, (1.54) xk+1 = FT (xk , uk ) = xk + tk where uk is the input taken at sample time tk , and φ is the continuous-time solution of the system under the sample-and-hold input. The function FeT is known as the exact discrete-time map, and is generally difficult to compute due to requiring an integral of general nonlinear functions. Motivated by the work in [81], I will consider design using an approximation FaT : Rn ×Rm → Rn : FaT (xk , uk ) ≈ FeT (xk , uk ). (1.55) In order to achieve theoretical stability and safety properties, the approximation will need to satisfy a one-step consistency property defined as: ∥FeT (xk , uk ) − FaT (xk , uk )∥ ≤ T ρ(T ), (1.56) for sufficiently small sample periods T , where ρ : R≥0 → R≥0 is monotonically increasing and satisfies ρ(0) = 0. This property dictates the accuracy required 27 by the approximation, stating that it must more accurately represent the exact discrete-time map for smaller sample periods. I will demonstrate that standard approximation schemes such as Euler approximations and Runge-Kutta approximations satisfy this property, and thus can be used in control design while preserving stability and safety properties. In particular, I will develop results of the form: Proposition 9 (Informal). If the approximate discrete-time map FaT is onestep consistent with the exact discrete-time map FeT , then for controllers of the form: kT (xk ) = argmin ∥u − knom (xk )∥2 u∈Rm (1.57) s.t. C(FaT (xk , u)) − C(xk ) ≤ −T αC(xk ), we have stability properties of the form: lim |C(φ(tk ))| ≤ R(T ), k→∞ (1.58) and safety properties of the form: C(φ(tk )) ≤ R(T ), (1.59) for all k ∈ Z≥0 . There are a number of interesting observations about this result. First observe that the controller considers a finite difference of the value of C across one sample period using an approximate discrete-time map, and the inequality with the difference scales with the sample period. This differs from the continuoustime formulations used in the previous chapters. Next, the stability and safety properties are of a partial robustness form, as the violation of stability and safety scales with the sample period, and can be made arbitrarily small with sufficiently small sample periods. Lastly, the stability and safety guarantees hold at the sample-times, and do not make statements about inter-sample behavior (this can be done with additional analysis of this framework if so desired). The unique variations in these controllers and properties arise because rather than trying to view input sampling as a perturbation in continuous-time dynamics, a completely different paradigm of design is taken using approximations of discrete-time systems. Lastly, a key focus in this work will be in 28 ensuring that the controller in the preceding proposition is specified by a convex optimization problem, and I will observe that different classes of convex optimization problems arise due to the class of approximation and geometry of stability and safety constraints, which is not seen in continuous-time. Section 6.5: Event-Triggered Safety In Section 6.5 I will explore a different paradigm of input-sampling known as event-triggered control. In this setting, input-sampling is actually introduced to a continuous-time system (it is assumed the controller could be implemented continuously in time) in an effort to conserve actuation resources. In particular, this problem setting is concerned with minimizing costs that happen due to switching the input to the system. The goal in event-triggered control will be to determine a trigger-law that dictates how sampling times are specified: tk+1 = min{t ≥ tk | ρ(∥x − x(tk )∥) = 0}, (1.60) where ρ : R≥0 → R is defined to ensure that: Ċ(x(t), uk ) ≤ −cαC(x(t)), (1.61) where x(t) is the continuous-time evolution of the state at a time t ≥ tk using the constant input uk , and c ∈ (0, 1). This states that a continuous-time controller implementation that meets the original inequality on Ċ is implemented with a sample-and-hold implementation that is updated to meet a relaxed inequality. In this sense, inherent robustness in the continuous-time controller is being traded for more efficient use of actuation resources. The burden of proof in this setting is to ensure the existence of a minimum interevent time (MIET), such that the trigger law leads to the existence of some τ ∈ R>0 such that tk+1 ≥ tk + τ . This is not only important for ensuring there is some efficiency benefit to an event-triggered implementation, but it also is important for eliminating undesirable Zeno-like behavior, where infinitely many events occur in a finite amount of time. Exploring this challenge will lead to the following result: Proposition 10 (Informal). There exists a function ρ : R≥0 → R such that the corresponding trigger-law leads to a minimum interevent time and a safety guarantee of the form: C(φ(t)) ≤ ϵ, (1.62) 29 where ϵ ∈ R>0 may be taken arbitrarily small at the expense of a smaller minimum interevent time. This result states that it is possible to use event-triggered implementations of CBF-based controllers and satisfy a partial robustness property that can be tuned to be an arbitrarily small degradation of the original safety property. The cost for making this degradation smaller is a potentially smaller minimum interevent time, which may degrade resource efficiency. The key challenge that is addressed in this result is regulating the behavior of the system when it is on the boundary between safe and unsafe states (C(x) = 0, where motion along the boundary can lead to a failure to have an MIET. 30 Chapter 2 BACKGROUND In this chapter I will present various background materials in nonlinear control systems that underlie my contributions presented later in this thesis. I will begin by reviewing nonlinear dynamics and the properties of stability and safety that are often an objective of control design. I next review Control Lyapunov Functions (CLFs) as a tool for stabilizing control synthesis, followed by feedback linearization as a constructive tool for synthesizing both stabilizing controllers and CLFs. I then proceed to review Control Barrier Functions (CBFs) as a tool for safe control synthesis. Following this, I review nonlinear dynamics with disturbances, the robustness properties of Input-to-State Stability (ISS) and Input-to-State Safety (ISSf), and corresponding variants of CLFs and CBFs for achieving these properties. 2.1 Nonlinear Dynamics Let E ⊆ Rn be an open set and consider a nonlinear control-affine system: ẋ = f (x) + g(x)u, (2.1) with state x ∈ E, input u ∈ Rm , and functions f : E → Rn and g : E → Rn×m assumed to be locally Lipschitz continuous on E. This system is referred to as an open-loop system, as the input u is unspecified. Let k : E → Rm be a controller that is locally Lipschitz continuous on E. This controller yields the following closed-loop system: ẋ = f (x) + g(x)k(x). (2.2) As the functions f , g, and k are locally Lipschitz continuous on E, for any initial condition x0 ∈ E there exists a maximal time interval I(x0 ) = [0, tmax (x0 )) and a unique continuously differentiable solution φ : I(x0 ) → E satisfying: φ̇(t) = f (φ(t)) + g(φ(t))k(φ(t)), (2.3) φ(0) = x0 , (2.4) for all t ∈ I(x0 ) [100], [101]. 31 A specific set of nonlinear dynamics that I will consider in various contributions in this thesis are robotic system dynamics. The state of a robotic system will be described by a set of nc generalized coordinates q ∈ Q ⊆ Rnc , where Q is referred to as the configuration space, and a set of nc generalized coordinate h i⊤ nc ⊤ ⊤ rates q̇ ∈ R , yielding the state x ≜ q q̇ ∈ Q × Rnc ≜ X ⊆ Rn . The dynamics of a robotic system are governed by the Euler-Lagrange equations: D(q)q̈ + C(q, q̇) + G(q) = B(q)τ , (2.5) c , Coriolis matrix C : X → with positive definite inertia matrix D : Q → Sn≻0 nc nc R , gravitational matrix G : Q → R , actuation matrix B : Q → Rnc ×m , and input torques τ ∈ Rm . The functions D, G, and B are assumed to be locally Lipschitz continuous on Q and the function C is assumed to be locally Lipschitz continuous on X . These dynamics can be written in the form of (2.1) as follows: # " # " # " q̇ 0nc d q = + τ , (2.6) dt q̇ D(q)−1 B(q) |{z} −D(q)−1 (C(q, q̇) + G(q)) | {z } | {z } | {z } u ẋ f (x) g(x) noting that the functions f and g are locally Lipschitz continuous on X as the matrix inverse is a locally Lipschitz continuous function [102, Theorem 9.8]. Lastly, a robotic system is said to be fully-actuated if m = nc and the matrix B(q) is full-rank for each q ∈ Q. 2.2 Stability & Safety The concept of stability has long been a focus in studying dynamic systems [103], and is often an important objective in control design. Stability properties of the closed-loop system (2.2) are defined with respect to an equilibrium point, which is captured in the following definition: Definition 1 (Equilibrium Point). A point xe ∈ E is said to be an unforced equilibrium point of the open-loop system (2.1) if: f (xe ) = 0n . (2.7) A point xe ∈ E is said to be a forced equilibrium point of the open-loop system (2.1) if there exists a ue ∈ Rm such that: f (xe ) + g(xe )ue = 0n . (2.8) 32 A point xe ∈ E is said to be an equilibrium point of the closed-loop system (2.2) if: f (xe ) + g(xe )k(xe ) = 0n . (2.9) This enables the following definition of exponential stability [100]: Definition 2 (Exponential Stability). Let xe ∈ E be an equilibrium point of the closed-loop system (2.2). The closed-loop system (2.2) is said to be locally exponentially stable with respect to xe if there exist an open set D ⊆ E with xe ∈ D and M, λ ∈ R>0 such that: x0 ∈ D =⇒ = ∥φ(t) − xe ∥ ≤ M ∥x0 − xe ∥e−λt (2.10) for all t ∈ R≥0 . Note the requirement for local exponential stability that the unique continuously differentiable solution φ exists for all time. If it is possible to find an open set D′ ⊆ E with xe ∈ D′ such that: x0 ∈ D′ =⇒ = ∥φ(t) − xe ∥ ≤ M ∥x0 − xe ∥e−λt . (2.11) for all t ∈ I(x0 ), then an open set D ⊆ D′ with xe ∈ D and a compact set K with D ⊂ K ⊂ D′ can be found such that for any initial condition x0 ∈ D, φ(t) ∈ K for all t ∈ I(x0 ), and thus I(x0 ) = R≥0 . [101, Chapter 2.4, Corollary 2]. This is a useful requirement in the definition of stability as it establishes a well-posedness when discussing the evolution of the system (2.2) for all time. The notion of safety I will consider for nonlinear dynamic systems is codified through the concept of forward invariance [104]: Definition 3 (Forward Invariance & Safety). A set C ⊆ E is said to be forward invariant for the closed-loop system (2.2) if for any initial condition x0 ∈ C, we have that φ(t) ∈ C for all t ∈ I(x0 ). If a set C is forward for the closed-loop system (2.2), then the closed-loop system (2.2) is said to be safe with respect to C. If a closed-loop system (2.2) is safe with respect to a compact set C, then I(x0 ) = R≥0 for any initial condition x0 ∈ C [101, Chapter 2.4, Corollary 2], and thus φ(t) ∈ C for all t ∈ R≥0 . 33 2.3 Control Lyapunov Functions Lyapunov theory is a fundamental tool for both certifying stability properties of nonlinear systems and constructively synthesizing stabilizing controllers. I will begin by reviewing exponential Lyapunov functions and their connection with exponential stability [100]: Definition 4 (Exponential Lyapunov Function). Let xe ∈ E be an equilibrium point of the closed-loop system (2.2). A function V : E → R≥0 that is continuously differentiable on E is said to be a local exponential Lyapunov function for the closed-loop system (2.2) and equilibrium point xe if there exist an open set D ⊆ E with xe ∈ D and k1 , k2 , k3 , a ∈ R>0 such that: k1 ∥x − xe ∥a ≤ V (x) ≤ k2 ∥x − xe ∥a , ∂V ∂V V̇ (x) ≜ (x)f (x) + (x)g(x) k(x) ≤ − k3 ∥x − xe ∥a , ∂x ∂x | {z } | {z } Lf V (x) (2.12) (2.13) Lg V (x) for all x ∈ D. Exponential Lyapunov functions enable the certification of exponential stability, as codified in the following theorem [100, Theorem 4.10]: Theorem 1. Let xe ∈ E be an equilibrium point of the closed-loop system (2.2). If there exists a local exponential Lyapunov function V : E → R≥0 for the closed-loop system (2.2) and equilibrium point xe , then the closed-loop system (2.2) is locally exponentially stable with respect to xe . A converse result to this preceding theorem establishes the existence of a local exponential Lyapunov function for a system that is locally exponentially stable under mild regularity assumptions [100, Theorem 4.14]: Theorem 2. Let xe ∈ E be an equilibrium point of the closed-loop system (2.2), and suppose that the functions f , g, and k are continuously differentiable on E. If the closed-loop system (2.2) is locally exponentially stable with respect to xe , then there exists a local exponential Lyapunov function V : E → R≥0 for the closed-loop system (2.2) and equilibrium point xe with a = 2 and corresponding open set D ⊆ E. Furthermore, there exists a k4 ∈ R>0 such that: ∂V (x) ≤ k4 ∥x − xe ∥, ∂x for all x ∈ D. (2.14) 34 A result arising from the existence of a local exponential Lyapunov function pertains to the existence and uniqueness of solutions when a controller is not locally Lipschitz continuous at the equilibrium point xe [105, Theorem 1.2]: Theorem 3. Let xe ∈ E be an equilibrium point of the closed-loop system (2.2), and suppose that the function k : E → Rm is continuous on E, but only locally Lipschitz continuous on E \ {xe }. If there exists a local exponential Lyapunov function V : E → R≥0 for the closed-loop system (2.2) and equilibrium point xe , then for the initial condition x0 = xe , there exists a unique continuously differentiable solution φ : R≥0 → E to the closed-loop system (2.2) given by φ(t) = xe for all t ∈ R≥0 . This result states that if a controller is not locally Lipschitz continuous at the equilibrium point xe , but a local exponential Lyapunov function exists for the closed-loop system (2.2) and equilibrium point xe , then an issue with nonuniqueness of solutions for the closed-loop system (2.2) does not arise from the initial condition xe . This precludes the pedagogical closed-loop system: 2 ẋ = f (x) = x 3 , (2.15) which is not locally Lipschitz continuous at the equilibrium point xe = 0, and has two solutions from the initial condition x0 = 0 given by φ1 (t) = 0 and
3 φ2 (t) = 3t for all t ∈ R≥0 . It is straightforward to see this system is not locally exponentially stable (it is unstable), and thus we could not find a local exponential Lyapunov function V that would certify that φ1 (t) = 0 for all t ∈ R≥0 is the unique solution from x0 = 0. This further shows how stability and Lyapunov functions play a role in ensuring the existence and uniqueness of solutions to the differential equations describing the system dynamics. A powerful tool for jointly synthesizing both an exponentially stabilizing controller and a corresponding local exponential Lyapunov function as a certificate of that exponential stability is the notion of a local exponential Control Lyapunov Function (CLF). CLFs were first introduced in [8] and were subsequently used to constructively synthesize stabilizing controllers in [9] and [106]. Their use in synthesizing exponentially stabilizing controllers through convex optimization was first considered in [26], [107]. I will explore how CLFs can be synthesized using feedback linearization in Section 2.4, but other recent work has synthesized CLFs using proportional-derivative (PD) control [108]. 35 Definition 5 (Control Lyapunov Function (CLF)). Let xe ∈ E be an unforced or forced equilibrium point of the open-loop system (2.1). A function V : E → R≥0 that is continuously differentiable on E is said to be a local exponential Control Lyapunov Function (CLF) for the open-loop system (2.1) and equilibrium point xe if there exist an open set D ⊆ E with xe ∈ D, and k1 , k2 , k3 , a ∈ R>0 such that: k1 ∥x − xe ∥a ≤ V (x) ≤ k2 ∥x − xe ∥a , inf V̇ (x′ , u) ≜ Lf V (x′ ) + Lg V (x′ )u < − k3 ∥x′ − xe ∥a , u∈Rm (2.16) (2.17) for all x ∈ D and all x′ ∈ D \ {xe }. There are a few notable observations to make about the definition of a local exponential CLF. First, I note that a local exponential CLF is constructed with respect to the open-loop system (2.1), rather than the closed-loop system (2.2). Because of this, local exponential CLFs serve as a statement about how a dynamic system could be stabilized. Defining stability properties still requires specifying a controller k and producing a closed-loop system. This distinction will be important in Chapter 3 when understanding the well-posedness of addressing model error using local exponential CLFs. Second, I note the strict inequality used in (2.17). This strict inequality is consistent with the original definition of CLFs in [8, Theorem 4.1], and the more recent definition in [51, Definition 1]. The importance of this strict inequality is two-fold. First, the infimum in (2.17) may not be attained, and thus if the inequality was non-strict, it may be impossible to produce a controller that meets the exponential Lyapunov condition in (2.13) (for a particular value of k3 ). Second, the strictness of this inequality is important when establishing local Lipschitz continuity of controllers synthesized via local exponential CLFs, as seen in [51, Theorem 1]. Furthermore, this strict inequality contrasts with the non-strict inequality appearing in (2.13). Satisfying the non-strict inequality is sufficient for ensuring stability of a closed-loop system, while the importance of the strict inequality is in synthesizing controllers to go from an open-loop system to a closed-loop system that satisfies the non-strict inequality. Lastly, the positive definiteness of a local exponential CLF as enforced by (2.16) requires that the gradient of the local exponential CLF with respect to the state vanishes at the equilibrium point xe , implying that V̇ (xe , u) = 0 for all u ∈ Rm . Thus, the strict inequality in (2.13) cannot be enforced at this 36 point, and thus it is omitted from consideration. This omission is acceptable because any controller k : E → Rm satisfies V̇ (xe , k(xe )) = 0, meeting the condition (2.13) for stability at the point xe . Third, I note that the infimum condition in (2.17) can be equivalently rewritten as the following implication: Lg V (x′ ) = 0m =⇒ Lf V (x′ ) < −k3 ∥x′ − xe ∥a , (2.18) for all x′ ∈ D \ {xe }. This alternative perspective provides a geometrical interpretation of the local exponential CLF condition (2.17). In the set of states which the time derivative of the exponential CLF can not be manipulated through the choice of control input (x′ satisfies the precedent), the unactuated dynamics of the system (given by f ) must display the necessary stability properties (the antecedent). This interpretation will be useful later in Section 3.8 for proving feasibility results for optimization-based controllers synthesized via local exponential CLFs in the presence of model error. The utility of local exponential CLFs is in their characterization of exponentially stabilizing inputs and subsequent role as local exponential Lyapunov functions. Given a local exponential CLF for the open-loop system (2.1) and equilibrium point xe , define the following pointwise set: n o m a KCLF (x) = u ∈ R | V̇ (x, u) ≤ −k3 ∥x − xe ∥ . (2.19) This set leads to the following result [62], [109]: Theorem 4. Let xe ∈ E be an unforced or forced equilibrium point of the open-loop system (2.1), and let V : E → R≥0 be a local exponential CLF for the open-loop system (2.1) and equilibrium point xe with corresponding open set D ⊆ E. Then, the set KCLF (x) is non-empty for all x ∈ D, and for any controller1 k : D → Rm that is continuous on D, locally Lipschitz continuous on D\{xe }, renders xe an equilibrium point of the closed-loop system (2.2), and satisfies k(x) ∈ KCLF (x) for all x ∈ D, the function V is a local exponential Lyapunov function for the closed-loop system (2.2) and equilibrium point xe . 1 The feedback linearization and optimization-based controllers I will work with in this thesis will often not be naturally defined outside of an open set D ⊂ E, nor often can they be extended beyond D while meeting regularity properties on all of E. When working with these controllers it is natural to restrict the notion of the solutions to the closed-loop system (2.2) to initial conditions x0 ∈ D and solutions φ : I(x0 ) → D. Note that D is not the set typically referred to as the “region of attraction” but rather contains the region of attraction. 37 This result characterizes requirements on the controller k for which the exponential stability of (2.2) with respect to the equilibrium point xe can be certified via the local exponential CLF V serving as a local exponential Lyapunov function. Producing such a controller meeting the required regularity properties was first considered in [9], wherein the notion of the small control property was proposed for unforced equilibrium points, and generalized in [106, Definition 4.6] to forced equilibrium points in the form of the continuous control property: Definition 6 (Small Control Property (SCP)). Let xe ∈ E be an unforced equilibrium point of the open-loop system (2.1), and let V : E → R≥0 be a local exponential CLF for the open-loop system (2.1) and equilibrium point xe . The local exponential CLF V is said to satisfy the small control property (SCP) if for any ε ∈ R>0 , there exists an open set D ⊆ E with xe ∈ D such that for any x ∈ D \ {xe }, there exists a u ∈ Rm satisfying ∥u∥ < ε and: V̇ (x, u) < −k3 ∥x − xe ∥a . (2.20) Definition 7 (Continuous Control Property (CCP)). Let xe ∈ E be a forced equilibrium point of the open-loop system (2.1) for equilibrium input ue , and let V : E → R≥0 be a local exponential CLF for the open-loop system (2.1) and equilibrium point xe . The local exponential CLF V is said to satisfy the continuous control property (CCP) if for any ε ∈ R>0 , there exists an open set D ⊆ Rn with xe ∈ D such that for any x ∈ D \ {xe }, there exists a u ∈ Rm satisfying ∥u − ue ∥ < ε and: V̇ (x, u) < −k3 ∥x − xe ∥a . (2.21) These properties provide a statement about the behavior of a local exponential CLF V as it approaches the equilibrium point xe . To highlight the importance of the SCP in achieving the necessary regularity properties on a local exponential CLF-based controller, consider the following simple example: Example 1. Consider the simple scalar open-loop system: ẋ = x + x2 u. (2.22) The origin of this system is an unforced equilibrium point (xe = 0), and the quadratic function V (x) = 21 x2 is a local exponential CLF for the open-loop 38 system (2.22) and equilibrium point xe , as we have that: inf V̇ (x, u) = inf x2 + x3 u = −∞ u∈R u∈R (2.23) for all x ∈ R \ {0}. The exponential CLF V does not satisfy the SCP. To see this, consider an x ∈ R>0 , and observe that the SCP condition (2.20) requires that there exists a u ∈ Rm satisfying: x2 + x3 u < −k3 xa ⇐⇒ u < −k 3 xa−3 − x−1 . (2.24) As x approaches 0 from the right, the input u must become unbounded in the negative direction, and thus can not be made to satisfy |u| < ε for arbitrarily small ε ∈ R>0 in some open set containing 0. Intuitively, we observe that in the absence of input, the system (2.22) is an unstable linear system. Furthermore, its input is multiplied by a quadratic term of the state. Thus, as x approaches the origin, the magnitude of the input must become unbounded to compensate for the faster decay of the quadratic term compared to the unstable linear term. As I will discuss when exploring feedback linearization in the next section, establishing that a local exponential CLF meets the small or continuous control property is often most easily done by constructing the local exponential CLF through a controller k that is continuous at xe and for which xe is an equilibrium point of the closed-loop system (2.2). Let xe ∈ E be an unforced (ue = 0m ) or forced equilibrium point with equilibrium input ue for the open-loop system (2.1), and let V be a local exponential CLF for the open-loop system (2.1) and equilibrium point xe with corresponding open set D ⊆ E. Consider the following optimization-based controller kCLF : D → Rm , first introduced in [26], which is synthesized directly using the local exponential CLF V : kCLF (x) = argmin ∥u − ue ∥2 u∈Rm (CLF-QP) s.t. V̇ (x, u) ≤ −k3 ∥x − xe ∥a . I make the following observations regarding this controller. First, for x = xe , the constraint of the optimization problem is trivially satisfied for any value of u, and thus the input that minimizes the cost function is u = ue , which renders xe an equilibrium point of the closed-loop system (2.2). Second, the cost function of the optimization problem defining the constraint is convex 39 with respect to the input, and the function V̇ is affine in the input, making this optimization problem a convex quadratic program (QP). The minimizing solutions of these problems are well understood (there is a unique solution to this particular optimization problem as the cost is strictly convex with respect to the input), and they may be efficiently solved numerically [29], [110], permitting their use on real-world systems such as bipedal walking platforms [24], [107]. This optimization-based control framework will serve as a cornerstone in the contributions of this thesis. Lastly, by construction, the resulting closed-loop system (2.2) satisfies the local exponential Lyapunov function condition (2.13), such that the closed-loop system is locally exponentially stable by design. This is codified in the following result [51, Theorem 1]: Theorem 5. Let xe ∈ E be an unforced (forced) equilibrium point of the open-loop system (2.1) with equilibrium input ue , let V : E → R≥0 be a local exponential CLF for the open-loop system (2.1) and equilibrium point xe with corresponding open set D ⊆ E, let the function ∂V : E → Rn be locally ∂x Lipschitz continuous on D, and assume that V satisfies the small (continuous) control property. Then the controller kCLF : D → Rm is continuous on D, locally Lipschitz continuous on D \ {xe }, renders xe an equilibrium point of the closed-loop system (2.2), and satisfies kCLF (x) ∈ KCLF (x) for all x ∈ D. This result states that the CLF-QP controller meets the conditions of Theorem 4, and thus locally exponentially stabilizes the closed-loop system (2.2) with respect to the equilibrium point xe . 2.4 Feedback Linearization Feedback linearization is a geometric approach for synthesizing stabilizing controllers for a special class of nonlinear systems [111], [112]. A feedback linearizing controller eliminates the nonlinear components in a subset of the system dynamics, and replaces them with stable linear dynamics. While eliminating nonlinear dynamics through control may result in less efficient controllers [109], be fragile to model uncertainty [113], and induce unstable internal dynamics [114], feedback linearization is an important constructive tool in nonlinear control design as it enables the synthesis of local exponential CLFs with which efficient and robust control can be performed. My review of feedback linearization will focus on only the elements necessary to support the contributions of this thesis, and thus will begin from an advanced starting point rather than 40 the usual starting point of defining a relative degree, but I note that a thorough introduction to the material can be found in [111]. Following this definition, I will provide a concrete example in the form of a fully-actuated robotic system. I now provide the following definition of feedback linearizability [115]: Definition 8 (Feedback Linearizability). Let xe ∈ E be an unforced or forced equilibrium point of the open-loop system (2.1) with equilibrium input ue . The open-loop system (2.1) is said to be locally feedback linearizable with respect to the equilibrium point xe if there exists an open set D ⊆ E with xe ∈ D, a function Φ : D → Rn that is a diffeomorphism2 between D and Φ(D) satisfying Φ(xe ) = 0n , constants γ, k ∈ Z>0 satisfying γ ≤ n and k ≤ m, a controller kfbl : D × Rk → Rm that is locally Lipschitz continuous on D × Rk and satisfies kfbl (xe , 0k ) = ue , a controllable pair (A, B) ∈ Rγ×γ × Rγ×k , and functions fη : Φ(D) → Rγ , gη : Φ(D) → Rγ×m and ω : Φ(D) → Rn−γ that are continuously differentiable3 on Φ(D) such that: 1. The functions fξ : Φ(D) → Rn and gξ : Φ(D) → Rn×m defined as: fξ (ξ) ≜ ∂Φ −1 (Φ (ξ))f (Φ−1 (ξ)), ∂x gξ (ξ) ≜ ∂Φ −1 (Φ (ξ))g(Φ−1 (ξ)), ∂x (2.25) for all ξ ∈ Φ(D) satisfy: " # fη (ξ) fξ (ξ) = , ω(ξ) " # gη (ξ) gξ (ξ) = , 0n−γ (2.26) for all ξ ∈ Φ(D). 2. We have that: fη (ξ) + gη (ξ)kfbl (Φ−1 (ξ), ν) = Aη + Bν, (2.27) for all ξ ∈ Φ(D) and ν ∈ Rk , where η ∈ Rγ and z ∈ Rn−γ satisfy (η, z) = ξ. Φ : D → Rn is a diffeomorphism between D and Φ(D) if it is smooth on D, and there exists an inverse function Φ−1 : Φ(D) → D that is smooth on Φ(D). Note that this requires that ∂Φ ∂x (x) is full rank for all x ∈ D. 3 This assumption on fη , gη , and ω is typically stronger than necessary. It is made here only to reason about stability using linearizations of ω. I note that if f and g are continuously differentiable on E, then the smoothness of Φ implies these properties. 2 41 If γ = n, the open-loop system (2.1) is said to be locally full-state feedback linearizable with respect to the equilibrium point xe , and the function ω does not appear in (2.26). The corresponding normal-form open-loop system is: " # " # " # η̇ fη (ξ) gη (ξ) ξ̇ = ≜ + u = fξ (ξ) + gξ (ξ)u. (2.28) ż ω(ξ) 0n−γ The term ν is referred to as an auxiliary input to the controller kfbl . The requirement that k ≤ m means that the number of outputs being feedback linearized is less than or equal to the number of control inputs. The controllable pair (A, B) are the linear dynamics achieved through feedback linearization, and for which standard linear control design techniques such as pole-placement, linear quadratic regulation (LQR), or robust control designs [99] can be used. The constant γ captures the notion of a vector relative degree for the system, while the function ω captures the internal dynamics that arise due to creating a linear relationship between control inputs and outputs through feedback when γ < n. I will refer to ξ as the normal state, η as the output coordinates, and z as the zero coordinates, and note I consider a specialized setting of feedback linearization in which the dynamics of the zero coordinates do not depend on the input u, which occurs frequently in robotics [116]. If ν is specified by an auxiliary controller kaux : Φ(D) → Rk that is locally Lipschitz continuous on Φ(D) and satisfies kaux (0n ) = 0k , then the definitions in (2.25) imply that 0n is an equilibrium point of the closed-loop system: ξ̇ = fξ (ξ) + gξ (ξ)kfbl (Φ−1 (ξ), kaux (ξ)). (2.29) Defining k′fbl : D → Rm as: k′fbl (x) = kfbl (x, kaux (Φ(x))), (2.30) for all x ∈ D, observe that k′fbl is locally Lipschitz continuous on D by the smoothness of Φ and: f (xe ) + g(xe )k′fbl (xe ) = 0n , (2.31) such that xe is an equilibrium point of the closed-loop system (2.2) using the controller k′fbl . More generally, (2.25) and the fact that ∂Φ (xe ) is full rank implies that xe is an ∂x unforced (forced) equilibrium point of the open-loop system (2.1) if and only 42 if the point 0n is an unforced (forced) equilibrium point of the normal-form open-loop system (2.28), noting that this requires ω(0n ) = 0n−γ whether the equilibrium point is unforced or forced. Given a controller kξ : Φ(D) → Rm , define a normal-form closed-loop system: ξ̇ = fξ (ξ) + gξ (ξ)kξ (ξ). (2.32) Similarly, 0n is an equilibrium point of the normal-form closed-loop system if and only if xe is an equilibrium point of the closed-loop system (2.2) using the controller k : D → Rm defined as k(x) = kξ (Φ(x)). The diffeomorphic relationship between the closed-loop system (2.2) and normal-form closed-loop system (2.32) allows for the following results regarding the perseverance of local exponential stability4 : Theorem 6. Let the open-loop system (2.1) be locally feedback linearizable with respect to an equilibrium point xe with corresponding open set D ⊆ E with xe ∈ D and function Φ : D → Rn that is a diffeomorphism between D and Φ(D) and satisfies Φ(xe ) = 0n . Given a controller kξ : Φ(D) → Rm that renders 0n an equilibrium point of the normal-form closed-loop system (2.32), the normal-form closed-loop system is locally exponentially stable with respect to the equilibrium point 0n if and only if the closed-loop system (2.2) with the controller k : D → Rm defined as k(x) = kξ (Φ(x)) is locally exponentially stable with respect to the equilibrium point xe . Considering the normal-form open-loop system (2.28), let K ∈ Rk×γ be a gain matrix such that the matrix Acl ≜ A − BK is Hurwitz (all of its eigenvalues have a negative real part). Such a gain matrix is guaranteed to exist as the pair (A, B) is controllable [117]. Using the auxiliary controller kaux (ξ) = −Kη implies the output coordinates of the closed-loop system (2.29) evolve as the following output closed-loop system: η̇ = fη (ξ) + gη (ξ)kfbl (Φ−1 (ξ), −Kη) = Acl η. (2.33) Momentarily neglecting the dynamics of the zero coordinates, this output closed-loop system is locally exponentially stable with respect to the unique equilibrium point 0γ . In this way, a subset of the system dynamics can be 4 I prove a similar result to this in Section 6.3, for which the proof steps can be followed to establish this theorem. 43 made into a stable linear system through an appropriate choice of feedback. To achieve more efficient or robust control, this linearizing feedback can serve as a starting point for synthesizing a CLF for the output open-loop system: η̇ = fη (ξ) + gη (ξ)u, (2.34) and equilibrium point 0γ . A core result in linear systems theory is that for any positive definite matrix Q ∈ Sγ≻0 , there exists a unique positive definite matrix P ∈ Sγ≻0 satisfying the continuous-time Lyapunov equation (CTLE): A⊤ cl P + PAcl = −Q. (2.35) For a given Q ∈ Sγ≻0 with a corresponding solution P ∈ Sγ≻0 to the CTLE, the function V : Rγ → R≥0 defined as: V (η) = η ⊤ Pη, (2.36) λmin (P)∥η∥2 ≤ V (η) ≤ λmax (P)∥η∥2 , (2.37) V̇ (ξ, kfbl (Φ−1 (ξ), −Kη)) ≤ −λmin (Q)∥η∥2 , (2.38) satisfies: ∂V (η) ∂η ≤ λmax (P)∥η∥, (2.39) for all ξ ∈ Φ(D), where (2.39) is an instance of the bound in (2.14). Furthermore, note that ∂V is locally Lipschitz continuous on its domain. Consequently, ∂η V serves as a local exponential Lyapunov function for the output closed-loop system (2.33) and the equilibrium point 0γ . As in [118], to ensure that V̇ satisfies the strict inequality necessary for a local exponential CLF, consider a positive definite matrix Q2 ∈ Sγ≻0 satisfying Q2 ≺ Q, yielding: V̇ (ξ, kfbl (Φ−1 (ξ), −Kη)) < −λmin (Q2 )∥η∥2 , (2.40) for all ξ ∈ Φ(D) \ ({0γ } × Rn−γ ). Consequently, we have that: inf V̇ (ξ, u) < −λmin (Q2 )∥η∥2 , u∈Rm (2.41) for all ξ ∈ Φ(D) \ ({0γ } × Rn−γ ), and thus V is a CLF for the output openloop dynamics (2.34) and equilibrium point 0γ . We can define the non-empty pointwise set: KCLF (ξ) ≜ {u ∈ Rm | V̇ (ξ, u) ≤ −λmin (Q2 )∥η∥2 }, (2.42) 44 for all ξ ∈ Φ(D), noting that kfbl (Φ−1 (ξ), −Kη) ∈ KCLF (ξ) for all ξ ∈ Φ(D), and synthesize an optimization-based controller kCLF : Φ(D) → Rm as follows: kCLF (ξ) = argmin ∥u − kfbl (Φ−1 (ξ), 0k )∥2 u∈Rm (Output-CLF-QP) s.t. V̇ (ξ, u) ≤ −λmin (Q2 )∥η∥2 . Using the techniques in [51] with the local Lipschitz continuity of kfbl on D × Rk , this controller can be shown to be locally Lipschitz continuous on Φ(D) (including Φ(D) ∩ ({0γ } × Rn−γ )). While the techniques in [51] cover the case when ξ ∈ Φ(D) \ ({0γ } × Rn−γ ), I will briefly sketch why this is true for ξ ∈ Φ(D) ∩ ({0γ } × Rn−γ ). Note that for such ξ, we have that kCLF (ξ) = kfbl (Φ−1 (ξ), 0k ). Let ξ ′ = (η ′ , z′ ) ∈ Φ(D). Observe that kfbl (Φ−1 (ξ ′ ), −Kη ′ ) ∈ KCLF (ξ ′ ) implies: ∥kCLF (ξ ′ ) − kCLF (ξ)∥ = ∥kCLF (ξ ′ ) − kfbl (Φ−1 (ξ), 0k )∥, (2.43) ≤ ∥kfbl (Φ−1 (ξ ′ ), −Kη ′ ) − kfbl (Φ−1 (ξ), 0k )∥, " # Φ−1 (ξ ′ ) − Φ−1 (ξ) , ≤ Lkfbl −Kη ′ (2.44) ≤ Lkfbl LΦ−1 ∥ξ ′ − ξ∥ + Lkfbl ∥K∥∥η ′ ∥, (2.46) ≤ Lkfbl (∥K∥ + LΦ−1 )∥ξ ′ − ξ∥, (2.47) (2.45) where Lkfbl , LΦ−1 ∈ R≥0 are local Lipschitz constants that hold by restricting ξ ′ to a sufficiently small neighborhood of ξ, and the last inequality used the fact that ∥η ′ ∥ ≤ ∥ξ ′ − ξ∥. Because Φ is smooth, the controller k′CLF : D → Rm defined as k′CLF (x) = kCLF (Φ(x)) is locally Lipschitz continuous on D. While the output closed-loop dynamics may be locally exponentially stable with respect to 0γ , local exponential stability of the zero coordinate dynamics is still necessary for the full normal-form closed-loop system (2.2) to be locally exponentially stable with respect to 0n . This is important to consider because if the zero coordinates do not display stability properties, they can lead to solutions φ that exit the domain D where the controllers kfbl and kCLF are well-defined (the interval of existence of the solution would be finite). To this end, define the zero dynamics system as: ż = ω((0γ , z)), (2.48) which correspond to the evolution of the zero coordinates when the output η = 0γ . Note that 0n−γ is an equilibrium point for this system as xe is assumed 45 to be an unforced or forced equilibrium point of the open-loop system (2.1), implying ω(0n ) = 0n−γ , yielding the following result [111, Proposition 4.4.2]: Theorem 7. If the zero dynamics system (2.48) is locally exponentially stable with respect to 0n−γ , then for any controller kξ : Φ(D) → Rm that is continuous on Φ(D), locally Lipschitz continuous on Φ(D) \ {0n }, renders 0n an equilibrium point of the normal-form closed-loop system (2.32), and satisfies kξ (ξ) ∈ KCLF (ξ) for all ξ ∈ Φ(D), the normal-form closed-loop system (2.32) is locally exponentially stable with respect to the equilibrium point 0n . Combining Theorems 6 and 7, local exponential stability of the zero dynamics in conjunction with a locally exponentially stabilizing controller kξ yields that the closed-loop system (2.2) using k(x) = kξ (Φ(x)) is locally exponentially stable with respect to the equilibrium point xe . To provide a concrete example of feedback linearization, consider a fullyactuated robotic system governed by (2.5) with a forced equilibrium point xe = (qe , 0nc ) ∈ X and corresponding equilibrium torque: τ e = B(qe )−1 (C(qe , 0nc ) + G(qe )). Let us define the function Φ : X → Rn as: " # q − qe Φ(x) = = ξ. q̇ (2.49) (2.50) It is easy to see that Φ is smooth on X and has an inverse Φ−1 : Φ(X ) → X that is smooth on Φ(X ). Consider a controller kfbl : X × Rnc → Rm given by: kfbl (x, ν) = B(q)−1 D(q)(D(q)−1 (C(q, q̇) + G(q)) + ν), (2.51) which is locally Lipschitz continuous on X ×Rnc , and observe that kfbl (xe , 0nc ) = τ e . This controller yields: " " # # 0nc ×nc Inc 0nc ×nc x+ ν, (2.52) ẋ = Inc 0nc ×nc 0nc ×nc {z } | {z } | A B where (A, B) ∈ Rn×n × Rn×nc are a controllable pair. Noting that Axe = 0n and denoting η = x − xe , these dynamics can be rewritten as: η̇ = Aη + Bν, (2.53) 46 mirroring the form in (2.27) with functions fη (ξ) = f (Φ−1 (ξ)) and gη (ξ) = g(Φ−1 (ξ)) with f and g defined in (2.6), and no function ω, such that γ = n and k = nc . Consequently, a fully-actuated robotic system is locally full-state feedback linearizable with respect to the equilibrium point xe with D = X . A detail that will appear in Chapter 3 when considering the impact of model error on CLF-based controllers is the model dependence of the feedback linearizing controller (2.51). 2.5 Control Barrier Functions Historically, barrier methods were first developed in the context of constrained optimization [119], wherein constraint satisfaction could be achieved through increasingly large penalties on constraint violation. The idea to use barrier functions (BFs) in the context of nonlinear dynamical systems was first proposed in [120] with the goal of certifying the forward invariance of a set for a closed-loop system, similar to the way in which local exponential Lyapunov functions serve as a certificate of local exponential stability. This idea was further developed in [10], yielding the first definition of Control Barrier Functions (CBFs) as a tool for simultaneously synthesizing a safety-critical controller and a BF for the corresponding closed-loop system. The controller in this work was based on a structured design developed with CLFs for stabilization in [9]. A consequence of this structured design was that the controller’s only goal was safety, which, while of vital importance, is often not the only control objective that needs to be achieved (such as tracking performance). An important pair of changes to the formulation of CBFs that increased their potential to provide safety-critical yet performant control were proposed in [11]. The first change was incorporating an extended class-K function (defined below) into the CBF time derivative condition required for safety. This change allowed the system state to approach the boundary of the forward invariant set as long as it displayed a safe degree of “braking”, reducing the conservative nature of the original definition of CBFs. The second change was realizing that the CBF time derivative was affine in the control input, and thus could be directly incorporated as a constraint in a convex optimization problem, as was done with CLFs in Section 2.3. This resulted in a way to minimally “filter” a controller designed for performance such that it meets safety requirements. Subsequently, the notion of zeroing CBFs were introduced in [62], which reframed the preceding definition of CBFs in a way that was well- 47 defined outside of a set being kept safe [121, Remark 3]. I note that in this thesis my definition of CBFs will correspond to zeroing CBFs. Before defining BFs and CBFs, I recall the following useful definitions: Definition 9 (Class K Function [122]). Let a ∈ R>0 . A function α : [0, a) → R≥0 that is continuous on [0, a) is said to be class-K (α ∈ K) if α(0) = 0 and α(r1 ) < α(r2 ) for all r1 , r2 ∈ [0, a) satisfying r1 < r2 . The function α is said to be class-K∞ (α ∈ K∞ ) if a = ∞ and limr→∞ α(r) = ∞. Definition 10 (Extended Class K Function [121]). Let a, b ∈ R>0 . A function α : (−b, a) → R that is continuous on (−b, a) is said to be extended class-K (α ∈ Ke ) if α(0) = 0 and α(r1 ) < α(r2 ) for all r1 , r2 ∈ (−b, a) satisfying e r1 < r2 . The function α is said to be extended class-K∞ (α ∈ K∞ ) if a = ∞, b = ∞, limr→∞ α(r) = ∞, and limr→−∞ α(r) = −∞. A barrier function is used to describe the safety of the closed-loop system (2.2) with respect to a set C ⊂ E specified as the 0-superlevel set of a function h : E → R that is continuously differentiable on E: C ≜ {x ∈ E | h(x) ≥ 0}, (2.54) Int(C) ≜ {x ∈ E | h(x) > 0}, (2.55) ∂C ≜ {x ∈ E | h(x) = 0}. (2.56) Given this construction, barrier functions are defined as in [62]: Definition 11 (Barrier Function (BF)). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. The function h is a barrier function (BF) for the closed-loop system (2.2) on C if there exists an α ∈ Ke such that: ḣ(x) ≜ ∂h ∂h (x)f (x) + (x)g(x) k(x) ≥ −α(h(x)), ∂x ∂x | {z } | {z } Lf h(x) (2.57) Lg h(x) for all x ∈ E. Note that the function α ∈ Ke in this definition must be defined on the interval:   inf h(x), sup h(x) ⊆ R. (2.58) x∈E x∈E 48 The following result establishes how the existence of a barrier function for the closed-loop system (2.2) on C certifies the safety of the closed-loop system (2.2) with respect to C: Theorem 8. Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. If h is a BF for the closed-loop system (2.2) on C, then the closed-loop system (2.2) is safe with respect to the set C. This result was first established in [62, Proposition 1] only requiring that the BF inequality (2.57) be satisfied for x ∈ C by using Nagumo’s theorem [104], [123] under the requirement that the function h has 0 as a regular value: h(x) = 0 =⇒ ∂h (x) ̸= 0n , ∂x (2.59) for all x ∈ E. Subsequently, an alternative proof of this result was provided in [124, Theorem 1] that does not require h have 0 as regular value, but rather uses the fact that α ∈ Ke and the barrier function inequality (2.57) is satisfied for x ∈ E \ C. The definition of CBFs that I next consider will implicitly require that h has 0 as a regular value. CBFs provide a tool for synthesizing both a safety-critical controller and a BF that certifies the corresponding closed-loop system (2.2) is safe with respect to the set C: Definition 12 (Control Barrier Function (CBF)). Let C ⊂ E be the 0superlevel set of a function h : E → R that is continuously differentiable on E. The function h is a Control Barrier Function (CBF) for the open-loop system (2.1) on C if there exists an α ∈ Ke such that: sup ḣ(x, u) ≜ sup Lf h(x) + Lg h(x)u > −α(h(x)), u∈Rm u∈Rm (2.60) for all x ∈ E As with CLFs, the strict inequality in (2.60), which appears in [51], is important not only for ensuring a controller k satisfying (2.57) can exist, but also that a controller meeting suitable regularity properties can be found. Note that the strictness also requires h to have 0 as a regular value. Similarly, an equivalent requirement to (2.60) is given by: Lg h(x) = 0m =⇒ Lf h(x) > −α(h(x)), (2.61) 49 for all x ∈ E. Given a CBF h for the open-loop system (2.1) on C and a corresponding α ∈ Ke , define the point-wise set: o n m ḣ(x, u) ≥ −α(h(x)) . (2.62) KCBF (x) = u ∈ R This yields the following result [62]: Theorem 9. Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. If h is a CBF for the open-loop system (2.1) on C, then the set KCBF (x) is non-empty for all x ∈ E, and for any controller k that is locally Lipschitz continuous on E with k(x) ∈ KCBF (x) for all x ∈ E, the function h is a BF for the closed-loop system (2.2) on C. This result states that for a given α ∈ Ke , a CBF describes the set of inputs that a controller can take such that the function h serves as a BF for the closedloop system (2.2) on C. To see how CBFs can minimally modify a performant controller while ensuring safety, consider a nominal controller knom : E → Rm that is locally Lipschitz continuous on E, and define the optimization-based controller as was first presented in [27]: kCBF (x) = argmin ∥u − knom (x)∥2 u∈Rm (CBF-QP) s.t. ḣ(x, u) ≥ −α(h(x)). As with CLFs, because ḣ is affine with respect to the input u, this controller is defined by a convex QP which can be efficiently solved. Moreover, if at a given state x ∈ E the nominal controller knom satisfies ḣ(x, knom (x)) ≥ −α(h(x)), then kCBF (x) = knom (x). Thus the nominal controller can be designed to meet some performance criteria, and the controller (CBF-QP) can be used to modify the nominal controller only when it is unsafe, as specified by the BF requirement (2.57). The following result describes the properties of this controller [51, Theorem 1]: Theorem 10. Let C ⊂ E be the 0-superlevel set of a function h : E → R ∂h : E → Rn be that is continuously differentiable on E, let the function ∂x locally Lipschitz continuous on E, let the nominal controller knom : E → Rm be locally Lipschitz continuous on E, and let h be a CBF for the open-loop system (2.1) on C with α locally Lipschitz continuous on its domain. Then the 50 controller kCBF : E → Rm is locally Lipschitz continuous on E and satisfies kCBF (x) ∈ KCBF (x) for all x ∈ E. Suppose that beyond ensuring the closed-loop system (2.2) is safe with respect to the set C, an objective of control design is to locally exponentially stabilize the (2.2) system with respect to an unforced equilibrium point or forced equilibrium point xe ∈ Int(C) with equilibrium input ue . Furthermore, suppose that there exists a CBF h for the open-loop system (2.1) on C with corresponding α ∈ Ke , and a local exponential CLF V for the open-loop system (2.1) and equilibrium point xe with a corresponding open set D ⊆ E that satisfies the small or continuous control property. A controller k : D → Rm that incorporates both the CLF and CBF can be specified as follows [62]: k(x) = argmin ∥u − ue ∥2 + cδ 2 (u,δ)∈Rm+1 (CLF-CBF-QP) s.t. V̇ (x, u) ≤ −k3 ∥x − xe ∥a + δ, ḣ(x, u) ≥ −α(h(x)), (2.63) for c ∈ R>0 . This controller ensures the closed-loop system (2.2) is safe with respect to C, and attempts to locally exponentially stabilize the closed-loop system (2.2) with respect to the equilibrium point xe . Because it may not be possible to simultaneously satisfy both the CLF and CBF inequality constraints, the CLF constraint is relaxed with the slack variable δ which is highly penalized by a large value of c in the cost function. While this controller does not endow the system with a formal stability guarantee, it can often effectively stabilize a system while ensuring safety. Furthermore, alternative relaxation schemes can be considered that provide local exponential stability results [51, QP Problem −γm Version]. The continuity on D and local Lipschitz continuity on D \ {xe } of this controller can be established along the lines of [51, Theorem 1] under assumptions on the small or continuous control property ∂h and the local Lipschitz continuity of ∂V and ∂x . ∂x I will now make a useful observation regarding the relationship between local exponential CLFs and CBFs. First, as noted in [62, Proposition 2], outside of the set C, a barrier function h : E → R for the closed-loop system (2.2) can be used to construct a Lyapunov-like function for the closed-loop system (2.2) and the set C, such that the closed-loop system (2.2) displays asymptotic stability properties with respect to the set C. Considering the other direction, suppose 51 that V : E → R≥0 is a local exponential CLF for the open-loop system (2.1) and an unforced or forced equilibrium point xe ∈ E with corresponding open set D. In this development, I will restrict the open set E in the definition of forward invariance and CBFs to be E = D. The lower bound in (2.12) implies the existence of a c ∈ R>0 such that for any c ∈ [0, c), the set C defined as: C ≜ {x ∈ E | V (x) ≤ c}, (2.64) satisfies C ⊂ D. Next, define function the h : D → R as follows: h(x) = c − V (x), (2.65) noting that C is the 0-superlevel set of h. We then have that: sup ḣ(x, u) = sup −V̇ (x, u), u∈Rm u∈Rm (2.66) = − infm V̇ (x, u), u∈R > k3 ∥x − xe ∥a , k3 ≥ V (x), k2 k3 k3 > (V (x) − c) = − h(x), k2 k2 (2.67) for all x ∈ D \ {xe }. We also have that V̇ (xe , u) = ḣ(xe , u) = 0 for all u ∈ Rm , and h(xe ) = c > 0, and consequently: sup ḣ(xe , u) > − u∈Rm k3 h(xe ). k2 (2.68) Thus we have that h is a CBF for the open-loop system (2.1) on the set C. This is a useful observation as constructive tools such as feedback linearization can be used to produce local exponential CLFs, and from these CBFs can be constructed. The key step in this process occurs in (2.67), where the inequality is relaxed by the offset − kk32 c. This term allows a system using a CBF to approach the boundary of the safe set, rather than explicitly converge to the point xe as is achieved with an exponential CLF and exponential stability. 2.6 Nonlinear Dynamics with Disturbances The preceding developments of CLFs, feedback linearization, and CBFs were presented in the context of the ideal control system seen in Figure 1.1. For many real-world systems, control design is often complicated by challenges 52 such as unmodeled actuator dynamics or external perturbations. These challenges can be studied by considering nonlinear systems with disturbances. I will consider two classes of systems with disturbances. The first class is the setting of unmatched disturbances described by the open-loop system: ẋ = f (x) + g(x)u + d(t), (2.69) where the function d : R≥0 → Rn is piecewise continuous5 on R≥0 and reflects a disturbance signal. A controller k : E → Rm that is locally Lipschitz on E yields the closed loop system: ẋ = f (x) + g(x)k(x) + d(t). (2.70) As before, for any initial condition x0 ∈ E and piecewise continuous disturbance signal d : R≥0 → Rn , there exists a maximal time interval Id (x0 , d) ≜ [0, tmax (x0 , d)) and a unique piecewise continuously differentiable6 solution φd : Id (x0 , d) → E to the closed-loop system (2.70) satisfying: φ̇d (t) = f (φd (t)) + g(φd (t))k(φd (t)) + d(t), (2.71) φd (0) = x0 , (2.72) for almost all t ∈ Id (x0 , d) [100]. Note that the interval of existence and uniqueness Id (x0 , d) depends not only on the initial condition x0 as it did in the absence of disturbances, but now it also depends on the particular disturbance signal the system experiences. The second class I will consider is the setting of matched disturbances, described by the open-loop system: ẋ = f (x) + g(x)(u + d(t)), (2.73) where the function d : R≥0 → Rm is piecewise continuous on R≥0 and reflects a disturbance in the input signal to the system. I will later show that this setting permits specialized robust control designs to be developed using the previous formulations of CLF and CBFs. A controller k : E → Rm that is locally Lipschitz on E yields the closed loop system: ẋ = f (x) + g(x)(k(x) + d(t)). 5 (2.74) This definition is taken as in [100], where for every compact interval [a, b] ⊂ R≥0 , there exists a finite partition of the interval [a, b], T = {t0 , t1 , . . . , tN }, such that d is continuous on the domain (ti−1 , ti ) and the one-sided limits limδ→0+ d(ti−1 + δ) and limδ→0− d(ti + δ) exist for i = 1, . . . , N . 6 Piecewise continuous differentiability is defined such that the function φd is continuous on Id (x0 , d) with a derivative that is piecewise continuous on Id (x0 , d). 53 As before, for any initial condition x0 ∈ E and piecewise continuous disturbance signal d : R≥0 → Rm , there exists a maximal time interval Id (x0 , d) ≜ [0, tmax (x0 , d)) and a unique piecewise continuously differentiable solution φd : Id (x0 , d) → E to the closed-loop system (2.74) satisfying: φ̇d (t) = f (φd (t)) + g(φd (t))(k(φd (t)) + d(t)), (2.75) φd (0) = x0 , (2.76) for almost all t ∈ Id (x0 , d) [100]. 2.7 Input-to-State Stability & Input-to-State Safety When studying nonlinear systems in the presence of disturbances, there are two perspectives of robust stability and safety I will consider. The first approach is to ensure that exponential stability or safety are achieved no matter what the disturbance signal may be, as is done with CBFs in [51]. In the setting of safety, this may induce significant conservativeness as the system is robust to worst-case disturbance signals that it may not experience. Achieving exponential stability of an equilibrium point in the presence of disturbances is often impossible, simply because it is impossible to choose one control input that will enforce xe is an equilibrium point for all possible disturbance signals without extremely restrictive assumptions [109, Section 3.4.1]. This particular challenge with stability in the presence of disturbances led to the proposal of Input-to-State Stability (ISS) [56], [57], [125], and later in the setting of safety, Input-to-State Safety (ISSf) [58]. These definitions consider the degradation of stability and safety in the presence of disturbances. In particular, systems that display ISS or ISSf properties gracefully degrade in the presence of disturbances, such that the magnitude of degradation is correlated with the magnitude of the disturbance that is experienced. This can provide for a powerful design paradigm when working with real-world systems in which disturbances are present [21], [126]. In this thesis I will explore several ways in which this perspective on robust control synthesis can be powerful for synthesizing performant yet robust controllers, including settings involving residual learning error, disturbances, and input-sampling effects. I will now mathematically define these robustness properties. I first note these definitions are the same for both the unmatched and matched disturbance settings, with the distinction between those settings arising when considering 54 control synthesis in the next section. First, I will denote the following: ∥d∥∞ = ess sup ∥d(t)∥, (2.77) t∈R≥0 where the essential supremum ess sup allows us to neglect the values of the piecewise continuous disturbance signal d at the times at which it is discontinuous when assessing a bound on the disturbance. This permits the following definition of Input-to-State Stability: Definition 13 (Input-to-State Stability (ISS)). Let xe ∈ E be an equilibrium point of the closed-loop system (2.2). The closed-loop system with unmatched disturbances (2.70) or matched disturbances (2.74) is said to be locally exponentially Input-to-State Stable (ISS) with respect to xe if there exist an open set D ⊆ E with xe ∈ D, constants M, λ, d ∈ R>0 , and γ ∈ K such that for all d ∈ [0, d]: x0 ∈ D =⇒ ∥φd (t) − xe ∥ ≤ M ∥x0 − xe ∥e−λt + γ(d), (2.78) for all t ∈ R≥0 and disturbance signals d : R≥0 → Rn (Rm for matched disturbances) that are piecewise continuous on R≥0 and satisfy ∥d∥∞ ≤ d. This definition modifies the definition of local exponential stability by incorporating a class-K function of the worst-case magnitude disturbance experienced by the system into the bound on the solution’s deviation from the equilibrium point. If d = 0, such that the disturbance signal satisfies d(t) = 0n (0m for matched disturbances) for all t ∈ R≥0 , we recover that the closed-loop system (2.2) without disturbances must be locally exponentially stable with respect to the equilibrium point xe . In the presence of disturbances, the system must converge exponentially to a ball of radius γ(d) centered at xe , and remain within that ball. ISS is inherently a statement about the behavior of the closed-loop systems (2.70) or (2.74) across the spectrum of disturbances which are essentially bounded by the maximum disturbance magnitude d. If the disturbance the system experiences is small, then the ball the system converges to is small, and likewise, if it is large (but satisfies the bound), the ball it will converge to is bounded. Producing a concrete statement about the size of the ball a closed-loop system with disturbances will actually remain in during its evolution requires producing an estimate of the worst-case possible disturbance signal the system will experience. Before defining Input-to-State Safety, I present the following definition: 55 Definition 14 (Disturbed Forward Invariance & Safety). Let d ∈ R≥0 . A set C ⊂ E is said to be forward invariant up to d for the closed-loop system with unmatched disturbances (2.70) or closed-loop system with matched disturbances (2.74) if for any initial condition x0 ∈ C and disturbance signal d : R≥0 → Rn (Rm for matched disturbances) that is piecewise continuous on R≥0 and satisfies ∥d∥∞ ≤ d, we have φd (t) ∈ C for all t ∈ Id (x0 , d). In this case, we call the system (2.70) or (2.74) safe up to d with respect to the set C. The first perspective on robustness to disturbances, where a system must satisfy safety properties with respect to the set C for any disturbance it may experience, is described by the notion of the system being safe up to d with respect to C. Input-to-State Safety will relax this definition as follows: Definition 15 (Input-to-State Safety (ISSf )). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. The closed-loop system with unmatched disturbances (2.70) or closed-loop system with matched disturbances (2.74) is said to be Input-to-State Safe (ISSf ) with respect to the set C if there exist d ∈ R>0 and γ ∈ K such that for all d ∈ [0, d], the set Cd ⊂ E defined as: Cd = {x ∈ E | h(x) + γ(d) ≥ 0} , (2.79) is forward invariant up to d for the closed-loop system (2.70) or (2.74). If the system (2.70) or (2.74) is ISSf with respect to the set C, the set C is referred to as an Input-to-State Safe set (ISSf set). This definition states that the set that is kept forward invariant scales with the magnitude of the disturbance up to some maximum disturbance, and that in the absence of disturbances, safety of the original set C is retained. Note that ISSf is explicitly constructed with a set that is the 0-superlevel set of a function h : E → R that is continuously differentiable on E, rather than just a general set C ⊂ E. 2.8 Input-to-State Stable Control Lyapunov Functions Establishing ISS will be achieved through corresponding Lyapunov functions that accommodate the disturbance signal. Producing these certificate functions will be achieved through corresponding modifications to CLFs, where 56 the importance of unmatched and matched disturbances will become apparent. I now define Input-to-State Stable Lyapunov functions (ISS-LFs) as in [125, Definition 2.2]: Definition 16 (Input-to-State Stable Lyapunov Function (ISS-LF)). Let xe be an equilibrium point of the closed-loop system (2.2). A function V : E → R≥0 that is continuously differentiable on E is said to be a local exponential Input-to-Stable Lyapunov function (ISS-LF) for the closed-loop system with unmatched disturbances (2.70) and equilibrium point xe if there exist an open set D ⊆ E with xe ∈ D, constants k1 , k2 , k3 , a, d ∈ R>0 , and ρ ∈ K such that: k1 ∥x − xe ∥a ≤ V (x) ≤ k2 ∥x − xe ∥a , ∂V (x)d ≤ −k3 ∥x − xe ∥a , V̇ (x, d) ≜ Lf V (x) + Lg V (x)k(x)+ ∂x (2.80) (2.81) for all x ∈ D and d ∈ Rn such that ∥d∥ ≤ d and ∥x − xe ∥ ≥ ρ(∥d∥). The function V is said to be a ISS-LF for the closed-loop system with matched disturbances (2.70) and equilibrium point xe if (2.81) is replaced with: V̇ (x, d) ≜ Lf V (x) + Lg V (x)k(x) + Lg V (x)d ≤ −k3 ∥x − xe ∥a , (2.82) for all x ∈ D and d ∈ Rm such that ∥d∥ ≤ d and ∥x − xe ∥ ≥ ρ(∥d∥). This definition of ISS-LFs requires that the Lyapunov decay conditions (2.81) or (2.82) be met only for disturbances that are smaller (with the class-K function ρ) than the distance of the state from the equilibrium point xe . A useful alternative understanding of the requirement in (2.81) and (2.82) is captured in the following result [125, Remark 2.4]: Theorem 11. Let xe be an equilibrium point of the closed-loop system (2.2). A function V : E → R≥0 satisfying (2.80) for some k1 , k2 , a ∈ R>0 is a local exponential ISS-LF for the closed-loop system with unmatched disturbances (2.70) (matched disturbances (2.74)) with corresponding open set D ⊆ E if and only if there exist k3′ , d ∈ R>0 and σ ∈ K such that: V̇ (x, d) ≤ −k3′ ∥x − xe ∥a + σ(∥d∥), (2.83) for all x ∈ D and d ∈ Rn (d ∈ Rm ) such that ∥d∥ ≤ d. Note that this alternative condition uses the quantity k3′ ∈ R>0 which is potentially different than the k3 in (2.81) and (2.82). ISS-LFs serve as certificate 57 of local exponential ISS of a closed-loop system with respect to an equilibrium point as follows [125, Theorem 1]: Theorem 12. Let xe ∈ E be an equilibrium point of the closed-loop system (2.2). If there exists a local exponential ISS-LF V : E → R≥0 for the closedloop system with unmatched disturbances (2.70) (matched disturbances (2.74)) and equilibrium point xe , then the closed-loop system with unmatched disturbances (2.70) (matched disturbances (2.74)) is locally exponentially ISS with respect to xe . Extending the notion of an ISS-LF to an ISS Control Lyapunov Function (ISS-CLF) requires more care than the extension of Lyapunov functions to CLFs. One potential way for defining an ISS-CLF in the setting of matched disturbances would be to require that: inf V̇ (x, u, d) ≜ infm Lf V (x) + Lg V (x)(u + d) < −k3 ∥x − xe ∥a + σ(∥d∥), u∈Rm u∈R (2.84) for all x ∈ D \ {xe } and d ∈ R (d ∈ R ) such that ∥d∥ ≤ d. The issue with such a definition is that it is unclear how control inputs should be chosen, given the appearance of d, which is not a decision variable, but rather an unknown perturbation. This issue of defining ISS-CLFs in a way that permits controller design has been considered in [127, Theorem 2] and [106, Section 6]. In this thesis, I will consider the following definition of an ISS-CLF: n m Definition 17 (Input-to-State Stable Control Lyapunov Function (ISS-CLF)). Let xe ∈ E be an unforced or forced equilibrium point of the open-loop system (2.1). A function V : E → R≥0 that is continuously differentiable on E is said to be a local exponential Input-to-State Stable Control Lyapunov Function (ISS-CLF) for the open-loop system with unmatched disturbances (2.69) if there exist an open set D ⊆ E with xe ∈ D and k1 , k2 , k3 , ϵ ∈ R>0 such that: k1 ∥x − xe ∥2 ≤ V (x) ≤ k2 ∥x − xe ∥2 , infm Lf V (x′ ) + Lg V (x′ )u + u∈R 1 ∂V ′ (x ) ϵ ∂x (2.85) 2 < −k3 ∥x′ − xe ∥2 , (2.86) for all x ∈ D and x′ ∈ D \ {xe }. The function V is said to be a local exponential ISS-CLF for the open-loop system with matched disturbances (2.73) and equilibrium point xe if (2.86) is replaced with: infm Lf V (x′ ) + Lg V (x′ )u + u∈R 1 2 ∥Lg V (x′ )∥ < −k3 ∥x′ − xe ∥2 , ϵ (2.87) 58 for all x′ ∈ D \ {xe }. First observe that I have specialized this definition to use a value of a = 2. Second, observe that the ISS-CLF condition for the matched disturbance setting (2.87) can be rewritten as the alternative CLF condition (2.18). This implies that if V is a local exponential CLF for the open-loop system (2.1) and equilibrium point xe , it is a local exponential ISS-CLF for the open-loop system with matched disturbances (2.73) and equilibrium point xe . This is a very useful property given the ability to synthesize CLFs through feedback linearization demonstrated in Section 2.4, and is a significant difference between the unmatched and matched disturbance settings. Given a local exponential ISS-CLF V for the open-loop system with unmatched disturbances (2.69) and equilibrium point xe , define the following pointwise set: ( ) L V (x) + L V (x)u f g KISS (x) = u ∈ Rm , (2.88) 2 + 1ϵ ∂V (x) ≤ −k3 ∥x − xe ∥2 ∂x and given a local exponential ISS-CLF V for the open-loop system with matched disturbances (2.73) and equilibrium point xe , define the following pointwise set: ) ( L V (x) + L V (x)u f g . (2.89) KISS (x) = u ∈ Rm + 1ϵ ∥Lg V (x)∥2 ≤ −k3 ∥x − xe ∥2 This set leads to the following result7 : Theorem 13. Let xe ∈ E be an unforced or forced equilibrium point of the open-loop system (2.1), and let V : E → R≥0 be a local exponential ISSCLF for the open-loop system with unmatched disturbances (2.69) (matched disturbances (2.73)) and equilibrium point xe with corresponding open set D ⊆ E. Then, the set KISS (x) is non-empty for all x ∈ D, and for any controller k : D → Rm that is continuous on D, locally Lipschitz continuous on D \ {xe }, renders xe an equilibrium point of the closed-loop system (2.2), and satisfies k(x) ∈ KISS (x) for all x ∈ D, the function V is a local exponential ISSLF for the closed-loop system with unmatched disturbances (2.70) (matched disturbances (2.74)) and equilibrium point xe with k3′ = k3 and σ ∈ K defined 2 as σ(r) = ϵr4 . 7 This result can be constructed along the proof steps used in Section 4.3. 59 This theorem not only states that an ISS-CLF V characterizes the set of inputs that allow V to serve as an ISS-LF, but it also provides a characterization of the constant k3′ and class-K function σ in the alternative definition of an ISS-LF given in (2.83). Note that the impact of the disturbance can be made smaller through small values of ϵ, but this requires that the controller provide more aggressive stabilizing control inputs. This is an analog to the fundamental balance between mitigating disturbances and high-gain control commonly studied in linear systems [128]. Building controllers using ISS-CLFs meeting the conditions of Theorem 13 will require the following variants of the small and continuous control properties: Definition 18 (ISS Small Control Property (ISS-SCP)). Let xe ∈ E be an unforced equilibrium point of the open-loop system (2.1), and let V : E → R≥0 be a local exponential ISS-CLF for the open-loop system with unmatched disturbances (2.69) (matched disturbances (2.73)) and equilibrium point xe with constant ϵ ∈ R>0 . The local exponential ISS-CLF V is said to satisfy the ISS small control property (ISS-SCP) if for any ε ∈ R>0 , there exists an open set D ⊆ E with xe ∈ D such that for any x ∈ D \ {xe }, there exists a u ∈ Rm satisfying ∥u∥ < ε such that: 1 ∂V Lf V (x) + Lg V (x)u + (x) ϵ ∂x 2 < −k3 ∥x − xe ∥2 , (2.90) for the unmatched disturbance setting, or such that: Lf V (x) + Lg V (x)u + 1 ∥Lg V (x)∥2 < −k3 ∥x − xe ∥2 , ϵ (2.91) for the matched disturbance setting. Definition 19 (ISS Continuous Control Property (ISS-CCP)). Let xe ∈ E be a forced equilibrium point of the open-loop system (2.1) for equilibrium input ue , and let V : E → R≥0 be a local exponential ISS-CLF for the open-loop system with unmatched disturbances (2.69) (matched disturbances (2.73)) and equilibrium point xe with constant ϵ ∈ R>0 . The local exponential ISS-CLF V is said to satisfy the ISS continuous control property (ISS-CCP) if for any ε ∈ R>0 , there exists an open set D ⊆ E with xe ∈ D such that for any x ∈ D \ {xe }, there exists a u ∈ Rm satisfying ∥u − ue ∥ < ε such that: Lf V (x) + Lg V (x)u + 1 ∂V (x) ϵ ∂x 2 < −k3 ∥x − xe ∥2 , (2.92) 60 for the unmatched disturbance setting, or such that: Lf V (x) + Lg V (x)u + 1 ∥Lg V (x)∥2 < −k3 ∥x − xe ∥2 , ϵ (2.93) for the matched disturbance setting. I note that in the matched disturbance setting, the relationship between a local exponential CLF satisfying the small or continuous property and a local ISSCLF satisfying the small or continuous control property is non-trivial. Rather, it is necessary, but not sufficient, for the CLF to satisfy the small or continuous control property for the ISS-CLF to satisfy the ISS small or continuous control property. Lastly, I will define the convex optimization-based controllers: (ISS-CLF-QP) kISS (x) = argmin ∥u − ue ∥2 u∈Rm 1 ∂V s.t. Lf V (x) + Lg V (x)u + (x) ϵ ∂x 2 ≤ −k3 ∥x − xe ∥2 , and: (ISS-CLF-QP) kISS (x) = argmin ∥u − ue ∥2 u∈Rm s.t. Lf V (x) + Lg V (x)u + 1 ∥Lg V (x)∥2 ≤ −k3 ∥x − xe ∥2 , ϵ for the unmatched and matched disturbance settings, respectively. This leads to the following result, which can be proven similarly to [51, Theorem 1]: Theorem 14. Let xe ∈ E be an unforced (forced) equilibrium point of the open-loop system (2.1) with equilibrium input ue , let V : E → R≥0 be a local exponential ISS-CLF for the open-loop system with unmatched disturbances (2.69) or matched disturbances (2.73) and equilibrium point xe with corre: E → Rn be locally Lipschitz sponding open set D ⊆ E, let the function ∂V ∂x continuous on D, and assume that V satisfies the ISS small (continuous) control property. Then the controller kISS : D → Rm is continuous on D, locally Lipschitz continuous on D\{xe }, renders xe an equilibrium point of the closedloop system (2.2), and satisfies kISS (x) ∈ KISS (x) for all x ∈ D. 2.9 Input-to-State Safe Control Barrier Functions Establishing ISSf will be achieved through corresponding barrier functions that accommodate the disturbance signal. Producing these certificate functions will 61 be achieved through corresponding modifications to CBFs, where the importance of unmatched and matched disturbances will become apparent. I now define Input-to-State Safe barrier functions (ISSf-BFs) as in [58, Definition 4]: Definition 20 (Input-to-State Safe Barrier Function (ISSf-BF)). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. The function h is said to be an Input-to-State Safe barrier function (ISSf-BF) for the closed-loop system with unmatched disturbances (2.70) on C if there exist α ∈ Ke , ι ∈ K, and d ∈ R>0 such that: ḣ(x, d) ≜ Lf h(x) + Lg h(x)k(x) + ∂h (x)d ≥ −α(h(x)) − ι(∥d∥), ∂x (2.94) for all x ∈ E and d ∈ Rn such that ∥d∥ ≤ d. The function h is said to be an ISSf-BF for the closed-loop system with matched disturbances (2.74) on C if (2.94) is replaced with: ḣ(x, d) ≜ Lf h(x) + Lg h(x)k(x) + +Lg h(x)d ≥ −α(h(x)) − ι(∥d∥), (2.95) for all x ∈ E and d ∈ Rm such that ∥d∥ ≤ d. ISSf-BFs serve as a certificate of ISSf of a closed-loop system with respect to a set C as follows [58, Theorem 1]: Theorem 15. Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. If h is an ISSf-BF for the closed-loop system with unmatched disturbances (2.70) (matched disturbances (2.74)) on C, then the closed-loop system with unmatched disturbances (2.70) (matched disturbances (2.74)) is ISSf with respect to the set C. In the same way that extending ISS-LFs to ISS-CLFs required considering the difference between the unmatched and matched disturbances, the notion of an ISSf-CBF will depend on what disturbance setting is being considered: Definition 21 (Input-to-State Safe Control Barrier Function (ISSf-CBF)). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. The function h is said to be an Input-to-State Safe Control Barrier Function (ISSf-CBF) for the open-loop system with unmatched disturbances (2.69) on C if there exist α ∈ Ke and ϵ ∈ R>0 such that: 1 ∂h sup Lf h(x) + Lg h(x)u − (x) ϵ ∂x u∈Rm 2 > −α(h(x)), (2.96) 62 for all x ∈ E. The function h is said to be an ISSf-CBF for the open-loop system with matched disturbances (2.73) on C if (2.96) is replaced with: sup Lf h(x) + Lg h(x)u − u∈Rm 1 ∥Lg h(x)∥2 > −α(h(x)), ϵ (2.97) for all x ∈ E. As with ISS-CLFs, the condition for the matched disturbance setting (2.97) can be rewritten as the alternative CBF condition (2.61). This implies that if h is a CBF for the open-loop system (2.1) on C, it is an ISSf-CBF for the open-loop system with matched disturbances (2.73) on C. Given an ISSf-CBF h for the open-loop system with unmatched disturbances (2.69) on C, define the following pointwise set: ) ( L h(x) + L h(x)u f g , (2.98) KISSf (x) = u ∈ Rm 2 ∂h (x) ≥ −α(h(x)) − 1ϵ ∂x and given an ISS-CBF h for the open-loop system with matched disturbances (2.73) on C, define the following pointwise set: ( ) Lf h(x) + Lg h(x)u m KISSf (x) = u ∈ R . (2.99) − 1ϵ ∥Lg h(x)∥2 ≥ −α(h(x)) This set leads to the following result: Theorem 16. Let h : E → R be an ISSf-CBF for the open-loop system with unmatched disturbances (2.69) (matched disturbances (2.73)) on C. Then, the set KISSf (x) is non-empty for all x ∈ E, and for any controller k : E → Rm that is locally Lipschitz continuous on E and satisfies k(x) ∈ KISSf (x) for all x ∈ E, the function h is an ISSf-BF for the closed-loop system with unmatched disturbances (2.70) (matched disturbances (2.74)) on C with ι ∈ K defined as 2 ι(r) = ϵr4 . This theorem not only states that an ISSf-CBF h characterizes the set of inputs that allow h to serve as an ISS-BF, but it also provides a characterization of the class-K function ι in the definition of an ISSf-BF. Again, note that the impact of the disturbance can be made smaller through small values of ϵ, but this requires that the controller provide inputs that meet a stronger safety criterion. This will be a trade-off that is studied in Chapter 4. Lastly, given a 63 nominal controller knom : E → Rm that is locally Lipschitz continuous on E, I will define the following convex optimization-based controllers: (ISSf-CBF-QP) kISSf (x) = argmin ∥u − knom (x)∥2 u∈Rm s.t. Lf h(x) + Lg h(x)u − 1 ∂h (x) ϵ ∂x 2 ≥ −α(h(x)), and: kISSf (x) = argmin ∥u − knom (x)∥2 u∈Rm s.t. Lf h(x) + Lg h(x)u − (ISSf-CBF-QP) 1 ∥Lg h(x)∥2 ≥ −α(h(x)), ϵ for the unmatched and matched disturbance settings, respectively. This leads to the following result, which can be proven similarly to [51, Theorem 1]: Theorem 17. Let C ⊂ E be the 0-superlevel set of a function h : E → R ∂h that is continuously differentiable on E, the function ∂x : E → Rn be locally Lipschitz continuous on E, the nominal controller knom : E → Rm be locally Lipschitz continuous on E, and h be an ISSf-CBF for the open-loop system with unmatched disturbances (2.69) or matched disturbances (2.73) on C. Then the controller kISSf : E → Rm is locally Lipschitz continuous on E and satisfies kISSf (x) ∈ KISSf (x) for all x ∈ E. 64 Chapter 3 LEARNING & ADAPTIVE CONTROL FOR NONLINEAR SYSTEMS The construction of controllers for nonlinear systems through Control Lyapunov Functions (CLFs), feedback linearization, and Control Barrier Functions (CBFs) is inherently a model-based design paradigm. In particular, the functions f and g describing the open-loop system (2.1) appear in the inequality constraints in the optimization-based controllers (CLF-QP), (CBF-QP), (CLF-CBF-QP), (ISS-CLF-QP), and (ISSf-CBF-QP), and similarly appear in the feedback linearizing controller (2.51) synthesized for robotic systems. Consequently, if there is error in a nominal model of the system (which will be mathematically described in Section 3.2), these controllers will not necessarily lead their respective true closed-loop systems to possess corresponding Lyapunov and barrier functions that certify local exponential stability and safety. Robustness to model error is one of the core challenges in control theory that the use of feedback is known to solve [99], [128]. A limitation of a purely robust approach is that it requires a characterization of the possible model errors, and a control design that meets stability or safety requirements for all possible model errors. Importantly, the resulting controller will be robust to model errors that are not actually present between the nominal model of the system and the true dynamics of the system. Thus, a coarse characterization of possible model error can lead to unnecessary conservativeness which significantly degrades the performance of a controller Alternatively, one may try to identify the actual model error between a nominal model of the system and true system, and design a controller using a corrected version of the nominal model. Classically, control theory has explored this question through the tools of system identification [129] and adaptive control [130]. System identification seeks to reduce model error by fitting models that more accurately capture the functions f and g using data, while adaptive control seeks to identify parametric model errors online while modifying parameters of a controller to ensure that stability and safety properties are 65 met as the model errors are identified. Recently, the advent of machine learning techniques for identifying complex models from data has led to significant development of data-driven techniques for control design beyond the classical tools of system identification [131]–[133], including both learning model errors and constructing updated model-based controllers [134]–[137], as well as direct model-free controller design [138]–[142]. My work presented in this chapter focuses on data-driven techniques for reducing model errors in a way that is amenable to the convex optimization-based control framework using CLFs and CBFs discussed in Chapter 2, as well as an adaptive control approach for ensuring safety in the presence of parametric model error. In Section 3.1 I discuss related work in the area of integrating learning techniques with CLFs and CBFs, as well as some of the historical and recent work in adaptive control for safety-critical systems. In Section 3.2 I will describe the mathematical structure of model errors that I consider. In Sections 3.3-3.6 I will present algorithms for learning model errors as they impact the time derivative of a CLF or CBF with simulation and experimental demonstrations, as well as provide a mathematical framework for understanding the impact of residual learning error on stability and safety guarantees through the lens of ISS and ISSf. In Section 3.7 I will consider some structural challenges that occur with the learning approaches in Sections 3.3 and 3.5 for high-dimensional systems, and explore an alternative learning framework using projected disturbances that can overcome these challenges. This work culminates in Section 3.8, wherein I will draw upon the lessons learned in the previous sections to propose an alternative approach for data-driven control using CLFs and CBFs that is constructed through robust convex optimization. This leads to a controller specified via a second-order cone program (SOCP), which is a convex optimization problem of a greater complexity than the QP-based controllers considered in Chapter 2 and Sections 3.3-3.7. Lastly, in Section 3.9, I will propose a framework for achieving safety of systems with parametric model error through the notion of adaptive CBFs. Key contributions of this work are described at the beginning of each respective section. 3.1 Related Work Learning with Control Lyapunov Functions The certification of stability provided by Lyapunov functions is an appealing tool for learning and control methods that seek to ensure a controller built 66 using learning is stabilizing. Early work in the topic of learning for dynamic systems used Lyapunov functions as a tool to ensure the stable learning of dynamics and regions of attraction of nonlinear systems [143], [144]. Similarly, Lyapunov functions were paired with model-based reinforcement learning (RL) to synthesize optimal controllers that provide stability guarantees [145], [146]. Further work has looked how existing models and controllers can be refined online while being kept stable using Lyapunov functions [147], and how Lyapunov functions can be learned from counterexamples [148]. A comprehensive survey on methods for learning Lyapunov functions can be found in [45]. The combination of learning with CLFs in an effort to synthesize stabilizing controllers, rather than just certify stability, was first considered in [17] to ensure stability of robotic reaching motions. The work in [38] proposed an uncertainty-based CLF using Gaussian Process Regression (GPR) that allowed the system to be driven into regions of the state space where there was high confidence in the learning model’s ability to predict model error in the system dynamics. Other work has looked at how CLFs can be learned from counterexamples [39], or how CLFs can be used to in the cost function of an RL problem to reduce sample-complexity [40]. My work in [149], which is presented in Section 3.3, was the first to look at learning model error as it directly impacts the time derivative of a CLF. This approach was motivated by the observation that only model errors appearing in the time derivative of a CLF impacted the local exponential stability of the system, and a complete characterization of model errors in the full state dynamics was unnecessary to certify stability. To ensure the well-posedness of this approach, I develop an understanding of structural assumptions that can be made about the true system that allow transferring a CLF for a nominal model to the true system. This enables enforcing a specific structure upon the learning models that allow them to be integrated into the optimization-based control framework described in Section 2 without destroying convexity. An analysis of the impact of residual learning error on local exponential stability guarantees was developed in my work in [150], which is presented in Section 3.4. Subsequent approaches have built upon the idea of learning model error directly as it impacts a CLF time derivative, such as in the context of RL [151]. Other work used affine GPR models to learn model errors in a CLF time derivative and produce a robust optimization-based controller [46] based on 67 the optimization-based control framework described in Section 2. My work in [152], discussed in Section 3.8, also takes a robust optimization-based approach for synthesizing a stabilizing controller directly using data, but takes a different perspective on quantifying model error. A comprehensive survey of CLFs being integrated with learning methods can be found in [41]. Learning with Control Barrier Functions A significant amount of recent work has looked at utilizing learning with CBFs for dynamic systems. An indicator of this is the several recent survey papers focused on the two topics, including one focused on learning CBFs [45], one focused on learning applications using CBFs [41], one looking at safe learning in robotics [133], and my own work surveying connections between datadriven safety-filter techniques developed through Hamilton-Jacobi Reachability, CBFs, and predictive control [153]. One key reason for this magnitude of development compared to learning with CLFs is the permissive behaviors that can be achieved when enforcing safety of systems with CBFs. In particular, rather than stabilizing a system to a particular point or a trajectory, CBFs can allow a system to explore a safe region as it acquires data, builds learning models, and discovers optimal behaviors. The first, and perhaps most obvious, way in which CBFs and learning can be integrated is by learning model errors that impact CBF-based controllers. Some approaches seek to do this task online, with the system required to remain safe during the process of collecting data and building learning models. This challenge was first explored in [16], where the dynamics of a quadrotor were safely learned using GPR by enforcing CBF constraints during the data collection process, while further work has looked at this task of safely learning dynamics online, either through Bayesian learning approaches [43], [154], [155] or online disturbance estimation [156]. Other approaches have taken an offline approach in which the system collects data, learning models are built offline, and the models are integrated into subsequent controller designs. The work in [42] looks at learning nonlinear dynamics using GPR and integrating the learning models into a CBF-based controller. Similar approaches using GPR and CBFs have been explored for learning in the context of decentralized collision avoidance [157] and elastic joint robots [158]. Additional approaches use Koopman operators to learn open-loop dynamics to be used in CBF-based controller synthesis [159], or learn the trajectories of closed-loop systems under 68 back-up set CBF-based controllers to reduce online computations [160]. Inspired by my work on learning model error as it impacts the time derivatives of CLFs presented in Sections 3.3 and 3.4, my work in [161], discussed in Section 3.5, looked at how model error could be learned directly as it impacts the time derivative of a CBF. This was an offline learning approach, in which data is episodically collected, integrated into structured learning models that preserve convexity of downstream control synthesis, and iteratively improve the safety of a system. Based on this approach, I developed a rigorous analysis of the impact of residual learning error on the safety of the system through the lens of ISSf in my work in [162], discussed in Section 3.6. This idea of learning model error only as it impacts a CBF time derivative was extended to the RL setting in [151], and used in online learning with extreme learning machines [163]. This idea was further explored in my work in [25], discussed in Section 3.7, which looked at learning unstructured model errors in a CBF time derivative when working with high-dimensional systems. One direction of research that extended on the idea of learning model uncertainty directly as it impacts the time derivative of a CBF was through the use of robust optimization. This work began in the context of CLFs in [46], but my work in [152] done in parallel provided an alternative perspective that was applied to both CLFs and CBFs. The work in [46] was later extended to the setting of CBFs with a focus on verifying the feasibility of the robust optimization-based controllers in [164], and was subsequently utilized in an online learning setting in [165]. I note that the robust control approach developed in my work [152] has been specialized for certain cases of model uncertainty that allow for more efficient robust optimization problems in [166] and [167]. Another area that has seen rapid development in integrating learning with CBFs is safe RL. First steps in this area occurred in [44], [168], where CBFs were used to restrict policy updates such that the RL algorithm would only return safe policies. Work beyond this has sought to improve optimality and relax requirements on knowledge of a conservative but safe controller when starting the RL algorithm [169], [170]. Other work has looked at using approximate dynamic programming methods for safely solving the optimal control problem posed by RL online [171], multi-agent RL using multiple CBFs [172], and safe inverse RL [173]. Though not explicitly RL, the recent work in [174] looks at how online imitation learning can be safely performed using CBFs. 69 Instead of learning model errors or controllers that optimize reward functions, another avenue of recent work has explored how CBFs can be learned for nonlinear control systems. The first work considering this challenge was [175], which looked to learn CBFs for motion planning of robotic manipulators from human demonstrations. Following this, a framework for building CBFs from data was proposed in [176], and extended to hybrid nonlinear systems in [177]. Other developments have looked at the construction of CBFs online for robotic systems operating in unknown environments using distance measurements [178], using RL to synthesize CBFs that capture tolerable levels of failure risk [179], integrating learned CBFs into Nonlinear Model Predictive Control (NMPC) [180], and iteratively learning linear CBFs to approximate a complex set that is to be kept forward invariant [181]. Learning and CBFs have been integrated in several other creative research directions. One direction has looked at the tuning of CBFs and CBF-based controllers, such as using RL to tune CBF parameters to enable efficient collision avoidance of robotic manipulators [182], using RL to ensure input constraints are satisfied [183], or my own work in [23], discussed in Section 5.4, which uses Preference-Based Learning (PBL) to tune robustness parameters of a robust CBF controller in an effort to meet performance requirements. Another direction has been in ensuring safety of controllers using learned vision modules, such as my own work [184], or work on safe self-supervised online learning using stereo vision [185]. Other ideas have looked at how CBFs can be used to define cost functions for learning problems that produce bipedal walking gaits [186], how CBFs can be used to quantify the safety of learning-based autonomous driving (without controlling the system itself) [187], or how CBF safety filters can be cheaply approximated using learning models [188]. Adaptive Safety My work in adaptive safety through CBFs was motivated by the existing notion of adaptive CLFs [47] that were used to stabilize systems in the presence of parametric model error. Adaptive CLFs were extended into a modular framework focused on ISS [189], considered in an inverse optimal control problem [48], and integrated with sums-of-squares (SOS) programming [49]. Recently, they have been used in an experimental setting on a brachiating robot [20]. Absent from these works was a focus on adaptively enforcing the safety of a particular set in the state space, with the set being kept invariant using adap- 70 tive CLFs lying in the composite state and parameter estimate space. My work in [190] sought to translate much of the framework for adaptive CLFs into the setting of adaptive safety, yielding a definition of adaptive CBFs. This work explored how enforcing safety of a particular set in the state space in the presence of parametric model error is more difficult than stabilizing a system in the presence of parametric model error. A key extension of this work integrated previously collected data by the system into the adaptation algorithm [191], similarly to concurrent learning [192], allowing for a reduction in conservativeness with respect to safety as a system evolves. Further extensions have looked at fixed time adaptation laws [193], hybrid adaptation laws [194], multiple CBFs [195], higher-order CBFs [196], utilizing the certainty-equivalence principle to simplify requirements for adaptive safety [197], dynamic regressor extension and mixing [198], modular adaption law design [199], and approaches for permitting parametric model error in the actuation matrix g [200]. 3.2 Model Errors Recall the open-loop system (2.1). In many practical settings, a control designer may not have perfect knowledge of the functions f and g. This lack of knowledge may arise due to unmodeled effects such as friction, or due to errors in system parameters such as inertias. Instead, control design is done with a nominal model that estimates the true open-loop system (2.1): ḃ = b b(x)u, x f (x) + g (3.1) b : E → Rn×m are assumed to be locally where the functions b f : E → Rn and g Lipschitz continuous on E. I note that this construction implicitly assumes that the true system can be described by the state vector x, and that the true open-loop system (2.1) is control-affine and time-invariant. I further note that this permits error in how the actuation matrix g is modeled. Many existing approaches assume certainty in how control inputs enter the system [38], [201], [202]. These structural assumptions will ensure that the learning problem in Sections 3.3 and 3.5 is well-posed, while they are necessary for the stability and safety guarantees achieved by the data-driven control approach in Section 3.8. Adding and subtracting the nominal model open-loop system (3.1) from the true open-loop system (2.1) yields the following description of the true 71 open-loop system: b(x) + f (x) − b b(x)) u, ẋ = b f (x) + g f (x) + (g(x) − g | {z } | {z } (3.2) e(x) g e f (x) e : E → Rn×m are locally Lipschitz continuous on where e f : E → Rn and g b) and capture the error between the E (by the assumptions on f , g, b f , and g nominal model open-loop system (3.1) and the true open-loop system (2.1). Example 2. Consider an inverted pendulum with input attenuation near the upright configuration described by the true open-loop system: " # " # 0 x2 ẋ = g + 1−0.75 exp(−x2 ) u, (3.3) 1 sin(x1 ) ℓ mℓ2 with state x ∈ R2 , input u ∈ R, and gravitational acceleration, length, and mass parameters g, ℓ, m ∈ R>0 , respectively. I will consider a nominal model of the true open-loop system given by: # " # " 0 x 2 ḃ = + 1 u, x (3.4) g b sin(x ) 1 2 b b ℓ m bℓ bm where gb, ℓ, b ∈ R>0 are estimates of the gravitational acceleration, length, and e are given by: mass parameters. The corresponding functions e f and g # " # " 0 0 e e(x) = 1−0.75 exp(−x2 ) , g . (3.5) f (x) =  g gb  1 1 − ℓb sin(x1 ) −m 2 ℓ b mℓ2 bℓ A special case of model error that is typically considered in adaptive control is parametric model error in the drift dynamics, such that the true open-loop system (2.1) can be written as: b(x) u, ẋ = b f (x) + F(x)θ ⋆ + g | {z } |{z} f (x) (3.6) g(x) where the function F : E → Rn × Rn×p is locally Lipschitz continuous on E and θ ⋆ ∈ Rp . The function F captures a collection of basis functions that are known to parameterize the model error, and θ ⋆ is the vector of true parameters that describe the system. Note that this construction does not permit error in the actuation matrix (e g(x) = 0n×m for all x ∈ E). Recalling the open-loop robotic system (2.5), this can be a limiting setting as the mass and inertia terms which appear in the matrix D are often estimated imperfectly, leading b. to error between g and g 72 Example 3. Considering the inverted pendulum with input attenuation in Example 2, suppose that m b = m, ℓb = ℓ, and the attenuation term −0.75 exp(−x21 ) is absent. Then the true open-loop system is of the form in (3.6) with: " # 0 F(x) = , (3.7) sin(x1 ) g . and θ⋆ = g−b ℓ 3.3 Learning with Control Lyapunov Functions In this section I will present an episodic learning approach for mitigating model errors as they directly impact the time derivative of a local exponential CLF. The motivation for learning model error in this way is the idea that not all of the model error needs to be identified to achieve local exponential stability, but rather, only the model uncertainty that impacts the time derivative of a corresponding local exponential CLF. This learning approach will utilize a structured learning model that can be directly integrated into the convex optimization-based (CLF-QP) controller to improve the stability of a system, without destroying convexity. One challenge in developing learning-based methods for controller improvement is how best to collect training data that accurately reflects the desired operating environment and control goals. In particular, exhaustive data collection typically scales exponentially with dimensionality of the joint state and control input space, and so should be avoided. But first pre-collecting data upfront can lead to poor performance as downstream control behavior may enter states that are not present in the pre-collected training data. I will leverage the episodic learning idea of Dataset Aggregation (DAgger) [203] to address these challenges in a data-efficient manner, and yield iteratively refined controllers. The contributions of this section are as follows: • An episodic learning algorithm that identifies model errors as they directly impact the time derivative of a local exponential CLF using a structured learning model that permits use of the convex optimizationbased control framework described in Section 2.3. • A characterization of the well-posedness of learning and control problems using CLFs by justifying the transfer of a local exponential CLF for a 73 nominal model open-loop system to the true open-loop system using assumptions on structural properties of the true open-loop system. The text for this section is adapted from: A. J. Taylor, V. D. Dorobantu, H. M. Le, Y. Yue, and A. D. Ames, “Episodic learning with control lyapunov functions for uncertain robotic systems,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Macau, China, 2019, pp.6878-6884. A. J. Taylor participated in the conception of the project, algorithm design, simulation code implementation, and writing of the article. Uncertainty in Control Lyapunov Functions Let xe ∈ E be an unforced or forced equilibrium point with equilibrium input b e ∈ Rm for the nominal model open-loop system (3.1). Consider a local u exponential CLF V : E → R≥0 for the nominal model open-loop system (3.1) and equilibrium point xe ∈ E, with corresponding open set D ⊆ E with xe ∈ D, and k3 , a ∈ R>0 , such that: ∂V b(x)u) < −k3 ∥x − xe ∥a , (x)(b f (x) + g infm Vḃ (x, u) ≜ infm u∈R ∂x u∈R (3.8) for all x ∈ D \ {xe }. The time derivative of the function V uses the true open-loop system (2.1), and thus using (3.2), we have that: ∂V ∂V V̇ (x, u) = Vḃ (x, u) + (x)e f (x) + (x)e g(x) u. |∂x {z } |∂x {z } b(x) (3.9) a(x)⊤ Consequently, choosing an input u ∈ Rm such that the inequality in (3.8) is satisfied is insufficient to conclude that the time derivative of the local exponential CLF V will meet the required inequality to guarantee local exponential stability because of the functions b : E → R and a : E → Rm . Note that the e only as they impact the functions b and a capture the model errors in e f and g evolution of the local exponential CLF V , and consequently, local exponential stability. This motivates considering a learning-based approach to identify the functions b and a such that their effects on the CLF time derivative can be accounted for in the control design process, and local exponential stability can be achieved. Note that by explicitly learning the functions b and a, instead 74 of learning their combined effect in a single function δ(x, u) = b(x) + a(x)⊤ u that takes both the state and control input as inputs, highly accurate nonlinear learning models can be used without destroying the control-affine nature of V̇ that permits control synthesis through convex optimization. To ensure the well-posedness of this approach, I make the following assumptions: Assumption 1. The point xe ∈ E is an unforced or forced equilibrium point of true open-loop system (2.1) with known equilibrium input ue . Assumption 2. The function V is a CLF for the true open-loop system (2.1) and equilibrium point xe with corresponding open set D ⊆ E with xe ∈ D, and constants k3 , a ∈ R>0 . The first assumption states that trying to stabilize the true open-loop system (2.1) to the point xe is well-posed, as xe can be rendered an equilibrium point of the true open-loop system (2.1). It also states that the equilibrium input ue needed to render xe an equilibrium point of the true open-loop system (2.1) is b e ̸= ue ). known (it may be different than the one for the nominal model, i.e., u The second assumption states that V is a local exponential CLF for the true open-loop system (2.1) and equilibrium point xe with the same open set D and constants k3 , a ∈ R>0 used for the nominal model open-loop system (3.1). It is important to note that this is an assumption on how the true open-loop system (2.1) can be controlled. In particular, this assumption states that for a given x ∈ D \ {xe }, one knows that there exists an input u ∈ Rm such that: V̇ (x, u) = Vḃ (x, u) + b(x) + a(x)⊤ u < −k3 ∥x − xe ∥a , (3.10) but not necessarily which value of u satisfies the inequality. If one knew the functions b and a perfectly, the value of u satisfying this inequality could be found. Thus, if one could learn these functions from data, the control synthesis problem through CLFs would be well-posed. To better under this assumption consider the following example of a fully-actuated robotic system. Example 4. Consider the following nominal model for a fully-actuated robotic system: b b b b D(q)q̈ + C(q, q̇) + G(q) = B(q)τ , (3.11) b : Q → Snc , C b : X → Rnc , G b : Q → Rnc , and B b : Q → Rm are locally where D ≻0 Lipschitz continuous functions on their respective domains. These functions 75 are nominal models of the corresponding functions D, C, G, and B in the true open-loop robotic system (2.5). Let (qe , 0) ∈ X be a forced equilibrium point of the nominal model open-loop robotic system (3.11) with equilibrium input: b e )−1 (C(q b e , 0nc ) + G(q b e )). τb e = B(q (3.12) Let us suppose that the true open-loop robotic system is fully-actuated, such that B(q) is invertible for each q ∈ Q. Then (qe , 0) is a forced equilibrium point of the true open-loop robotic system (2.5) with equilibrium input: τ e = B(qe )−1 (C(qe , 0nc ) + G(qe )). (3.13) i⊤ h bfbl : X × Rnc → Let η = (q − qe )⊤ q̇⊤ . A feedback linearizing controller k Rm using the nominal model open-loop robotic system (3.11) can be designed: −1 b bfbl (x, −Kη) = B(q) b −1 D(q)( b b b k D(q) (C(q, q̇) + G(q)) − Kη), (3.14) with K ∈ Rnc ×n . For the nominal model open-loop robotic system (3.11), this yields the linear closed-loop output dynamics: η̇ = Acl η, (3.15) which are locally exponentially stable for an appropriate choice of K. As in Section 2.4, for a given Q ∈ Sn≻0 , the CTLE can be solved to produce a matrix P ∈ Sn≻0 that defines a local exponential CLF V for the nominal model open-loop robotic system (3.11) and equilibrium point (qe , 0). Considering the true open-loop robotic system (2.5), a feedback linearizing controller kfbl : X × Rnc → Rm can be specified as: kfbl (x, −Kη) = B(q)−1 D(q)(D(q)−1 (C(q, q̇) + G(q)) − Kη). (3.16) Similarly, for the true open-loop robotic system (2.5), this controller leads to the same closed-loop linear output dynamics in (3.15), for which the CTLE produces the same matrix P ∈ Sn≻0 that defined the local exponential CLF V . While the controller (3.16) can not be evaluated as the functions D, C, G, and B are not precisely known, the input it would return exists, and thus it is sufficient to conclude that V is a local exponential CLF for the true open-loop robotic system (2.5) and equilibrium point (qe , 0). Thus, the assumption of V being a local exponential CLF for the true open-loop robotic system (2.5) is justified under other structural assumptions such as the robotic system being fully-actuated. This ensures that successfully learning the functions b and a will lead to a well-posed control problem. 76 Data-Driven Episodic Learning Framework Given that the formulation in (3.2) and (3.9) defines a general class of model error (rather than parametric), it is natural to consider a data-driven method to estimate the unknown functions b and a on the domain D. To motivate a learning-based framework, first consider a simple approach of learning b and a via supervised regression [204]: the true closed-loop system (2.2) is operated using a given controller k : D → Rm , data points are gathered along the system’s evolution, and functions that approximate b and a are produced via supervised learning. Concretely, let x0 ∈ D be an initial state. An experiment is defined as the evolution of the true closed-loop system over a finite time interval from the initial condition x0 using a sample-hold implementation of the controller k. A resulting discrete-time state history is obtained, which is then transformed with the local exponential CLF V and finally differentiated numerically to estimate V̇ throughout the experiment. This yields a data set composed of input-output pairs: m D = {((xi , ui ), V̇i )}N i=1 ⊆ (D × R ) × R. (3.17) Consider a class Hb of nonlinear functions mapping Rn to R and a class Ha of nonlinear functions mapping Rn to Rm . For a given bb ∈ Hb and b a ∈ Ha , ċ define the structured estimator W as: ċ (x, u) = Vḃ (x, u) + bb(x) + b W a(x)⊤ u, (3.18) and let H be the class of all such estimators mapping Rn × Rm to R. Observe that the structure of this estimator ensures it is affine in the control input, and thus can be directly integrated into the convex optimization-based control framework presented in Section 2.3. Defining a loss function L : R × R → R≥0 , the supervised regression task is then to find a function in H via empirical risk minimization (ERM): N 1 X ċ inf L(W (xi , ui ), V̇i ). b∈Ha N a i=1 (3.19) b b∈Hb This experiment protocol can be executed either in simulation or directly on hardware. While being simple to implement, supervised learning critically assumes independently and identically distributed (i.i.d) training data. Each 77 experiment violates this assumption, as the regression target of each data point is coupled with the input data of the previous time step. As a consequence, standard supervised learning with sequential, non-i.i.d data collection often leads to error cascades [142]. Episodic learning refers to learning procedures that iteratively alternate between executing an intermediate controller (also known as a roll-out in reinforcement learning [132]) to collect data from that roll-out, and designing a new controller using the newly collected data. My approach integrates learning b and a with improving the performance and stability of the controller in such an iterative fashion. First, assume we are given a nominal controller knom : E → Rm , which may not locally exponentially stabilize the resulting true closed-loop system (2.2) to the equilibrium point xe , but satisfies ċ ∈ H defined in (3.18), specify a learningknom (xe ) = ue . With an estimator W informed (CLF-QP) controller as: k(x) = argmin ∥u − knom (x)∥2 u∈Rm (3.20) ċ (x, u) ≤ −k ∥x − x ∥a . s.t. W 3 e This learning-informed (CLF-QP) controller finds the closest input to knom (x) such that the local exponential CLF inequality is met using the estimator ċ degrades the local exponential ċ for V̇ . However, error in the estimator W W stability of the true closed-loop system (2.2) with respect to the equilibrium point xe . ċ , an experiment In an effort to reduce the remaining error in the estimator W can be run using the learning-informed (CLF-QP) controller to produce data by which to obtain better estimates of b and a. To overcome the limitations in needing i.i.d data for conventional supervised learning, I leverage reduction techniques: a sequential prediction problem is reduced to a sequence of supervised learning problems over multiple episodes [203], [205]. In particular, in each episode, an experiment generates data using a different controller. The data set is aggregated and a new ERM problem is solved after each episode. This episodic learning implementation is inspired by the Data Aggregation (DAgger) algorithm [203], with some key differences: • DAgger is a model-free policy learning algorithm, which trains a policy directly in each episode using optimal control input oracles. As such 78 an oracle is not available in my problem setting, my algorithm defines a controller indirectly via a local exponential CLF as in (3.20) to explicitly drive the system towards stable behavior. • The ERM problem is underdetermined, i.e., different approximations (bb, b a) may achieve similar loss for a given data set while failing to accurately model b and a. This issue will be explored more in Sections 3.7 and 3.8. This potentially introduces error in estimating V̇ for control inputs not reflected in the training data, and necessitates the use of exploratory control action to constrain the estimators bb and b a. Such exploration can be achieved by randomly perturbing the controller used in an experiment at each time step. This need for exploration is an analog to the notion of persistent excitation for adaptive systems [206]. Algorithm 1 specifies a method of computing a sequence of local exponential ċ and augmented1 controllers k . During the k th CLF derivative estimates W k k th episode, the k − 1 augmented controller is used in an experiment, data is collected and aggregated with the data set D and an estimate of the local ċ is formed. After this, a corresponding learningexponential CLF derivative W k informed (CLF-QP) controller as in (3.20) is produced, scaled by a heuristically chosen factor wk ∈ [0, 1] reflecting trust in the estimate and added to the nominal controller k0 = knom for use in the subsequent experiment as an augmented controller kk . The trust coefficients form a monotonically nondecreasing sequence. Importantly, this experiment need not take place in simulation; the same procedure may be executed directly on hardware. At a high level, this episodic approach makes progress by gathering more data in relevant regions of the state space, such as states close to xe . This extends the generalizability of the estimator in its use by subsequent controllers. There are a few practical concerns with this algorithm that should be observed. First, the system will not necessarily be locally exponentially stable during the episodic learning process. In general, one will often begin the learning process with a controller that is unstable, or displays desirable behavior but possess no guarantees of local exponential stability (such as a Lyapunov function). This algorithm instead tolerates instability during the episodic learning process to 1 I refer to these as augmented controllers if wk ̸= 1, as the learning-informed (CLF-QP) controller in (3.20) is not fully used, but blended with the nominal controller k0 . 79 Algorithm 1 Dataset Aggregation for Control Lyapunov Functions (DaCLyF) ċ = Vḃ , model Require: Local exponential CLF V , derivative estimate W 0 classes Ha and Hb , loss function L, nominal controller k0 = knom , number of experiments T , sequence of trust coefficients 0 ≤ w1 ≤ · · · ≤ wT ≤ 1. D=∅ for k = 1, . . . , T do x0 ← sample D Dk ← experiment(x0 , kk−1 ) D ← D ∪ Dk ċ ) b ak , bbk ← ERM(Ha , Hb , L, D, W 0 ċ ċ Wk ← W0 + b a⊤ u + bbk ▷ Initialize data set ▷ Get initial condition ▷ Run experiment ▷ Aggregate data set ▷ Fit estimators ▷ Update derivative estimator ċ ▷ Update controller kk ← (1 − wk ) · k0 + wk · augment(k0 , W k ) end for ċ , k return W T T k ensure that the system performs adequate exploration in the data acquisition process and return a controller at the end of the episodic learning for which meaningful stability properties can be established, as explored in Section 3.4. Second, there can be issues that arise with the constraint in the learninginformed (CLF-QP) controller (3.20). In particular, while we know that: inf Vḃ (x, u) < −k3 ∥x − xe ∥a , u∈Rm (3.21) for all x ∈ D \ {xe }, and we assume that: inf V̇ (x, u) < −k3 ∥x − xe ∥a , u∈Rm (3.22) for all x ∈ D \{xe }, such that the local exponential CLF constraint can be met for the nominal model open-loop system (3.1) and the true open-loop system (2.1), we do not know that: ċ (x, u) < −k ∥x − x ∥a , infm W k 3 e u∈R (3.23) for all x ∈ D \ {xe } for k = 1, . . . , T . In particular, the learned models bb and b a can lead to this condition being violated. This means that the learninginformed (CLF-QP) controller will not necessarily be feasible for all x ∈ D. This issue is noticeable at the equilibrium point xe , as we must have: bbk (xe ) + b ak (xe )⊤ ue ≤ 0, (3.24) 80 ċ (x , u ) ≤ 0 (as Vḃ (x , u ) = 0), and or the equilibrium input will not satisfy W k e e e e thus the controller will not use the equilibrium input at the equilibrium point. In practice, the relaxation technique used in the (CLF-CBF-QP) controller is utilized, such that the problem always remains feasible at the expense of rigorous stability guarantees. This highlights a particular challenge in learning and control, which is the analytic nature of the stability and safety guarantees that come from CLFs and CBFs, and the numerical nature of learned models. In reality, one will likely never be able to learn the functions b and a perfectly. Instead, stability and safety guarantees will need to be built keeping in mind that residual learning error arising from numerical learning methods will be present in learning models, as explored in Sections 3.4 and 3.6. Simulation Results I apply this episodic learning algorithm to a planar Segway platform seen in Figure 3.1. In particular, I consider a 4-dimensional planar Segway model based on the simulation model in [27]. The system states consist of horizontal position x ∈ R, horizontal velocity ẋ ∈ R, pitch angle θ ∈ R, and pitch angle rate θ̇ ∈ R. Control is specified as a single voltage input supplied to both wheel motors. The parameters of the model (including mass, inertias, and motor parameters but excluding gravity) are randomly modified by up to 10% of their nominal values and then held fixed for the simulations. Note that even b and g, though this is parametric model error, it introduces error between g which is typically not considered by adaptive control approaches. I seek to stabilize the pitch angle and pitch angle rate to a trajectory generated using the GPOPS-II Optimal Control Software [207] for the nominal model2 . The nominal controller k0 = knom is a linear proportional-derivative (PD) controller on angle and angle rate error. A set of T = 20 experiments are conducted with trust values wk varying from 0.01 to 0.99 in a sigmoid fashion. The exploratory control is drawn uniformly at random between −20% and 20% of the norm of the control input specified by the augmented controller kk for 2 More precisely, the definition of local exponential stability in Definition 2 can easily be extended to use a desired trajectory xd : R≥0 → E that satisfies the differential equation (2.1) for some input signal ud : R≥0 → Rm instead of an equilibrium point xe . Furthermore, the Segway system is underactuated, and thus the CLF considered is the one for the outputs (the pitch angle tracking error and pitch angle rate tracking error) (2.36) produced using feedback linearization. The goal is thus to drive the outputs to 02 , but the stability of the zero dynamics (and full system stability) are not considered. 81 t=0 t=1 t=2 t=3 t=4 t=5 Figure 3.1. (Left) Nominal model-based (CLF-QP) controller fails to stabilize the system to the trajectory. (Right) Improvement in angle tracking of system with learning-informed (CLF-QP) controller over nominal PD controller. (Bottom) Corresponding visualizations of state data. Video of this simulation is found at https://youtu.be/cB5MY_8vCrQ. the first ten episodes. The percentages decay linearly to zero in the remaining ten episodes. The model classes selected are sets of two-layer neural networks with Recitified Linear Unit (ReLU) nonlinearities with a hidden layer width of 2000 nodes that are implemented in Keras [208]. Failure of the nominal model-based (CLF-QP) controller is seen in the left panel of Figure 3.1. The nominal PD controller k0 = knom and the final learning-informed (CLF-QP) controller k20 produced using w20 = 1 can be seen in the right panel of Figure 3.1. Corresponding visualizations of the Segway states are displayed in the bottom panel of Figure 3.1. The final learning-informed (CLF-QP) controller exhibits a notable improvement over the nominal model-based (CLF-QP) controller and nominal PD controller in its ability to track the desired trajectory. To verify the robustness of the learning algorithm, the 20 experiment process was conducted ten times. After each experiment the augmented controller kk was tested without exploratory perturbations. For the last three experiments and a test of the final learninginformed (CLF-QP) controller k20 , the minimum, mean, and maximum angles across all ten instances are displayed for each time step in Figure 3.2. The mean trajectory consistently improves in these later episodes as the trust fac- (rad) (rad) 82 Figure 3.2. Augmented controllers consistently improve stabilization to trajectory across episodes. Ten instances of the algorithm are executed with the shaded region formed from minimum and maximum angles for each time step within an episode. The corresponding average angle trajectories are also displayed. tor increases. The variation increases but remains small, indicating that the learning problem is robust to randomness in the initialization of the neural networks, in the network training algorithm, and in the noise added during the experiments. The performance of the controller in the earlier episodes displayed negligible variation from the nominal PD controller k0 = knom due to small trust factors. Conclusion In this section I have presented an episodic learning framework for learning model errors between a nominal model open-loop system and the true openloop system as they directly impact the time derivative of a local exponential CLF V . I provide a discussion on assumptions that ensure a local exponential CLF V for the nominal model open-loop system is a local exponential CLF for the true open-loop system, and provide justification of these assumptions 83 through an example in the context of a fully actuated robotic system. I then describe the episodic learning framework, and address practical concerns that arise when integrating the analytic tools of local exponential CLFs with the numerical tools of learning models. Lastly I provide a demonstration of the efficacy of the algorithm in simulation on a planar Segway system. 3.4 Projection-to-State Stability In this section I will present an analysis of the effects of residual learning error on the local exponential stability of the true closed-loop system (2.2) using a learning-informed (CLF-QP) controller. Building off of the episodic learning algorithm in Section 3.3, I will consider learning model errors as they impact the time derivative of a local exponential CLF. I will first demonstrate that the inherent robustness properties of local exponential CLF-based controllers yield a general robustness to residual learning errors in this learning and control setting. This will be achieved by looking at the evolution of a local exponential CLF as a disturbed dynamic system through a modification of ISS known as Projection-to-State Stability (PSS). Second, I will consider the possible ċ as residual learning errors that can occur given a data set D and estimator W in Section 3.3, and what this will mean for the behavior of the true closed-loop system (2.2) using an imperfect learning-informed controller. The contributions of this section are as follows: • A notion of robust stability in the form of Projection-to-State Stability (PSS), which captures the impact of disturbances on stability only as they impact the time derivative of a CLF, rather than the full-order dynamics, as is done by ISS discussed in Section 2.7. • A data-driven characterization of possible residual learning errors when learning model errors as they impact the time derivative of a local exponential CLF. The convex representations that arise from this characterization are the foundation of the work in Section 3.8. The text for this section is adapted from: A. J. Taylor, V. D. Dorobantu, M. Krishnamoorthy, H. M. Le, Y. Yue, and A. D. Ames, “A control lyapunov perspective on episodic learning 84 via projection to state stability,” in Proc. IEEE 58th Conf. on Decision and Control (CDC), Nice, France, 2019, pp.1448-1455. A. J. Taylor participated in the conception of the project, theoretical analysis, simulation code implementation, and writing of the article. Projection-to-State Stability and Residual Learning Error Let xe ∈ E be an equilibrium point of the nominal model open-loop system be (3.1) and the true open-loop system (2.1) with known equilibrium inputs u and ue , respectively. Suppose that V : E → R≥0 is a local exponential CLF for the nominal model open-loop system (3.1) and true open-loop system (2.1) and equilibrium point xe with corresponding open set D ⊆ E with xe ∈ D. This amounts to the requirements of Assumptions 1 and 2 made in Section 3.3. Furthermore, consider the nominal model-based (CLF-QP) controller: b k(x) = argmin ∥u − ue ∥2 u∈Rm (3.25) s.t. Vḃ (x, u) ≤ −k3 ∥x − xe ∥a . This controller corresponds to the (CLF-QP) controller using the nominal model open-loop system (3.1) for the time derivative of the local exponential b may be written as: CLF V . The true closed-loop system using the controller k b b b b(x)k(x) e(x)k(x). ẋ = f (x) + g(x)k(x) =b f (x) + g +e f (x) + g (3.26) For a given initial condition x0 ∈ D, denote the solution to the true closed-loop system (3.26) as φ : I(x0 ) → D. Alternatively, this system can be understood to evolve as a nominal model closed-loop system with unmatched disturbances: b b(x)k(x) ẋ = b f (x) + g + d(t), (3.27) where for any initial condition x0 ∈ D the disturbance signal is defined as: b e(φ(t))k(φ(t)). d(t) = e f (φ(t)) + g (3.28) Viewing model error as an unmatched disturbance in the nominal model closed-loop system motivates studying it through the lens of Input-to-State Stability (ISS). If the nominal model closed-loop system with unmatched disturbances (3.27) is ISS with respect to the equilibrium point xe , then small model errors will lead to small disturbances and a small degradation in the 85 local exponential stability that would be demonstrated by the nominal model closed-loop system without disturbances: (3.29) b b(x)k(x). ẋ = b f (x) + g In the context of learning the time derivative of a local exponential CLF as in Section 3.3, the disturbance signal is not as readily understood in the state dynamics. In particular, recall that: (3.30) V̇ (x, u) = Vḃ (x, u) + b(x) + a(x)⊤ u. ċ ∈ H as: Let bb ∈ Hb and b a ∈ Ha compose the estimator W (3.31) ċ (x, u) = Vḃ (x, u) + bb(x) + b W a(x)⊤ u, and let us assume that: (3.32) ċ (x, u) < −k ∥x − x ∥a , inf W 3 e u∈Rm ċ (x , u ) ≤ 0. Adding and subtracting the for all x ∈ D \ {xe }, and3 that W e e b estimators b and b a from V̇ yields: (3.33) ċ (x, u) + b(x) − bb(x) + (a(x) − b a(x))⊤ u. V̇ (x, u) = W {z } | {z } | e(x)⊤ a e b(x) The functions eb : E → R and e a : E → Rm capture residual learning error left after trying to learn the functions b and a with the estimators bb and b a. As these error terms appear in the time derivative of the local exponential CLF V , they will consequently have an impact on the stability properties of the system. Using the learning-informed (CLF-QP) controller: (3.34) k(x) = argmin ∥u − ue ∥2 u∈Rm ċ (x, u) ≤ −k ∥x − x ∥a , s.t. W 3 e 3 This assumption will lead the learning-informed controller in (3.34) to satisfy k(xe ) = ue , such that xe is an equilibrium point of the true closed-loop system (2.2). This assumption can often practically be enforced at the expense of additional residual learning error near the ċ (x , u ) = −ϵ equilibrium point x by providing “artificial” training data specifying that W e e e for some ϵ ∈ R>0 . A consequence of continuous learning models is that this artificial ċ (x, u ) ≤ −k ∥x − x ∥a for all x in a neighborhood of the modification will lead to W e 3 e equilibrium point xe , such that k(x) = ue , even if V̇ (x, ue ) > −k3 ∥x − xe ∥a for some values of x in that neighborhood. Thus, local exponential stability will be compromised. As we are interested in studying the effects of residual learning errors as disturbances in a dynamic system, we will only really be concerned with convergence to a neighborhood of the origin as captured by ISS, making the loss of local exponential stability tolerable. 86 yields: ċ (x, k(x)) + eb(x) + e V̇ (x, k(x)) = W a(x)⊤ k(x). (3.35) For any initial condition x0 ∈ D, let φ : I(x0 ) → D denote the solution to the true closed-loop system (2.2) using the learning-informed (CLF-QP) controller (3.34). The evolution of (3.35) can be viewed as a closed-loop system with unmatched disturbances: ċ (x, k(x)) + δ(t), V̇ (x, k(x)) = W (3.36) where for any initial condition x0 ∈ D the function δ : R≥0 → R defined as: δ(t) = eb(φ(t)) + e a(φ(t))⊤ k(φ(t)). (3.37) is a projected disturbance. As with model error in the state dynamics, this motivates studying residual learning error through the lens of ISS. Observe that the learning-informed controller enforces: V̇ (x, k(x)) ≤ −k3 ∥x − xe ∥a + |δ(t)|, k3 ≤ − V (x) + |δ(t)|. k2 (3.38) (3.39) Consequently, we have that the identity map ι : R≥0 → R≥0 defined as ι(r) = r for all r ∈ R≥0 satisfies: |V (x)| ≤ ι(V (x)) ≤ |V (x)|, ∂ι k3 k3 (V (x))V̇ (x, k(x)) ≤ − V (x) + |δ(t)| = − |V (x)| + |δ(t)|, ∂r k2 k2 (3.40) (3.41) and thus is a local exponential ISS-LF for the closed-loop system with unmatched disturbances (3.36) with respect to 0. Thus, there exist an open set D′ ⊆ D with xe ∈ D′ , constants M, λ, d ∈ R>0 , and a function γ ∈ K such that for any initial condition x0 ∈ D′ and d ∈ [0, d], if ∥δ∥∞ ≤ d, then the solution φ : R≥0 → D to the true closed-loop system (2.2) satisfies: V (φ(t)) ≤ M V (x0 )e−λt + γ(d), (3.42) for all t ∈ R≥0 . Using the bounds on V in (2.16), we have that: ∥φ(t) − xe ∥a ≤ M k2 1 ∥x0 − xe ∥a e−λt + γ(d). k1 k1 (3.43) 87 Using the weak class-K triangle inequality [56, Equation 12], we have that: r ∥φ(t) − xe ∥ ≤ a λ 2M k2 ∥x0 − xe ∥e− a t + k1 ′ ≜ M ′ ∥x0 − xe ∥e−λ t + γ ′ (d), q 2M k2 , λ′ = λa , and γ ′ (r) = k1    a1 2 γ(d) , k1 (3.44) (3.45)  a1 where M = . This bound appears like a statement of ISS for the true closed-loop system (2.2) using the learninginformed controller (3.34), but it is defined now in terms of a bound on the projected disturbance signal δ rather than a disturbance signal d in the state dynamics. Consequently, the true closed-loop system (2.2) with the learninginformed controller (3.34) is said to be Projection-to-State Stable (PSS) with respect to the equilibrium point xe and projection V . The utility of PSS is that it consider the impacts of model error only as it impacts the time derivative of the local exponential CLF. By doing so, it provides a mathematical framework for understanding the impact of introducing learning models for the time derivative of a local exponential CLF, as it is not obvious how to represent residual learning error in learning models for a CLF time derivative as a disturbance in the full state dynamics (2.2), making an analysis by standard ISS difficult. Furthermore, it may potentially produce less conservative statements on the degradation of local exponential stability by neglecting model error that does not affect stability. ′ a 2 γ(r) k1 Quantifying Residual Learning Error In the preceding construction, I use a local exponential ISS-LF to establish the existence of some open set D′ ⊆ D with xe ∈ D′ , constants M, λ, d ∈ R>0 , and function γ ∈ K such that if ∥δ∥∞ ≤ d, then conclusions can be drawn about the PSS properties of the true closed-loop system (2.2). Recalling that given a data set D and corresponding estimators bb and b a, the system experiences the particular projected disturbance signal in (3.37), this in and of itself is not a particularly useful result if we do not actually know that ∥δ∥∞ ≤ d for all initial conditions x0 ∈ D′ . Consequently, an important question is whether or not we can find an open set D′ ⊆ D with xe ∈ D′ , constants M, λ, d ∈ R>0 , and a function γ ∈ K satisfying the relationship in (3.42) for all t ∈ R≥0 and d ∈ [0, d], and for which we know that the particular disturbance signal in (3.37) at least satisfies ∥δ∥∞ ≤ d for all initial conditions x0 ∈ D′ . This task can be reduced to finding just an open set D′ ⊆ D with xe ∈ D′ and a constant 88 d ∈ R>0 such that for any initial condition x0 ∈ D′ , the solution φ : R≥0 → D to the true closed-loop system (2.2) is defined for all time and the projected disturbance defined in (3.37) satisfies |δ(t)| ≤ d for all t ∈ R≥0 . With these, the constants M, λ and function γ can be established using the standard proof showing ISS through an ISS-LF [56, Theorem 1]. While these concerns seem technical, the concrete concern in the context of learning is if the residual learning error near the equilibrium point xe is large, it may cause the system to evolve into states not present in the data set D, leading to even more residual learning error and a larger projected disturbance, leading to further deviation from xe and repetition of this cycle. This motivates using a data-driven approach to quantify the projected disturbance signal with the goal of finding the desired open set D′ and constant d ∈ R>0 , for which we know that for all initial conditions x0 ∈ D′ , the solution φ : R≥0 → D to the true closed-loop system (2.2) is defined for all time and the projected disturbance defined in (3.37) satisfies |δ(t)| ≤ d for all t ∈ R≥0 . To this end, I will first describe the notion of a set-valued uncertainty function, and show how such a function can be used to produce a set D′ and constant d. I will then provide a data-driven construction of such an uncertainty function. Let P(R × Rm ) denote the powerset of R × Rm , and define the following: Definition 22 (Uncertainty Function). A function ∆ : D → P(R × Rm ) is an uncertainty function if (eb(x), e a(x)) ∈ ∆(x) for all x ∈ D. It is straightforward to see that given an uncertainty function ∆, we have that: ċ (x, k(x)) + V̇ (x, k(x)) ≤ W sup (b + a⊤ k(x)), (3.46) (b,a)∈∆(x) for all x ∈ D. To see how this can be used, let us denote the c-sublevel set of the local exponential CLF V as: Ω(c) = {x ∈ E | V (x) ≤ c}. (3.47) For a given c ∈ R≥0 , I will denote the closure of Ω(c) in Rn by Ω(c). For simplicity, I will assume that there exists some c ∈ R>0 such that Ω(c) ⊂ E for all c ∈ [0, c). This is a mild technical assumption that can generally be shown to hold with large values of c for commonly used local exponential CLFs using the lower bound in (2.12). This leads to the following result: 89 Theorem 18. If a value c ∈ (0, c) satisfies: sup sup b + a⊤ k(x) ≤ x∈Ω(c) (b,a)∈∆(x) k3 c, k2 (3.48) and Ω(c) ⊂ D, then for the set D′ = Int(Ω(c)) and constant d = kk32 c, we have that for any initial condition x0 ∈ D′ , the solution φ : R≥0 → D to the true closed-loop system (2.2) exists for all time and the projected disturbance in (3.37) satisfies |δ(t)| ≤ d for all t ∈ R≥0 . This theorem states that if the worst-case residual learning error permitted by an uncertainty set ∆ satisfies a bound over a sublevel set of the local exponential CLF V , and the sublevel set is contained in the domain D where the time derivative of the local exponential CLF V can be bounded, we can produce the desired open set D′ and constant d. Proof. Note that for any c ∈ (0, c), the continuity of V yields that Ω(c) is closed (Ω(c) = Ω(c)), and thus ∂Ω(c) ⊂ Ω(c). Additionally, the lower bound in (2.12) ensures that the set Ω(c) is bounded, such that Ω(c) is compact. Consider an x ∈ ∂Ω(c), noting that x ∈ Ω(c) and thus x ∈ D. First, observe that V (x) = c. To see this, assume that V (x) < c (x ∈ / Ω(c) if V (x) > c). By the continuity of V on E, there exists an open set U with x ∈ U such that V (y) < c for all y ∈ U , and thus U ⊂ Ω(c). Thus x ∈ Int(Ω(c)), and thus (x) ̸= 0n . To see x∈ / ∂Ω(c), yielding a contradiction. Next observe that ∂V ∂x this, observe that x ̸= xe , as V (x) = c > 0. As we have that: inf Vḃ (x, u) < −k3 ∥x − xe ∥a < 0, u∈Rm (3.49) for all x ∈ D, we must have that ∂V (x) ̸= 0n , otherwise the left-hand side ∂x would be 0. Now, we have that: b + a⊤ k(x), (3.50) k3 V (x) + sup sup b + a⊤ k(y), k2 y∈Ω(c) (b,a)∈∆(y) k3 k3 ≤ − c + c = 0. k2 k2 (3.51) V̇ (x, k(x)) ≤ −k3 ∥x − xe ∥a + sup (b,a)∈∆(x) ≤− (3.52) Thus V̇ (x, k(x)) ≤ 0 for any x ∈ ∂Ω(c), and by Nagumo’s theorem [104], [123], the set Ω(c) is forward invariant for the true closed-loop system (2.2). 90 Consequently, by the compactness by Ω(c), for any initial condition x0 ∈ D′ ≜ Int(Ω(c)), we have that the solution φ : R≥0 → Ω(c) for the true closed-loop system (2.2) exists for all time (I(x0 ) = R≥0 ) [101, Chapter 2.4, Corollary 2]. Furthermore, from any initial condition x0 ∈ D′ we have that the actual projected disturbance encountered by the system satisfies: |δ(t)| ≤ k3 c ≜ d, k2 (3.53) for all t ∈ R≥0 , such that ∥δ∥∞ ≤ d. This theorem highlights a critical challenge when working with local exponential stability and data-driven methods. As the value of c increases, the bound on the worst-case residual learning error is allowed to grow. This corresponds to the fact that the inequality enforced in the learning-informed controller (3.34) drives the system to the equilibrium point xe more aggressively the further the state of the system is from xe , overpowering the projected disturbance. At the same time, the worst-case residual learning error at additional states (as Ω(c1 ) ⊂ Ω(c2 ) for c1 < c2 ) must be bounded. Furthermore, the value of c must be small enough that Ω(c) ⊂ D, where the learning-informed controller (3.34) is feasible. This creates a “racing” behavior, where the learning error needs to grow slowly enough with c such that a small enough sublevel set Ω(c) meeting the inequality in the theorem can be found inside D. While it may be possible to change the learning-informed controller k to ensure that the conditions of this theorem are met, one straightforward approach is to improve the uncertainty function ∆, as captured in the following theorem: Theorem 19. Consider uncertainty functions ∆, ∆′ : D → P(R × Rm ), and let Ω(c) ⊂ D. If ∆′ (x) ⊆ ∆(x) for all x ∈ Ω(c), then: sup sup b + a⊤ k(x) ≤ sup x∈Ω(c) (b,a)∈∆′ (x) sup b + a⊤ k(x). (3.54) x∈Ω(c) (b,a)∈∆(x) Proof. The proof is straightforward as for any x ∈ Ω(c): sup (b,a)∈∆′ (x) b + a⊤ k(x) ≤ sup b + a⊤ k(x). (3.55) (b,a)∈∆(x) This theorem states that if the possible uncertainties can be reduced (such that ∆′ (x) ⊆ ∆(x) for all x ∈ Ω(c)), the bound on the worst-case projected 91 disturbance can be lowered, making it easier to satisfy the inequality in (3.48) and thus achieve the forward invariance that yields a valid D′ and d. I will now explore how an uncertainty function ∆ can be built from a data set D of the form in (3.17), restated here: m D = {((xi , ui ), V̇i )}N i=1 ⊆ (D × R ) × R. (3.56) For notational convenience denote: D0 = {(xi , ui ) ∈ D × Rm | ((xi , ui ), V̇i ) ∈ D}. (3.57) This leads to the following result: e are bounded on D by known Theorem 20. Suppose4 that the functions e f, g constants Mef and Mge , respectively, and are Lipschitz continuous on D with known Lipschitz constants Lef and Lge , respectively. Then an uncertainty function ∆ can be constructed as: ∆(x) = {(b, a) ∈ R × Rm | ± (b + a⊤ ui ) ≤ ϵ(x, xi , ui ) for all (xi , ui ) ∈ D0 }, (3.58) m for all x ∈ D, where ϵ : D × D × R → R≥0 is continuous on its domain. Observe that for each x ∈ D, the set ∆(x) is a closed polyhedron, and bounded given sufficiently diverse control inputs in the data set (as will be made rigorous in Section 3.8). In this case, ∆(x) is a compact, convex set, and the supremum in (3.46) can be computed via a convex linear program (LP) that yields a finite optimal value. Proof. Define the observed learning error as: ċ (x , u ) , ℓ(xi , ui ) = V̇i − W i i (3.59) ċ . Observe which can be computed from the data set D and the estimator W that we can rewrite this observed learning error as: ℓ(xi , ui ) = eb(xi ) + e a(xi )⊤ ui . 4 (3.60) Recall that D is a set associated with a local exponential CLF, and thus these bounds and Lipschitz properties generally only need to hold over a small neighborhood of the equilibrium point xe . 92 Given a test point x ∈ D, we have that: a(x)⊤ ui + eb(xi ) − eb(x) + (e a(xi ) − e a(x))⊤ ui , ℓ(xi , ui ) = eb(x) + e (3.61) ≥ eb(x) + e a(x)⊤ ui − eb(xi ) − eb(x) − ∥e a(xi ) − e a(x)∥ ∥ui ∥, (3.62) following from the reverse triangle inequality, triangle inequality, and CauchySchwarz inequality. This can be rearranged as: eb(x) + e a(xi ) − e a(x)∥ ∥ui ∥. a(x)⊤ ui ≤ ℓ(xi , ui ) + eb(xi ) − eb(x) + ∥e (3.63) We have that: ∂V eb(xi ) − eb(x) = ∂V (xi )e f (xi ) − bb(xi ) − (x)e f (x) + bb(x) , ∂x ∂x ∂V ∂V ≤ (xi )e f (xi ) − (x)e f (x) + bb(x) − bb(xi ) . ∂x ∂x (3.64) (3.65) We can then perform the following manipulation: ∂V ∂V ∂V ∂V (xi )e f (xi ) − (x)e f (x) = (xi )e f (xi ) − (x)e f (xi ) ∂x ∂x ∂x ∂x ∂V ∂V (x)e f (xi ) − (x)e f (x) , + ∂x ∂x ∂V ∂V f (xi ) ≤ e (xi ) − (x) ∂x ∂x ∂V + f (xi ) − e f (x) , (x) e ∂x ≤ Mef ϵ∞ (x, xi ) + Lef ϵL (x, xi ), (3.66) (3.67) (3.68) where: ∂V ∂V (xi ) − (x) , ∂x ∂x ∂V ϵL (x, xi ) = (x) ∥x − xi ∥ . ∂x ϵ∞ (x, xi ) = (3.69) (3.70) This yields: eb(xi ) − eb(x) ≤ Meϵ∞ (x, xi ) + LeϵL (x, xi ) + bb(x) − bb(xi ) . f f (3.71) A similar decomposition can be performed to establish: ∥e a(xi ) − e a(x)∥ ≤ Mge ϵ∞ (x, xi )∥ui ∥ + Lge ϵL (x, xi )∥ui ∥ + ∥b a(x) − b a(xi )∥ ∥ui ∥. (3.72) 93 Denoting: ϵH (x, xi , ui ) = bb(x) − bb(xi ) + ∥b a(x) − b a(xi )∥ ∥ui ∥, (3.73) we have that: eb(x) + e a(x)⊤ ui ≤ ℓ(xi , ui ) + (Mef + Mge ∥ui ∥)ϵ∞ (x, xi ) + (Lef + Lge ∥ui ∥)ϵL (x, xi ) + ϵH (x, xi , ui ), ≜ ϵ(x, xi , ui ). (3.74) (3.75) Note that if the estimators bb and b a are continuous functions on D, the function ϵ is continuous on its domain. Breaking the absolute value on the left-hand side in to two cases, we have:   a(x)⊤ ui ≤ ϵ(x, xi , ui ). (3.76) ± eb(x) + e Consequently, defining the uncertainty function ∆ as in (3.58), we have that (eb(x), e a(x)) ∈ ∆(x) as required. Observe that given a test point x ∈ D, the function ϵ can be computed from ċ , and constants M , M , L , and L , and thus it the data set D, estimator W e e e e g g f f is possible to evaluate the supremum in (3.46) at the test point by solving a linear program. Furthermore, we have that: ϵ(xi , xi , ui ) = ℓ(xi , ui ). (3.77) This means that close to a data point, the bound on the uncertainty is only determined by the observed learning error ℓ(xi , ui ). Thus, the uncertainty set can be kept small by sufficiently accurate learning models and sufficiently dense data covering the set D. ċ trained on a data set D. Suppose that Lastly, let us consider an estimator W additional data is produced, and added to the data set D to produce a data ċ is not retrained. For uncertainty set D′ with D ⊂ D′ , but the estimator W functions ∆, ∆′ : D → P(R × Rm ) constructed using the data sets D and D′ , respectively, we have that ∆′ (x) ⊆ ∆(x) for all x ∈ D. This is because the function ϵ is the same between the two constructions, such that for a data point (xi , ui ) ∈ D0 , at a test point x ∈ D we have that ϵi (x, xi , ui ) returns the same value between the two constructions. But, by adding data points in 94 going from D to D′ , we have added additional constraints in the construction of ∆′ compared to ∆, thus reducing possible residual learning errors. Thus, continuing to add data allows the improvement of residual learning error with ċ is retrained with the new data, this may a fixed estimator. If the estimator W not necessarily be the case, as the observed residual learning error ℓ(xi , ui ) and the function ϵH (x, xi , ui ) can increase for data points (xi , ui ) ∈ D0 . This must be balanced with the benefits of achieving low observed residual learning error on the data points appearing in D′ \ D, which may be in regions of interest. Simulation Results In this section I apply the episodic learning DaCLyF algorithm in Algorithm 1 to an inverted pendulum subject to parametric model error. The nominal model open-loop system is given by: # " # " x 0 2 ḃ = + 1 u, (3.78) x g b sin(x1 ) ℓb m b ℓb2 while the true open-loop system is given by: " # " # x2 0 ẋ = gb + 1 u. sin(x1 ) ℓ mℓ2 (3.79) The true mass m and length ℓ are perturbations of the nominal model mass m b and length ℓb by up to 30% of the nominal model values. The nominal controller knom is a linear PD controller set to track angle and angle rate trajectories. The estimators bb and b a are chosen from the class of two layer neural networks with 200 hidden units and ReLU nonlinearities. The trust factors are chosen in a sigmoid fashion. Naive exploratory control is introduced as in Section 3.3, with perturbations chosen uniformly at random between -50% and 50% of the value of the augmented controller kk . A comparison of the nominal modelbased (CLF-QP) controller and final learning-informed (CLF-QP) controller demonstrating improved tracking performance is shown in Figure 3.3. After performing the episodic learning process, a final data set D and estimator ċ are produced, and an uncertainty function ∆ can be built. I evaluate the W following quantities across the state space: sup (b,a)∈∆(x) b b + a⊤ k(x), (3.80) 95 Figure 3.3. Comparison of tracking performance for PD controller and final learninginformed (CLF-QP) controller in inverted pendulum example. The final learning-informed (CLF-QP) controller tracks the desired angle trajectory significantly more accurately. and: sup b + a⊤ k(x). (3.81) (b,a)∈∆(x) The first quantity corresponds to the worst-case residual learning error when using the nominal model-based (CLF-QP) controller in (3.25), while the second quantity is the worst-case residual learning error using the learning-informed (CLF-QP) controller in (3.34). The closed-loop trajectories using these controllers from the same initial conditions can be seen in Figure 3.4, on the left and right, respectively. We see that the nominal model-based (CLF-QP) controller does not stabilize the system to the desired trajectory and fails to bring it to the origin, while the learning-informed (CLF-QP) controller does stabilize the system and brings it to the origin. The value of the computed supremums is indicated by the color-maps. Observe that there is some mild change between the values for the nominal model-based (CLF-QP) controller and learning-informed (CLF-QP) controller (in particular, in the top right of the plot), but that the changes throughout the state space are fairly small. This indicates that although the learning-informed (CLF-QP) controller is able to stabilize the system, it has not greatly reduced the worst-case residual learning error possible by the uncertainty function ∆. This suggests that the system is not actually experiencing the worst-case residual learning error, 96 Figure 3.4. Comparison of worst-case residual learning errors (3.80) with nominal modelbased (CLF-QP) controller (Left) and (3.81) with final learning-informed (CLF-QP) controller (Right). Closed-loop trajectories using the two controllers are displayed with dashed lines. The learning-informed (CLF-QP) controller keeps the system in regions with lower residual learning error, while the system leaves the region around the training data under the nominal model-based (CLF-QP) controller. The heat maps were generated by sampling states randomly about training data points and evaluating the upper bound for each sampled state. The results were then discretized for ease of visualization. Each bin is colored by the maximum disturbance observed in the bin. and that reducing the worst-case residual learning error is not necessary for successful stabilization if a controller does not “excite” the worst-case residual learning error. This observation is the basis for the work in Section 3.8. Conclusion In this section I have presented an approach for analyzing the impacts of residual learning errors when learning model errors between a nominal model open-loop system and the true open-loop system as they directly impact the time derivative of a local exponential CLF V . I propose a novel notion of robust stability based on ISS in the from of Projection-to-State Stability (PSS), which only considers the impacts of disturbances on local exponential stability as they affect the time derivative of a local exponential CLF. I then provide an analysis of closed-loop system behavior given a particular data set and learning models through a data-driven construction of uncertainty functions. Lastly I perform this data-driven analysis in simulation on an inverted pendulum, revealing that stabilization can be achieved without needing to explicitly reduce worst-case residual learning errors, which will motivate the work in Section 3.8. 97 3.5 Learning with Control Barrier Functions The inherent similarities between local exponential CLFs and CBFs suggest that the episodic learning algorithm developed in Section 3.3 could naturally be adapted to address model error directly as it impacts the time derivative of a CBF. In this section I will adapt the DaCLyF episodic learning algorithm in Algorithm 1 to the setting of CBFs, producing a corresponding algorithm named DaCBarF. I will explore how model uncertainty impacts the time derivative of a CBF, and how an ERM problem can be solved sequentially to overcome challenges that arise when deploying standard supervised learning techniques on non-i.i.d data sets. While these adaptions are fairly straightforward and provided for completeness, the main contribution of this section is the demonstration of the episodic learning algorithm on a Segway hardware platform. This result establishes the possibility of identifying error between a nominal model and a real-world system using real data, and constructing learning models that can reflect this model error while remaining computationally efficient enough to be evaluated in a closed-loop controller for a robotic system. The contributions of this section are as follows: • An adaption of the episodic learning algorithm presented in Section 3.3 for identifying model errors as they directly impact the time derivative of a CBF, including using structured learning models to enable using the convex optimization-based control framework described in Section 2.5. • An experimental demonstration of the proposed episodic learning algorithm on a Segway hardware platform. The text for this section is adapted from: A. J. Taylor, A. Singletary, Y. Yue, and A. D. Ames, “Learning for safetycritical control with control barrier functions,” in Proc. 2nd Learning for Dynamics and Control (L4DC), vol. 120, Berkeley, CA, USA, 2020, pp.708-717. A. J. Taylor participated in the conception of the project, algorithm design, simulation and experimental code implementation, conduction of experiments, and writing of the article. 98 Uncertainty in Control Barrier Functions Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. Suppose that h is a CBF for the nominal model open-loop system (3.1) on C with corresponding function α ∈ Ke , such that: ∂h b(x)u) > −α(h(x)), (x)(b f (x) + g u∈Rm ∂x ḃ u) ≜ sup sup h(x, u∈Rm (3.82) for all x ∈ E. The time derivative of the function h uses the true open-loop system (2.1), and thus using (3.2), we have that: ∂h ḃ u) + ∂h (x)e ḣ(x, u) = h(x, f (x) + (x)e g(x) u. |∂x {z } |∂x {z } b(x) (3.83) a(x)⊤ As with CLFs, simply choosing an input u such that the inequality in (3.82) is satisfied is insufficient to conclude that the time derivative of the CBF h will meet the required inequality to guarantee safety because of the functions b : E → R and a : E → Rm . Note the functions b and a capture the e only as they impact the evolution of the CBF h, and model errors in e f and g consequently, safety. Once again, I will consider a learning-based approach to identify the functions b and a, such that their effects on the CBF time derivative can be accounted for in the control design process, and safety can be achieved. To ensure the well-posedness of such an approach, I make the following assumption: Assumption 3. The function h is a CBF for the true open-loop system (2.1) on C with corresponding function α ∈ Ke . This assumption is similar to Assumption 2 and states that h is a CBF for the true open-loop system (2.1) on C with the same function α ∈ Ke used for the nominal model open-loop system (3.1). It is important to note that once again, this is an assumption on how the true open-loop system (2.1) can be controlled. In particular, this assumption states that for a given x ∈ E, one knows that there exists an input u ∈ Rm such that: ḃ u) + b(x) + a(x)⊤ u > −α(h(x)), ḣ(x, u) = h(x, (3.84) but not necessarily which value of u satisfies the inequality. If one knew the functions b and a perfectly, the value of u satisfying this inequality could be found. Thus, if one could learn these functions from data, the control synthesis 99 problem through CBFs would be well-posed. Recalling the construction of a CBF from a local exponential CLF in Section 2.5 and the construction of a local exponential CLF for both a fully-actuated robotic system nominal model open-loop system and true open-loop system in Section 3.3, assumptions on the structural properties of the true-open loop system can be sufficient to ensure this assumption is met. Data-Driven Episodic Learning Framework As with CLFs, I consider a learning-based framework that learns the functions b and a via supervised regression [204]. In particular, the true closed-loop system (2.2) is operated using a given controller k : E → Rm , data points are gathered along the system’s evolution, and functions that approximate b and a are produced via supervised learning. Concretely, let x0 ∈ E be an initial state. An experiment is defined as the evolution of the true closed-loop system over a finite time interval from the initial condition x0 using a sample-hold implementation of the controller k. A resulting discrete-time state history is obtained, which is then transformed with the CBF h and finally differentiated numerically to estimate ḣ throughout the experiment. This yields a data set comprised of input-output pairs: m D = {((xi , ui ), ḣi )}N i=1 ⊆ (E × R ) × R. (3.85) Consider a class Hb of nonlinear functions mapping Rn to R and a class Ha of nonlinear functions mapping Rn to Rm . For a given bb ∈ Hb and b a ∈ Ha , ḃ define S as: ḃ u) + bb(x) + b ḃ u) = h(x, a(x)⊤ u, (3.86) S(x, and let H be the class of all such estimators mapping Rn × Rm to R. Defining a loss function L : R × R → R≥0 , the supervised regression task is then to find a function in H via empirical risk minimization (ERM): N 1 X ḃ L(S(xi , ui ), ḣi ). inf b∈Ha N a i=1 (3.87) b b∈Hb This process can be executed either in simulation or directly on hardware. Once again, standard supervised learning with sequential, non-i.i.d data collection often leads to error cascades [142], motivating the use of an episodic learning framework. My approach integrates learning b and a with improving 100 the safety of the controller in such an iterative fashion. First, assume we are given a nominal controller knom : E → Rm , which may not render the set C ḃ ∈ H safe for the resulting true closed-loop system (2.2). With an estimator S defined in (3.18), specify a learning-informed (CBF-QP) controller as: k(x) = argmin ∥u − knom (x)∥2 u∈Rm (3.88) ḃ u) ≥ −α(h(x)). s.t. S(x, This learning-informed (CBF-QP) controller finds the closest input to knom (x) ḃ for ḣ. However, such that the CBF inequality is met using the estimator S ḃ degrades the safety of the true closed-loop system error in the estimator S (2.2) with respect to the set C. ḃ an experiment In an effort to reduce the remaining error in the estimator S, can be run using the learning-informed (CBF-QP) controller to produce data by which to obtain better estimates of b and a. In particular, in each episode, an experiment generates data using a different controller. The data set is aggregated and a new ERM problem is solved after each episode, once again inspired by the Data Aggregation (DAgger) algorithm [203]. The DaCBarF algorithm in Algorithm 2 specifies a method of computing a sequence of CBF ḃ and augmented controllers k . During the k th episode, derivative estimates S k k the learning-informed (CBF-QP) controller (3.88) associated with the estimate ḃ is scaled by a heuristically chosen factor w ∈ of the CBF time derivative S k k [0, 1] reflecting trust in the estimate and added to the nominal model-based (CBF-QP) controller k0 = kCBF for use in the subsequent experiment as an augmented controller kk . The trust coefficients form a monotonically nondecreasing sequence. Importantly, this experiment need not take place in simulation; the same procedure may be executed directly on hardware. At a high level, this episodic approach makes progress by gathering more data in relevant regions of the state space, such as states inside C. This extends the generalizability of the estimator in its use by subsequent controllers. There are a few practical concerns with this algorithm that should be observed. First, the system will not necessarily be safe with respect to the set C during the episodic learning process. In general, one will often begin the learning process with a controller that is unsafe, or displays desirable behavior but possess no guarantees of safety (such as a barrier function). This algorithm 101 Algorithm 2 Dataset Aggregation for Control Barrier Functions (DaCBarF) ḃ model classes H and H , loss ḃ = h, Require: CBF h, derivative estimate S 0 a b function L, nominal model-based (CBF-QP) controller k0 = kCBF , number of experiments T , sequence of trust coefficients 0 ≤ w1 ≤ · · · ≤ wT ≤ 1 D=∅ for k = 1, . . . , T do x0 ← sample E Dk ← experiment(x0 , kk−1 ) D ← D ∪ Dk ḃ ) b ak , bbk ← ERM(Ha , Hb , L, D, S 0 ḃ ḃ ⊤ b Sk ← S0 + b a u + bk ▷ Initialize data set ▷ Get initial condition ▷ Run experiment ▷ Aggregate data set ▷ Fit estimators ▷ Update derivative estimator ḃ kk ← (1 − wk ) · k0 + wk · augment(k0 , S k ) ▷ Update controller end for ḃ , k return S T T k instead tolerates unsafe behavior during the episodic learning process to ensure that the system performs adequate exploration in the data acquisition process and return a controller at the end of the episodic learning for which meaningful safety properties can be established, as explored in Section 3.6. Second, there can be issues that arise with the constraint in the learninginformed controller (3.88). In particular, while we begin knowing that: ḃ u) > −α(h(x)), sup h(x, u∈Rm (3.89) for all x ∈ E, and we assume that: sup ḣ(x, u) > −α(h(x)), u∈Rm (3.90) for all x ∈ E, such that the CBF constraint is met for the nominal model open-loop system (3.1) and the true open-loop system (2.1), we do not know: ḃ (x, u) > −α(h(x)), sup S k u∈Rm (3.91) for all x ∈ E for k = 1, . . . , T . In particular, it is possible that the learned models bbk and b ak can lead to this condition being violated. This means that the learning-informed controller will not necessarily be feasible for all x ∈ E. In practice, the relaxation technique used in (CLF-CBF-QP) is utilized, such that the problem always remains feasible at the expense of rigorous safety 102 guarantees. In Section 3.6 safety guarantees will be built keeping in mind that some amount of residual learning error arising from numerical learning methods will be present in learning models. Simulation & Experimental Results I apply the proposed episodic learning algorithm to the Segway platform considered in Section 3.3, which can be seen in Figures 3.5 and 3.6. The nominal controller knom : E → R is a hand-tuned proportional-derivative (PD) controller which locally exponentially stabilizes the system to an equilibrium configuration. A particular requirement for safety of the Segway system can be encoded as limitations on how far and how fast the Segway is allowed to tip from this equilibrium configuration. This appears mathematically as a CBF on the pitch angle and pitch angle rate:  1 2 (3.92) θmax − (θ − θe )2 − cθ̇2 , h(θ, θ̇) = 2 where θ ∈ R is the pitch angle, θ̇ ∈ R is the pitch angle rate, θmax ∈ R>0 is a limit on how far the pitch angle can deviate from the equilibrium pitch angle, θe ∈ R, and the coefficient c ∈ R>0 scales the importance of the pitch angle rate in defining safety. This construction of h is performed such that it is a CBF for the nominal model open-loop system (3.1), and thus the (CBF-QP) controller can be used to ensure the safety of the system (in the absence of model errors). To evaluate the impact of model errors on the ability of the nominal modelbased (CBF-QP) controller to ensure the safety of the system, I perturb the mass and electrical parameters of the Segway model by up to 15%, but withhold these artificial model errors from the nominal model-based (CBF-QP) controller. Figure 3.5 shows that the nominal model-based (CBF-QP) controller is unable to achieve safety, with the state of the system leaving the 0-superlevel set of h (left), and the value of h falling below 0 (right). I deploy the episodic learning framework on the Segway system to overcome this model error and improve safety. In particular, I conduct a sequence of ten experiments with varying initial conditions. Data from each experiment is aggregated into a data set D that is used to train learning models bb and b a ḃ and construct the estimator S. The learning models bb and b a are chosen to be neural networks with one hidden layer of 200 nodes and ReLU nonlinearities. 103 0.4 0.05 0.2 0 0 -0.05 -0.2 -0.1 -0.4 -0.15 -0.6 -0.2 -0.8 -0.25 -0.1 0 0.1 0.2 0.3 0.4 0 0.5 1 1.5 2 Figure 3.5. Simulation results using the nominal model-based (CBF-QP) controller (green) and learning-informed (CBF-QP) controller (3.88) (blue). Episodic data appears as red traces. (Left) Segway simulation CAD model. (Middle) The phase portrait showing the evolution of the state of the system leaves the safe set (black ellipse) with the nominal modelbased (CBF-QP) controller, while it remains within using the learning-informed (CBF-QP) controller. (Right) The value of h drops below zero (twice) using the nominal model-based (CBF-QP) controller, while it remains above zero using the learning-informed (CBF-QP) controller. Video can be found at https://youtu.be/tD8zH4GF_8U. These networks are implemented in Keras [208] and trained to minimize mean absolute error using stochastic gradient descent (SGD). After each episode the learning-informed (CBF-QP) controller (3.88) is blended with the nominal model-based (CBF-QP) using trust factor weights that grow linearly across the episodes. Figure 3.5 shows the behavior of the system when the learninginformed (CBF-QP) controller (3.88) is deployed after episodic learning. The state of the system is kept within the 0-superlevel set of h (left), with the value of h being kept above 0 (right). This indicates that the proposed learning approach is able to sufficiently identify model errors as they impact safety such that the learning-informed (CBF-QP) controller is able to keep the system safe. To further demonstrate the ability of this learning approach, I deploy it on the Segway hardware platform seen in Figure 3.6. The nominal controller knom was designed to track a desired horizontal velocity profile while satisfying a safety constraint on the pitch angle rate specified by the 0-superlevel set of a function h : E → R defined as: 1 (3.93) h(θ, θ̇) = (1 − cθ̇2 ). 2 The desired velocity profile is chosen to yield a violation of this safety constraint when using the nominal model-based (CBF-QP) controller, as seen in Figure 3.6. A series of three episodes were conducted tracking this velocity profile without introducing learning models into the controller, after which 104 2 0.5 1 0.4 0.3 0 0.2 -1 0.1 0 -2 -0.1 -1 -0.5 0 0.5 1 0 1 2 3 4 5 6 7 Figure 3.6. Experimental results using the nominal model-based (CBF-QP) controller (green) and learning-informed (CBF-QP) controller (3.88) (blue). (Left) Custom robotic Ninebot Segway platform. (Middle) The phase portrait showing the evolution of the state of the system leaves the safe set (black strip) with the nominal model-based (CBF-QP) controller, while it remains within using the learning-informed (CBF-QP) controller. (Right) The value of h drops below zero (at t = 3) using the nominal model-based (CBF-QP) controller, while it remains above zero using the learning-informed (CBF-QP) controller. Video can be found at https://youtu.be/YCesvD_Ae00 estimators b a and bb were implemented as neural networks with two hidden layers of 50 nodes and ReLU nonlinearities, trained on this collected data, and incorporated into the learning-informed controller (3.20). Note that this implementation deviates from the DaCBarF algorithm by not updating the controller between episodes, but performs an initial set of experiments to form a data set and learning just using this data. In this experimental configuration, natural variations in the initial condition of the system and behaviors of the hardware platform led to enough variation of data to support successful learning for this safety task. The closed-loop behavior of the system can be see in Figure 3.6. The state of the system is kept within the 0-superlevel set of h (left), with the value of h being kept above 0 (right). This indicates that the proposed learning approach is able to sufficiently identify model errors as they impact safety, the assumption of h being a CBF for the true-open loop system is justified, and the learning-informed (CBF-QP) controller is able to keep the system safe. Conclusion In this section I have presented an adaptation of the episodic learning framework in Section 3.3 for learning model errors between a nominal model openloop system and the true open-loop system as they directly impact the time derivative of a CBF h. I formalize how model errors appear in the time derivative of a CBF and adapt the episodic learning algorithm for the setting of CBFs, 105 yielding the DaCBarf algorithm. I demonstrate the algorithm in simulation, and as the main contribution of this section, experimentally on a Segway hardware platform. 3.6 Projection-to-State Safety In this section I will present an analysis of the effects of residual learning error on the safety of the true closed-loop system (2.2) using a learning-informed (CBF-QP) controller. This analysis will mirror the one presented in Section 3.4 that considered residual learning error as a disturbance in the time derivative of a local exponential CLF and culminated in the idea of Projection-to-State Stability (PSS). In the setting of safety, I will build off of the episodic learning algorithm in Section 3.5 and consider learning model errors as they impact the time derivative of a CBF. This will lead to a characterization of the inherent robustness of CBF-based controllers to residual learning errors in a modification of ISSf known as Projection-to-State Safety (PSSf). Given a data set and corresponding learning models, this will lead to a statement on the superlevel set of a CBF that can be kept forward invariant in the presence of residual learning errors. While the extensions in going from PSS to PSSf are relatively straightforward due to the similarities between CLFs and CBFs, a key contribution of this work will be the demonstration of PSSf behavior on a Segway hardware platform. The contributions of this section are as follows: • A notion of robust safety in the form of Projection-to-State Safety (PSSf ), which adapts the analysis via CLFs and PSS in Section 3.4 to CBFs, thereby capturing the impact of disturbances on safety only as they impact the time derivative of a CBF, rather than the full-order dynamics, as is done by ISSf discussed in Section 2.7. • An experimental demonstration of PSSf behavior with respect to residual learning errors on a Segway hardware platform, including a demonstration of how learning models can improve PSSf safety guarantees. The text for this section is adapted from: A. J. Taylor, A. Singletary, Y. Yue, and A. D. Ames, “A control barrier perspective on episodic learning via projection-to-state safety,” in IEEE 106 Control Sys. Let., vol. 5, no. 3, pp.1019-1024, 2021. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation and experimental code implementation, conduction of experiments, and writing of the article. Projection-to-State Safety In the context of learning model errors as they appear in the time derivative of a CBF as in Section 3.5, it can be difficult to formalize residual learning errors as a disturbance signal in the full state dynamics. Rather, as in Section 3.4, I will look at residual learning error as it appears directly in the time derivative of the CBF. In particular, recall that: ḃ u) + b(x) + a(x)⊤ u. ḣ(x, u) = h(x, (3.94) ḃ ∈ H as: Let bb ∈ Hb and b a ∈ Ha compose the estimator S ḃ u) + bb(x) + b ḃ u) = h(x, S(x, a(x)⊤ u, (3.95) and let us assume that: (3.96) ḃ u) > −α(h(x)), sup S(x, u∈Rm for all x ∈ E. Adding and subtracting the estimators bb and b a from ḣ yields: ḃ u) + b(x) − bb(x) + (a(x) − b ḣ(x, u) = S(x, a(x))⊤ u. | {z } | {z } e b(x) (3.97) e(x)⊤ a Once again, the functions eb : E → R and e a : E → Rm capture residual learning error left after trying to learn the functions b and a with the estimators bb and b a. As these error terms appear in the time derivative of the CBF h, they will consequently have an impact on the safety properties of the system. Letting knom : E → Rm be a nominal controller, using the learning-informed (CBF-QP) controller: k(x) = argmin ∥u − knom (x)∥2 u∈Rm (3.98) ḃ u) ≥ −α(h(x)), s.t. S(x, yields: ḃ k(x)) + eb(x) + e ḣ(x, k(x)) = S(x, a(x)⊤ k(x). (3.99) 107 For any initial condition x0 ∈ E, let φ : I(x0 ) → E denote the solution to the true closed-loop system (2.2) using the learning-informed (CBF-QP) controller (3.98). We can then view the evolution of (3.99) as a closed-loop system with unmatched disturbances: ḃ k(x)) + δ(t), ḣ(x, k(x)) = S(x, (3.100) where for any initial condition x0 ∈ E, the function δ : R≥0 → R defined as: δ(t) = eb(φ(t)) + e a(φ(t))⊤ k(φ(t)). (3.101) is a projected disturbance. This perspective motivates studying residual learning error through the lens of ISSf. Observe that the learning-informed (CBF-QP) controller enforces: ḣ(x, k(x)) ≥ −α(h(x)) − |δ(t)|. (3.102) Considering the identity map ι : R → R defined as ι(r) = r for all r ∈ R, note that the 0-superlevel set of ι is R≥0 . Furthermore, the identity map satisfies: ∂ι (h(x))ḣ(x, k(x)) ≥ −α(h(x)) − |δ(t)| = −α(ι(h(x))) − |δ(t)|, ∂r (3.103) and thus is a ISSf-BF for the closed-loop system (3.100) on R≥0 . Thus, noting the relationship between the 0-superlevel set of h and 0-superlevel set of the identity map ι, there exists a constant d ∈ R>0 and a function γ ∈ K such that for all d ∈ [0, d], the set: Cd = {x ∈ E | ι(h(x)) + γ(d) ≥ 0}, (3.104) is forward invariant for the closed-loop system (3.100) if the projected disturbance satisfies5 ∥δ∥∞ ≤ d for all initial conditions x0 ∈ Cd . This appears like a statement of ISSf for the true closed-loop system (2.2) using the learning-informed controller (3.98), but it is defined now in terms of a bound on the projected disturbance signal δ rather than a disturbance signal d in the state dynamics. Consequently, the true closed-loop system (2.2) with the learning-informed controller (3.98) is said to be Projection-to-State Safe 5 In the setting of forward invariance and safety, we only care about the solution φ being contained in a set for the interval on which it exists. Consequently, the particular projected disturbance signal δ defined in (3.101) may not be defined on all of R≥0 . In this case, the ∥ · ∥∞ norm is understood to be over the interval of existence of the solution. 108 (PSSf ) with respect to the set C and projection h. The utility of PSSf is that it consider the impacts of model error only as it impacts the time derivative of the CBF h. By doing so, it provides a mathematical framework for understanding the impact of introducing learning models for the time derivative of a CBF, as it is not obvious how to represent residual learning error in learning models for a CBF time derivative as a disturbance in the full state dynamics (2.2), making an analysis by standard ISSf difficult. Furthermore, it may potentially produce less conservative statements on the degradation of safety by neglecting model error that does not affect safety. Ensuring Projection-to-State Safety with Residual Learning Error As we saw with PSS in Section 3.4, the preceding analysis through PSSf produced some value of d which is not particularly useful if we do not know that the particular projected disturbance signal in (3.101) at least satisfies ∥δ∥∞ ≤ d. We may not simply increase the value of d to achieve this, as we must then consider initial conditions and subsequent projected disturbances from larger sets Cd , which can lead to the need for larger bounds on the projected disturbance, and a repetition of this cycle. This is particularly problematic noting that the data-driven construction of an uncertainty function in Section 3.4 revealed that proximity to training data is important for constructing meaningful bounds on the projected disturbance signal in (3.101). This means we need data further and further outside of the set C if we are to establish a viable bound d, which may not be tractable to acquire safely. I will use uncertainty functions (Definition 22) to establish conditions yielding a function γ ∈ K and a value of d such that the projected disturbance is bounded from any initial condition x0 in the set Cd defined using the function γ as in (3.104). While I will not establish this in the proof, it is straightforward to show using the same proof technique that this function γ can be used to evaluate the sets that are kept forward invariant if the projected disturbance is bounded by a value of d ∈ [0, d], capturing the behavior of PSSf if the actual projected disturbance the system experiences is bounded by a lower value than the worst-case bound d. Denote the −c-superlevel set of the CBF h as: C(c) = {x ∈ E | h(x) ≥ −c}, (3.105) noting that C(0) = C. I denote the closure of C(c) in Rn by C(c), and for simplicity, assume that there exists some c ∈ R>0 such that C(c) ⊂ E for all 109 c ∈ [0, c). This is generally a weak assumption in many practical cases. This leads to the following theorem: Theorem 21. Let ∆ : E → P(R × Rm ) be a uncertainty function and suppose there exists a c ∈ (0, c) such that: sup sup b + a⊤ k(x) ≤ −α(−c). (3.106) x∈C(c) (b,a)∈∆(x) Then for any initial condition x0 ∈ C(c), we have that the projected disturbance in (3.101) satisfies |δ(t)| ≤ d = −α(−c) for all t ∈ I(x0 ), and the function γ ∈ K appearing in (3.104) is defined as: (3.107) γ(r) = −α−1 (−r). Proof. First observe that by the continuity of h, the set C(c) is closed, such that C(c) = C(c). Consider a point x ∈ ∂C(c), noting that this implies x ∈ E by our assumption that C(c) ⊂ E. We can show by continuity of h that x ∈ ∂C(c) implies that h(x) = −c. As h is a CBF on C for the nominal model (3.1), we have: ∂h b(x)u) > −α(h(x)) = −α(−c) > 0, (x)(b f (x) + g m ∂x u∈R sup (3.108) ∂h as α ∈ Ke . This implies that ∂x (x) ̸= 0n , and thus −c is a regular value of the function h. Now, we have that: ḣ(x, k(x)) ≥ −α(h(x)) − b + a⊤ k(x), sup (3.109) (b,a)∈∆(x) ≥ −α(h(x)) − sup sup b + a⊤ k(y), (3.110) y∈C(c) (b,a)∈∆(y) ≥ −α(−c) + α(−c) = 0. (3.111) Thus ḣ(x, k(x)) ≥ 0 for any x ∈ ∂C(c), and by Nagumo’s theorem [104], [123], we may conclude that the set C(c) is forward invariant for the true closed-loop system (2.2). Consequently, for any initial condition x0 ∈ C(c), we have: |δ(t)| ≤ −α(−c) ≜ d, (3.112) for all t ∈ I(x0 ), such that ∥δ∥∞ ≤ d. Lastly, because the set C(c) is forward invariant for the disturbance bound d, we wish for Cd to be equal to C(c). Correspondingly, we have that: γ(−α(−c)) = c, (3.113) 110 from which we can manipulate to establish that: γ(r) = −α−1 (−r) (3.114) for any r ∈ R≥0 . While not repeated here, similar results to Theorem 19 and Theorem 20 regarding the refinement of uncertainty sets yielding improved bounds and the construction of an uncertainty function capturing residual learning errors when learning a CBF time derivative through data can easily be constructed. Simulation & Experimental Results To demonstrate the ability of learning to improve safety guarantees via PSSf, I deploy the episodic learning framework with CBFs presented in Section 3.5 on the Segway system described in Sections 3.3 and 3.5 in simulation6 and experimentally. In both simulation and experimentally I run a sequence of episodes to train the estimators bb and b a, represented by 4-layer neural networks with ReLU nonlinearities. In each episode the Segway was set to track a desired trajectory in the pitch angle space without violating a barrier function on a portion of its state, using an augmented (CBF-QP) controller. After the sequence of episodes, the Segway was ran once more with a learning-informed (CBF-QP) controller, and the projected disturbance δ as defined in (3.101) was computed. This computation is performed by looking at the difference between the measured value of ḣ (numerically approximated) and an estimated value of ḣ at sample times ti . The first estimate I use only uses the nominalḃ with no learning models, to produce a model based estimate of ḣ, given by h, projected disturbance signal: ḃ δ(ti ) = ḣ(φ(ti ), k(φ(ti ))) − h(φ(t i ), k(φ(ti ))). (3.115) ḃ produced after the episodic The second estimate I consider uses the estimator S learning process to produce a projected disturbance signal: ḃ δl (ti ) = ḣ(φ(ti ), k(φ(ti ))) − S(φ(t i ), k(φ(ti ))). (3.116) The worst case disturbances δ = ∥δ∥∞ and δ l = ∥δl ∥∞ were found, and a lower bound on h for that trajectory was determined using the function α ∈ Ke defined as α(r) = kr for k ∈ R>0 , to produce γ(r) = kr as in (3.104). 6 The simulation code (including an expression for the full system dynamics) can be found at https://github.com/DrewSingletary/cyberpod_sim_ros 111 2 0.15 1.5 0.1 1 0.05 0.5 0 0 0 1 2 3 0 1 2 3 Figure 3.7. Simulation results with Segway platform demonstrating improvement in PSSf behavior. (Left) Absolute value of the projected disturbance along the trajectory without learning models ((3.115), red) and with learning models ((3.116), blue), with learning reducing the worse case projected disturbance (δ/k). (Right) The value of the barrier satisfies the corresponding worst case lower bound with and without learning being used to compute δ. The worst case lower bound is raised with learning (the blue dashed line lies above the red dashed line). In simulation, the Segway was given a bound on its position in space, constraining it to a one meter distance from its starting location. A CBF on the entire system state that enforces this constraint was generated through the backup controller method [13] (reviewed in Section 5.3). The simulation was done in a Robot Operating System (ROS) based C++ environment similar to [209]. The simulation environment accurately simulates the physical system by adding input delay, sensor noise, and state estimation. The code is identical to that on the physical hardware for the state estimator, controller, and CBF, apart from the ROS functionality. Experimentally, a CBF was specified to limit the pitch angle and pitch angle rate of the Segway to an ellipse about the Segway’s equilibrium state. The desired pitch angle trajectory leads the Segway to tip quickly, leaving the safety set in the absence of the CBF and safety-critical controller. In both cases, we see that introducing learning estimators into the computation of the projected disturbance decreases the worse case disturbance (δ > δ l ). This leads to a greater lower bound on h, and thus a stronger guarantee on the PSSf behavior of the system. I note that the conservative nature of the lower bounds on h arise from the fact that the worst case disturbance δ along the trajectory is used, while the system experiences a much lower projected disturbance throughout most of its evolution. If the worst case disturbance 112 0.8 0.5 0.6 0 0.4 -0.5 0.2 0 0 2 4 6 0 2 4 6 Figure 3.8. Experimental results with Segway platform demonstrating improvement in PSSf behavior. (Left) Absolute value of the projected disturbance along the trajectory without learning models ((3.115), red) and with learning models ((3.116), blue), with learning reducing the worse case projected disturbance (δ/k). (Right) The value of the barrier satisfies the corresponding worst case lower bound with and without learning being used to compute δ. The worst case lower bound is raised with learning (the blue dashed line lies above the red dashed line). can be reduced (by data-aware control synthesis), stronger guarantees on safety can be made that reflect this behavior. Conclusion In this section I have presented an approach for analyzing the impacts of residual learning errors when learning model errors between a nominal model open-loop system and the true open-loop system as they directly impact the time derivative of a CBF h. I propose a notion of robust safety based on ISSf in the from of Projection-to-State Safety (PSSf), which only considers the impacts of disturbances on safety as they affect the time derivative of a CBF. I demonstrate the safety guarantees ensured by PSSf in both simulation and experimentally, with a focus on the fact that learning model errors directly as they impact the time derivative of a CBF can reduce the projected disturbance a system experiences and therefore lead to stronger conclusions on safety. 3.7 Projected Disturbance Learning with Control Barrier Functions In this section I will present an episodic learning approach for learning a projected disturbance directly as it impacts a CBF time derivative. In contrast to the learning approach taken in Section 3.5, I will not utilize structured learning models that attempt to learn the functions b and a individually in an effort to 113 preserve convexity of downstream control synthesis. Rather, I will learn the sum of these functions under a given feedback controller. I will first motivate the reasoning behind this learning approach by considering some challenges that arise from insufficiently rich data when trying to learn using structured models on high-dimensional systems. I will then describe the episodic learning algorithm for directly learning projected disturbances as they impact the time derivative of a CBF. I will conclude with an application of the learning algorithm to a bipedal walking system, both in simulation and experimentally. The contributions of this section are as follows: • An analysis of challenges in using the structured learning models considered in Sections 3.3 and 3.5, and an episodic learning algorithm for learning projected disturbances that circumnavigates these challenges. • An implementation of the proposed algorithm on a bipedal walking platform, both in simulation and experimentally. The text for this section is adapted from: N. Csomay-Shanklin, R. K. Cosner, M. Dai, A. J. Taylor and A. D. Ames, “Episodic learning for safe bipedal locomotion with control barrier functions and projection-to-state safety,” in Proc. 3rd Learning for Dynamics and Control (L4DC), vol. 144, Zürich, Switzerland, 2021, pp.1041-1053. A. J. Taylor participated in the conception of the project, algorithm design, and writing of the article. Challenges in Learning with Control Barrier Functions To understand some of the challenges that can arise when trying to learn structured models with insufficiently rich data, I will consider a simple singleinput example. Consider a data point ((xi , ui ), ḣi ) as in (3.85), and suppose ui ̸= 0. The model error we are trying to reduce can be expressed as: ė = ḣ − h(x ḃ , u ) = b(x ) + a(x )u . h i i i i i i i This can be represented in an affine form as: " # h i a(x ) i ė = h . ui 1 i b(xi ) (3.117) (3.118) 114 1   a(xi) b(xi)  1 − ui γ2 1   u γ1 i 1   ui 1 ui   b a(xi) bb(xi) Figure 3.9. Schematic for challenges in learning CBF time derivatives with structured models. The vectors (ui , 1) and (a(xi ), b(xi )) are visualized in Figure 3.9 (black and blue, respectively) We can decompose the vector (a(xi ), b(xi )) as follows: # " # " # " a(xi ) ui − u1i , (3.119) = γ1 + γ2 1 b(xi ) 1 for some γ1 , γ2 ∈ R, noting we have used an orthogonal basis. This decomposition can be seen in Figure 3.9 (red and green vectors). Noting this decomposition, we have that: " # 2 ui ė = γ h . (3.120) i 1 1 This indicates that in our data point we have information about the constant γ1 , but we do not have information about γ2 . Suppose that we build accurate learning models bb : Rn → R and b a : Rn → R such that: ė ≈ bb(x ) + b h a(xi )ui . i i This can be represented in affine form as: # " h i b a (x ) i ė ≈ h . ui 1 b i b(xi ) (3.121) (3.122) 115   a(xi) b(xi)   u β1 j 1 1   uj 1   u γ1 i 1   ui 1 ui Figure 3.10. Schematic for introducing additional data for resolving challenges in learning CBF time derivatives with structured models. We can perform a similar decomposition on the estimators: # " # " # " ui − u1i b a(xi ) =γ b1 +γ b2 , bb(xi ) 1 1 (3.123) for some γ b1 , γ b2 ∈ R. What this reveals is that an accurate learning model only needs to satisfy γ b1 ≈ γ1 , and there are no requirements on γ b2 ≈ γ2 . This can be seen in Figure 3.9, where the estimator (pink) satisfies γ b1 = γ1 (lies on the dotted black line), but is dramatically different than the true value of the error terms (the blue vector). Consequently, inserting this estimator into the optimization-based controllers as in Section 3.3 and 3.5 can lead to poor performance and failure to satisfy stability and safety requirements. Reducing this problem can be achieved through richness in the inputs composing the data sets. Suppose that we have another data point at the same state, but with a different input uj ̸= 0, and that the learning models are trained accurately on this data to yield: ė ≈ bb(x ) + b h a(xi )uj . j i (3.124) This additional data point is visualized in Figure 3.10. Considering the two 116 data points together, we have that:   " # #" ė b a (x ) h u 1 i  i ≈ i . ė b uj 1 b(xi ) hj | {z } (3.125) A Accurately identifying the values of b(xi ) and a(xi ) with the corresponding estimators can thus be seen as solving a least-squares problem locally at the state xi . While learning is often done numerically through gradient-based updates rather than taking a (psuedo)inverse of the matrix A, the numerical stability in solving the learning problem accurately is effectively determined by the condition number of the matrix A. If the input uj = −ui , then the cond(A) = 1. In practice, taking dramatically different inputs at the same state may be unsafe or damaging to a system, making it difficult to acquire data that leads to a well conditioned matrix A. If instead uj = ui + ϵ for some small non-zero value ϵ ∈ R, we have that: lim cond(A) = ∞. ϵ→0 (3.126) Thus small perturbations to a nominal input do not significantly improve the numerical stability of the estimation problem at this state. This highlights an important challenge in learning with structured models that ensure an affine dependence on the control input. Data collection that does not damage a system is often limited to small perturbations around baseline inputs which are already stabilizing or safe, but this does not provide strong numerical foundations for the downstream learning problem. Moreover, this challenge is exacerbated for high-dimensional systems with multiple inputs. The same geometric decomposition can be performed in high dimensions, revealing the need for m + 1 data points with varying inputs at the same state to make sure the learning problem is well conditioned. Additionally, for high-dimensional systems, such as bipedal walking platforms, there are strong requirements on what the input must be at a given state to achieve walking, prohibiting large exploratory perturbations to inputs. In Section 3.8 I will explore how this challenge can be approached through robust convex optimization. In the remainder of this section, I will consider learning projected disturbances without using structured learning models in an effort to avoid this problem. 117 Learning Projected Disturbances To motivate learning projected disturbances with unstructured models, I will first indicate some challenges that can arise, and what assumptions justify taking such a learning approach. First, note that for a given controller k : E → Rm that is locally Lipschitz continuous on E, in the absence of learning models bb and b a, the value of the projected disturbance the system experiences at any given state is defined in (3.101) and given by: δ(x) = b(x) + a(x)⊤ k(x). (3.127) Importantly, this projected disturbance depends on the particular controller k. If another controller k′ : E → Rm was designed to account for this projected disturbance, then the value of the new projected disturbance would be: δ ′ (x) = b(x) + a(x)⊤ k′ (x). (3.128) This introduces a type of feedback loop in which modifying the controller based on an estimate of the projected disturbance leads to a new projected disturbance for which the estimator may not be accurate. To understand why one may still consider learning the projected disturbance directly, we will understand the impacts of introducing such a learning model into the optimization-based controllers in Section 3.5. In particular, the nominal model-based (CBF-QP) controller can be expressed as [21]: ⊤ ∂h (x)b g(x) , kCBF (x) = knom (x) + λ(x) ∂x | {z }  (3.129) Lgb h(x)⊤ for a function λ : Rn → R≥0 that is locally Lipschitz continuous on E. Suppose that we have a function δb : E → R that is locally Lipschitz continuous on E, and we define a (CBF-QP) controller k′CBF : E → Rm as: k′CBF (x) = argmin ∥u − knom (x)∥2 u∈Rm b (δ-CBF-QP) ḃ u) + δ(x) b s.t. h(x, ≥ −α(h(x)). This controller can be expressed as: k′CBF (x) = knom (x) + λ′ (x)Lgb h(x)⊤ , (3.130) 118 for a function λ′ : Rn → R≥0 that is locally Lipschitz continuous on E. It can be shown that: b δ(x) ≥ 0 =⇒ λ′ (x) ≤ λ(x), b δ(x) ≤ 0 =⇒ λ′ (x) ≥ λ(x). (3.131) (3.132) ∂h Denoting Lbf h(x) = ∂x (x)b f (x), the time derivative of the CBF h using both of these controllers is given by: ḣ(x, kCBF (x)) = Lbf h(x) + b(x) + (Lgb h(x) + a(x)⊤ )knom (x) + λ(x)Lgb h(x)Lgb h(x)⊤ + λ(x)a(x)⊤ Lgb h(x), (3.133) and: ḣ(x, k′CBF (x)) = Lbf h(x) + b(x) + (Lgb h(x) + a(x)⊤ )knom (x) + λ′ (x)Lgb h(x)Lgb h(x)⊤ + λ′ (x)a(x)⊤ Lgb h(x)⊤ . (3.134) Let us suppose that: ∥Lgb h(x)∥ ≥ ∥a(x)∥, (3.135) for all x ∈ E. This assumptions states that the impact of model error in how the input affects the time derivative of h is less significant than the impact of the input on the time derivative of h captured by our nominal model. Consequently, we have that: Lgb h(x)Lgb h(x)⊤ ≥ a(x)⊤ Lgb h(x)⊤ . (3.136) b Now let us suppose that for a particular state x ∈ E, we have δ(x) ≤ 0, such that λ′ (x) ≥ λ(x). If the function δb is an (accurate) estimator for the projected disturbance (3.127) using the controller kCBF , this indicates that: ḃ k ḣ(x, kCBF (x)) ≤ h(x, CBF (x)), (3.137) such that the system may not satisfy the safety requirement: ḣ(x, kCBF (x)) ≥ −α(h(x)). (3.138) Because λ′ (x) ≥ λ(x), we have that: ḣ(x, k′CBF (x)) ≥ ḣ(x, kCBF (x)), (3.139) such that the introduction of δb to the controller is doing more to ensure the safety requirement is met. In this way by learning the projected disturbance 119 using one controller, we can construct another controller that improves upon the previous controller with respect to safety. As this modifies the projected disturbance, this process can be iterated on to identify the new projected disturbance and improve the safety of the controller further. More concretely, consider the following setting: in an experiment, the system is allowed to evolve forward in time from a particular initial condition x0 ∈ E under a given controller k : E → Rm that is locally Lipschitz continuous on E. During this experiment, state data xi ∈ E is collected at discrete sampletimes ti ∈ R≥0 , providing a discrete-time history of the CBF, hi ≜ h(xi ). This time history is smoothed and numerically differentiated to compute an approximate time history of the true value of the time derivative of the CBF, ḣi ≜ ḣ(xi , k(xi )). This yields a collection of input-output pairs: n oN D = ((xi , k(xi )), ḣi ) ⊆ (E × Rm ) × R. (3.140) i=1 Given a nonlinear function class H : Rn → R and a loss function L : R × R → R, a learning problem can be specified as finding a function δb ∈ H to estimate δ via empirical risk minimization (ERM): N  1 X ḃ b i ), ḣi . L h(xi , k(xi )) + δ(x b N i=1 δ∈H inf (ERM) b A controller can then be synthesized as in (δ-CBF-QP). I directly build upon the episodic learning framework from Sections 3.3 and 3.5 as outlined in Algorithm 3. In each episode, the algorithm runs the current controller to collect data, learns a new estimator δb using the newly collected data, and synthesizes a new controller. A notable difference is that data is not aggregated during this process. This is because data from a previous episode corresponds to a projected disturbance using a different controller, and thus would conflict with data collected using a modified controller. Bipedal System Application Note that the contents of this subsection were primarily written by collaborators, edited by Andrew Taylor for their appearance in conference proceedings, and edited by Andrew Taylor for their appearance in this thesis. We will now consider learning projected disturbances in the setting of bipedal locomotion on stepping-stones. We will briefly introduce the theory of bipedal 120 Algorithm 3 Projected Disturbance Learning (PDL) ḃ model class H, loss function L, Require: CBF h, CBF derivative estimate h, nominal model-based (CBF-QP) controller k0 = kCBF , number of episodes T for k = 1, . . . , T do x0 ← sampleE D ← experiment(x0 , kk−1 ) ḃ δbk ← ERM(H, L, D, h) kk ← augment(k0 , δbk ) end for return δbT , kT ▷ Sample initial condition ▷ Run experiment ▷ Fit estimator ▷ Update controller locomotion and then describe the barrier function formulations which allow achieving safe bipedal locomotion across stepping-stones. A deeper exploration of this material may be found in [210]. The bipedal robotic system we consider is the AMBER-3M robotic platform seen in Figure 3.11, modeled as an underactuated, planar five-link robot with point feet [211] and physical parameters reported in [212, Table 1]. The configuration coordinates q ∈ Q ⊂ R5 are given by q = [qsf , qsk , qsh , qnsh , qnsk ]⊤ , with stance foot angle qsf , stance knee angle qsk , stance hip angle qsh , nonstance hip angle qnsh and non-stance knee angle qnsk . The full system state is given by x = (q, q̇) ∈ Q × R5 . The continuous-time equations of motion are given by the Euler-Lagrange equations as in (2.5). Note that this is the pinned model of the robot dynamics; for the unpinned model, refer to [213]. For AMBER-3M, the number of inputs is one fewer than the degrees of freedom, meaning the system has one degree of underactuation. Taking pv : Q → R to represent the vertical position (height) of the swing foot, the admissible states are given by the domain D = {(q, q̇) ∈ Q × R5 | pv (q) ≥ 0}. The switching surface on which the impact events occur, also known as the guard, is defined by: S = {(q, q̇) ∈ Q × R5 | pv (q) = 0, ṗv (q, q̇) < 0} ⊂ D. (3.141) Assuming that an impact at happens at time t ∈ R≥0 , the impact dynamics7 7 I do not discuss the notion of solutions for hybrid systems in this thesis, but an introduction to the topic can be found in [214]. The relevant detail for this example is that the corresponding solution φ will be discontinuous at impact times. 121 Figure 3.11. (Left) Schematic diagram of the AMBER-3M robot with position coordinates. (Center) Schematic of the foot placement in the stepping-stone problem. The boundaries of virtual stepping-stones are captured via the blue and orange vertical lines. (Right) Virtual stepping stone width as function of the phase variable τ (q). [215] are defined by a reset map ∆ : S → D relating pre-impact states x− ≜ limτ →t− φ(τ ) and post-impact states x+ = limτ →t+ φ(τ ) via x+ = ∆(x− ). Combining these concepts yields the following hybrid control system:  ẋ = b b(x)u x− ∈ D \ S, f (x) + g HC = (3.142) x+ = ∆(x− ) x− ∈ S. Control of bipedal systems uses a phasing variable, τ : Q → [0, 1], given by: τ (q) = + δhip (q) − δhip − + δhip − δhip , (3.143) where δhip : Q → R defined as δhip (q) = [−lt − lf , −lf , 0, 0, 0]q is the linearized hip position with lt and lf the length of the robots tibia and femur, + − respectively. The constants δhip and δhip are the linearized hip positions at the beginning and the end of a step, ensuring that τ (q) increases monotonically in time within a step. Desired trajectories resulting in walking gaits for the robot can be synthesized via a hybrid zero dynamics framework [107], [210]. We are now equipped to define the relative degree 2 ([112]) outputs y : Q × Rp → R4 as the difference between the actual output ya : Q → R4 and the desired output trajectory yd : R × Rp → R4 : y(q, α) ≜ ya (q) − yd (τ (q), α), (3.144) 122 with α ∈ Rp being the coefficients of a Bézier polynomial coming from the trajectory generation step. The actual output is given by the actuated coordinates: h i ya (q) = 04×1 I4×4 q. (3.145) The nominal controller for this system is then given by the proportionalderivative controller knom (x) = −KP y(q) − KD ẏ(q, q̇) with proportional gain KP ∈ S4≻0 and derivative gain KD ∈ S4≻0 . The stepping-stone problem uses virtual stepping-stones, which shrink over the course of a step to confine foot placement to a targeted stone [22]. The CBFs used to specify these foot position constraints are given by: h1 (q) = R(τ (q)) − (Ox − Fx (q)), (3.146) h2 (q) = R(τ (q)) + (Ox − Fx (q)), (3.147) where Fx (q) is the horizontal position of the swing foot, Ox > 0 is the horizontal position of the center of stepping-stone, and the virtual stone width is given by the function R : R → R defined as: R(τ (q)) = ar − 1 + 1 + r, 1 + ar(e−λ(τ (q)−1) − 1) (3.148) where λ ∈ R>0 determines the decay rate of the barrier function, (1 + a)r is half of the targeted stone width, and 1 + r defines the half the width of the virtual stepping-stone when τ = 0. These functions are visualized in Figure 3.11. The safety constraints can be interpreted as keeping the swing foot horizontal position in an interval centered at the middle of the stepping-stone, where the interval shrinks as τ increases. As this formulation of CBFs is position-based and therefore relative degree two, we employ the exponential CBF (ECBF) extension technique [216] to both CBFs to attain the relative degree 1 CBFs: he,i (x) ≜ Lbf hi (x) + αe hi (q) with αe ∈ R>0 . The final Stepping Stone QP b (SS-QP) controller combines the robustifying term of the (δ-CBF-QP) controller with the stepping-stone ECBF extensions of (3.146) and (3.147): k(x) = argmin u∈Rm 1 ∥u − knom (x)∥2 2 (SS-QP) s.t. Lb2f h1 (x) + Lgb Lbf h1 (x)u + αe Lbf h1 (x) + δb1 (x) ≥ −α(he,1 (x)), Lb2 h2 (x) + Lgb Lbh2 (x)u + αe Lbh2 (x) + δb2 (x) ≥ −α(he,2 (x)). f f f 123 Figure 3.12. Simulation (S) and Hardware (H) data where model mismatch causes violations. (Far Left) Simulation where the barrier functions h1 (solid blue) and h2 (solid orange) are enforced via a nominal model-based (CBF-QP) controller. A conservative controller using a worst-case upper bound on the projected disturbance is also shown (dashed blue, dashed orange), which results in more conservative behavior over many steps. (Mid Left) After three episodes of learning using the (SS-QP) controller in simulation, the maximum barrier violation decreases from 2.0 to 0.3 [cm]. (Mid Right) Hardware where the barrier functions h1 (blue) and h2 (orange) are enforced via a nominal model-based (CBF-QP) controller. (Far Right) After two episodes of learning on hardware, the maximum barrier violation decreases from 9.2 to 1.9 [cm] using the (SS-QP) controller. Simulation & Experimental Results Note that the contents of this subsection were primarily written by collaborators, edited by Andrew Taylor for their appearance in conference proceedings, and edited by Andrew Taylor for their appearance in this thesis. We now apply the episodic learning algorithm in Algorithm 3, to the AMBER3M platform in both simulation with artificial model error and on hardware with the model error inherent to real-world systems. In each instance the estimator δb was implemented as a neural network with two hidden layers of 50 hidden units using the ReLU activation function. The network was trained minimizing mean absolute error using mini-batch gradient descent. Mean absolute error was chosen over other loss functions for its robustness to outliers. The SS-QP controller was deployed in the RaiSim [217] simulation environment and on the AMBER-3M hardware platform. The controllers and learning algorithm were first validated in simulation. Model error was introduced by increasing the inertia of all limbs on the nominal model by a factor of ten while maintaining constant mass to produce the true system. Due to the underactuated nature of the robot and the relationship between step length and zero dynamics stability, not every set of stepping stones 124 Figure 3.13. Gait tiles for Episode 2 of learning showing the AMBER-3M robot safely traversing a set of stepping stones. Notice the change in step width and added lean of the torso induced by the barrier functions. Video can be found at https://vimeo.com/ 481809664 is navigable, even if safety is perfectly enforced with respect to the CBFs. Therefore, a feasible stepping stone configuration was first generated for the robot to traverse with stones of 4 [cm] in width. Without knowledge of the modified model (δb1 (x) = δb2 (x) = 0 for all x ∈ E), the controller did not satisfy the CBF constraints, resulting in a maximum violation at foot placement of 2.0 [cm], causing the robot to miss the stepping stone and fall over. Three episodes of the PDL algorithm were run, after which the maximum violation was reduced to be 0.3 [cm], only 15% of the original violation. Additionally, a controller using a worst-case global upper bound on the projected disturbance was implemented, which ensured safety but resulted in extremely conservative behavior, resulting in poor qualitative walking, i.e. harsh foot strikes and an over-bending torso. A comparison of the barrier functions h1 and h2 over the steps with these controllers can be seen in Figure 3.12. The same nominal model for the robot was used in the hardware experiments as in simulation, with model uncertainty presenting itself as significant friction in the joints, as well as imperfect mass and inertia measurements. The PDL algorithm was implemented on the AMBER-3M robot across a sequence of two episodes. The controllers ran on an off-board i7-6700HQ CPU at 2.6 [GHz] with 16 GB RAM, which computed desired torques and communicated to ELMO motor drivers on the 137 [cm] tall, 22 [kg] robot. The motor driver communication ran at 2 [kHz], and the (SS-QP) ran at 1 [kHz]. The stepping stone configuration was specified to the controller with stones of 8 [cm] in width. The nominal model-based (CBF-QP) controller resulted in a maximum 125 violation of the barriers of 9.2 [cm] due to model error. After running the PDL algorithm for two episodes, the maximum violation of the barriers was 1.9 [cm], only 21% of the original violation, as depicted in Figure 3.12. Gait tiles for this improved traversal of the stepping stones are shown in Figure 3.13. Conclusion In this section I have presented an approach for learning projected disturbances in the time derivative of a CBF with unstructured learning models. I begin by analyzing some of the challenges that can occur with learning projected disturbances with structured learning models that preserve a control-affine relationship. I then motivate why directly learning a projected disturbance produced using a particular controller and modifying the controller can lead to improved safety, and produce an episodic learning algorithm. I then present the problem of bipedal walking on stepping-stones, and present results from deploying the proposed learning algorithm in simulation and experimentally on the AMBER-3M bipedal platform. 3.8 Data-Driven Nonlinear Control In this section I will present a data-driven control framework. This framework directly incorporates data into a convex optimization problem that ensures that a CLF or CBF condition is met for all possible model errors permitted by the data. This approach is fundamentally different than the supervised learning approaches in Sections 3.3, 3.5, and 3.7. Rather than building learning models offline and deploying them in a controller online, which typically leads to residual learning errors (that may be quite large as discussed in Section 3.7) that impact stability or safety (as studied in Sections 3.4 and 3.6), this approach actively chooses control inputs that work for all possible model errors allowed by a data set. A key result of this work will be the ability to synthesize stabilizing or safe controllers without requiring the degree of variety in inputs in a data set that was needed to ensure a well-conditioned learning problem as discussed in Section 3.7. Rather, the use of robust convex optimization will allow relying only on inputs that have been seen in the data set. I will begin by considering the data-driven characterization of uncertainty in Section 3.4, and establish some intuition behind the approach with geometric visualizations. I will then introduce Control Certificate Functions (CCFs), which is a generalization that captures both CLFs and CBFs and permits 126 a single unifying framework. Next I will describe how model errors will be characterized, (it will differ slightly than in Section 3.4) and how this leads to a robust controller specified by a second-order cone program (SOCP), which is a convex optimization problem of a more complex class than the quadratic programs considered in Chapter 2 and Sections 3.3-3.7. After this, I will address questions regarding the boundedness of the possible model errors given properties of a data set, and provide a thorough proof of the feasibility of the robust controller by generalizing the conditions required by CLFs and CBFs to accommodate a set of model errors. Lastly, I will present simulation results utilizing the robust controller, demonstrating that performance can be achieved with limited variation of the inputs in the data set. The contributions of this section are as follows: • A data-driven control framework that is robust to model errors permitted by a data set through the use of robust convex optimization. • A characterization of feasibility of a robust controller by generalizing the conditions required of CLFs and CBFs to include a set of model errors. The text for this section is adapted from: A. J. Taylor, V. D. Dorobantu, S. Dean, B. Recht, Y. Yue, and A. D. Ames, “Towards robust data-driven control synthesis for nonlinear systems with actuation uncertainty,” in Proc. IEEE 60th Conf. on Decision and Control (CDC), Austin, TX, USA, pp.6469-6476, 2021. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. Data-Driven Intuition I will begin by developing some intuition behind how partial characterizations of model error from data can still permit controllers that meet stability or safety requirements. Consider the true open-loop system (2.1), and the nominal model open-loop system (3.1), and suppose there is a single input (m = 1). Suppose that xe is an unforced or forced equilibrium point of both the true open-loop system (2.1) and the nominal model open-loop system (3.1), and 127   Lg V (x) Lf V (x)   Lgb V (x) Lbf V (x) 1 1 ui   ui 1 min (Lg V (x)) ≤ Lg V (x) max (Lf V (x)) ≥ Lf V (x)   uj 1 1 1 uj ui Figure 3.14. Schematic of model error set arising from multiple data points. the function V : E → Rn is a local exponential CLF for both the true-open system (2.1) and nominal model (3.1) and equilibrium point xe . This amounts to the requirements of Assumptions 1 and 2 in Section 3.3. Recall that: V̇ (x, u) = Lf V (x) + Lg V (x)u, (3.149) Vḃ (x, u) = Lbf V (x) + Lgb V (x)u. (3.150) For a given x ∈ D, the vectors (Lg V (x), Lf V (x)) and (Lgb V (x), Lbf V (x)) can be visualized in Figure 3.14 (Top, Left). Model error between the true openloop system (2.1) and the nominal model (3.1) leads to a mismatch between the two vectors. 128 Consider the data-driven model error characterization in Section 3.4 in the ċ (x, u) = Vḃ (x, u) for all x ∈ D and absence of estimators bb and b a (such that W u ∈ R and ϵH does not appear). Given a data point ((xi , ui ), Vi ), we have that: |b(x) + a(x)ui | ≤ ϵ(x, xi , ui ). (3.151) This data point tells us that the vector (Lg V (x), Lf V (x)) lies in a strip that runs perpendicular to the vector (ui , 1), is centered about the vector (Lgb V (x), Lbf V (x)), and has a width determined by ϵ(x, xi , ui ), as visualized in Figure 3.14 (Top, Right). If we consider another data point ((xj , uj ), Vj ) with uj ̸= ui , we similarly have that: |b(x) + a(x)uj | ≤ ϵ(x, xj , ui ). (3.152) This again defines another strip that the vector (Lg V (x), Lf V (x)) must lie in, as seen in Figure 3.14 (Bottom, Left). Thus the vector (Lg V (x), Lf V (x)) must lie in the intersection of the two strips, which is a compact polytope as visualized in Figure 3.14 (Bottom, Right). In this way, multiple data points (even without dramatically different inputs) can lead to bounded sets of possible model errors, as will be explored later in this section. Intuitively, the greater the difference between ui and uj (such as if uj = −ui ), the smaller the set of possible model errors will be. This is a manifestation of the wellconditioning of the learning problem explored in Section 3.7. Still, even a small amount of variation is sufficient to introduce a bounded set of possible model errors, even if it would not lead to a well conditioned learning problem. Observe that for all values of Lf V (x) and Lg V (x) permitted by the set of possible model errors, we have that: max(Lf V (x)) ≥ Lf V (x), (3.153) 0 < min(Lg V (x)) ≤ Lg V (x), (3.154) as seen in Figure 3.14 (Bottom, Right). Consequently, if a negative input u ∈ R<0 is chosen to satisfy: max(Lf V (x)) + min(Lg V (x))u ≤ −k3 ∥x − xe ∥a , (3.155) then it satisfies: Lf V (x) + Lg V (x)u ≤ −k3 ∥x − xe ∥a , (3.156) 129 and thus the LF condition needed for local exponential stability is satisfied for the true closed-loop system. In this way, inputs can chosen based on worst-case values in the possible set of model errors permitted by a data set to ensure the desired stability or safety condition is met. Adding more data (with more variety in the inputs) serves to reduce these worst-case model errors and would yield a potentially less conservative (or more optimal) choice of u, but importantly, is not needed for feasibility, just optimality. The choice of inputs that are robust to the worst-case model errors will be rigorously codified via a convex optimization problem later in this section. Control Certificate Functions and Model Error To develop a control framework that can flexibly accommodate both CLFs and CBFs, I will use the notion of a Control Certificate Function (CCF), which was first explored in [218], [219]: Definition 23 (Control Certificate Function (CCF)). A function C : E → R that is continuously differentiable on E is said to be a Control Certificate Function (CCF) for the open-loop system (2.1) on the set D ⊆ E if there exists a function α : R → R such that: ∇C(x)⊤ z }| { ∂C ∂C (x) f (x) + (x)g(x) u < −α(C(x)), infm Ċ(x, u) ≜ infm u∈R u∈R ∂x | {z } |∂x {z } Lf C(x) (3.157) Lg C(x) for all x ∈ D. It is straightforward to see that if xe is an unforced or forced equilibrium point of the open-loop system (2.1) and V : E → R≥0 is a local exponential CLF for the open-loop system (2.1) and equilibrium point xe with a corresponding open set D, then V is a CCF for the open-loop system (2.1) on the set D = D \ {xe } with a function α(r) = − kk23 r. Showing that a CBF h : E → R for the open-loop system (2.1) on a set C defined as the 0-super level set of h is a bit more difficult. It is easier if C is defined as the 0-sublevel set, such that x ∈ C ⇐⇒ h(x) ≤ 0, in which case the CBF condition is replaced by: inf ḣ(x, u) < −α(h(x)), u∈Rm (3.158) for some α ∈ Ke and all x ∈ E. It is straightforward to transfer all of the results relating BFs and CBFs to safety to this sign convention, and using 130 CCFs is done simply to unify CLFs and CBFs under one mathematical tool. In particular, h is a CCF for the open-loop system (2.1) on the set D = E. As was done with both CLFs and CBFs, given a CCF C for the open-loop system (2.1) on the set D ⊆ E and a function α, define the point-wise set:  KCCF (x) ≜ u ∈ Rm Ċ(x, u) ≤ −α(C(x)) . (3.159) Any nominal controller knom : E → Rm that is locally Lipschitz continuous E can be modified to take values in the set KCCF (x) via following controller: 1 k(x) = argmin ∥u − knom (x)∥2 (CCF-QP) u∈Rm 2 s.t. ∇C(x)⊤ (f (x) + g(x)u) ≤ −α(C(x)). Considering a nominal model as in (3.1) and the dynamics decomposition in (3.2), we can see that the time derivative of a CCF for the open-loop system (2.1) on the set D can be written as: ḃ C(x,u) }| { z ⊤b ⊤ b(x) u + ∇C(x)⊤e e(x) u. f (x) + ∇C(x)⊤ g Ċ(x, u) = ∇C(x) f (x) + ∇C(x) g | {z } | {z } | {z } | {z } Lbf C(x) Lgb C(x) Lef C(x) Lge C(x) (3.160) Consequently, even if C is a CCF for the nominal model (3.1) on the set D, it is not necessarily a CCF for the open-loop system (2.1) on D. To ensure that the data-driven control problem is well-posed, I will make the following assumption as was seen in Sections 3.3 and 3.5: Assumption 4. If the function C is a CCF for the nominal model (3.1) on the set D, then it is a CCF for the open-loop system (2.1) on the set D. Lastly, I will also make the following assumption: e are Lipschitz continuous on E with Assumption 5. The functions e f and g known Lipschitz constants Lef and Lge . Observe that this preceding assumption is about Lipschitz constants on the e. This is an important detail because it suggests that error terms e f and g developing a better nominal model (reducing the error terms, and (likely) their Lipschitz constants) will make the subsequent constructions less conservative, and thus work invested in developing a good model-based controller before employing data-driven techniques will require the data-driven techniques to accomplish less. 131 Data-Driven Robust Control Synthesis Consider a dataset consisting of N tuples of states, inputs, and corresponding m state time derivatives, D = {(xi , ui , ẋi )}N i=1 , with xi ∈ E, ui ∈ R , and ẋi ∈ Rn for i = 1, . . . , N . I now show how data allows us to reduce model e directly, error by constraining the possible values of the functions e f and g without a parametric estimator. Considering the true open-loop system (2.1) evaluated at a state and input pair (xi , ui ) in the dataset yields: e i ≜ ẋi − (b b(xi )ui ) = e e(xi )ui , F f (xi ) + g f (xi ) + g (3.161) e i ∈ Rn can be interpreted as the error between the true state time where F derivative and the nominal model (3.1) evaluated at the state and input pair (xi , ui ). Considering a state x ∈ E (not necessarily present in the dataset D), the second equality in (3.161) implies: e ei = e e(x)ui − F e(xi )) ui . f (x) + g f (x) − e f (xi ) + (e g(x) − g (3.162) This expression provides a relationship between the possible values of the e at the state x and the values of the unmodeled unmodeled dynamics e f and g e are dynamics at the data point xi . Using the fact the functions e f and g Lipschitz continuous on E yields the following bound: e ei e(x)ui − F f (x) + g e(xi )) ui , f (x) − e f (xi ) + (e g(x) − g = e
≤ Lef + Lge ∥ui ∥ ∥x − xi ∥ ≜ ϵi (x), (3.163) (3.164) where ϵi : E → R≥0 for i = 1, . . . , N . The bound grows with the magnitude of the Lipschitz constants Lef and Lge and distance of the state x from the data point xi . The values of Lef and Lge are not explicitly data dependent, and thus the bound can be improved for a given dataset by reducing the possible model error through improved modeling. Given this construction, I define the point-wise error set:   n×m n e Ui (x) ≜ (A, b) ∈ R ×R b + Aui − Fi ≤ ϵi (x) ⊂ Rn×m × Rn , (3.165) noting that (e g(x), e f (x)) ∈ Ui (x) and Ui (x) is closed and convex. Performing this construction over the entire dataset D yields a point-wise error set: U(x) ≜ N \ i=1 Ui (x) ⊂ Rn×m × Rn , (3.166) 132 noting that (e g(x), e f (x)) ∈ U(x) and U(x) is closed and convex. Therefore, U(x) consists of all possible model errors that are consistent with the observed data. This allows us to pose the following robust control problem: Definition 24 (Data Robust CCF Optimization Problem). 1 kROB (x) = argmin ∥u − knom (x)∥2 (DR-CCF-OP) u∈Rm 2 s.t. ḃ C(x, u) + ∇C(x)⊤ (b + Au) ≤ −α(C(x)) for all (A, b) ∈ U(x). By construction we have that (e g(x), e f (x)) ∈ U(x), implying that kROB (x) ∈ KCCF (x) when the problem is feasible. I next present one the main results of this section by using robust optimization [220] to yield a convex problem for synthesizing such a robust controller. Theorem 22. Let C : E → R be a CCF with on the set D ⊂ E with corresponding function α : R → R. The robust controller (DR-CCF-OP) is equivalently expressed as: kROB (x) = argmin u∈Rm λi ∈Rn C(x, u) − s.t. ḃ 1 ∥u − knom (x)∥2 2 (DR-CCF-SOCP) N   X e i − ∥λi ∥ϵi (x) ≤ −α(C(x)), F λ⊤ i i=1 N X i=1 ⊤ λi u⊤ i = −∇C(x)u , N X λi = −∇C(x). i=1 Observe that this optimization problem is not a quadratic program, but rather, a second-order cone program (SOCP) [29]. This is due to the fact that the terms ∥λi ∥ appear in an inequality constraint. SOCPs are a more complex type of convex optimization problem (every QP can be written as an SOCP), but may still be efficiently solved through various numerical solvers such as ECOS [35] and MOSEK [36]. An important observation that I will also show in other contributions of this thesis synthesizing robust controllers for various modeling challenges through convex optimization is that robustifying an optimization problem of a particular class (a QP) will generally make it more complex, such as an SOCP. While I do not explore the local Lipschitz continuity properties of the controller above, I will explore some local Lipschitz continuity results for 133 SOCP-based controllers in Section 5.2. A meaningful area of future work in robust convex optimization-based control through CLFs and CBFs will seek to study the regularity properties of controllers specified through more complex optimization problems than QPs (which have been well covered in [51], [221], [222]) using concepts from variational analysis [223], [224]. Another interesting feature of this form of the controller is the dependence of the control input u on the inputs ui seen in the data set. In particular, the last two constraints establish a relationship between u and ui where u must be in some sense a linear combination of inputs ui in the data set. This relationship is best captured by the colloquial description of do what has worked before, and avoid what you don’t know. Lastly, the size of this optimization problem scales with the numbers of data points N , as each data point adds a decision variable λi . Solving this optimization problem in a computationally efficient manner may require data segmentation for higher-dimensional problems [225], but a detailed consideration is outside the scope of this thesis, which is focused on theoretical foundations. Proof. An input u ∈ Rm is feasible if the optimal value of the optimization problem: ḃ C(x, u) + ∇C(x)⊤ (b + Au) sup A∈Rn×m b∈Rn (3.167) e i ∥ ≤ ϵi (x) for all i ∈ 1, . . . , N, s.t. ∥b + Aui − F is less than or equal to −α(C(x)). Each inequality constraint can be rewritten as set membership in a second-order cone. Thus we can rewrite this optimization problem as: sup ḃ C(x, u) + ∇C(x)⊤ (b + Au) A∈Rn×m b∈Rm s.t.  (3.168)  e b + Aui − Fi , ϵi (x) ∈ Qn for all i ∈ 1, . . . , N, where Qn ⊂ Rn+1 denotes the Lorentz (second-order) cone: Qn = {(y, t) ∈ Rn × R : ∥y∥ ≤ t} . (3.169) 134 The associated Lagrangian for this optimization problem is given by: " #⊤ " # N ei X λi b + Aui − F ḃ ⊤ C(x, u) + ∇C(x) (b + Au) + νi ϵi (x) i=1 = ḃ C(x, u) + ∇C(x)⊤ b + ∇C(x)u⊤ , A N   X ⊤e ⊤ ⊤ + λi ui , A + λi b − λi Fi + νi ϵi (x) , (3.170) i=1 * = ∇C(x)u⊤ + + ḃ C(x, u) − N X i=1 N X + λi u⊤ i ,A + ∇C(x) + N X !⊤ λi b i=1  e i − νi ϵi (x) , F λ⊤ i (3.171) i=1 where (λi , νi ) ∈ Qn are Lagrange multipliers corresponding to the constraints imposed by the data point (xi , ui ) and ⟨·, ·⟩ denotes the trace inner product. The dual optimization problem is therefore: C(x, u) − inf n ḃ λi ∈R νi ∈R s.t. N  X  e i − νi ϵi (x) λ⊤ F i (3.172) i=1 N X ⊤ λi u⊤ i = −∇C(x)u , i=1 N X λi = −∇C(x), i=1 ∥λi ∥ ≤ νi for all i ∈ 1, . . . , N. For any (xi , ui ), choosing Lagrange multipliers such that νi > ∥λi ∥ increases the corresponding value of the problem compared to choosing νi = ∥λi ∥. I therefore simplify the problem as:  X ⊤e ḃ inf n C(x, u) − λi Fi − ∥λi ∥ϵi (x) (3.173) λi ∈R s.t. i=1 N X ⊤ λi u⊤ i = −∇C(x)u , i=1 N X λi = −∇C(x). i=1 This optimization problem can be substituted in for the robust constraint in (DR-CCF-OP) to yield the optimization problem (DR-CCF-SOCP). Controller Feasibility I now provide an analysis of the feasibility of the preceding controller. The feasibility of the controller (DR-CCF-SOCP) at a given state x is determined by the structure of the model error set U(x) defined in (3.166). The following 135 lemma provides a condition on the inputs in the dataset that implies U(x) is bounded for all x ∈ E: Lemma 1. Consider a dataset D with N data points satisfying N ≥ m + 1. If there exists a set of data points {(xi , ui , ẋi )}m+1 i=1 ⊆ D such that the set of vectors: (" #)m+1 ui M≜ , (3.174) 1 i=1 are linearly independent, then the model error set U(x) is bounded (and thus compact) for any x ∈ E. Proof. Consider the set M, and define: ( Mu ≜ ui ∈ Rm " # ) ui ∈M . 1 (3.175) For arbitrary x ∈ E, let (A, b) ∈ U(x). By definition of the model error sets Ui (x) and U(x) given in (3.165) and (3.166), respectively, we have that: e i ≤ ϵi (x), Aui + b − F (3.176) e i , we have that: for ui ∈ Mu . Defining vi ≜ Aui + b − F m+1 X 2 ei Aui + b − F i=1 = m+1 n XX |Vji |2 = ∥V∥2F , (3.177) i=1 j=1 h i where ∥ · ∥F is the Frobenius norm and V = v1 · · · vm+1 . Noting that ∥V∥F = ∥V⊤ ∥F , factoring and using the fact ∥P∥ ≤ ∥P∥F for any P ∈ Rm+1×n in conjunction with (3.176) yields:  u⊤  .1  ..  u⊤ m+1    2 e⊤ 1 " ⊤# F 1 m+1 X  .  A ..     . − ≤ ϵi (x)2 . . ⊤ .   b i=1 e⊤ 1 F (3.178) m+1 Taking the square root of both sides and employing the reverse triangle inequality we arrive at:  u⊤  .1  ..  u⊤ m+1 | {z U  v 1 " ⊤# um+1 uX  ..  A ≤t ϵi (x)2 + . ⊤ b i=1 1 }   e⊤ F  .1   ..  .   ⊤ e Fm+1 (3.179) 136 Noting that U has full rank by the linear independence of the vectors in the set M, we have that:   ⊤ " # " # u 1  .1  A⊤ A⊤ . . ..  σm+1 (U) ≤  (3.180)  .  b⊤ , b⊤ u⊤ m+1 1 where σm+1 (U) > 0 is the smallest singular value of U. Combining the two previous inequalities shows that the model error set U(x) is bounded. This result shows that variety in input directions is sufficient to assert boundedness of the model error set as a uniform property over the entire set E. As seen in the proof, the bound on this set may be very large if the values of ϵi (x) are large (as is the case when x is far away from the data points xi associated with the inputs used to construct the bound). Alternatively, the model error set will be small for a point x if there is local variety in input directions within the training dataset. To understand how the size of U(x) impacts feasibility of the optimization problem, I define the following set: n UeC (x) ≜ (a, b) ∈ Rm × R | ∃ (A, b) ∈ U(x) s.t. o a = (∇C(x)⊤ A)⊤ , b = ∇C(x)⊤ b . (3.181) The set UeC (x) can be interpreted as the projection of the dynamics model error set U(x) along the gradient of the CCF C, creating an m + 1 dimensional set. Additionally define the set: UC (x) ≜  Lgb C(x)⊤ , Lbf C(x)
⊕ UeC (x), (3.182) where ⊕ denotes a Minkowski sum. The set UC (x) is the recentering of UeC (x) around the nominal model terms (Lgb C(x)⊤ , Lbf C(x)) ∈ Rm+1 , such that it captures the possible values of (Lg C(x)⊤ , Lf C(x)). As multiplication by ∇C(x)⊤ is a linear transformation, UeC (x) and UC (x) are convex, and if U(x) is bounded (and therefore compact), then UeC (x) and UC (x) are compact. I now present the second main result of this section in the form of a necessary and sufficient condition for feasibility of (DR-CCF-OP): Theorem 23. For a state x ∈ E, let the sets U(x) and UC (x) be defined as in (3.166) and (3.182), respectively. Define the ray R ⊂ Rm+1 as R = 137 {0m }×(−α(C(x)), ∞). Assuming that U(x) is bounded, the data-driven robust controller (DR-CCF-SOCP) is feasible if and only if: UC (x) ∩ R = ∅. (3.183) Intuitively, the ray R represents scenarios in which the CCF time derivative can not be modified by actuation, and the CCF condition is not satisfied without actuation. By Assumption 4, the tuple (Lg C(x)⊤ , Lf C(x)) is not contained in R, but the possible model errors permitted by data need not necessarily reflect this. If one possible model error in the set UC (x) is contained in R, it is impossible to meet the CCF condition for that model error. Observe that the error set not intersecting this ray is a direct analog of the alternative CLF and CBF condition in (2.18) and (2.61), respectively. As those conditions enable proving feasibility of the resulting optimization-based controller, the analog in this setting will also permit proving feasibility. Proof. The proof proceeds from the original structure of the robust control problem (DR-CCF-OP). The constraint on the input u∗ specified by this controller is given by: Lbf C(x) + ∇C(x)⊤ b + (Lgb C(x) + ∇C(x)⊤ A)u∗ ≤ −α(C(x)), (3.184) for all (A, b) ∈ U(x), Given the definitions of UeC (x) and UC (x) in (3.181) and (3.182), this constraint can be expressed as: q + p⊤ u∗ ≤ −α(C(x)), (3.185) for all (p, q) ∈ UC (x). Necessity Assume that UC (x) ∩ R = ̸ ∅. This implies that there exists a (p, q) ∈ UC (x) such that q > −α(C(x)) and p = 0m . Thus for any input u ∈ Rm , we have that: q + p⊤ u = q > −α(C(x)), (3.186) violating the constraint in (3.185). Thus the optimization problem is infeasible. 138 Sufficiency i⊤ h Begin by defining the hyperplane H ⊂ Rm+1 with unit normal n = 0⊤ 1 m and offset −α(C(x)) and define the set: *" # " #+ ) ( 0m p , ≥ −α(C(x)) , (3.187) U C (x) = (p, q) ∈ UC (x) 1 q which corresponds to the tuples in the set UC (x) that do not meet the CCF condition (3.185) strictly under no input (u∗ = 0m ). Note that if u∗ = 0m satisfies (3.185) for all (p, q) ∈ U C (x), then u∗ = 0m satisfies (3.185) for all (p, q) ∈ UC (x). Also note that the set U C (x) is the closed subset of the compact and convex set UC (x) that is also contained in a (convex) half-space defined by the hyperplane H, meaning U C (x) is compact and convex (as it is the intersection of convex sets). Therefore, we can define: *" # " #+ 0m p q ⋆⋆ = max q = max , , (3.188) (p,q)∈U C (x) (p,q)∈U C (x) 1 q h i⊤  , · : U C (x) → R is continuous on a which exists as the function 0⊤ m 1 compact domain. We consider the case that q ⋆⋆ > −α(C(x)), as otherwise u∗ = 0m satisfies (3.185). Define the projection function Π : Rm ×R → Rm such that for (p, q) ∈ Rm ×R we have: Π((p, q)) = p. (3.189) As Π is a linear transform, the image of the compact and convex set U C (x) under Π, denoted as Π(U C (x)), is compact and convex and by assumption satisfies: Π(U C (x)) ∩ {0m } = ∅. (3.190) We may use the strict separating hyperplane theorem [29] to separate the set Π(U C (x)) from {0m } with the hyperplane Hβ with unit normal s ∈ S m−1 and offset β ∈ R>0 . This hyperplane can also be shifted to pass through the origin, given by H0 (with unit normal s ∈ S m−1 and offset 0). This results in the configuration seen in Figure 3.15. These hyperplanes can be extended back into the ambient space Rm × R by defining hyperplanes Hβ′ ⊂ Rm × R and h i⊤ H0′ ⊂ Rm × R with normal vector s′ = s⊤ 0 and respective offsets β and 0. 139 Rm H0 Hβ s Π(UC(x)) s Π(U C(x)) Figure 3.15. Projected view of model error set used in proving feasibility of the (DR-CCF-OP). The green region corresponds to Π(UC (x)) while the orange region highlights the set U C (x) lying above the hyperplane H. The line Hβ represents the strictly separating hyperplane with normal s and offset β, while H0 is the hyperplane shifted to pass through the origin. These hyperplanes serve to separate the vertical axis 0m × R from the cylinder defined by Π(U C (x)) × R. Define the two following open halfspaces: ( * " #+ ) p H0+ ≜ (p, q) ∈ Rm × R s′ , = p⊤ s > 0 , (3.191) q ( * " #+ ) p H0− ≜ (p, q) ∈ Rm × R s′ , = p⊤ s < 0 . (3.192) q We will now find a feasible input that lies anti-parallel to s, i.e., we will consider inputs of the form u∗ = −γs for γ ∈ R≥0 throughout the rest of this proof. Finding a feasible input satisfying constraint (3.185) amounts to finding a γ ∗ such that q − γ ∗ p⊤ s ≤ −α(C(x)) for all (p, q) ∈ UC (x). By definition we have that for any (p, q) ∈ U C (x): * " #+ p s′ , = s⊤ p ≥ β. (3.193) q Let γ0 ∈ R>0 be defined as: q ⋆⋆ + α(C(x)) γ0 = > 0. β (3.194) This value of γ0 implies that for any (p, q) ∈ U C (x) we have: q − γ0 p⊤ s ≤ q ⋆⋆ − γ0 β ≤ −α(C(x)). (3.195) 140 Define the set:  VC (x, γ) = (p, q − γp⊤ s) ∈ Rm × R | (p, q) ∈ U C (x) , (3.196) and the function: ψ(γ) = max q, (p,q)∈VC (x,γ) (3.197) that satisfies ψ(0) = q ⋆⋆ and ψ(γ0 ) ≤ −α(C(x)). I later show that ψ is continuous, but note the intermediate value theorem implies the existence of a γ ∗ ∈ (0, γ0 ] with: ψ(γ ∗ ) = max ∗ q = −α(C(x)). (3.198) (p,q)∈VC (x,γ ) Fixing u∗ = −γ ∗ s, we have that condition (3.185) is satisfied for all points (p, q) ∈ H0+ ∩ U C (x). Likewise, consider a point (p, q) ∈ H0+ ∩ (UC (x) \ U C (x)), for which the condition (3.185) is satisfied with no input as q < −α(C(x)), and note that p⊤ s > 0. We have that: q + p⊤ u∗ = q − γ ∗ p⊤ s < −α(C(x)). (3.199) Combining these results, we have that condition (3.185) is satisfied for all (p, q) ∈ H0+ ∩ UC (x). Additionally, for any (p, q) ∈ H0′ ∩ UC (x), p⊤ s = 0, as s′ is normal to H0′ . As strict separation implies that q < −α(C(x)) for any (p, q) ∈ H0′ ∩ UC (x), we have q − γ ∗ p⊤ s < −α(C(x)) and condition (3.185) is satisfied. Lastly, we must consider points (p, q) ∈ H0− ∩ UC (x). To this end, define the set:  WC (x, γ ∗ ) = (p, q − γ ∗ p⊤ s) ∈ Rm × R | (p, q) ∈ UC (x) . (3.200) This set can be interpreted as an invertible linear transformation of the set UC (x) as we have: " # " # " #" # p′ p Im×m 0m p = = , (3.201) q′ q − γ ∗ p⊤ s −γ ∗ s⊤ 1 q implying that WC (x, γ ∗ ) is compact and convex. Similarly, define the set: *" # " #+   0m p ∗ ∗ W C (x, γ ) = (p, q) ∈ WC (x, γ ) , ≥ −α(C(x)) . (3.202) q 1 The set W C (x, γ ∗ ) is convex and contains points in WC (x, γ ∗ ) that are in or above the hyperplane H, or points that meet with equality or violate (3.185), 141 respectively. By the specific choice of γ ∗ , there exists at least one point v+ ∈ H0+ ∩W C (x, γ ∗ ). Furthermore, by the preceding strict separation, we have that H0′ ∩ W C (x, γ ∗ ) = ∅. Now assume for contradiction that H0− ∩ W C (x, γ ∗ ) ̸= ∅, or that there exists v− ∈ H0− ∩ W C (x, γ ∗ ). As the set W C (x, γ ∗ ) is convex and ⟨s′ , v+ ⟩ > 0 and ⟨s′ , v− ⟩ < 0, there is a λ∗ ∈ (0, 1) satisfying: ⟨s′ , (1 − λ∗ )v+ + λ∗ v− ⟩ = 0, (3.203) implying (1 − λ∗ )v+ + λ∗ v− ∈ H0′ ∩ W C (x, γ ∗ ). This contradicts the fact H0′ ∩ W C (x, γ ∗ ) = ∅, implying that H0− ∩ W C (x, γ ∗ ) = ∅, or that condition (3.185) is satisfied for all points (p, q) ∈ H0− ∩ UC (x). Combining all previous results, condition (3.185) is satisfied for all points (p, q) ∈ UC (x) for the input u∗ = −γ ∗ s, ensuring feasibility. I now show that ψ is continuous. For γ, γ ′ > 0 and (p, q) ∈ U C (x), we have: " # " # p p′ min (3.204) − (p′ ,q ′ )∈VC (x,γ ′ ) q − γp⊤ s q′ " # " # p p ≤ − ≤ p⊤ s|γ − γ ′ |, (3.205) q − γp⊤ s q − γ ′ p⊤ s and similarly, for (p′ , q ′ ) ∈ U C (x), we have: " # " # p p′ min ≤ (p′ )⊤ s|γ − γ ′ |. − ′ (p,q)∈VC (x,γ) q q − γ ′ (p′ )⊤ s (3.206) Therefore, the Hausdorff distance between VC (x, γ) and VC (x, γ ′ ) is bounded:   ′ ⊤ dH (VC (x, γ), VC (x, γ )) ≤ max p s |γ − γ ′ |, (p,q)∈U C (x) implying VC (x, γ) is Lipschitz continuous (with respect to the Hausdorff metric) as a function of γ. The support function of a nonempty, compact, and convex set A ⊂ Rm × R given by hA : Rm+1 → R is defined as: * " #+ p hA (v) = max v, . (3.207) (p,q)∈A q h i⊤ Recalling n = 0⊤ , the Hausdorff distance between two nonempty, com1 m pact, and convex sets A, B ⊂ Rm × R satisfies: dH (A, B) = max |hA (v) − hB (v)| ≥ |hA (n) − hB (n)|. m v∈S 142 Noting that ψ can be expressed in terms of the support function as a composition, with: ψ(γ) = max q = hVC (x,γ) (n), (p,q)∈VC (x,γ) implies ψ is a continuous function. Simulation Results To demonstrate the proposed controller, I simulate the inverted pendulum with input attenuation in (3.3) with gravitational acceleration g = 10, length ℓ = 0.7, and mass m = 0.7. The nominal model for the system is given by (3.4) with gravitational acceleration estimate ĝ = 10, length estimate ℓ̂ = 0.63, and mass estimate m̂ = 0.63. Consider functions V : R2 → R≥0 and h : R2 → R, given by V (x) = x⊤ Px and h(x) = x⊤ Px − c with: # "√ 3 1 √ , (3.208) P= 3 1 and a constant c = 0.2. Noting that both the nominal model open-loop system (3.4) and the true open-loop system (3.3) are feedback linearizable, V and h satisfy the CCF condition (3.157) for the comparison function α(r) = σr/λmax (P) with σ ∈ (0, 1), for both the nominal model open-loop system (3.4) and the true open-loop system (3.3) (implying Assumption 4 is met). In particular, V may serve as a CLF, and h as a CBF. I explore both data-driven stability and safety with this system. I compare the robust data-driven controller with an oracle controller, which is a true model-based (CCF-QP) controller, and a baseline controller, which is a nominal model-based (CCF-QP) controller. In each setting, the system model underestimates the pendulum mass and length and assumes that the input gain is independent of the state. As a result, the Lipschitz constants of the √ errors can be bounded by Lef = g|ℓ − ℓ̂|/(ℓℓ̂) and Lge = 0.75 2 exp(− 21 )/(mℓ2 ). For the stability experiment, I generate data sets by gridding the state and input spaces. I consider x1 in the interval [0, 1] and x2 in the interval [−0.25, 0.25], 1 with grid sizes ϵθ = ϵθ̇ = 40 . I generate a sparse data set by considering u ∈ {−5, −1} and a dense data set by considering u ∈ {−5, −3, −1, 1, 3, 5}. I set the controller knom to be a feedback linearizing controller designed using √ the nominal model and linear gains kp = 1/2 and kd = 3/2. The system is simulated from the initial condition x0 = [0.8, 0.1]⊤ for 10 [s], with control 143 Stability u(t) C(x(t)) 2 1 0 0 −5 0 2 4 6 8 10 6 8 10 C(x(t)) Safety 0.2 oracle baseline R-dense R-sparse 0.0 −0.2 u(t) 0 −2 −4 0 2 4 t (sec) Figure 3.16. The value of the certificate function C(x) (Top) and input u (Bottom) over time for the stability (Top Pair) and safety (Bottom Pair) experiments. I compare the behavior of a true model-based (CCF-QP) controller (oracle, black square), a nominal-model based (CCF-QP) controller (baseline, red diamond), and the proposed (DR-CCF-SOCP) controller using dense input data (R-dense, blue triangle) and sparse input data (R-Sparse, green circle). For stability, the desired behavior for C(x) is to converge towards 0, while for safety it is to remain non-positive. inputs specified at 100 [Hz]. For the safety experiment, I consider a similar pair of sparse and dense data sets with x1 in the interval [0, 0.25] and x2 in the same interval as the stability experiment, the same grid sizes, and the same sets of control inputs. The system is simulated with no nominal controller (knom = 0 for all x ∈ E) from the initial condition x0 = [0.1, 0.1]⊤ for the same amount of time and the same control input frequency. The results of the simulations may be seen in Figure 3.16. In both experiments, we see that the baseline controller fails to achieve the specified objective, while the oracle controller succeeds. Furthermore, we see that the data-driven controllers perform nearly identically for both the sparse and dense input data sets. This similarity indicates that greater variety in input directions and cov- 144 erage of the input space by the data set are not needed to achieve satisfactory closed-loop behavior. Conclusion In this section I have presented an approach for data-driven robust control through convex optimization. I first provide a geometric understanding of model error as it impacts the time derivative of a CLF, and indicate how data can be used to bound potential model errors in a way that permits robust control. I then consider Control Certificate Functions as a generalization that captures both CLFs and CBFs. This leads to a robust optimization problem with an infinite number of constraints which can be shown to be equivalent to a finite dimensional second-order cone program through a duality argument. I then provide a statement about the boundedness of model error sets given sufficiently rich input data, and use this to provide a proof on the feasibility of the robust controller. I demonstrate the proposed control approach in simulation on the inverted pendulum with input attenuation in Section 3.2. 3.9 Adaptive Safety via Control Barrier Functions In this section I will present work on the problem of adaptive safety-critical control through CBFs. I will first review the work in [47] which describes the adaptive control setting considered in this work, and defines adaptive CLFs as a tool for stabilization in the presence of parametric model error. Building off of the ideas in this work, I will propose a novel notion of an adaptive barrier function (aBF) and Control Barrier Function (aCBF) that must meet an inequality condition over both states and possible parameter values. I will then show how an aBF yields safety of a closed-loop system and how an aCBF can be used to synthesize a controller that yields an aBF for a closedloop system. I will then analyze some of the conservative conditions that define aBFs and aCBFs through a rigorous counterexample, demonstrating how a naïve approach to defining aCBFs can lead to a failure of rigorous safety guarantees. Lastly, I demonstrate the tool of aCBFs in simulation on an adaptive cruise control (ACC) example. The contributions of this section are as follows: • The definition of adaptive Control Barrier Functions for ensuring the safety of a nonlinear control system with parametric model error. 145 • A detailed analysis of the strong requirements needed for forward invariance of a set in the presence of parametric model error through a theoretically rigorous counterexample. The text for this section is adapted from: A. J. Taylor and A. D. Ames, “Adaptive safety with control barrier functions,” in Proc. IEEE American Control Conf. (ACC), Denver, CO, USA, 2020, pp.1399-1405. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. Adaptive Systems Recall the formulation of parametric model error described in Section 3.2 yielding the open-loop system (3.6) which is restated here: b(x)u, ẋ = b f (x) + F(x)θ ⋆ + g (3.209) where θ ⋆ ∈ Rp is a vector of unknown parameters and F : E → Rn×p is a collection of known basis functions and is assumed to be locally Lipschitz continuous on E. The challenge of designing explicit controllers that are robust to a wide range of unknown parameters suggests considering a larger class of controllers to ensure the safety of (3.209). In particular, controllers that update an estimate of the unknown parameters. These are called adaptive controllers, and take the form: b u = k(x, θ), b˙ = Γτ (x, θ), b θ (3.210) (3.211) b ∈ Rp represents an estimate of the unknown parameters θ ⋆ maintained where θ by the controller, k : E × Rp → Rm is locally Lipschitz continuous on E × Rp , Γ ∈ Sp≻0 is a matrix adaptive gain, and τ : E × Rp → Rp is a adaptation law that is locally Lipschitz continuous on E × Rp . Introducing this parameter update results in a composite closed-loop system: " # " # b b ẋ b(x)k(x, θ) f (x) + F(x)θ ⋆ + g = . (3.212) b b˙ Γτ (x, θ) θ 146 I refer to the state portion of this system as the state closed-loop system, and the parameter portion as the parameter closed-loop system. The composite b0 ) ∈ E × system evolves in E × Rp , such that for any initial condition (x0 , θ b0 ) = [0, tmax (x0 , θ b0 )) and a unique Rp , there exists a maximal interval I(x0 , θ b0 ) → E × Rp satisfying: continuously differentiable solution (φx , φθb ) : I(x0 , θ " # " # b b(φx (t))k(φx (t), φθb (t)) φ̇x (t) f (φx (t)) + F(φx (t))θ ⋆ + g = , (3.213) φ̇θb (t) Γτ (φx (t), φθb (t)) b0 ). For simplicity, in this work I will assume that the closedfor all t ∈ I(x0 , θ loop system (3.212) is forward complete, such that for any initial condition b0 ) ∈ E × Rp , the interval I(x0 , θ b0 ) = R≥0 . (x0 , θ Adaptive Control Lyapunov Functions Stabilization of a system with parametric model errors has long been a goal of adaptive control, though the work in [47] was the first to consider achieving it through CLFs. It is useful to review this work as it will guide the formulation taken for achieving safety in the presence of parametric model errors through the use of adaptive CBFs. In this setting we are interested in driving the system to a forced or unforced equilibrium xe ∈ E of the open-loop system (3.209). Because we are considering potentially unbounded parameter errors, we will restrict our attention to global stability results. To ensure the wellposedness of the stabilization problem, I will assume that F(xe ) = 0n , such that xe only needs to be an unforced or forced equilibrium point of the nominal model open-loop system: b(x)u. ẋ = b f (x) + g (3.214) Next, I will relax the assumption that the controllers I consider be locally Lipschitz continuous on E × Rp , but rather require that they only be locally Lipschitz continuous on (E \ {xe }) × Rp . Furthermore, the controllers will need to render xe an equilibrium point of the state portion of the closed-loop system (3.212), such that: b b = 0n , b(xe )k(xe , θ) f (xe ) + g (3.215) b ∈ Rp . This leads to the following notion of adaptive stability: for all θ Definition 25 (Adaptive Stability). Let xe ∈ E be an unforced or forced equilibrium point of the open-loop system (3.209). The composite closed-loop 147 system (3.212) is said to be adaptively stable with respect to xe if for any initial b0 ) ∈ E ×Rp , we have that the solution (φx , φb ) : R≥0 → E ×Rp condition (x0 , θ θ is bounded on R≥0 and satisfies limt→∞ φx (t) = xe . Note that the requirements for adaptive stability are rather weak in the sense that φθb (t) is not required to converge to θ ⋆ . We will see, in fact, that convergence of φθb (t) to θ ⋆ is not necessary for φx (t) to converge to xe . Establishing that a system is adaptively stable with respect to an equilibrium point xe can be achieved through an adaptive Lyapunov Function (aLF): Definition 26 (Adaptive Lyapunov Function (aLF)). Let xe ∈ E be an equilibrium point of the state closed-loop system of (3.212). A function a Va : E → R≥0 that is continuously differentiable on E with ∂V : E → Rn ∂x locally Lipschitz continuous on E is said to be an adaptive Lyapunov function (aLF) for the state closed-loop system in (3.212) and equilibrium point xe if there exists k1 , k2 , k3 , a ∈ R>0 such that: k1 ∥x − xe ∥a ≤ Va (x) ≤ k2 ∥x − xe ∥a , (3.216)   b+g b ≤ − k3 ∥x − xe ∥a , (3.217) b ≜ ∂Va (x) b b(x)k(x, θ) f (x) + F(x)θ V̇a (x, θ) ∂x for all x ∈ E and θb ∈ Rp . a The importance of ∂V being locally Lipschitz continuous on E will be impor∂x tant for ensuring the update law τ is locally Lipschitz continuous on E × Rp . An aLF can be used to certify the adaptive stability of the closed-loop system (3.212), as captured in the following theorem: Theorem 24. Let xe ∈ E be an equilibrium point of the state closed-loop system in (3.212). If there exists an adaptive Lyapunov function Va : E → R≥0 for the state closed-loop system in (3.212) and equilibrium point xe , then for any Γ ∈ Sp≻0 , the update law:  b = τ (x, θ) ⊤ ∂Va (x)F(x) , ∂x (3.218) renders the closed-loop system (3.212) adaptively stable with respect to xe . It is useful to review this proof, as similar steps will be taken when considering adaptive safety through adaptive barrier functions: 148 Proof. Denote the parameter error: (3.219) e = θ ⋆ − θ, b θ to be the difference between the actual and estimated parameters. Consider now the candidate composite Lyapunov function: b = Va (x) + 1 θ eT Γ−1 θ, e V (x, θ) 2 (3.220) noting that it satisfies: k1 ∥x − xe ∥a + 1 2 2λmax (Γ) e θ b ≤ k2 ∥x − xe ∥a + ≤ V (x, θ) 1 2λmin (Γ) 2 e θ . (3.221) Computing its derivative we can obtain: b = V̇a (x, θ ⋆ ) − θ eT Γ−1 θ, b˙ V̇ (x, θ) (3.222)  eT ≤ −k3 ∥x − xe ∥ + θ a ! T ∂Va b . (x)F(x) − τ (x, θ) ∂x (3.223) It is now easy to see that using the update law: T ∂Va (x)F(x) , ∂x (3.224) b ≤ −k3 ∥x − xe ∥a , V̇ (x, θ) (3.225)  b = τ (x, θ) implies: from which we conclude that the solution (φx , φθb ) is bounded on R≥0 . It now follows from LaSalle’s invariance principle [100, Theorem 4.4] that φx converges to the largest invariant subset of the collection of points x ∈ E satisfying k3 ∥x − xe ∥a = 0, which is x = xe , such that limt→∞ φx (t) = xe . The proof utilizes LaSalle’s invariance principle, which has two effects. First, it does not allow us to draw conclusions about the convergence of φθb to the true parameter θ ⋆ . This is generally achieved by requiring conditions such as persistence of excitation [206], but this will not be considered in this thesis. Second, it only provides a weak bound on the transient behavior of the state of the system. In particular, the time derivative of V can be made negative by decreasing the parameter error, while increasing the deviation of φx from the equilibrium point xe . It is this behavior that will have to be carefully 149 considered when considering forward invariance of a set in the state space when developing adaptive CBFs. The notion of adaptive LFs naturally generalizes to adaptive CLFs, as captured in the following definition: Definition 27 (Adaptive Control Lyapunov Function (aCLF)). Let xe ∈ E be an unforced or forced equilibrium point of the open-loop system (3.209). A function Va : E → R≥0 that is continuously differentiable on E is said to be an adaptive Control Lyapunov Function (aCLF) for the open-loop system (3.209) and equilibrium point xe if there exists k1 , k2 , k3 , a ∈ R>0 such that: k1 ∥x − xe ∥a ≤ Va (x) ≤ k2 ∥x − xe ∥a , (3.226)   b u) ≜ inf ∂Va (x′ ) b b+g b(x′ )u < − k3 ∥x′ − xe ∥a , f (x′ ) + F(x′ )θ infm V̇a (x′ , θ, m u∈R u∈R ∂x (3.227) for all x ∈ E, x′ ∈ E \ {xe } and θb ∈ Rp . Given an aCLF for the open-loop system (3.209) and equilibrium point xe ∈ E, we can define the following pointwise set: b = {u ∈ Rm | V̇a (x, θ, b u) ≤ −k3 ∥x − xe ∥a }. KaCLF (x, θ) (3.228) This leads to the following theorem typically seen with CLFs: Theorem 25. Let xe ∈ E be an unforced or forced equilibrium point of the open-loop system (3.209) and let Va : E → R≥0 be an aCLF for the open-loop b is nonsystem (3.209) and equilibrium point xe . Then, the set KaCLF (x, θ) b ∈ Rp , and for any controller k : E × Rp → Rm empty for all x ∈ E and θ that is continuous on E × Rp , locally Lipschitz continuous on (E \ {xe }) × Rp , renders xe an equilibrium point of the state closed-loop system in (3.212), and b ∈ KaCLF (x, θ) b for all x ∈ E and θ b ∈ Rp , the function Va is satisfies k(x, θ) an aLF for the state closed-loop system in (3.212) and equilibrium point xe . Naturally, this motivates synthesizing a controller through convex optimization as follows: b = argmin ∥u − ue ∥2 kaCLF (x, θ) u∈Rm b u) ≤ −k3 ∥x − xe ∥a . s.t. V̇a (x, θ, (aCLF-QP) 150 While not provided in the thesis, continuity and local Lipschitz continuity results for controllers built through CLFs and optimization in Section 2.3 can be extended to this controller under appropriate small or continuous control b in some property assumptions on Va that hold for all parameter values θ bounded set (noting that the proof of adaptive stability through aLFs yields that φθb (t) is bounded). Adaptive Control Barrier Functions The strong parallels between CLFs and CBFs motivate the consideration of achieving safety for systems with parametric model error through the use of adaptive control in conjunction with CBFs. In spite of these parallels, one challenge that is hidden in the proof relating an aLF and adaptive stability is the transient behavior of the solution φx . In particular, one does not have that the sublevel sets of the aLF Va are forward invariant, but rather that the sublevel sets of the composite LF V are forward invariant. Because of this, the system state may leave sublevel sets of Va while V decreases because the error between the parameter estimates and true parameters is sufficiently decreasing. This will be an issue if it is necessary to keep the state of the system within a particular set during the evolution of the system. In an effort to translate the work on aCLFs to the setting of safety and CBFs and overcome the aforementioned challenges, I will propose a novel notion of adaptive safety: Definition 28 (Adaptive Safety). Let C ⊂ E be the 0-superlevel set of a function ha : E → R that is continuously differentiable on E. The composite closed-loop system (3.212) is said to be adaptively safe with respect to C if there exists a δ ∈ R>0 such that the set: Cδ ≜ {x ∈ Rn | ha (x) ≥ δ}, (3.229) b0 ) ∈ Cδ × Θ where Θ ⊂ Rp is is non-empty and for any initial condition (x0 , θ some collection of possible initial parameter estimates, we have that φx (t) ∈ C for all t ∈ R≥0 . There are a few notable observations about this definition. First, it is posed for a set C that can be represented as the 0-superlevel set of a function ha that is continuously differentiable on E, rather than any general set in E. This 151 structure allows us to construct a contraction of C by shifting the value of the superlevel set up to produce Cδ . The state initial conditions are restricted to this set Cδ to provide a buffer between when the system begins evolving (when adaptation begins) and when the system can encounter the boundary of the set C. We will see when proving that an adaptive barrier function renders a system adaptively safe with respect to the set C, the size of this buffer determines requirements on the adaption rate (the magnitude of the adaptive gain Γ). Second, we consider a restricted initial parameter set Θ, rather than all of Rp . This initial parameter set typically contains the true parameters θ ⋆ , its size will be related to possible initial parameter errors, and it will similarly yield requirements on the adaptive gain Γ. Given this definition, I now provide a novel definition of adaptive barrier functions (aBFs): Definition 29 (Adaptive Barrier Function (aBF)). Let C ⊂ E be the 0superlevel set of a function ha : E → R that is continuously differentiable a a : E → Rn locally Lipschitz continuous on E, ∂h (x) ̸= 0n on E, with ∂h ∂x ∂x when ha (x) = 0, and suppose there exists an x ∈ E such that ha (x) > 0. The function h is an adaptive barrier function (aBF) for the state closed-loop system in (3.212) on C if: b ≜ ḣa (x, θ)   ∂ha b + g(x)k(x, θ) b ≥ 0, f (x) + F(x)θ (x) b ∂x (3.230) b ∈ Rp . for all x ∈ E and θ Observe that the typical extended class-K function that lower bounds the time derivative has been replaced with 0. It is natural to wonder if this could be replaced with −α(ha (x)) for some α ∈ Ke . I will show that this weakening of the lower bound will not lead to adaptive safety in the next subsection through a rigorous counterexample. The following theorem relates adaptive safety and aBFs: Theorem 26. Let C ⊂ E be the 0-superlevel set of a function ha : E → R. If ha is an aBF for the state closed-loop system in (3.212) on C, then for any c ∈ R>0 , there exists an adaptive gain Γ ∈ Sp≻0 such that the update law:  b =− τ (x, θ) ∂ha (x)F(x) ∂x ⊤ , (3.231) 152 renders the closed-loop system (3.212) adaptively safe with respect to C with: n o b0 ∈ Rp | ∥θ b0 − θ ⋆ ∥ ≤ c . Θ= θ (3.232) This theorem states that for any maximum initial parameter error c, there exists an adaptive gain Γ and update law τ that renders the composite closedloop system (3.212) adaptively safe with respect to C. In contrast to adaptive stabilization, the same adaptive gain may not work for all possible initial parameter errors, as shown in the following proof: Proof. By assumption, there exists an x ∈ C such that ha (x) ≜ δ > 0. Let Γ ∈ Sp≻0 satisfy: c2 (3.233) λmin (Γ) ≥ . 2δ Note that this inequality requires the adaptive gain to grow with larger possible initial parameter errors, as well as grow as the buffer between the set C and Cδ becomes smaller. Notice that for a fixed adaptive gain, the size of the buffer can be smaller for smaller initial parameter errors. Consider a continuously differentiable function h : E × Rp → R defined as: b ⊤ Γ−1 (θ ⋆ − θ). b b = ha (x) − 1 (θ ⋆ − θ) h(x, θ) 2 (3.234) Observe that rather than add the quadratic function of the parameter error as was done with aCLFs, I have subtracted it. To see that 0 is a regular value of h, consider its gradient: # " # " ∂h ∂ha b (x, θ) (x) ∂x ∂x = . (3.235) ⋆ ∂h b b ⊤ Γ−1 (x, θ) (θ − θ) b ∂θ b = 0. If ha (x) = 0, then ∂h (x, θ) b = ∂ha (x) ̸= 0n . If Suppose that h(x, θ) ∂x ∂x b ̸= 0p , such that ∂h (x, θ) b = (θ ⋆ − θ) b ⊤ Γ−1 ̸= 0p as Γ−1 ha (x) ̸= 0, then θ ⋆ − θ b ∂θ is full rank. Thus 0 is a regular value of h. Next, note that by construction: b ⊤ Γ−1 (θ ⋆ − θ) b ≤ λmax (Γ−1 )∥θ ⋆ − θ∥ b 2, (θ ⋆ − θ) 1 b 2. = ∥θ ⋆ − θ∥ λmin (Γ) (3.236) (3.237) 153 b0 ) ∈ Cδ × Θ, we have that: Thus for any initial condition (x0 θ b0 ) = ha (x0 ) − 1 (θ ⋆ − θ b0 )⊤ Γ−1 (θ ⋆ − θ b0 ), h(x0 , θ 2 1 b0 ∥2 , ≥δ− ∥θ ⋆ − θ 2λmin (Γ) 2δ b0 ∥2 ≥ 0. ≥ δ − 2 ∥θ ⋆ − θ 2c (3.238) (3.239) (3.240) Thus, the adaptive gain Γ has been chosen large enough to ensure that the value of the composite barrier function h is positive for all initial conditions b0 ) ∈ Cδ × Θ. The time derivative of h is given by: (x0 , θ   ∂ha b˙ b +θ e⊤ Γ−1 θ, b(x)k(x, θ) (x) b f (x) + F(x)θ ⋆ + g ∂x  ∂h  ∂ha a ⋆ b b b b(x)k(x, θ) + = (x) f (x) + F(x)θ + g (x)F(x)θ ∂x ∂x ∂ha b b+θ eT τ (x, θ), (x)F(x)θ − ∂x   ∂ha b + g(x)k(x, θ) b = (x) f (x) + F(x)θ ∂x ∂ha e+θ e⊤ τ (x, θ), b + (x)F(x)θ ∂x ! ⊤  ⊤ ∂h a b , e (x)F(x) + τ (x, θ) ≥ θ ∂x b = ḣ(x, θ) ≥ 0, where the first inequality follows as ha is an aBF for the state closed-loop system in (3.212) on C and the second inequality follows by the choice of update law τ . Thus by Nagumo’s theorem [104], [123], we have that h(φx (t), φθb (t)) ≥ 0 for all t ∈ R≥0 . Lastly, we may conclude that: 1 ⋆ (θ − φθb (t))⊤ Γ−1 (θ ⋆ − φθb (t)), 2 ≥ 0, ha (φx (t)) ≥ (3.241) (3.242) such that φx (t) ∈ C for all t ∈ R≥0 as desired. The proof reveals that all superlevel sets of h are forward invariant, which is a potentially conservative requirement, but also a non-trivial one to circumnavigate. As h can not be computed without knowing the true parameters θ ⋆ , it is b ≥ −α(h(x, θ)) b for some extended class-K function not possible to set ḣ(x, θ) b for all α as is typical with CBFs. Furthermore, we have that ha (x) ≥ h(x, θ) 154 b ∈ Rp , implying that −α(ha (x)) ≤ −α(h(x, θ)) b for all x ∈ E, x ∈ E and θ b ∈ Rp , and α ∈ Ke . Thus setting ḣ(x, θ) b ≥ −α(ha (x)) does not yield the deθ b One may note that setting ḣ(x, θ) b ≥ −α(ha (x)) sired lower bound on ḣ(x, θ). b ≥ 0 when ha (x) = 0, or when the state is on the boundleads to ḣ(x, θ) ary of C. This fact is concurrent with the common forward invariance proof technique utilizing Nagumo’s theorem [104], [123]. Despite this, it is in fact possible to construct simple examples (in R2 ) such that the state must leave the set C defined by ha for any choice of differentiable α ∈ Ke and Γ ∈ Sp≻0 , as shown in the following subsection. The reason these challenges are avoided with adaptive CLFs is because the proof concludes by using LaSalle’s invariance principle, and neglecting the transient behavior of φx . It is precisely the transient behavior which describes whether or not the state remains within a particular set during the course of its evolution. Extending the notion of adaptive BFs to adaptive CBFs requires a small amount of care. In particular, we must be careful when translating the nonstrict inequality in (3.244) to a strict inequality as is typical when defining a a (x) = 0n , then CBF variant. If there exists a state x ∈ Int(C) such that ∂h ∂x it will be impossible to satisfy the strict inequality at that state through the choice of input. This eliminates considering any set C that is compact, as any continuously differentiable function ha must8 have a critical point in the b u) = 0 for interior of C. Fortunately, at any such state x ∈ Int(C), ḣa (x, θ, b ∈ Rp and u ∈ Rm , so the aBF property is still satisfied at these states all θ as required for adaptive safety even if the strict inequality can’t be satisfied there. Note that such states must not appear on the boundary of C so that Nagumo’s theorem may be used. To this end, I will denote the following set:   ∂ha (x) = 0n , (3.243) AC = x ∈ E | x ∈ Int(C) and ∂x and propose the following definition: Definition 30 (Adaptive Control Barrier Function (aCBF)). Let C ⊂ E be the 0-superlevel set of a function ha : E → R that is continuously differentiable a a on E with ∂h : E → Rn locally Lipschitz continuous on E, ∂h (x) ̸= 0n when ∂x ∂x ha (x) = 0, and suppose there exists an x ∈ E such that ha (x) > 0. The function ha is said to be an adaptive Control Barrier Function (aCBF) for the 8 I do not offer a proof of this in this thesis, but it is relatively easy to show through first-order optimality conditions. 155 open-loop system (3.209) on C if:   ∂ha b b b b(x)u > 0, sup ḣa (x, θ, u) ≜ sup (x) f (x) + F(x)θ + g u∈Rm u∈Rm ∂x (3.244) for all x ∈ E \ AC and θb ∈ Rp . Given an aCBF for the open-loop system (3.209) on C, we can define the following pointwise set: b = {u ∈ Rm | ḣ(x, θ, b u) ≥ 0}. KaCBF (x, θ) (3.245) This leads to the following theorem: Theorem 27. Let C ⊂ E be the 0-superlevel set of a function ha : E → R and let ha be an adaptive CBF for the open-loop system (3.209) on C. Then, the b is non-empty for all x ∈ E and θ b ∈ Rp , and for any controller set KaCBF (x, θ) k : E × Rp → Rm that is locally Lipschitz continuous on E × Rp , and satisfies b ∈ KaCBF (x, θ) b for all x ∈ E and θ b ∈ Rp , the function ha is an aBF k(x, θ) for the state closed-loop system in (3.212) on C. Letting knom : E × Rp → Rm be locally Lipschitz continuous on E × Rp , we may consider the following controller: b = argmin ∥u − knom (x, θ)∥ b 2 kaCBF (x, θ) u∈Rm (aCBF-QP) b u) ≥ 0. s.t. ḣa (x, θ, This controller can be shown to be locally Lipschitz continuous on E×Rp under b ∈ a variant of the continuous control property that requires for each (x, θ) p AC × Rp , there exists an open set D(x,θ) b ⊆ E × R and a constant L(x,θ) b ∈ R≥0 b ∈ D b , there exists an input u ∈ Rm such that: such that for any (y, ψ) (x,θ) b ≤ L b ∥y − x∥ + L b ∥ψ b − θ∥, b ∥u − knom (x, θ)∥ (x,θ) (x,θ) (3.246) and: b u) ≥ 0. ḣa (y, ψ, (3.247) I note that it is important that kaCBF be locally Lipschitz continuous on E×Rp , and not just locally Lipschitz continuous on (E \ AC ) × Rp and continuous on E×Rp . An important requirement of Nagumo’s theorem [104], [123] used in the 156 proof of adaptive safety are that solutions are unique. Theorem 3 establishes the existence of a unique solution to the closed-loop system (2.2) from the initial condition xe ∈ E at which the controller k is only continuous (but not locally Lipschitz continuous) by requiring that xe be an equilibrium point of the closed-loop system (2.2) and the existence a local exponential Lyapunov function. If kaCBF was only locally Lipschitz continuous on (E \ A) × Rp , using Theorem 3 would require that each point xa ∈ AC be an equilibrium point of the state closed-loop system in (3.212). Furthermore, if x ∈ AC is a local minimum of the function ha we would require an open set D(x,θ) b such that b ≤ 0 for all (y, ψ) b ∈ D b . Given the requirements of an aBF, this ḣa (y, ψ) (x,θ) b = 0 for all (y, ψ) b ∈ D b . We can avoid inducing would require that ḣa (y, ψ) (x,θ) these unwanted stable equilibrium points throughout the set C by requiring the Lipschitz-like continuous control property noted above. Counterexample I now analyze the aCBF condition to verify that, in fact, it is not overly b ≥ 0 to conservative. In particular, changing the aCBF condition ḣa (x, θ) b ≥ −α(ha (x)) does not necessarily lead to adaptive safety. Consider ḣa (x, θ) the simple open-loop system given by: ẋ = θ⋆ + u, (3.248) with θ⋆ ∈ R unknown and the function ha : R → R defined as: ha (x) = 1 − x2 , (3.249) yielding the set C = {x ∈ R | x2 ≤ 1}. It is easy to see that ha is an aCBF for the open-loop system (3.248) on C. Let δ ∈ (0, 1), c ∈ R≥0 , and consider a composite barrier function given by: 1 h(x, θ̂) = ha (x) − γ −1 θ̃2 , 2 (3.250) with γ ∈ R>0 satisfying: c2 . 2δ Define the following sets, which are seen in Figure 3.17: γ≥ U = {(x, θ̃) ∈ R2 | x ∈ C}, b ≥ 0}. H0 = {(x, θ̃) ∈ R2 | h(x, θ) (3.251) (3.252) (3.253) 157 θ̃ U √ ∂U 2γ H0 x C 1 Figure 3.17. Schematic of aCBF counterexample. Note that the set U extends infinitely along the θ̃-axis, and completely contains H0 . Furthermore, H0 ∩ ∂U = {(−1, 0), (1, 0)}. The time derivative of the composite barrier function h is given by: b u) = −2x(θ̂ + u) + θ̃(−2x + τ (x, θ)), b ḣ(x, θ, (3.254) ˙ b Choose the update law τ (x, θ) b = 2x and controller k(x, θ) b = for θb = γτ (x, θ). −θb + 12 xα(ha (x)), with continuously differentiable extended class-K function α. Note that both τ and k are locally Lipschitz continuous on E × Rp . These choices yield: ḣ(x, θ̂) = −x2 α(ha (x)) ≥ −α(ha (x)), (3.255) as when α(ha (x)) ≥ 0, x2 ≤ 1, and when α(ha (x)) ≤ 0, x2 ≥ 1. Noting the b ≤ construction of U , we have the implication that (x, θ̃) ∈ U =⇒ ḣ(x, θ) 0. Instead of considering the composite dynamics in (3.212), we will instead consider an equivalent composite closed-loop system built using the parameter error as a state: " # " # " # ẋ θ̃ + 21 xα(ha (x)) θ̃ − F (x) = = , (3.256) −2γx −g(x) θ̃˙ which has an unstable equilibrium point at the origin. This system is an example of a Liénard system (like the Van der Pol oscillator) as in [101, Section 158 3.8], with F (x) = − 12 xα(ha (x)) and g(x) = 2γx. For systems of this form, the following theorem [101, Section 3.8, Theorem 1], attributed to Liénard, provides the existence of a unique, stable limit cycle: Theorem 28. Under the assumption that F, g are continuously differentiable on R, F and g are odd functions of x, xg(x) > 0 for x ̸= 0, F (0) = 0, F ′ (0) < 0, F has a single positive zero at x = a, and F increases monotonically to infinity for x ≥ a as x → ∞, it follows that the Liénard system (3.256) has exactly one limit cycle and it is stable. As α is continuously differentiable in addition to being extended K∞ , the assumptions of this theorem are met by the functions given in (3.256). Note that a = 1 in this given example. Thus we can conclude that the system (3.256) has a stable periodic orbit, which is denoted Φ ⊂ R2 . Denote the open set in R2 enclosed by the limit cycle as int(Φ). Additionally, the proof of this theorem as in [101, Section 3.8] implies the following corollary: Corollary 1. The stable limit cycle Φ is symmetric about the origin and passes through a point, denoted as P2 = (x2 , θ̃2 ), such that x2 > a. Given that a = 1, this corollary reveals that the stable limit cycle leaves the set U , for which the state is considered safe. Additionally, as the limit cycle is symmetric about the origin, and the origin is an unstable equilibrium, the origin is contained in int(Φ). This corollary also implies that H0 ⊂ (Φ ∪ int(Φ)). To see this, note that as the limit cycle encircles the origin, it must reenter the set U after leaving the point P2 . At any point v = (v1 , v2 ) ∈ U that the limit cycle enters, we must have h(v1 , v2 ) ≤ 0, given the two points in H0 ∩ ∂U . Once the limit b ≤ 0 until the limit cycle leaves U as previously cycle enters U , we have ḣ(x, θ) b ≤ 0 along the portion of the limit cycle contained in noted. Thus, h(x, θ) U , implying H0 ⊂ (Φ ∪ int(Φ)). To complete the proof, we will employ the following definition [100, p. 127] and lemma [100, Lemma 4.1]: Definition 31. The positive limit set L+ of a solution (φx , φθ̃ ) is defined as all points p ∈ R2 such that there is a sequence {tn } with tn → ∞ as n → ∞, and (φx (tn ), φθ̃ (tn )) → p as n → ∞. 159 4 2 0 -2 -4 -1.5 -1 -0.5 0 0.5 1 1.5 Figure 3.18. Simulation trajectory of aCBF counterexample. The system state leaves the prescribed safe set as it converges to the limit cycle. Lemma 2. If a solution (φx , φθ̃ ) of (3.256) is bounded on R≥0 , then its positive limit set L+ is a nonempty, compact, invariant set, and (φx , φθ̃ ) approaches L+ as t → ∞. Note that the unstable equilibrium point is not contained within the positive limit set L+ of a solution (φx , φθ̃ ) starting from any initial condition (x0 , θb0 ) ∈ H0 \ {02 }. As the 0-superlevel set of h, and thus all possible initial conditions given our bound on θ̃0 , are contained inside the limit cycle, all solutions to (3.256) are bounded (by the limit cycle). Furthermore, for the solution (φx , φθ̃ ) starting from any initial condition (x0 , θb0 ) ∈ H0 \ {02 }, we have that L+ = Φ, and thus all solutions starting in the 0-superlevel of h except for the origin approach Φ. As the point P2 ∈ Φ, and P2 ∈ / U , we see that any solution starting in the 0-superlevel set of h leaves the desired state safe set C. Hence, the relaxation does not achieve safety of the state, as seen in Figure 3.18. Simulation Results To demonstrate how an aCBF can be used to render a system adaptively safe, I consider the problem of adaptive cruise control (ACC) as posed in [11]. The 160 dynamics of the system9 are given by:   # f0 " # " # " " # 1 0 d v 1 1 v v2   = − f1  + m u, dt D m 0 0 0 0 v0 − v f2 (3.257) with v the velocity of the vehicle, D the distance between the vehicle and a leading vehicle traveling at a fixed velocity v0 , m the vehicle’s mass, and f0 , f1 , and f2 unknown parameters associated with rolling frictional force. Define the state as x = (v, D). In this problem, I seek to drive the velocity to a desired velocity, vd , while simultaneously ensuring the distance between the vehicles satisfies a safety constraint given by: D ≥ 1.8v. (3.258) The parameters f0 , f1 , and f2 are often determined empirically, and if they are not accurate, the desired velocity may not be accurately tracked. Furthermore, if the parameters do not exactly match the true parameters, it may not be possible to certify that the system will satisfy the safety constraint. The control objective of tracking a desired velocity can be achieved with a handdesigned controller knom , or encoded using a CLF, and the safety constraint can be encoded using a CBF. Additional constraints on the maximum acceleration and deceleration can be enforced to maintain passenger comfort. To handle error in the parameters, I utilize the tool of aCBFs to maintain and update estimates of these parameters. An aCBF that yields desirable results is defined as the following continuously differentiable function:  β 2 if D − 1.8v ≥ β, ha (v, D) = (3.259) β 2 − (D − 1.8v − β)2 if D − 1.8v < β, for β > 0. This particular construction of ha is constant away from the safety boundary and diminishes to 0 (quadratically to preserve continuous differentiability) as the boundary is approached. In practice, this is to handle the fact that superlevel sets of the composite safety function h are forward a invariant. In regions where ha is constant, ∂h (x) = 0n , and thus the update ∂x b = 0. law in (3.231), is 0p , thus making ḣ(x, θ) This system will violate the assumption that F(xe ) = 02 for the point xe ∈ R2 that I try to stabilize the system to. In general, this will compromise stability guarantees, which are not well-posed. Still, for this example, I am able to achieve interesting behavior showing the benefit of using adaptive elements. 9 161 100 26 24 50 22 0 20 18 -50 16 -100 14 -150 0 5 10 15 12 20 0 5 10 15 20 60 40 20 0 -20 0 5 10 15 20 Figure 3.19. Comparison of different adaptive and non-adaptive control methodologies. (Top, Left) An aCBF controller is able to enforce safety of the nominal controller knom . (Top, Right) An aCLF-aCBF controller is able to track the desired velocity with zero steady state error (Top, Right). (Bottom) Both aCBF controllers are able to keep the vehicle within the safe region for all time. aCBF-QP Controller: A simple proportional controller knom on tracking error v − vd can be implemented and achieve good tracking performance, but is not necessarily safe. A CBF alone would not ensure the safety of this controller with model parametric model error, but by using the proportional controller knom in a (aCBF-QP) controller, safety can be achieved. aCLF-aCBF-QP Controller: Additionally, we can unify an aCLF and an aCBF in a quadratic program based controller to receive the benefits of optimal and adaptive tracking while remaining safe. Separate estimates of the parameters are maintained for the aCLF and the aCBF, as the form of the update laws in (3.218) and (3.231) may not be simultaneously satisfiable for only one estimate of the parameters. The CLF in [11] on the velocity tracking error v − vd , given by Va = (v − vd )2 , also satisfies the aCLF condition (3.227). b and ψ b be parameter estimates associated with the aCLF and aCBF, Letting θ 162 respectively, I formulate a QP-based controller: b ψ) b = argmin J(u) + cV (x)δV + cp (x)δp k(x, θ, u∈Rm b u) ≤ −k3 (v − vd )2 + δV , s.t. V̇a (x, θ, b u) ≥ 0, ḣa (x, ψ, u ≤ umax + δp , u ≥ −umax − δp , δV , δp ≥ 0, b and ψ b as in (3.224) and (3.231), respectively. with parameter updates for θ The function J is a cost function on the input, and the variables δV and δp are relaxations to the optimization problem to ensure its feasibility, while safety is ensured. The functions cV and cp are Lipschitz continuous and are used to ensure the optimization problem is well-conditioned. i h i h With initial parameter estimates fb0 fb1 fb2 = 10 f0⋆ f1⋆ f2⋆ (less friction than modeled), the results of this controller appear in Figure 3.19. We see that the proportional controller fails to keep the vehicle safe, but filtering it with the aCBF-QP keeps it safe (with D ≥ 1.8v for all time) even with parametric model error. A CLF-CBF controller with no adaptive elements fails to either track the desired velocity (with steady state error) or keep the vehicle safe. The CLF-aCBF controller keeps the vehicle safe but has steady state tracking error, while an aCLF-aCBF controller accurately tracks the desired velocity with no steady state error, and keeps the vehicle safe. Conclusion In this section I have presented a novel definition of adaptive Control Barrier Functions as a tool for safety-critical control synthesis in the presence of parametric model error. I begin by reviewing adaptive CLFs as formulated in [47] as a starting point for developing adaptive CBFs. I then propose a notion of adaptive safety and a definition of adaptive BFs that ensure adaptive safety. Subsequently, I introduce a notion of adaptive CBFs and explain some of the challenges that must be considered when ensuring the controllers producing using adaptive CBFs satisfy regularity properties. I then show that the conservative nature of adaptive CBFs is not unnecessary through a rigorous counter example. I lastly highlight the usefulness of adaptive elements in simulation with an adaptive cruise control system. 163 3.10 Conclusion and Future Work In this chapter I have presented a collection of results for addressing general model error and parametric model error. In Section 3.3 I presented an episodic learning framework for learning general model errors as they impact the time derivative of a CLF and local exponential stability. In Section 3.4 I explored how residual learning error could be understood as a disturbance in a closedloop system, and how robustness to these disturbances can be codified with Projection-to-State Stability. I additionally explore how possible model errors can be restricted through the use of a data set. In Section 3.5 I extend the preceding episodic learning algorithm to the setting of safety by considering how model error can be learned directly as it impacts the time derivative of a CBF, with an experimental demonstration. In Section 3.6 I rigorously codify safety guarantees that can be achieved in the presence of residual learning error through the notion of Projection-to-State Safety, and demonstrate the ability to improve safety guarantees through learning in simulation and experimentally. Building off of these previous results, in Section 3.7 I consider some of the challenges that arise when learning with structured models that preserve convexity of downstream control synthesis. To resolve these challenges, I present an episodic learning framework that uses an unstructured learning model, and demonstrate the ability to achieve safe walking on stepping-stones with a bipedal platform both in simulation and experimentally. This work culminates in Section 3.8 in which I propose a data-driven control framework that ensures robustness to all model errors permitted by a data set. I propose a convex optimization-based controller specified via a second-order cone program, and develop a rigorous theoretical understanding of this controller’s feasibility using an analog of the alternative CLF and CBF condition. Lastly, in Section 3.9 I propose a notion of adaptive CBFs for achieving safety in the presence of parametric model error. I further highlight how the conservative nature of the proposed definition is necessary through a rigorous counterexample. I believe there are two major directions of future work worthwhile pursuing based on the contributions of this chapter. The first direction pertains to the data-driven control framework in Section 3.8. In the context of data-driven control, there are challenges with implementing the proposed controller with large data sets. This is due to the fact that the size of the optimization problem scales with the size of the data set. A similar challenge is seen in nonparametric learning methods, such as GPR, where evaluating a model scales 164 in complexity with the size of the data set [226]. One potential approach for resolving this issue is developing heuristics for selecting relevant data close to a test point while achieving necessary input variance, similar to the notion of k-nearest neighbors with diversity [227]. Another approach would be to build outer-approximations of the model error sets offline that can be quickly utilized in online control synthesis. There is perhaps great potential in this direction because the model error sets are defined by a collection of convex constraints, and there exist efficient ways for approximating such sets [30, Chapter 11]. Another worthwhile direction is extending this framework to the sampleddata setting in Chapter 6, as well as capturing robustness to disturbances and measurement error discussed in Chapters 4 and 5, respectively. The other direction of future work looks at further developing the notion of adaptive CBFs established in Section 3.9, with some work already being considered in this direction [191], [196], [197], [199], [200]. The main objective of much of this work is in reducing the conservative nature of the adaptive CBFs presented in this thesis through the use of data. In particular, these approaches seek to find ways to introduce the extended class-K function back into the aCBF formulation, at the expensive of potentially more conservativeness until adequate data can be collected. Building upon this work with methods for collecting relevant data necessary for adaptation is one future direction. Another direction of work is to address the challenge of parametric model error in the function g, which has not been considered until only very recently in [200]. This opens up the classes of systems that adaptive systems can be deployed on, such as robotic systems that have error in their mass and inertia parameters. 165 Chapter 4 DISTURBANCE-ROBUST SAFETY-CRITICAL CONTROL In this chapter I will present results from my work on developing methods for safety-critical control that are both performant and robust to disturbances. Disturbances, either unmatched or matched, can serve as a sort of catch-all for various types of challenges in nonlinear control design. In fact, model errors, measurement errors, and sampling effects, as presented in Chapters 3, 5, and 6, respectively, can all be cast as disturbances in a nonlinear system. Often times though, this perspective can obscure important structural properties of these different types of challenges that can be used to synthesize controllers that are both robust and performant. Instead, disturbances are best for capturing challenges for which there is no underlying structure that can easily be understood. For robotic systems, this may include things such as perturbing external forces created by the environment (objects, people, or other robots bumping into the robot), or complex actuator dynamics that are abstracted away when assuming that the inputs to the system are torques (for systems with electric motors, the inputs are typically voltage or current commands which are passed to a motor controller that attempts to produce a certain motor torque). In the former case, there may be no reasonable model that could be constructed to capture the myriad of potential unexpected interactions, even from data, while in the latter case, constructing a first-principles model may require significant work, and collecting data to build a data-driven model may require extensive additional sensing capabilities. Moreover, if a robot is subjected to multiple challenges, such as both external forces and actuator dynamics (which they likely will be), the modeling challenges becomes increasingly difficult. Instead, one may simply capture the collective effects as a disturbance that the control design must be robust to. Thus, disturbance robustness is a general form of robustness to the many different challenges that can be captured as disturbances (even if they can be captured using more structure, as in the other chapters of this thesis). Instead of trying to ascribe structure to the system’s environment or produce models for complex underlying structures, designs that consider disturbances generally require the control designer to produce a bound on the potential 166 disturbance signal. This in and of itself can be a difficult problem, and producing a weak bound can have significant consequences for the performance of a system. Intuitively, for larger potential disturbances, a larger degree of robustness is required. If the system never actually experiences such a large disturbance, then the system is potentially conceding a significant amount of performance for robustness to something it never encounters. Moreover, if it were to encounter this disturbance, robustness in the controller would be most important near the boundary of a set that is to be kept forward invariant, rather than deep inside the set, as the disturbance must take time to make the system unsafe if it is deep inside the safe set, and feedback control can be used to respond during this time. My work in this chapter focuses on balancing robustness to disturbances with performance in safety-critical control through a modification to Input-to-State Safety, as well as a characterization of safety guarantees in the presence of stochastic disturbance signals. In Section 4.1 I discuss related work in the setting of ISSf, disturbance robust control, and stochastic safety-critical control. In Section 4.2 I will discuss some of the limitations of ISSf and ISSf-CBFs as presented in Section 2.7 and 2.9. In Section 4.3 I will resolve some of these limitations through the proposal of a novel modification of ISSf known as Tunable Input-to-State Safety (TISSf ) that allows the prioritization of robustness to disturbances near the boundary of a set that is to be kept forward invariant, rather than over the entire set. I will then highlight the ability for TISSf to balance performance with robustness through an inverted pendulum example. In Section 4.4 I demonstrate the efficacy of TISSf with an application on an autonomous semi-trailer truck. I first present a model, safety specification captured via a CBF, and nominal controller design for this system. I show that the simplified model for this system neglects important dynamics in the actuation of an experimental system, leading to violations of safety when using the CBF-QP controller presented in Section 2.5. Using the tool of TISSf, I design a robust yet performant controller that successfully keeps the experimental autonomous semi-trailer truck system safe. Lastly, in Section 4.5, I will consider CBFs for discrete-time systems subjected to stochastic disturbances. I develop a notion of probabilistic safety related to the idea of ISSf, and show how CBFs satisfying convexity properties can be used to meet the necessary requirements to produce probabilistic safety guarantees. Key contributions of this work are described at the beginning of each respective section. 167 4.1 Related Work Disturbance Robust Safety-Critical Control The inherent robustness properties of CBFs were first observed in [221]. In particular, this work observed that if the CBF property was satisfied on the set E, including outside of the set C, then asymptotic stability of the set C could be established under a CBF-based controller. This asymptotic stability property was then used to establish an ISS result with respect to the set C for a system experiencing unmatched disturbances [221, Proposition 5]. This idea was codified through the concept of Input-to-State Safety (ISSf) in [58] for systems with matched disturbances. This work introduced the notion of ISS-BFs and ISSf-CBFs as tool for certifying and constructively achieving ISSf. Importantly, this work established a paradigm for safety-critical control design that focuses on controlling the enlargement of the set C that is kept forward invariant, rather than trying to explicitly enforce the forward invariance of the set C itself. It is this paradigm that will be utilized in my work. My contributions in this chapter focus on improving the balance of performance and robustness to disturbances that is achieved using ISSf-CBFs. In particular, my work in [228], presented in Sections 4.2 and 4.3, modifies the definitions of ISSf and ISSf-CBFs from those in [58] to permit controllers that achieve rigorous safety guarantees without greatly degrading performance. This is accomplished by relaxing the robustness terms in an ISSf-CBF deep in the interior of the set C, and only requiring robustness to be enforced near the boundary of C. My work in [21] demonstrates the effectiveness of this approach on the challenging realworld problem of keeping an automated semi-trailer truck at a safe following distance behind a lead vehicle, including an experimental demonstration. In parallel to the development of ISSf, other notions of robust CBFs that ensure the forward invariance of the set C in the presence of disturbances were formulated in [50]–[55], [229]–[231]. These approaches require knowledge of a bound on the disturbance that is utilized directly in the controller. Consequently, if the bound is weak, the system is more robust than necessary and displays conservative behavior, as was seen in the results of [50]. The work in [52] models dynamics of disturbances through the use of differential inclusions in which the disturbance is seen as taking values in some set (rather than just satisfying a bound on its magnitude). The work in [55] considers sector-bounded uncertainties affecting the input of the system (such as saturation) as a matched disturbance signal that takes a worst-case value based on 168 the input to the system, while the work in [232] considers unmodeled actuator dynamics leading to a matched disturbance. An interesting direction that could be paired with much of this work is the synthesis of CBFs that can be rendered robust to disturbances through control by utilizing a modified form of Hamilton-Jacobi reachability [229]. Other work that could be paired with many of these approaches considers using a disturbance observer to reduce the worst-case disturbance the system needs to be robust to [230], [231]. Stochastic Safety-Critical Control Instead of considering disturbances simply as piecewise continuous signals, it is often interesting to ascribe more structure through the form of a statement on the probability distribution of the disturbance value, leading to a stochastic disturbance signal. This perspective allows one to weight the frequency that a certain disturbance value occurs more highly than others. This contrasts much of the work studying disturbances through ISSf and other robustness approaches mentioned above, which typically assumes that the disturbance can (and will) always take the worst-case value when establishing safety guarantees. Because it is still possible for a stochastic disturbance signal to take these worst-case values (perhaps with low-probability), it is important that the resulting safety guarantees be designed to account for not only this situation, but make use of the fact that a stochastic disturbance is frequently not taking worst-case values. Results using CBFs for safety-critical control in the setting of stochastic disturbances can generally be divided along two axes, the first being the use of continuous versus discrete-time dynamics, and the second being chanceconstraints versus finite-time safety guarantees. Results in continuous-time consider stochastic differential equations, with the standard closed-loop system (2.2) perturbed by a Brownian motion [233]. This assumption typically implies that the stochastic disturbances experienced by the system at two different points in time are correlated based on how far those points in time are spaced apart, imparting significant structure that can be utilized in proving safety guarantees. Early work on this challenge developed results using Lyapunov functions [59], but work building off of this in the context of barrier functions focused on autonomous continuous-time systems (without control inputs) [60], and subsequently, synthesizing stochastic barrier certificates through Sums-of-Squares (SOS) programming [61]. The advent of CBFs in 169 [62] led to the rapid development of CBFs designed to enable control synthesis for stochastic differential equations [43], [63]–[71]. Results in discrete-time also stem from the early work [59] which additionally studied safety guarantees using Lyapunov functions for discrete-time systems with stochastic disturbances. Typically, stochastic disturbances in this setting are not assumed to be correlated across discrete-time instances in contrast to continuous-time. The work in [61] also considered verifying safety for discrete-time systems with stochastic disturbances using SOS with the tools in [59]. Developed shortly after continuous-time CBFs, discrete-time CBFs [234] provided an analog for studying safety-critical control synthesis for discrete-time systems with stochastic disturbances [69], [72]. My work in [235], discussed in Section 4.5, also takes a discrete-time CBF-based approach for studying safety-critical control synthesis, and utilizes the framework in [59] to establish rigorous safety guarantees. One direction that has remained fairly unexplored is the mixture of continuous and discrete-time stochastic systems. In particular, control systems are often based on discrete-time measurements with a sample-and-hold control implementation, as I will discuss in Chapter 6. The discrete-time measurements are often susceptible to noise, suggesting a discrete-time approach for modeling stochastic disturbances. Underlying the sample-and-hold inputs to the system is the continuous-time evolution of the system dynamics (with the input held constant). The work I present in Section 4.5 is a first step in this direction, with future work beyond this thesis seeking to unify it with the developments in Chapter 6. The other major axis in stochastic safety-critical control pertains to chanceconstraints versus finite-time safety guarantees. Chance-constraints typically require that a CBF-based inequality constraint be satisfied at any given time instance with high probability as a means for prescribing safety [43], [70], [72]. One challenge with this approach is that one instantaneous failure of the CBF constraint in closed-loop does not necessarily imply the system is unsafe, but rather that the change in the value of the CBF is larger than desired. Instead, the failure of safety typically arises from the compounding of several successive failures to satisfy the CBF inequality. Understanding this compounding behavior has been the focus of stochastic safety-critical control seeking finite-time safety guarantees. Much of this work focuses on bounding the probability that the state of the system leaves the set 170 C over some finite time horizon. I note that this differs from the typical notion of safety, which considers whether or not the state remains in the set C over the entire interval on which a solution exists (potentially for all time). Considering a finite time horizon allows for considering potentially unbounded disturbances (such as a discrete-time disturbance signal with a Gaussian distribution), for which considering an infinite horizon leads to a system being unsafe with probability 1 [68, Footnote 7]. This pursuit of finite-time safety guarantees using Lyapunov functions can be seen in the early work [59], which established safety guarantees using Martingale theory and has served as a source of motivation for many subsequent works. Examples of such work include [60], [61], which were the first to use barrier functions to certify finite-time safety properties in the presence of stochastic disturbances. Subsequently, the introduction of CBFs in [62] led to the rapid development of stochastic variants for achieving finite-time safety guarantees by synthesizing CBFs and polynomial controllers for continuous and discrete-time systems through SOS [63], [69], considering online optimization-based control for continuous-time systems [67], considering continuous-time hybrid systems [65], and proposing definitions for continuoustime stochastic CBFs that achieve less conservative guarantees [71]. My work in [235], discussed in Section 4.5, also develops off of the work in [59] in conjunction with the work in [69] to develop online optimization-based CBF-based controllers for discrete-time systems that provide finite-time safety guarantees. The focus of this work is on the properties that need to be satisfied by a CBF and the dynamic system to permit robust control synthesis through convex optimization. Additionally, this work takes an approach akin to ISSf by providing probabilistic safety guarantees based on enlargements of the set C, rather than only C. An alternative perspective that forgoes Martingale-based analysis but focuses on finite-time safety guarantees is proposed in [68]. This approach focuses on the continuous-time evolution of the entire distribution of the value of a CBF, which can be captured by the solution to a convectiondiffusion partial differential equation. Alternatively, the tools developed in this work were considered again in [64] to propose a method for achieving long-horizon safety guarantees using the forward invariance of level sets of the safety probability. Lastly, the work in [66] considers reciprocal CBFs (as defined in [11]) and continuous-time dynamics, allowing for strong claims on infinite horizon safety with probability 1. 171 4.2 Limitations of Input-to-State Safety Note that the contents of this section were primarily written by collaborators, edited by Andrew Taylor for their appearance in journal proceedings, and edited by Andrew Taylor for their appearance in this thesis. In this section we will explore some of the limitations of ISSf and ISSf-CBFs as proposed in [58] and presented in Section 2.9. The text for this section is adapted from: A. Alan, A. J. Taylor, C. R. He, G. Orosz, and A. D. Ames, “Safe controller synthesis with tunable input-to-state safe control barrier functions", IEEE Control Sys. Let., vol. 6, pp. 908-913, 2022. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, and writing of the article. This will be done through a simple illustrative example. Consider a open-loop system: " # " # " # −x2 0 d x1 = + u, dt x2 0 1 (4.1) h i⊤ with state x = x1 x2 ∈ R2 and input u ∈ R. A controller k : R2 → R that is locally Lipschitz continuous on R2 can be specified as follows: k(x) = x1 − 2x2 − 1, (4.2) for which the resulting closed-loop system (2.2) has a locally exponentially h i⊤ stable equilibrium point xe = 1 0 . Next consider a function h : R2 → R that is continuously differentiable on R2 that is defined as: h(x) = x1 − x2 , (4.3)  C = x ∈ R2 | x1 − x2 ≥ 0 . (4.4) yielding the set C defined as: The time derivative of h is given by: ḣ(x) = −x1 + x2 +1 > −h(x). | {z } −h(x) (4.5) 172 Thus for α ∈ Ke defined as α(r) = r, we have that: ḣ(x) ≥ −α(h(x)), (4.6) and thus h is a BF for the closed-loop system (2.2) on C. Simulation results for the closed-loop system can be seen in Figure 4.1(a), where all of the trajectories initiated from different initial conditions x0 ∈ C approach the locally exponentially stable equilibrium point xe while remaining inside C. Now introduce a matched disturbance to the system (4.1), yielding: # " # " # " # " −x2 0 0 d x1 = + u+ d(t), dt x2 0 1 1 (4.7) where d : R≥0 → R is piecewise continuous on R≥0 . Using a harmonic disturbance signal d(t) = δ sin t with δ = 3, we see the evolution of the closed-loop system with matched disturbances (2.74) from the previous initial conditions x0 ∈ C in Figure 4.1(b). Observe that the disturbance makes the state trajectories leave the set C periodically. Note that Lg h(x) = −1 for all x ∈ R2 . Inspired by the definition of ISSf-CBFs in Section 2.9, consider a controller k ′ : R2 → R that is locally Lipschitz continuous on R2 that is defined as: k ′ (x) = k(x) + Lg h(x) , ϵ0 (4.8) for ϵ0 ∈ R>0 , noting that for the corresponding closed-loop system with matched disturbances (2.74), we have that: ḣ(x, t) ≥ −α(h(x)) + |Lg h(x)|2 − Lg h(x)d(t), ϵ0 (4.9) for all x ∈ R2 and t ∈ R≥0 . Completing the square, we have that: ḣ(x, t) ≥ −α(h(x)) − ϵ0 |d(t)|2 , 4 (4.10) for all x ∈ R2 and t ∈ R≥0 . Consequently, the closed-loop system with matched disturbances (2.74) is ISSf with respect to the set C with γ ∈ K defined as γ(r) = −α−1 (− ϵ40 r2 ). Observe that we can make the enlargement of the set C that is kept forward invariant smaller with smaller values of ϵ0 at the expense of the controller k ′ specifying larger inputs. Using the harmonic disturbance with δ = 3, the set Cδ (the set Cd defined in (2.79) with d = δ), we can see the evolution of the closed-loop system with matched disturbances (2.74) using the 173 Figure 4.1. Simple example demonstrating limits of ISSf in achieving performance and meaningful theoretical safety guarantees. (a) Safe controller with no disturbances. (b) Impact of harmonic disturbance signal without modifications to controller design. (c) ISSf based controller designs for different values of the parameter ϵ0 . Large values of ϵ0 lead to weak theoretical guarantees, while small values lead to conservative behavior. controller k ′ with different values of ϵ0 in Figure 4.1(c). Observe that for both values of ϵ0 , the system remains in the corresponding set Cδ as required by ISSf. For the value of ϵ0 = 1, the boundary of the set Cδ is relatively far away from the boundary of the set C compared to how close the solutions remain to the set C. In this case, a small violation of the original safety requirement is occurring, but the ISSf safety guarantee is fairly weak and does not accurately describe the closed-loop behavior. The weakness of this guarantee can make it useless as a guiding principle when designing robust controllers. In contrast, for ϵ0 = 0.1, the boundary of Cδ is very close to the boundary of C, providing a rigorous theoretical statement on small violations of safety. Unfortunately, with this strong guarantee comes undesirable closed-loop behavior, as the system evolves deep into the safe set, potentially degrading performance if the system is to remain near the equilibrium point xe . In fact, this introduces an even wider gap between the theoretical guarantee on the closed-loop behavior and the actual 174 realized closed-loop behavior, suggesting that once again, this is a vacuous guiding principle in producing a robust control design. The developments in the next sections in this chapter will seek to balance this tradeoff between strong theoretical guarantees and performant closed-loop behavior by replacing the parameter ϵ0 with a function that prioritizes safety only at the boundary and outside of the set C. 4.3 Tunable Input-to-State Safety In this section I will present work on balancing performance with robustness to disturbances through a modification of ISSf in the form of Tunable Inputto-State Safety (TISSf ). I will first present the idea of TISSf, and establish what theoretical safety guarantees are achieved using it. Next I will consider its application on the simple example presented in Section 4.2. Following this, I will present a detailed inverted pendulum example that shows how the parameters of a TISSf-based controller can be used to balance performance and theoretical safety guarantees. The contributions of this section are as follows: • The definition of Tunable Input-to-State Safety (TISSf), which modifies the original definition of ISSf in [58] to permit performant behavior with meaningful theoretical safety guarantees. • A detailed application of TISSf-based control design to an inverted pendulum with a focus on how the parameters of a TISSf controller determine performance and theoretical safety guarantees. The text for this section is adapted from: A. Alan, A. J. Taylor, C. R. He, G. Orosz, and A. D. Ames, “Safe controller synthesis with tunable input-to-state safe control barrier functions", IEEE Control Sys. Let., vol. 6, pp. 908-913, 2022. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, and writing of the article. A. Alan, A. J. Taylor, C. R. He, A. D. Ames, and G. Orosz, “Control barrier functions and input-to-state safety with application to automated 175 vehicles", conditionally accepted as a Full Paper in IEEE Trans. on Control Sys. Tech., 2022. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. Tunable Input-to-State Safety I will begin by defining the notion of Tunable Input-to-State Safety, which will lead to corresponding variants of ISSf-BFs and ISSf-CBFs. Definition 32 (Tunable Input-to-State Safety (TISSf )). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. The closed-loop system with unmatched disturbances (2.70) or closed-loop system with matched disturbances (2.74) is said to be Tunable Input-to-State Safe (TISSf ) with respect to the set C if there exists a function γT : R×R≥0 → e R≥0 with γT (a, ·) ∈ K∞ for all a ∈ R≥0 and γT (·, b) continuously differentiable on R for all b ∈ R≥0 such that defining the function hT : E × R≥0 → R as: hT (x, d) = h(x) + γT (h(x), d), (4.11) for all d ∈ R≥0 the set Cd,T ⊂ E defined as: Cd,T = {x ∈ E | hT (x, d) ≥ 0} , (4.12) is forward invariant up to d for the closed-loop system (2.70) or (2.74). If the system (2.70) or (2.74) is TISSf with respect to the set C, the set C is referred to as a Tunable Input-to-State Safe set (TISSf set). TISSf simply modifies the original definition of ISSf presented in Section 2.7 by allowing the function γT to not only depend on the disturbance signal, but to also depend on the value of the function h itself. While the utility of this modification is not immediately clear, it will be more easily understood when considering control synthesis. Another small modification is dropping the maximum disturbance magnitude d, and just requiring the property to hold for disturbances of arbitrarily large magnitudes. This is a simplifying assumption, and is not necessary if one keeps track of the appropriate domains for various functions throughout subsequent proofs. The objective of control synthesis will be producing a type of ISSf-BF, as captured in the following: 176 Definition 33 (Tunable Input-to-State Safe Barrier Function (TISSf-BF)). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously ∂h differentiable on E with ∂x (x) ̸= 0n when h(x) = 0. The function h is said to be a Tunable Input-to-State Safe barrier function (TISSf-BF) for the closedloop system with unmatched disturbances (2.70) on C if there exist functions e e α ∈ K∞ with α−1 ∈ K∞ continuously differentiable on R and ϵ : R → R>0 that is continuously differentiable on R such that: ∂h 1 (x) Lf h(x) + Lg h(x)k(x) − ϵ(h(x)) ∂x 2 ≥ −α(h(x)), (4.13) for all x ∈ E, and: ∂ϵ (r) ≥ 0, (4.14) ∂r for all r ∈ R. The function h is said to be a TISSf-BF for the closed-loop system with matched disturbances (2.74) on C if (4.13) is replaced with: Lf h(x) + Lg h(x)k(x) − 1 ∥Lg h(x)∥2 ≥ −α(h(x)), ϵ(h(x)) (4.15) for all x ∈ E. I note this definition looks closer to the definition of ISSf-CBFs than ISSfBFs in Section 2.9. This is done as it will allow explicitly highlighting the importance of the function ϵ, which has replaced the constant parameter ϵ0 that appears in the definition of ISSf-CBFs and was shown in Section 4.2 to heavily dictate performance and theoretical safety guarantees in the presence of disturbances. Note that the function ϵ must always take positive values, thereby requiring that the controller k always meet a stricter requirement than the standard CBF-QP condition (2.60). It also must increase (or not decrease) with its argument. This suggests that when h(x) is larger, or the system state is deep inside the set C, the modification to the standard CBF inequality (2.60) is reduced, as ϵ(h(x)) is larger. As h(x) gets smaller, such as when the state of the system approaches the boundary of C or moves further away from C, the value of ϵ must get smaller, and the importance of the robustifying term increases. This reflects the intuitive idea that robustness is more important near the boundary of C, and less important when the system is sufficiently far inside C. Examples of such functions can be seen in Figure 4.2 TISSf-BFs serve as a certificate of TISSf of a closed-loop system with respect to a set C as follows: 177 3 2 1 0 -1 -0.5 0 0.5 1 Figure 4.2. Examples of the function ϵ that satisfy the condition in (4.14). Theorem 29. Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. If h is a TISSf-BF for the closed-loop system with unmatched disturbances (2.70) (matched disturbances (2.74)) on C, then the closed-loop system with unmatched disturbances (2.70) (matched disturbances (2.74)) is TISSf with respect to the set C with a function γT : R × R≥0 → R≥0 defined as:   ϵ(r)d2 −1 . (4.16) γT (r, d) ≜ −α − 4 I will prove this theorem for the matched disturbance setting here, but a similar proof for the unmatched disturbance case is provided in Appendix A. Proof. Observe that by assumption α−1 and ϵ are continuously differentiable on R, such that the function γT (·, b) is continuously differentiable for any e b ∈ R≥0 . Similarly, as ϵ(r) > 0 for all r ∈ R and α−1 ∈ K∞ , the function e γT (a, ·) ∈ K∞ for any a ∈ R as required. Our next goal is to verify that for all d ∈ R≥0 , the set Cd,T defined in (4.12) is forward invariant up to d. Consider a value of d ∈ R≥0 and consider a disturbance signal d : R≥0 → Rm that is piecewise continuous on R≥0 and satisfies ∥d∥∞ ≤ d. As h is a TISSf-BF for 178 the closed-loop system with matched disturbances (2.74), we have that: ḣ(x, t) ≜ Lf h(x) + Lg h(x)k(x) + Lg h(x)d(t), 1 ≥ −α(h(x)) + ∥Lg h(x)∥2 + Lg h(x)d(t). ϵ(h(x)) (4.17) (4.18) Noting that: Lg h(x)d(t) ≥ − ∥Lg h(x)∥ ∥d∥∞ ≥ − ∥Lg h(x)∥ d, (4.19) for all x ∈ E and t ∈ R≥0 and ϵ(h(x)) > 0 for all x ∈ E, adding and 2 subtracting ϵ(h(x))d , and completing the squares yields: 4 ḣ(x, t) ≥ −α(h(x)) − ϵ(h(x))d2 . 4 (4.20) Next, observe that: ∂γT d2 ∂α−1 (r, d) = ∂r 4 ∂r  ϵ(r)d2 − 4  ∂ϵ (r) ≥ 0, ∂r (4.21) for all r ∈ R. Taking the time derivative of the function hT defined in (4.11) yields:   ∂γT (4.22) (h(x), d) ḣ(x, t). ḣT (x, d, t) = 1 + ∂r noting that d is a fixed constant in this proof. By (4.21), we have that: 1+ ∂γT (h(x), d) > 0, ∂r for all x ∈ E. Substituting (4.20) into (4.22), we obtain:    ∂γT ϵ(h(x))d2 ḣT (x, d, t) ≥ 1 + (h(x), d) −α(h(x)) − . ∂r 4 (4.23) (4.24) Next, consider a state x ∈ ∂Cd,T , such that hT (x, d) = 0, for which (4.11) and (4.16) imply: ϵ(h(x))d2 −α(h(x)) − = 0, (4.25) 4 yielding: ḣT (x, d, t) ≥ 0, (4.26) for all t ∈ R≥0 . In the case that d = 0, such that hT (x, 0) = h(x), we have that ∂h e hT (x, 0) = 0 implies ∂x (x) ̸= 0n . Considering d > 0, because γ(a, ·) ∈ K∞ for 179 all a ∈ R, when hT (x, d) = 0, we have −α(h(x)) > 0. Thus, the inequality in ∂h (4.15) requires that ∂x (x) ̸= 0n for x ∈ ∂Cd,T . Finally, we have:   ∂γT ∂h ∂hT (x, d) = 1 + (h(x), d) (x) ̸= 0n , (4.27) ∂x ∂r ∂x | {z } >0 using (4.23). Therefore, Nagumo’s theorem [104], [123] implies the set Cd,T is forward invariant as hT (x, d) = 0 implies ḣT (x, d, t) ≥ 0 for all t ∈ R≥0 , and ∂hT (x, d) ̸= 0n . ∂x Let us now consider a TISSf-based controller for the simple example we considered in Section 4.2. For the open-loop system with matched disturbances in (4.7) with the particular harmonic disturbance signal d(t) = δ sin t with δ = 3, I pick the following differentiable function: ϵ(h(x)) ≜ ϵ0 eλh(x) , (4.28) with constants ϵ0 , λ ∈ R>0 . Considering a controller k ′ : R2 → R that is locally Lipschitz continuous on R2 that is defined as: k ′ (x) = k(x) + 1 Lg h(x) = x1 − 2x2 − 1 − , ϵ(h(x)) ϵ0 eλ(x1 −x2 ) (4.29) the function h as defined in (4.3) is a TISSf-BF with α(r) = r for corresponding closed-loop system on C. Thus, the set:   ϵ0 eλ(x1 −x2 ) δ 2 2 ≥0 , (4.30) x 1 − x2 + Cδ,T = x ∈ R 4 is forward invariant up to δ (which the system experiences). It is noted that λ = 0 recovers the ISSf-based controller with ϵ(h(x)) = ϵ0 for all x ∈ R2 , whereas a larger λ pulls ∂Cδ,T closer to ∂C, and decreases the effect of the corresponding term in the controller (4.29) for h(x) > 0. The set Cδ,T is depicted in Figure 4.3(d) with ϵ0 = e−2 , λ = 2 chosen such that ∂Cδ,T corresponds to ∂Cδ using ϵ0 = 0.1. In this sense, it is possible to achieve the same theoretical safety guarantee using both TISSf and ISSf with small values of ϵ0 . Additionally, solution trajectories from the same set of initial conditions are also shown. Observe that all solution trajectories stay within the set Cδ,T as required by TISSf. Moreover, the system is allowed to evolve close to the boundary of the 180 Figure 4.3. Simple example using TISSf to achieve both meaningful theoretical safety guarantees and non-conservative behavior. set Cδ,T and is not forced deep into the set C. Thus, both meaningful theoretical guarantees and non-conservative behavior can be achieved using TISSf. Given a value of d ∈ R≥0 , the boundary of the set Cd,T that is rendered forward invariant is defined as the 0-level set of the TISSf-BF h. The value of h on this level set, denoted as h∗ ≤ 0, can be found by solving the equation:   ϵ(h∗ )d2 ∗ −1 = 0. (4.31) h −α − 4 | {z } γT (h∗ ,d) By definition, γT (h∗ , d) must be strictly positive for d > 0, implying that h∗ < 0 in the presence of disturbances. Moreover, as d increases, h∗ must get more negative, implying that the boundary of Cd,T falls farther from the boundary of C. Control over this degradation in safety can be achieved by modifying the function ϵ to yield different values of h∗ as specified in (4.31). From a practical perspective, h(φd (t)) ≥ h∗ can be seen as a design objective with h∗ chosen to satisfy a minimum tolerable safety requirement in the presence of disturbances. With this mindset, a control designer must appropriately choose ϵ to ensure h∗ meets these minimum tolerable safety requirements. It is straightforward to extend the idea of an TISSf-BF to a TISSf-CBF: Definition 34 (Tunable Input-to-State Safe Control Barrier Function (TISSf-CBF)). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is ∂h continuously differentiable on E with ∂x (x) ̸= 0n when h(x) = 0. The function h is said to be a Tunable Input-to-State Safe Control Barrier Function 181 (TISSf-CBF) for the open-loop system with unmatched disturbances (2.69) on e e C if there exist functions α ∈ K∞ with α−1 ∈ K∞ continuously differentiable on R and ϵ : R → R>0 that is continuously differentiable on R such that: sup Lf h(x) + Lg h(x)u − u∈Rm 1 ∂h (x) ϵ(h(x)) ∂x 2 > −α(h(x)), (4.32) for all x ∈ E, and: ∂ϵ (r) ≥ 0, (4.33) ∂r for all r ∈ R. The function h is said to be a TISSf-CBF for the open-loop system with matched disturbances (2.73) on C if (4.32) is replaced with: sup Lf h(x) + Lg h(x)u − u∈Rm 1 ∥Lg h(x)∥2 > −α(h(x)), ϵ(h(x)) (4.34) for all x ∈ E. As with the notion of ISSf-CBFs presented in Section 2.9, in the setting of matched disturbances, if h is a CBF for the open-loop system (2.1) with a e e that is continuously differthat has an inverse α−1 ∈ K∞ function α ∈ K∞ entiable on R, then for any ϵ : R → R>0 that is continuously differentiable on R and satisfies (4.33), the function h is a TISSf-CBF for the open-loop system with matched disturbances on C. This makes it relatively easy to robustify standard CBF-based controllers to matched disturbances if necessary. As typical, given a nominal controller knom : E → Rm that is locally Lipschitz continuous on E, an optimization-based controller can be specified as follows: kTISSf (x) = argmin ∥u − knom (x)∥2 u∈Rm s.t. Lf h(x) + Lg h(x)u − (TISSf-CBF-QP) 1 ∥Lg h(x)∥2 ≥ −α(h(x)), ϵ(h(x)) with a similar specification possible for the unmatched disturbance setting. The optimization problem defining this controller remains a quadratic program, as there is no modification to how the constraint depends on the decision variable u from the standard CBF-QP controller. As with the CBFQP controller, under assumptions of local Lipschitz continuity of the function ∂h : E → Rn and α this controller can be shown to be locally Lipschitz ∂x continuous on E. 182 Inverted Pendulum Example Consider a control system for an inverted pendulum described by the model: " # " # " # θ̇ 0 d θ = g + 1 u, (4.35) dt θ̇ sin θ 2 l ml | {z } | {z } f (x) g(x) with pendulum angle θ ∈ R and angular velocity θ̇ ∈ R defining the state x = [θ, θ̇]⊤ , and parameters given by the mass m ∈ R>0 , length l ∈ R>0 , and gravitational acceleration constant g ∈ R>0 . In this example I will use the parameter values m = 2 [kg], l = 1 [m] and g = 10 [m/s2 ]. The single input u ∈ R is a torque applied on the pendulum. A set C that we wish to keep safe for the inverted pendulum that restricts the angular position and velocity is given by the 0-superlevel set of a function h : R2 → R defined as: h(θ, θ̇) = 1 − θ2 θ̇2 θθ̇ − − , a2 b 2 ab (4.36) with parameters a, b ∈ R>0 . In this example I will use the parameter values a = 0.25 [rad] and b = 0.5 [rad/s]. The resulting set: ) (" # 2 2 θ θ̇ θ θ̇ θ ≥0 , (4.37) C= ∈ R2 1 − 2 − 2 − a b ab θ̇ is an ellipse as depicted in Figure 4.4 by a gold region. The function h given as in (4.36) is a CBF for the inverted pendulum system (4.35) on C. To see e defined as α(r) = αc r with αc > 0 satisfying this, consider a function α ∈ K∞ αc ≤ b/a. In this example I use the parameter value αc = 0.2 [1/s]. The CBF condition defined in (2.61) yields that: Lg h(θ0 , θ̇0 ) = 0 =⇒ θ̇0 = − b θ0 . 2a (4.38) This equation defines a line as depicted in Figure 4.4 by a solid black line. We have that on this line:   b 3 Lf h(θ0 , θ̇0 ) + α(h(θ0 , θ̇0 )) = αc + 2 − αc θ02 > 0, (4.39) 4a a such that the condition (2.61) is met for our choice of αc . I note that if we consider a set C˜ defined as the 0-superlevel set of a function h̃ : R2 → R given by: θ2 θ̇2 h̃(θ, θ̇) = 1 − 2 − 2 , (4.40) a b 183 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 -0.8 -0.4 -0.2 0 0.2 0.4 Figure 4.4. Simulation results for the inverted pendulum system. The gold ellipse is the set C as defined in (4.37). The black line is the set where Lg h(x) = 0 as defined in (4.38). The dashed blue line is the trajectory of the system evolving under knom as defined in (4.43), which leaves the set C. The green and purple regions indicate where the controller knom meets and fails to meet the CBF condition, respectively. The dashed red line is the trajectory of the system evolving under kCBF which remains inside the set C. which does not include the term θθ̇/ab, then the function h̃ is not a CBF for ˜ To see this, note that: the system (4.35) on C. Lg h̃(θ0 , θ̇0 ) = 0 =⇒ θ̇0 = 0. (4.41) e : In turn, we have that for any α ∈ K∞
Lf h̃(θ0 , θ̇0 ) + α(h̃(θ0 , θ̇0 )) = α 1 − θ02 /a2 . (4.42) The condition (2.61) is not satisfied for |θ0 | ≥ a, including |θ0 | = a which is ˜ Thus, it is important to choose the safe set and design the CBF to in ∂ C). be compatible with the system dynamics, eliminating points where the CBF condition is not met. Consider a controller knom : R2 → R that is locally Lipschitz continuous on R2 that locally exponentially stabilizes the pendulum to an upright position, given by the feedback linearization controller:   g knom (θ, θ̇) = ml2 − sin θ − Kp θ − Kd θ̇ , (4.43) l 184 with controller gains Kp , Kd ∈ R>0 . This controller yields the closed-loop system: " # " #" # 0 1 θ d θ = , (4.44) dt θ̇ −Kp −Kd θ̇ h i⊤ such that the equilibrium point xe = 0, 0 is locally exponentially stable. In this example I will use the parameter values Kp = 0.6 [1/s2 ] and Kd = 0.6 [1/s]. I simulate the system from the initial condition x0 = [−0.1, 0.5]⊤ ∈ C. This trajectory is depicted in Figure 4.4 by a dashed blue curve. Although the controller knom stabilizes the system to the upright position, in doing so it causes the state of the system to leave the set C. I next deploy the (CBF-QP) controller kCBF : R2 → R for the inverted pendulum system using the nominal controller knom defined in (4.43). I simulate the system from the initial condition x0 = [−0.1, 0.5]⊤ ∈ C. This trajectory is depicted in Figure 4.4 by a dashed red curve. Observe that the (CBF-QP) controller kCBF ensures that the system state remains within the set C. To see how this controller is impacted by disturbances, I will consider a matched disturbance signal specified as: d(t) = M (1 − s(t − 5) − s(t − 10) + s(t − 15)), where M ∈ R≥0 and s : R → R is the Heaviside function:  0 if τ < 0, s(τ ) = 1 if τ ≥ 0. (4.45) (4.46) With this disturbance we have ∥d∥∞ = M . In this example I use the parameter value M = 0.75 [N·m] and the corresponding disturbance signal is depicted in Figure 4.5. I deploy the (CBF-QP) controller with the nominal controller knom as in (4.43) to the inverted pendulum system without considering the disturbance d defined in (4.45). We see in Figure 4.6 that this controller fails to keep the system in the set C, and deviates from it significantly. I will next consider a design through TISSf. In particular, I will use the (TISSf-CBF-QP) controller kTISSf with a function ϵ defined as: ϵ(r) = ϵ0 eλr , (4.47) 185 1 0.5 0 -0.5 -1 0 5 10 15 20 Figure 4.5. Disturbance signal for the inverted pendulum system example as defined in (4.45). with parameters ϵ0 ∈ R>0 and λ ∈ R≥0 . Using this controller will render h a TISSf-BF for the corresponding closed-loop system, and thus given the choice of α, Theorem 29 yields the sets Cd,T defined in (4.12) are forward invariant up to d with the function γT defined as: ϵ(h(θ, θ̇))d2 . (4.48) 4αc As the disturbance signal is bounded by M , the forward invariant set we will be interested in studying is determined by considering d = M . Thus, I use d = 0.75 [N·m]. Next, we have that (4.31) reduces to: γT (h(θ, θ̇), d) = ∗ h∗ + ϵ0 eλh d2 = 0. 4αc (4.49) Once ϵ0 and λ are specified, (4.49) can be solved for h∗ to find the value of h that corresponds to the boundary ∂Cd,T . The left panel of Figure 4.7 shows the value of h∗ for the different choices of ϵ0 and λ specified in Table 4.1. The boundary ∂Cd,T corresponding to each set of parameters is shown in the right panel of Figure 4.7. The black and red parameter sets return (approximately) the same value of h∗ , and thus the produce the same boundary ∂Cd,T . In contrast, the green parameter set yields a larger set Cd,T as indicated by the smaller value of h∗ in Table 4.1. 186 0.5 0 -0.5 -0.4 -0.2 0 0.2 0.4 Figure 4.6. (Right) Simulation results for the inverted pendulum system with disturbances. The gold ellipse is the safe set C defined in (4.37). The blue line is the trajectory of the system evolving under kCBF defined in (CBF-QP), which is not robust to disturbances and leaves the safe set. The black, green, and dashed red lines are trajectories of the system evolving under kTISSf defined in (TISSf-CBF-QP) with different values for ϵ0 and λ. Color ϵ0 [ N2 1m2 s ] λ h∗ Black 0.15 0 −0.1055 Red 0.5 12 −0.1026 Green 0.5 0 −0.3516 Table 4.1. Parameters sets used in (4.47) and (4.49) for the inverted pendulum example. I next deploy the (TISSf-CBF-QP) controller kTISSf with the nominal controller knom as in (4.43) in simulation with the exponential function given in (4.47) utilizing the parameter pairs specified in Table 4.1. I first observe in Figure 4.6 that with the black parameter set, which corresponds to a standard ISSf design, the system is very conservative and the trajectory of the system rapidly converges to the equilibrium. Increasing the value of ϵ0 returns the green parameter set, which is also a standard ISSf design. We see in the right panel of Figure 4.7 that a larger value of ϵ0 leads to a larger set Cd,T as specified by the lesser value of h∗ in Table 4.1, while in Figure 4.6 we see that a larger value of ϵ0 allows the system to evolve less conservatively. This highlights that larger values of ϵ0 generally lead to less conservative closed-loop behavior and an expansion of the set possessing theoretical safety guarantees. 187 15 .1 -0 10 0.5 5 -0.1 -0. 1 0 5 -0.2 .15 -0 -0.5 2 -0. -0.3 -0.5 0 0 0.2 0.4 0.6 0.8 1 -0.4 -0.2 0 0.2 0.4 Figure 4.7. (Left) Curves corresponding to the value of h∗ solving (4.49) across the (ϵ0 , λ) parameter space for the inverted pendulum example. (Right) The boundary of the set Cd,T rendered forward invariant for different choices of the parameters ϵ0 and λ for the inverted pendulum example. Note that the Cd,T contains the set C for each parameter choice. Comparing the red and black parameter sets, we can see in the left panel of Figure 4.7 that the red parameters lie on the same h∗ -level-set curve as the black parameters, and thus return the same expanded safe set Cd,T as seen in the right panel of Figure 4.7. In contrast, we see in the right panel of Figure 4.7 that the closed-loop system is much less conservative and approaches the boundary of the safe set. This behavior indicates that ϵ0 and λ can be jointly tuned to return the same theoretical guarantees as a small value of ϵ0 without inducing conservativeness. The red parameter set has the same value of ϵ0 as the green parameter set, allowing us to observe directly the impact of λ. We see in the right panel of Figure 4.7 that the red parameters yield a smaller set Cd,T , implying stronger theoretical safety guarantees, while in Figure 4.6 we see that the closed-loop behavior of the red parameter set is much less conservative. Thus, the introduction of λ not only allows one to improve the theoretical safety guarantees for a fixed value of ϵ0 , but also allows one to reduce the conservativeness of the system. Lastly, we see in Figure 4.6 that for all three parameter sets, the controller keeps the trajectories within C, and thus, within Cd,T , that is, it guarantees h(φd (t)) ≥ h∗ . Conclusion In this section I have presented an approach for robust safety-critical control synthesis in the presence of disturbances. This approach builds off the intuition that robustness to disturbances is most important when near the boundary of the set C to propose a novel modification of ISSf in the form 188 of Tunable Input-to-State Safety (TISSf). The definition of TISSf allows for static parameters in ISSf-CBF-based controllers to be replaced by functions that weight robustness to disturbances according to the value of a CBF h. I then show how this allows for meaningful theoretical safety guarantees while permitting a significant reduction in conservativeness of a closed-loop system on an inverted pendulum. 4.4 Tunable Input-to-State Safety on Automated Truck In this section I will present work deploying TISSf-based controllers on an automated semi-trailer truck. I will begin by describing a simplified model for the system dynamics, followed by codifying a safety specification of maintaining a safe following distance behind a lead vehicle in the form of a CBF. Next, I will design a nominal controller for this system which tries to track a desired velocity while maintaining ride comfort. I show in simulation that this nominal controller does not satisfy safety constraints, while the controller (CBF-QP) does meet safety constraints. In deploying the controller (CBF-QP) on the experimental system as seen in Figure 4.8, I find it fails to keep the system safe due to actuator dynamics that can be captured by a matched disturbance. I then deploy the TISSf-based control framework from Section 4.3 using different parameters, demonstrating the ability to achieve meaningful theoretical safety guarantees with desirable performance in both simulation and experimentally. The contributions of this section are as follows: • A design and evaluation of TISSf-based controllers for a heavy-duty semitrailer truck system, both in simulation and experimentally. The text for this section is adapted from: A. Alan, A. J. Taylor, C. R. He, A. D. Ames, and G. Orosz, “Control barrier functions and input-to-state safety with application to automated vehicles,” conditionally accepted as a Full Paper in IEEE Trans. on Control Sys. Tech., 2022. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. 189 Figure 4.8. Experimental configuration for heavy-duty semi-trailer truck problem. (Top) Controller design without robustifying element yields safety violation. (Bottom) Robust safety-critical controller ensures semi-trailer truck brakes early and aggressively enough to maintain safe distance. System Model, Safety Specification, & Control Design Note that the contents of this subsection were primarily written by collaborators, edited by Andrew Taylor for their appearance in journal proceedings, and edited by Andrew Taylor for their appearance in this thesis. Andrew Taylor’s particular contributions in this subsection are the verification of the CBF from equation (4.57) to (4.58). We will begin by designing a safety-critical longitudinal controller for a connected automated semi-trailer truck. We first introduce the physical system and define a safe set via a CBF. Following this, we will present a nominal performance-based controller, and synthesize a safety-critical controller that modifies this nominal controller in a minimally invasive way while ensuring safety. We note that these constructions do not yet explicitly consider disturbances in the system model. In this work we consider a rear-axle-driven truck without a trailer. Assuming the truck’s tires roll without slipping and the truck travels on a flat road with no headwind, the longitudinal dynamics of the truck are described by the following model: kd v 2 + mgσ T − . (4.50) v̇ = meff R meff Here the state is given by the truck’s longitudinal speed v ∈ R, the input is the torque applied on the rear axle T ∈ R, and the parameters in the model are the mass of the truck m ∈ R>0 , the mass moment of inertia of the rotating elements I ∈ R>0 , the tire radius R ∈ R>0 , the effective mass meff = m + RI2 , the air drag constant kd ∈ R>0 , gravitational acceleration g ∈ R>0 , and rolling resistance coefficient σ ∈ R>0 . Note that the second term in (4.50) is dissipative 190 Figure 4.9. A connected automated truck following a connected vehicle. in nature, and slows down the vehicle when it has a positive velocity. This term may be accounted for in the control design via feedback linearization, or may be ignored as its omission introduces a factor of conservativeness to the controller in terms of safety. The torque input to the system is computed from a desired longitudinal acceleration command u ∈ R via feed-forward maps. This torque input command is provided by a drive-by-wire system to the braking systems that produce the actual torque T ; see Figure 4.9. With these feed-forward maps in mind, we simplify the longitudinal dynamics model to: v̇ = u. (4.51) Now let us consider the scenario when the truck follows a connected vehicle as depicted in Figure 4.9. Using the truck model in (4.51), the dynamics of this connected system are given by: Ḋ = vL − v, v̇ = u, (4.52) v̇L = aL , where vL , aL ∈ R are the speed and acceleration of the lead vehicle, respectively, and D ∈ R denotes the bumper-to-bumper headway distance between the truck and the lead vehicle, yielding the state x = [D, v, vL ]⊤ ∈ R3 . The truck and lead vehicle are outfitted with vehicle-to-everything (V2X) communication systems, permitting the truck to receive motion information from 191 the lead vehicle such as its Global Positioning System (GPS) position which yields the distance D, its speed vL , and its acceleration aL . We assume that the leader’s behavior satisfies: aL ∈ [−aL , aL ], vL ∈ [0, v L ], (4.53) where aL , aL , v L ∈ R>0 reflect a city-driving scenario; see Table 4.2 below. The safety task for the truck is to maintain a safe distance behind the leader. This task motivates a CBF h : R3 → R of the form: h(D, v, vL ) = D − ρ(v, vL ), (4.54) where the headway function ρ : R2 → R describes the minimum safe distance between the vehicles given their current velocities, v and vL . Motivated by [62] and [236], we define the headway function as: ρ(v, vL ) = c0 + c1 v + c2 vL + c3 v 2 + c4 vvL + c5 vL2 , (4.55) with parameters ci ∈ R for i = 0, . . . , 5; see Table 4.2. The value of the function ρ is seen in Figure 4.10. The corresponding set C defined by h is given by:      D     3 (4.56) D ≥ ρ(v, vL ) . C = v∈R     vL To verify that the function h is a CBF for (4.52), observe that: Lg h(D0 , v0 , vL,0 ) = 0 =⇒ c1 + 2c3 v0 + c4 vL,0 = 0, (4.57) which describes a line in (v, vL ) space where the CBF condition (2.61) must be met for all D ∈ R. We consider α(r) = αc r with αc ∈ R>0 , yielding: Lf h(D, v0 , vL,0 ) + α(h(D, v0 , vL,0 )) = vL,0 − v0 − aL (c2 + c4 v0 + 2c5 vL,0 ) + αc (D − ρ(v0 , vL,0 )). (4.58) Since checking the alternative CBF condition (2.61) analytically may be cumbersome using (4.58), we graphically evaluate it over a range of D and vL,0 (and v0 defined implicitly through (4.57)) that are states of interest in this problem, while taking the worst case value of aL making (4.58) as negative 192 40 30 20 10 100 0 40 50 20 0 0 Figure 4.10. (Left) The value of the function ρ defined in (4.55), which defines the minimum safe following distance as a function of the leader’s velocity vL and truck velocity v. (Right) The value of Lf h(x) + α(h(x)) as defined in (4.58) when Lg h(x) = 0 for various distances. As the function is strictly positive over the domain of interest, h is a CBF on C for (4.52). Figure 4.11. (Top) Range policy V defined in (4.60). (Bottom) Speed policy W defined in (4.61). as possible; see Figure 4.10. For αc = 0.1 [1/s], the value of (4.58) is strictly positive, ensuring the CBF condition (2.61) is met. Beyond the task of safety, we wish for the controller to have other desirable properties such as plant stability and string stability in the presence of communication delay [237], or optimal performance regarding energy efficiency and passenger comfort [238]. To accomplish this, we first design a nominal controller that prioritizes performance. In particular, we design a connected cruise controller (CCC) for the truck that utilizes information about the lead vehicle 193 0 -5 -10 0 5 10 Figure 4.12. Example profile for acceleration aL of lead vehicle used in simulation. available through V2X connectivity. We propose the controller structure: knom (D, v, vL ) = A(V (D) − v) + B(W (vL ) − v), (4.59) with parameters A, B ∈ R>0 and functions V : R≥0 → R≥0 and W : R≥0 → R≥0 . The first term in (4.59) specifies a distance-based speed error with:  V (D) = max 0, min{κ(D − Dst ), v} , (4.60) depicted in the top panel of Figure 4.11, producing a desired speed based on the distance D. Here Dst ∈ R>0 is the desired stopping distance, 1/κ > 0 is the desired time headway, and Dgo = v/κ + Dst . The second term in (4.59) specifies the error related to the relative speed with the speed policy: (4.61) W (vL ) = min{vL , v}, depicted in the bottom panel of Figure 4.11, which bounds the speed error if the lead vehicle violates vL ≤ v. aL = 5 [m/s2 ] aL = 10 [m/s2 ] v L = 20 [m/s] δ = 4.5 [m/s2 ] ϵ0 = 0.5 [s3 /m] λ = 0.4 [1/m] c0 = 2 [m] c1 = 1.1 [s] c2 = 0.6 [s] c3 = 0.03 [s2 /m] c4 = −0.03 [s2 /m] c5 = −0.03 [s2 /m] κ = 0.8 [1/s] αc = 0.1 [1/s] Dst = 5 [m] Dgo = 30 [m] A = 0.4 [1/s] B = 0.5 [1/s] Table 4.2. Parameter values used in controller design. 194 20 30 15 20 10 10 5 0 0 -10 0 2 4 6 8 10 0 2 4 6 8 10 Figure 4.13. (Left) Velocity vL of lead vehicle (black) and velocity of the truck using the nominal controller (4.59) (blue) and (CBF-QP) controller (red). (Right) Following distance D and value of CBF h using the nominal controller (4.59) and (CBF-QP) controller. We simulate both the nominal controller knom and (CBF-QP) controller from the initial condition x0 = [27.4, 16, 16]⊤ ∈ C. We use parameter values as specified in Table 4.2. The acceleration aL of the lead vehicle is given by a time profile reflecting a hard braking event, as seen in Figure 4.12. The velocity of the truck converges to zero and a crash does not occur for both controllers, but only the (CBF-QP) controller ensures the truck maintains a safe distance (indicated by hCBF (φ(t)) ≥ 0) as seen in Figure 4.13. Observe that the nominal controller brakes less aggressively than the (CBF-QP) controller, and thus does not react quickly enough to avoid violating the safe following distance requirement. Robust Control Design & Experimental Results We now provide a description of the automated semi-trailer truck experimental configuration and present results using the nominal controller (4.59) and (CBF-QP) controller. Subsequently, we will deploy TISSf-based controllers and demonstrate their advantages experimentally. Note that the contents of the following paragraph were primarily written by collaborators, edited by Andrew Taylor for their appearance in journal proceedings, and edited by Andrew Taylor for their appearance in this thesis. The automated truck used in the experiments is an International ProStar+ Class-8 truck developed by the Navistar [239]; see Figure 4.14(a). Both the automated truck and the lead vehicle are equipped with a V2X Onboard Unit (OBU) developed by Commsignia [240]. These units are equipped with an 195 accelerometer, gyroscope, magnetometer, and GPS unit. Furthermore, these OBUs support peer-to-peer communication such that the automated truck may receive position, velocity, and acceleration data from the lead vehicle through V2X antennas shown in Figure 4.14(a). The automated truck is additionally equipped with a Mobile Real-Time Targeting Machine developed by Speedgoat [241], which interfaces with the V2X OBU and the truck’s Engine Controller Unit (ECU) through a Controller Area Network (CAN) bus. The Speedgoat runs the controller for the system given a stream of values for D, v, vL , and aL coming from the V2X OBUs. It computes a desired acceleration input and converts it to a corresponding torque value through a feed-forward map. A drive-by-wire system controls the engine and the brake torques accordingly. The steering of the truck is done by a human driver in the experiments. In an effort to evaluate the repeatability of the experiments, it is necessary to eliminate variation in the lead vehicle’s behavior, which is being driven by a human. To achieve this, we use a recorded time profile of position, velocity, and acceleration of the lead vehicle while it performs a hard braking event. This profile for aL and vL is seen in Figure 4.12 and the left panel of Figure 4.13, respectively, and was used to produce the simulation results. During the experiments, this data is played back to the truck controller as a perceived lead vehicle, which the truck uses in it control computations. Experiments also include a physical lead vehicle for visualization purposes which travels on the other lane for safety reasons; see Figure 4.14(b)-(e) and notice the “collision” in panel (c). We note that this lead vehicle closely follows the previously recorded time profiles, but does not provide any data to the truck during the actual experiments. Importantly, the quantitative analysis of safety is performed by evaluating the CBF using the truck’s measured state with the recorded time profiles of the perceived lead vehicle. The nominal controller and (CBF-QP) controller are deployed on the automated truck with results as seen in Figure 4.15. Observe that not only does the nominal controller consistently fails to meet the safety requirements imposed by the CBF h, but the (CBF-QP) controller also consistently fails to meet the safety requirements. The top row in Figure 4.8 illustrates an experimental run with the nominal controller. To understand why the (CBF-QP) controller fails, we examine the discrepancy between the commanded acceler- 196 Figure 4.14. (a) Vehicles used in experiments. (b) Image from the dashboard of the truck during an experimental run. (c,d,e) Final configurations of separate experiments. See https://youtu.be/9dJtC1TCBbA for a video. ation and actual acceleration of the automated truck, as seen in the left panel of Figure 4.16. One may observe a delay between the commanded acceleration and the achieved acceleration. This delay in acceleration is due to the fact that the brakes of the truck are a complex nonlinear dynamical system that has been imperfectly abstracted away by the feed-forward maps that allow the 197 30 20 10 0 -10 0 5 10 Figure 4.15. Mean value (lines) and standard deviations (fills) of the distance D and the CBF h when using the nominal controller (4.59) (blue) and the (CBF-QP) controller (red) in the truck experiment. The experiments with these controllers are highly consistent. simplified model in (4.51). Rather than working with the braking dynamic system and improving the feed-forward maps, we describe the discrepancy in commanded and actual acceleration as a disturbance in the simplified model: v̇ = u + d(t), (4.62) where d : R≥0 → R reflects the difference between commanded acceleration and actual acceleration. As the disturbance d is caused by the complicated interactions of the drive-by-wire system and brake dynamics, it may be difficult to use model-based techniques to construct a meaningful bound δ ∈ R≥0 for the worst-case disturbance magnitude. Instead, we estimate the worstcase disturbance empirically by comparing the actual acceleration v̇(t) to the commanded acceleration u(t). Observe in the right panel of Figure 4.16 that the largest difference in the commanded and actual acceleration is around 4 [m/s2 ]. Thus, we study the degradation of safety taking a slightly larger value δ = 4.5 [m/s2 ]. To overcome this disturbance and improve the safe behavior of the truck, we deploy the tool of TISSf-CBFs described in Section 4.3. As h satisfies the CBF condition (2.61), and our disturbance it is matched, it also satisfies the TISSf-CBF condition (4.34), where we take: ϵ(r) = ϵ0 eλr , (4.63) 198 4 5 2 0 0 -2 -4 -6 0 5 10 -5 0 5 10 Figure 4.16. (Left) Discrepancy between acceleration commanded by safety-critical controller (black) and actual acceleration of the automated truck (red). (Right) Disturbance signal in input seen by the truck used to define the model (4.62). with ϵ0 ∈ R>0 and λ ∈ R≥0 . The parameter λ introduces a measure of flexibility by allowing one to require a greater degree of robustness when the truck is close to the leading vehicle, and less robustness when the distance is greater. Given (4.63) and the particular value of δ we consider, the smallest set that is forward invariant up to δ is given by:       D ϵ0 eλh(D,v,vL ) δ 2    3 . (4.64) Cδ =  v  ∈ R h(D, v, vL ) ≥ −   4αc   vL As in the inverted pendulum example in Section 4.3, the set Cδ being forward invariant up to δ implies that h(φd (t)) ≥ h∗ , where h∗ is the value of the TISSfCBF h on the boundary of Cδ , which can be calculated by solving (4.31). The value of h∗ for different choices of ϵ0 and λ can be seen in Table 4.3 below. We simulate the nominal controller, (CBF-QP) controller, and (TISSf-CBF-QP) controller from the initial condition x(0) = [27.4, 16, 16]⊤ ∈ C while disturbing the input using the signal shown in the right panel of Figure 4.16. We use parameter values as specified in Table 4.2. Observe in Figure 4.17 that introducing the disturbance signal into our simulation allows the recreation of the failures of the nominal controller and (CBF-QP) controller that were seen experimentally in Figure 4.15. Furthermore, observe that the (TISSf-CBF-QP) controller maintains the safety of the system even in the presence of the disturbance. 199 30 20 10 0 -10 0 2 4 6 8 10 Figure 4.17. Following distance D and value of h using the nominal controller (4.59) (blue), (CBF-QP) controller (red), and (TISSf-CBF-QP) controller (green) in the disturbed simulation. 30 20 10 0 -10 0 2 4 6 8 10 Figure 4.18. Mean value (lines) and standard deviations (fills) of the following distance D and h using the nominal controller (4.59) (blue), (CBF-QP) controller (red), and (TISSf-CBF-QP) controller (green) in experiment. We next deploy the TISSf-based control design on the experimental platform. Sets of three experimental runs were conducted using each parameter pair ϵ0 and λ shown in Table 4.3. The experimental results using the parameter set ϵ0 = 0.5 [s3 /m] and λ = 0.4 [1/m] (labeled as parameter pair (A)) can be seen in Figure 4.17 and are visualized in the bottom row of Figure 4.8. With these parameters the system is safe, as the value of h does not drop below 200 0.6 30 -4 B A 0.4 -8 C 10 -8 -4 0.2 0 1 -16 -16 -32 B 0 20 -8 2 3 A D -32 4 5 0 D C 101 102 Figure 4.19. (Left) Parameter values for ϵ0 and λ used in the truck experiments, with contours showing theoretical values of h∗ . Green markers denote parameter sets which achieve the original safety goal (h(φd (t)) ≥ 0), while red markers denote parameter sets for which the original safety goal is violated. (Right) Theoretical values of h∗ and the shift in steady-state tracking distance D̃ss , for the parameter sets used in the truck experiments. The blue markers denote parameter sets with λ > 0, while the black markers denote parameter sets with λ = 0. 0. Although the TISSf-based controller has a larger standard deviation across the experimental runs, it consistently satisfies the original safety requirement. To evaluate how the system behavior depends on the values of the parameters ϵ0 and λ, we first consider whether the original safety requirement is met, i.e., whether or not the value of h remains positive. While the TISSf-based controller does not provide a theoretical guarantee that h will remain nonnegative (it guarantees that h(φd (t)) ≥ h∗ ), for certain values of ϵ0 and λ the original safety requirement is still met. The minimum value hmin of the TISSfCBF observed during the experimental runs is shown in Table 4.3. In the left panel of Figure 4.19, green and red markers indicate parameter values for which the original safety requirement is and is not met, respectively. Safety is achieved using the ISSf-CBF formulation presented in Section 2.9 (with λ = 0) for small values of ϵ0 , but is also achieved using small values of λ. We remark that when changing the controller from kCBF to kTISSf , the equilibrium of the system is shifted as can be noticed when comparing the runs in Figure 4.18. We characterize this by the shift in the steady-state tracking distance error: exp D̃ss ≜ Dss − D∗ . (4.65) exp Here Dss is the steady-state distance captured in experiments when the 201 Label (B) (D) (A) (C) ϵ0 3 [s /m] 0.8 3 4 5 0.5 0.5 0.8 0.8 1.0 λ [1/m] 0 0 0 0 0.4 0.5 0.25 0.35 0.25 h∗ [m] −40.50 −151.88 −202.50 −253.13 −4.38 −3.80 −7.01 −5.64 −7.59 hmin [m] 22.09 2.99 1.02 −0.45 0.35 −1.27 0.78 −1.03 −0.86 D̃ss [m] 25.43 6.19 4.69 3.54 4.70 2.22 4.34 2.22 3.07 Table 4.3. Sets of parameter values used for the exponential function (4.63) in the automated truck experiments with theoretical safety guarantee h∗ , minimum experimental value of the TISSf-CBF hmin , and shift in the steady-state tracking distance D̃ss in (4.65). leader is moving with the steady-state speed v ∗ ∈ (0, v) before braking. The term D∗ = V −1 (v ∗ ) captures the desired steady-state distance given by the inverse of the range policy (4.60). In the experiments v ∗ = 16 [m/s], yielding D∗ = 25 [m]. The values of D̃ss corresponding to different parameter pairs are given in Table 4.3. In the right panel of Figure 4.19 we visualize the theoretical values of h∗ and the experimental values of D̃ss for different parameter sets. The black markers indicate parameter sets with λ = 0, while the blue markers show parameter sets with λ > 0. With λ = 0, the theoretical guarantees are nearly meaningless (observe the large negative values of h∗ ), and improving them requires dramatically increasing D̃ss . In contrast, the parameter sets with λ > 0 allow us to obtain significantly (an order of magnitude) stronger theoretical guarantees without greatly increasing D̃ss , thereby also achieving good performance. In the Figure 4.20, we give experimental results of three other parameter pairs labeled as (B), (C), and (D) in Table 4.3. The poor performance of case (B) is indicated by the large value of D̃ss . The results for cases (C) and (D) nearly overlap, but the introduction of λ yield a strong theoretical guarantee for case (C), which is missing for case (D). Conclusion In conclusion, in this section I have developed a robust yet performant safetycritical controller for a connected automated semi-trailer truck system. I begin by presenting a simple model for this system, a safety specification via a CBF, a nominal controller, and simulation and experimental results using a (CBF-QP) 202 60 40 20 0 0 2 4 6 8 10 Figure 4.20. Experimental results for parameter pairs (B), (C), and (D) in Table 4.3. Case (B) is conservative as indicated by the large steady-state tracking distance. Cases (C) and (D) display nearly identical behavior, but case (C) possesses a stronger theoretical guarantee. controller. Unmodeled actuator dynamics lead to a matched disturbance in the experimental system, resulting in the (CBF-QP) controller failing to keep the experimental system safe. I develop a collection of TISSf-based controllers and explore how the parameters of these controllers an be tuned to achieve both meaningful theoretical safety guarantees and strong performance. These conclusions are then realized on the experimental system. 4.5 Safety with Stochastic Disturbances In this section I will present work on safety-critical control in the presence of stochastic disturbances. This work will study conditions on a discretetime system and a corresponding CBF for which it is possible to synthesize controllers that return finite-time ISSf-like safety guarantees using convex optimization. I will begin by reviewing discrete-time dynamics and CBFs for discrete-time systems. I will then briefly review martingales, Ville’s Inequality, and Jensen’s Inequality, which are tools for analyzing stochastic systems. Next, I will propose an ISSf-like notion of finite-time safety for discrete-time systems with stochastic disturbances, and establish how a discrete-time BF meeting an expectation condition on its change across time steps certifies this safety property. Following this, I will explore how a discrete-time CBF satisfying certain convexity properties can be used to synthesize a safety-critical 203 controller that yields a discrete-time BF. Lastly, I show the proposed method in simulation on several examples, including a quadrupedal robot. The contributions of this section are as follows: • An ISSf-like characterization of finite-time safety for discrete-time systems experiencing stochastic disturbances. • An exploration of how convexity properties of a discrete-time CBF must be utilized with a stochastic disturbance signal to ensure the convexity of downstream control synthesis. The text for this section is adapted from: R. K. Cosner, P. Culbertson, A. J. Taylor, and A. D. Ames, “Robust safety under stochastic uncertainty with discrete-time control barrier functions,” accepted in Robotics: Science and Sys. (RSS) XIX, 2023. A. J. Taylor participated in the conception of the project, theoretical analysis, and writing of the article. Discrete Dynamics In this section I provide a review of safety-critical control systems for discretetime nonlinear systems via CBFs. Consider a discrete-time (DT) open-loop system with dynamics given by: xk+1 = F(xk , uk ), ∀k ∈ N, (4.66) with state xk ∈ E, input uk ∈ Rm , and dynamics F : E × Rm → Rn that are continuous on E × Rm . A controller k : E → Rm that is continuous on E yields the DT closed-loop system: xk+1 = F(xk , k(xk )), ∀k ∈ Z≥0 . (4.67) I assume for any x0 ∈ E, there exists a set I(x0 ) = {0, 1, . . . , kmax (x0 )} ⊆ N and a unique solution φ : I(x0 ) → E satisfying: φ(k + 1) = F(φ(k), k(φ(k))), φ(0) = x0 . ∀k ∈ I(x0 ), (4.68) (4.69) The notion of safety for systems of this form is similarly formalized using the concept of forward invariance [104]: 204 Definition 35 (Discrete-Time Forward Invariance & Safety). A set C ⊂ E is forward invariant for the closed-loop system (4.67) if x0 ∈ C implies that φ(k) ∈ C for all k ∈ I(x0 ). In this case, the closed-loop system (4.67) is said to be safe with respect to the set C. Discrete-time barrier functions (DT-BFs) are a tool for guaranteeing the safety of discrete-time systems. Consider a set C expressed as the 0-superlevel set of a function h : E → R that is continuous on E. I refer to such a function h as a DT-BF1 if it satisfies the following properties [234]: Definition 36 (Discrete-Time Barrier Function (DT-BF)). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuous on E. The function h is a discrete-time barrier function (DT-BF) for the closed-loop system (4.67) on C if there exists an α ∈ [0, 1] such that: h(F(x, k(x))) ≥ αh(x). (4.70) for all x ∈ E, DT-BFs serve as a certificate of forward invariance as captured in the following theorem [234]: Theorem 30. Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuous on E. If h is a DT-BF for the closed-loop system (4.67) on C, then the closed-loop system (4.67) is safe with respect to the set C. Intuitively, the value of h(φ(k)) can only decay as fast as the geometric sequence αk h(x0 ), which is lower-bounded by 0, thus ensuring the safety (i.e., forward invariance) of C. Discrete-time Control Barrier Functions (DT-CBFs) [234] provide a tool for constructively synthesizing controllers that yield closed-loop systems that possess a DT-BF: Definition 37 (Discrete-Time Control Barrier Function (DT-CBF)). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuous on E. 1 The state constraint xk ∈ C for all k ∈ I(x0 ), when expressed as h(xk ) ≥ 0 for all k ∈ I(x0 ), is the special case of a DT-BF with α = 0. 205 The function h is a discrete-time Control Barrier Function (DT-CBF) for the open-loop system (4.66) on C if there exists an α ∈ [0, 1] such that: sup h(F(x, u)) > αh(x), u∈Rm (4.71) for all x ∈ E. Given a CBF h for (4.66) and a corresponding α ∈ [0, 1], I define the point-wise set of control values: KCBF (x) = {u ∈ Rm | h(F(x, u)) ≥ αh(x)} . (4.72) This yields the following result [242]: Theorem 31. Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuous on E. If h is a DT-CBF for the open-loop system (4.66) on C, then the set KCBF (x) is non-empty for all x ∈ E, and for any controller k that is continuous on E with k(x) ∈ KCBF (x) for all x ∈ E, the function h is a DT-BF for the closed-loop system (4.67) on C. Given a nominal controller knom : E → Rm that is continuous on E and a DT-CBF h for the open-loop system (4.66) on C, a controller can be specified via the following optimization problem: kCBF (x) = argmin ∥u − knom (x)∥2 u∈Rm (DT-CBF-OP) s.t. h(F(x, u)) ≥ αh(x), noting that kCBF (x) ∈ KCBF (x) for all x ∈ E. I note that unlike the affine inequality constraint that arises in the continuous-time (CBF-QP), the DTCBF inequality constraint (4.71) is not necessarily convex with respect to the input, preventing it from being integrated into a convex optimizationbased controller. To solve this issue, it is often assumed that the function h ◦ F : E × Rm → R is concave with respect to its second argument [234], [243], [244]. This assumption will be studied later in Section 6.4. Martingales & Jensen’s Inequality In this section I review tools from probability theory that will allow utilizing information about the distribution of a stochastic disturbance signal in constructing a notion of stochastic safety and corresponding safety-critical controllers. I choose to provide this background material at the level necessary 206 to understand subsequent constructions of stochastic safety and safety-critical controllers, but refer to [245] for a precise measure-theoretic presentation of the following concepts. The key tool underlying the construction of a notion of stochastic safety is a nonnegative supermartingale, a specific type of expectation-governed random process: Definition 38 (Martingales). Let xk , k ∈ Z≥0 , be a sequence of random variables that take values in Rn , W : Rn × Z≥0 → R, and suppose that   E |W (xk , k)| < ∞ for all k ∈ Z≥0 . The process Wk ≜ W (xk , k) is a supermartingale if: E[Wk+1 | x0:k ] ≤ Wk almost surely for all k ∈ Z≥0 , (4.73) where x0:k indicates the random variables {x0 , x1 , . . . , xk }. If, additionally, Wk ≥ 0 for all k ∈ Z≥0 , Wk is a nonnegative supermartingale. If the process is non-decreasing in expectation, the process Wk is a submartingale. If the inequality (4.73) holds with equality, the process Wk is a martingale. An important result from martingale theory that I will use to develop probabilistic safety guarantees is Ville’s inequality, which allows bounding the probability that a nonnegative supermartingale will rise above a certain value [246]: Theorem 32. If Wk is a nonnegative supermartingale, then for all λ ∈ R>0 : ( ) E[W0 ] . (4.74) P sup Wk > λ ≤ λ k∈Z≥0 Intuitively, Ville’s inequality can be compared with Markov’s inequality [245] for nonnegative random variables; since the process Wk is non-increasing in expectation, Ville’s inequality allows for controlling the probability the process instead moves upward above λ. Lastly, as I will show when synthesizing safetycritical controllers in the presence of stochastic disturbances, conditions will need to be enforced on the expectation of a DT-CBF. In doing so, I will need to relate the expectation of the DT-CBF h(xk+1 ) to the expectation of the state xk+1 . This will be achieved using Jensen’s inequality [247]: Theorem 33. Consider a function h : E → R that is continuous on E and a random variable x that takes values in E with E[∥x∥] < ∞. We have that:  if h is convex, then E[h(x)] ≥ h(E[x]), (4.75) if h is concave, then E[h(x)] ≤ h(E[x]). 207 Safety of Stochastic Systems In this section I provide one of the main results of this work in the form of a bound on the probability that a system with stochastic disturbances will exit a given superlevel set of a DT-BF over a finite time horizon. Consider the following the DT open-loop system with additive disturbances: xk+1 = F(xk , uk ) + dk , ∀k ∈ Z≥0 , (4.76) with dk ∈ Rn . A controller k : E → Rm that is continuous on E yields the following closed-loop system: xk+1 = F(xk , k(xk )) + dk , ∀k ∈ Z≥0 . (4.77) I assume that for any initial condition x0 ∈ E and sequence of disturbances D = {dk } for k ∈ Z≥0 , there exists a set Id (x0 , D) = {0, 1, . . . , kmax (x0 , D)} ⊆ Z≥0 and a unique solution φd : I(x0 , D) → E satisfying: φd (k + 1) = F(φ(k), k(φ(k))) + dk , ∀k ∈ Id (x0 , D), (4.78) (4.79) φd (0) = x0 . I assume that x0 ∈ E is known and consider sequences of disturbances D = {dk } for k ∈ Z≥0 that are independently and identically distributed (with distribution D) random variables2 with (potentially unbounded) support on Rn , generating the random process x1:k ≜ {φd (1), φd (1), . . . , φd (k)} (this is done for notational convenience). To study the safety of this system, I will use the following definition: Definition 39 (K-Step Exit Probability). Let the function h : E → R be continuous on E. For any K ∈ Z≥0 , γ ∈ R≥0 , and initial condition x0 ∈ E, the K-step exit probability of the closed-loop system (4.77) is given by:   Pu (K, γ, x0 ) = P min h(xk ) < −γ . (4.80) k∈{0,...,K} This definition describes the probability that the state of the closed-loop system will leave the −γ superlevel set of h within K steps. This is inherently a finite-time safety guarantee, as it only considers finite values of K. The 2 This implies the dynamics define a Markov process, i.e. E[h(F(xk , k(xk ))+dk ) | x0:k ] = E[h(F(xk , k(xk )) + dk ) | xk ], since the state xk+1 at time k + 1 only depends on the state xk and disturbance dk at time k. 208 finite-time aspect of K-step exit probabilities is critical since systems exposed to unbounded disturbances will exit a bounded set with probability Pu = 1 over an infinite horizon [61], [68]. Intuitively, this is because a sufficiently large sample will eventually be drawn from the tail of the distribution that forces the system out in a single step. Additionally, observe that this definition is directly related ISSf as it reasons about all superlevel sets of the function h, rather than just the 0-superlevel set. For the remainder of this work, I will omit the dependence of Pu on K, γ, and x0 for notational simplicity. Given this definition, I now provide one of the main results of this work by relating DT-BFs to K-step exit probabilities. I note that this result is a reframing of the stochastic invariance theorem in [59], [63]. This reframing features three key components. First, it develops the results using the standard formulation of DT-BFs covered above. Second, it yields a probability bound not only for C (defined as the 0-superlevel set of h, such that γ = 0), but for all non-positive superlevel sets of h (γ ≥ 0), providing a stochastic variant of ISSf. Third, I present a complete proof of the result, with the goal of illuminating how to leverage tools from martingale theory to reason about the safety of discrete-time stochastic systems. Theorem 34. Let the function h : E → R be continuous and upper-bounded on E with upper bound M ∈ R>0 . Suppose there exists an α ∈ (0, 1) and3 δ ≤ M (1 − α) such that the closed-loop system (4.77) satisfies: E[h(F(x, k(x)) + d) | x] ≥ αh(x) + δ, (4.81) for all x ∈ E, with d ∼ D. For any K ∈ N and γ ∈ R≥0 , if δ < −γ(1 − α), we have that:   K M (1 − α) − δ X i−1 M − h(x0 ) Pu ≤ αK + α . (4.82) M +γ M +γ i=1 Alternatively if δ ≥ −γ(1 − α), then: h(x0 ) + γ Pu ≤ 1 − M +γ 3  Mα + γ + δ M +γ K . (4.83) The original presentation of Theorem 34 in [59] considers variable δk for k ∈ {0, . . . , K}, which are known a priori. In most practical applications, one assumes a lower bound that holds for all δk , motivating the use of a constant δ. Moreover, the use of a constant δ significantly clarifies the proof. 209 The upper bound δ ≤ M (1 − α) is relatively non-restrictive, as not only is δ typically negative, but it must hold such that, in expectation, h(xk+1 ) cannot rise above the upper bound M on h. The switching condition between (4.82) and (4.83) of δ = γ(1−α) corresponds to whether, in expectation, the one-step evolution of the system remains in the set Cγ = {x ∈ E | h(x) ≥ −γ} when it begins on the boundary of Cγ . To make the following argument clear at a high level, I begin with a short proof sketch before proceeding in detail. Proof sketch: The key tool in proving Theorem 34 is Ville’s inequality (4.74). Since h(xk ), in general, is not a super- or submartingale, I will first construct a nonnegative supermartingale, Wk ≜ W (xk , k), by scaling and shifting h(xk ). I can then apply Ville’s inequality (4.74) to bound the probability of Wk going above any λ > 0. Next I find a particular value of λ, denoted λ∗ , such that: max k∈{0,...,K} Wk ≤ λ∗ =⇒ min k∈{0,...,K} h(xk ) ≥ −γ. (4.84) Intuitively, this means that any sequence Wk that remains below λ∗ ensures that the corresponding sequence h(xk ) remains (safe) above −γ. This allows bounding the K-step exit probability Pu of the original process h(xk ) with the probability that Wk will rise above λ∗ :   W0 E[W0 ] ∗ = ∗, (4.85) Pu ≤ P max Wk > λ ≤ ∗ k∈{0,...,K} λ λ where the last equality will follow as it is assumed x0 is known a priori. Particular choices of W and λ∗ will yield the bounds stated in the theorem, completing the proof. Proof. Proof: Constructing a Nonnegative Supermartingale I will begin by constructing a nonnegative supermartingale, allowing the use of Ville’s inequality. To construct this supermartingale, first note that by rearranging terms in the inequality in (4.81), the process M − h(xk ) resembles a supermartingale: E[M − h(xk+1 ) | xk ] ≤ α(M − h(xk )) + M (1 − α) − δ, ≜ α(M − h(xk )) + φ, (4.86) but with a scaling α and additive term φ ≜ M (1 − α) − δ that makes E [M − h(xk+1 ) | xk ] ≰ M − h(xk ) in general. To remove the effects of α 210 and φ, consider the function W : Rn × Z≥0 → R defined as: K k X X W (xk , k) ≜ (M − h(xk ))θk − φ θi , θi + φ | {z } i=1 i=1 negate and scale | {z } | {z } (4.87) ensure W ≥ 0 cancel φ where θ ∈ [1, ∞) will be used to cancel the effect of α, but is left as a free variable that I will later use to tighten the bound on Pu . Denoting Wk ≜ W (xk , k), I now verify Wk is a nonnegative supermartingale. I first show that Wk ≥ 0 for all k ∈ {0, . . . , K}. Combining the two sums in (4.87) yields: k Wk = (M − h(xk ))θ + φ K X (4.88) θi , i=k+1 which is nonnegative as h(x) ≤ M for all x ∈ E, θ ≥ 1, and φ ≥ 0 since δ ≤ M (1−α) by assumption. I now show that Wk satisfies the supermartingale inequality (4.73): (4.89) E[Wk+1 | x0:k ] = E[Wk+1 | xk ], = (M − E[h(xk+1 ) | xk ])θ k+1 K X +φ (4.90) θi , i=k+2 ≤ (M − αh(xk ) − δ)θ k+1 +φ K X (4.91) θi , i=k+2 = αθ(M − h(xk ))θk + θk+1 ((1 − α)M − δ) +φ {z } | =φ k = |{z} αθ (M − h(xk ))θ + φ req.≤1 K X K X θi , i=k+2 θ i ≤ Wk , (4.92) i=k+1 where (4.89) is due to the Markovian nature of system (4.77), (4.90) comes from using (4.88) to write Wk+1 , (4.91) follows from (4.81), and (4.92) follows from the preceding line using the definition of φ and assuming the further requirement that θ ≤ α1 . Thus, I have shown that Wk is a nonnegative supermartingale. Proof: Bounding the Exit Probability via Ville’s inequality Since Wk is a nonnegative supermartingale, Ville’s inequality can be applied to establish:   E[W0 ] W0 P max Wk > λ ≤ = . (4.93) k∈{0,...,K} λ λ 211 for all λ ∈ R>0 . To relate this bound to the K-step exit probability Pu , I seek a value of λ, denoted λ∗ , such that: max k∈{0,...,K} Wk ≤ λ∗ =⇒ min k∈{0,...,K} h(xk ) ≥ −γ. (4.94) In short, I will choose a value of λ∗ such that all trajectories of Wk that remain below λ∗ must also have hk ≥ −γ. To this end, I use the geometric P k series identity4 ki=1 θi−1 = 1−θ to rewrite Wk as: 1−θ θK − θk . Wk = (M − h(xk ))θ + φθ θ−1 (4.95)   φθ φθ K λk = γ + M − θk + θ > 0, θ−1 θ−1 (4.96) k Define: which, intuitively, applies the same time-varying scaling and shift to a constant, −γ, that was applied to h(xk ) to yield Wk (4.95). I then choose: λ∗ ≜ min k∈{0,...,K} λk . (4.97) Since I assume maxk∈{0,...,K} Wk ≤ λ∗ , we can write, for all k ∈ {0, . . . , K}: 0 ≥ Wk − λ∗ ≥ Wk − λk = (−γ − hk )θk . (4.98) Since θ > 1, this implies that −γ − hk ≤ 0 for all k ∈ {0, . . . , K}, and thus mink∈{0,...,K} h(xk ) ≥ −γ, as needed. Proof: Choosing θ to Minimize the Ville’s Inequality Since the supermartingale Wk includes a free parameter θ ∈ (1, α1 ], I will choose a value of θ in this interval which provide the tightest bound on Pu . Case 1: Consider the first case where δ < −γ(1 − α), implying φ > (M + +γ γ)(1 − α). In this case α1 < MM+γ−φ and thus all of the allowable choices of M +γ 1 θ ∈ (1, α ) are such that θ < M +γ−φ . Denoting k ∗ such that λ∗ = λk∗ , we have that:   φθ K φθ ∗ ∗ λ = γ+M − θk + θ . (4.99) θ−1 θ−1 {z } | ≤0 k At θ = 1, the fraction 1−θ 1−θ is not well-defined. However, the proof can be carried out using the summation notation. In this case λ∗ = M + γ, and (4.93) yields Pu ≤ )+γ−φK 1 − h(x0M . +γ 4 212 Thus, we know mink∈{0,...,K} λk occurs at k ∗ = K and so:
φθ θK − 1 M − h(x0 ) + θ−1 W0 Pu ≤ ∗ = . λ (M + γ)θK (4.100) Since this bound is a decreasing function of θ (as shown in Lemma 3 following this proof), I choose the largest allowable value θ∗ = α1 to achieve the bound:
φ α−K − 1 M − h(x0 ) + 1−α W0 Pu ≤ ∗ = , (4.101) λ (M + γ)α−K   K M (1 − α) − δ X i−1 M − h(x0 ) K α + α , (4.102) = M +γ M +γ i=1 where I again use the geometric series identity. Case 2: Now consider the second case where δ ≥ −γ(1 − α), so φ ≤ (M + +γ γ)(1−α), which implies that the set [ MM+γ−φ , α1 ] is nonempty. Choosing a value of θ in this set ensures that:   φθ K φθ ∗ ∗ θ . (4.103) θk + λ = γ+M − θ−1 θ−1 | {z } ≥0 Thus mink∈{0,...,K} λk occurs at k ∗ = 0 and:
φθ (M − h(x0 )) + θ−1 θK − 1 W0 Pu ≤ = , φθ λ (M + γ) + θ−1 (θK − 1) (4.104) h(x0 ) + γ . φθ (θK − 1) M + γ + θ−1 (4.105) =1− Since this bound is increasing in θ (as shown in Lemma 4 following this proof), +γ I choose θ∗ = MM+γ−φ to achieve the bound:  K  Mα + γ + δ h(x0 ) + γ Pu ≤ 1 − . (4.106) M +γ M +γ  i M +γ If, alternatively, I choose θ ∈ 1, M +γ−φ , then the inequality in (4.99) holds, k ∗ = K, and the bound is decreasing in θ as in Case 1. Evaluating this bound +γ for the minimizing value θ∗ = MM+γ−φ again yields: M − h(x0 ) + (M + γ)(θK − 1) , (M + γ)θK   K h(x0 ) + γ Mα + γ + δ =1− . M +γ M +γ Pu ≤ (4.107) (4.108) 213 I now state and prove the lemmas used in the preceding proof to prove optimality of the bound in Theorem 34 Cases 1 and 2. These lemmas were originally stated without proof in [59]. Lemma 3. For M ∈ R>0 , γ, φ ∈ R≥0 , h(x0 ) ∈ [−γ, M ], and K ∈ Z≥1 , the function Ψ1 : (1, ∞) → R defined as:
φθ M − h(x0 ) + θ−1 θK − 1 Ψ1 (θ) = , (4.109) (M + γ)θK is monotonically decreasing. Proof. The geometric series identity yields: K X φ M − h(x0 ) −K θ + θi−K , Ψ1 (θ) = M +γ (M + γ) i=1 (4.110) K X M − h(x0 ) −K−1 (K − i)θi−K−1 dΨ1 =− Kθ −φ , dθ M +γ M +γ i=1 (4.111) ≤ 0, for all θ ∈ (1, ∞). Lemma 4. For M ∈ R>0 , γ, φ ∈ R≥0 , h(x0 ) ∈ [−γ, M ], and K ∈ Z≥1 , the function Ψ2 : (1, ∞) → R defined as: Ψ2 (θ) = 1 − h(x0 ) + γ , φθ M + γ + θ−1 (θK − 1) (4.112) is monotonically increasing. Proof. The geometric series identity yields: h(x0 ) + γ , P i M +γ+φ K θ  P i=1  K i−1 (h(x ) + γ) φ iθ 0 i=1 dΨ2 =  PK i 2 , dθ M + γ + φ i=1 θ Ψ2 (θ) = 1 − ≥ 0, for all θ ∈ (1, ∞). (4.113) (4.114) (4.115) 214 Practical Considerations in Enforcing Safety of Stochastic Systems Theorem 34 allows reasoning about the finite-time safety of systems governed by DT-BFs. To utilize the results of this theorem in a control setting, I aim to use DT-CBFs to develop controllers which enforce the expectation condition: E[h(F(xk , uk ) + dk ) | xk ] ≥ αh(xk ). (4.116) Like the DT-CBF-OP controller, I seek to enforce this constraint using an optimization-based controller that enforces safety while achieving pointwise minimal deviation from a nominal controller knom : E → Rm in the form of an Expectation-based DT-CBF (ED) controller: kED (xk ) = argmin ∥u − knom (xk )∥2 u∈Rm (ED) s.t. E[h(F(xk , u) + dk ) | xk ] ≥ αh(xk ). The expectation in (ED) adds complexity that is not generally considered in the application of deterministic DT-CBFs. Typically, CBF-based controllers solve “certainty-equivalent” optimization programs, like this CertaintyEquivalent DTCBF (CED) controller, that replace the expected barrier value E[h(xk+1 ) | xk ] with the barrier at the expected next state, h(E[xk+1 | xk ]): kCED (xk ) = argmin ∥u − knom (xk )∥2 u∈Rm (CED) s.t. h(F(xk , u) + E[dk ]) ≥ αh(xk ), where E[F(xk , uk ) | xk ] = F(xk , uk ) and E[dk |xk ] = E[dk ]. This constraint is often easier to evaluate than (4.116) since it allows control actions to be selected with respect to the expected disturbance E[dk ] without needing to model the disturbance distribution D. If the disturbance is zero-mean, then this form of the constraint is implicitly enforced by DT-CBF controllers such as those presented in [234], [244]. However, when replacing ED with CED it is important to consider the effect of Jensen’s inequality in Theorem 33. If the “certainty-equivalent” constraint in CED is strictly concave5 , then we can apply the results of Theorem 34 directly since Jensen’s inequality tightens the constraint and ensures satisfaction of the expectation condition (4.81). Unfortunately, using such a controller is a non-convex optimization program which 5 The constraint h(xk + u) ≥ αh(xk ) is concave in u when h is convex and it is convex in u when h is concave. 215 can be impractical to solve. If, instead, the constraint is convex, then CED is a convex program, but does not necessarily enforce the expectation condition (4.81) in Theorem (34) due to the gap introduced by Jensen’s inequality. In order to apply the results of Theorem 34 to controllers of the form (CED) with convex constraints, we must first provide a bound on the gap introduced by Jensen’s Inequality. In particular, for any concave function h : E → R and random variable d ∼ D, I seek to determine a value ψ ∈ R≥0 such that, for all x ∈ E and u ∈ Rm : E[h(F(x, u) + d) | x] ≥ h(F(x, u) + E[d]) − ψ, (4.117) thus quantifying the gap introduced by Jensen’s inequality. A large body of work has studied methods for finding the smallest possible ψ that satisfies (4.117). Here I adapt a result in [248] to achieve a relatively loose, but straightforward bound: Lemma 5. Consider a function h : Rn → R that is twice-continuously differentiable on Rn , concave, and satisfies supx∈Rn ∥∇2 h(x)∥2 ≤ λmax for some λmax ∈ R≥0 , and a random variable x that takes values in Rn with E[∥x∥] < ∞ and ∥cov(x)∥ < ∞. Then we have that: E[h(x)] ≥ h(E[x]) − λmax tr(cov(x)). 2 (4.118) Proof. Consider the function η : Rn → R which is twice-continuously differentiable on E and convex that is defined as η = −h. The Intermediate Value Theorem [249, Theorem 4.5.1] implies that for all y, z ∈ Rn , there exists an ω ∈ [0, 1] such that: 1 η(z) = η(y) + ∇η(y)⊤ e + e⊤ ∇2 η(c)e, 2 (4.119) where e ≜ z − y, c ≜ ωz + (1 − ω)z, and ∇2 η(c) is the Hessian of η evaluated at c. We then have that:
1 η(z) = η(y) + ∇η(y)⊤ e + tr ∇2 η(c)ee⊤ , (4.120) 2
1 ≤ η(y) + ∇η(y)⊤ e + ∥∇2 η(c)∥2 tr ee⊤ , (4.121) 2
λmax ≤ η(y) + ∇η(y)⊤ e + tr ee⊤ , (4.122) 2 where the first inequality is a property of the trace operator for positive semidefinite matrices [250] (and ∇2 η(c) is positive semi-definite as η is convex), 216 and the second inequality follows by our definition of λmax . Let x be a random variable taking values in Rn with probability density function p : Rn → R≥0 , and let µ ≜ E[x]. We then have that: Z (η(x) − η(µ))p(x)dx, (4.123) E[η(x)] − η(E[x]) = Rn Z
λmax ∇η(µ)⊤ e + ≤ (4.124) tr ee⊤ p(x)dx, 2 Rn λmax = tr(cov(x)), (4.125) 2 where e = x − µ. Replacing η with −h yields: E[h(x)] ≥ h(E[x]) − λmax tr(cov(x)). 2 (4.126) I first note that this theorem requires considering Rn instead of an open set E ⊂ Rn . This is important for ensuring the the first term in the integral in (4.124) evaluates to 0. This could likely be relaxed with additional assumptions on the nature of the distribution D. Next, I note that although this value tr(cov(x)) is easy to interpret, tighter bounds exist which have of ψ = λmax 2 less restrictive assumptions than a globally bounded Hessian [247]. Lastly, I note that sampling-based methods could be used to approximately satisfy the constraint (4.117) by estimating ψ empirically. Next, I present a controller which combines the mean-based control of the “certainty equivalent” (CED) while also accounting for Jensen’s inequality. This Jensen-Enhanced DT-CBF (JED) controller includes an additional control parameter cJ ≥ 0 to account for Jensen’s inequality: kJED (xk ) = argmin ∥u − knom (xk )∥2 u∈Rm (JED) s.t. h(F(xk , uk ) + E[dk ]) − cJ ≥ αh(xk ). Given this controller and a method for bounding ψ, we can now apply Theorem 34 while accounting for (or analyzing) the effects of Jensen’s inequality on the (JED) controller: Theorem 35. Consider the closed-loop system (4.77) and let the function h : Rn → R be twice-continuously differentiable on Rn , concave, and satisfy supx∈Rn h(x) ≤ M for M ∈ R>0 and supx∈Rn ∥∇2 h(x)∥2 ≤ λmax for λmax ∈ 217 R≥0 . Suppose there exists an α ∈ (0, 1) and a cJ ∈ [0, λmax tr(cov(d)) + M (1 − 2 α)] such that: h(F(x, k(x)) + E[d]) − cJ ≥ αh(x), (4.127) for all x ∈ Rn with d ∼ D. Then we have that: E[h(F(x, k(x)) + d) | x] ≥ αh(x) + δ, (4.128) tr(cov(d)). for all x ∈ Rn with d ∼ D and δ = cJ − λmax 2 Proof. Given x ∈ Rn , Lemma 5 ensures that: 0 ≤ h(F(x, k(x)) + E[d]) − cJ − αh(x), ≤ E[h(F(x, k(x)) + d) | x] + ψ − cJ − αh(x), (4.129) (4.130) where ψ = λmax tr(cov(d)). Letting δ = cJ − λmax tr(cov(d)) yields the desired 2 2 result. Simulation Results Note that the contents of this subsection were primarily written by collaborators, edited by Andrew Taylor for their appearance in conference proceedings, and edited by Andrew Taylor for their appearance in this thesis. We now consider a variety of simulation examples that highlight the key features of the proposed approach. Linear 1D System Here we analyze the finite-time safety probability bounds by considering the case of unbounded i.i.d. disturbances dk ∼ N (0, 1) (normal distribution with zero mean and variance of one) for the system: xk+1 = xk + 2 + uk + σdk , (4.131) with a function h : R → R defined as h = 1 − x2 yielding the set C = {x | 1 − x2 ≥ 0}. Observe that h is upper bounded by 1 on Rn and as a constant Hessian of value −2. The Jensen gap for this system and DT-CBF is bounded by ψ = σ 2 . For simulation, we employ the JED controller with cJ = σ 2 , α = 1 − σ 2 , and nominal controller knom (xk ) = 0. Figure 4.21 shows the results of 500 one second long trials run with a variety of σ ∈ [0, 0.2] and also displays how the bound on Pu decreases as γ increases. 218 Figure 4.21. The dashed lines represent the theoretical probability bounds for the system as in Theorem 34. The solid lines represent the Monte Carlo (MC) estimated Pu across 500 experiments. Simple Pendulum Next we consider an inverted pendulum about its upright equilibrium point with the DT dynamics: # " # # " " 0 θk + ∆tθ̇k θk+1 + u + dk , (4.132) = θ̇k + ∆t sin(θk ) ∆t θ̇k+1 with time step ∆t = 0.01, i.i.d disturbances6 dk ∼ N (02 , Diag([0.0052 , 0.0252 ]]), and set C defined as: # "   − 12 2 1 3 6 ⊤ n x≥0 , C= x∈R 1 − 2x (4.133) 1 π 1 3− 2 | {z } hpend (x) which is constructed using the CTLE as in [115] and for which |θ| ≤ π/6 for all x ∈ C. Figure 4.22 shows the results of 500 one second long trials for each x0 ∈ C using the JED controller with parameters α = 1 − ψ, cJ = ψ, where ψ = λmax tr(cov(dk )). This figure highlights the influence of x0 and shows how 2 the bound on Pu increases as h(x0 ) decreases. Quadruped Finally, we consider controlling a simulated quadrupedal robot locomoting along a narrow path. The simulation is based on a Unitree A1 robot as shown Diag: Rn → Rn×n generates a square diagonal matrix with its argument along the main diagonal. 6 219 Figure 4.22. (Top Left) System diagram of the inverted pendulum. (Top Right) 500 one second long example trajectories starting at x0 = 0. (Bottom Left) Monte Carlo estimates of Pu for γ = 0 using 500 one second long trials for each initial conditions represented by a black dot. (Bottom Right) The (conservative) theoretical bounds on Pu from Theorem 34. in Figure (4.23) which has 18 degrees of freedom and 12 actuators. An inverse dynamics-based (ID) QP controller designed using concepts in [251] and implemented at 1 [kHz] is used to track stable walking gaits with variable planar velocities and angle rate using the motion primitive framework presented in [252]. The full 18 degree of freedom quadruped dynamics are simulated at 1 [kHz], but the methodology in [253] is used by considering the following simplified discrete-time single-integrator system for DT-CBF-based control:    cos θ − sin θ 0 vkx    xk+1 = xk + ∆t  sin θ cos θ 0 vky  + dk , (4.134) 0 0 1 θk h i⊤ where xk = x, y, θ . In order to represent the error caused by uncertain terrain, zero mean Gaussian disturbances are added to the quadruped’s (x, y) body position and velocity with variances of 2.25 × 106 and 0.01 respectively. This random noise along with the dynamics-mismatch between the full-order 220 Figure 4.23. Safety of a simulated quadrupedal robot locomoting on a narrow path for a variety of controllers. (Top Left) The safe region that the quadruped is allowed to traverse. (Bottom Left) A system diagram depicting the reduced-order model states of the quadruped  ⊤ x y θ . (Top Right) 50 trajectories for 3 controllers: one without any knowledge of safety (knom ), one with the standard (DT-CBF-OP) controller, and finally the proposed method which accounts for stochasticity (JED). (Bottom Right) Plots of h(x), a scalar value representing safety. The system is safe (i.e., in the green safe region) if h(x) ≥ 0. quadrupedal dynamics and (4.134) is modeled as an i.i.d. random process dk . The quadruped is commanded to stand and then traverse a 7 [m] path that is 1 [m] meter wide, with the set C = {x ∈ Rn | 0.52 − y 2 ≥ 0}. For this simulation, three controllers are compared: a simple nominal controller knom (x) = h i⊤ 0.2, 0, −θ with no understanding of safety, the DT-CBF-OP controller with α = 0.99, and the proposed JED controller with α = 0.99 and cJ = ψ ush i⊤ ing the mean and covariance estimates, E[dk ] ≈ −0.0132, −0.0034, −0.0002 and tr(cov)(dk )) ≈ ψ = 0.000548, which were generated using 15 minutes of walking data controlled by knom . The results of 50 trials for each controller can be seen in Figure 4.23. As expected, knom generated the largest safety violations and JED the smallest and fewest safety violations. 221 Conclusion In conclusion, in this section I have developed a framework for safety-critical control of discrete-time systems with stochastic disturbances using CBFs and convex optimization. I begin by reviewing discrete-time nonlinear dynamics and corresponding variants of BFs and CBFs for such systems, followed by a review of tools from probability theory that are useful for studying systems with stochastic disturbances. I then consider nonlinear discrete-time systems with stochastic disturbances and propose a notion of finite-time safety akin to ISSf. I show how discrete-time BFs certify this finite-time safety property, and then show how convexity properties of discrete-time CBFs can be used to synthesize controllers that yield discrete-time BFs through convex optimization. I conclude with a set of simulations on various systems. 4.6 Conclusion In this chapter I have presented a collection of results on designing controllers that are performant while providing robustness to disturbances. In Section 4.2 I looked at some of the limitations of controllers designed through ISSf-CBFs as originally posed in [58] and presented in Section 2.9. This investigation focuses on balancing meaningful theoretical guarantees with conservative behavior. Drawing motivation from this analysis, in Section 4.3 I present a novel modification of ISSf in the form of Tunable Input-to-State Safety (TISSf) that permits controllers that provide not only meaningful theoretical guarantees, but also strong closed-loop performance. I then explore the capabilities of TISSf in the context of a simple inverted pendulum. In Section 4.4, I consider the significantly more challenging problem of designing a safety-critical controller for a connected automated semi-trailer truck. A simplified model for this system neglects important actuator dynamics in an experimental system that lead to the failure of standard CBF-based controllers to render the system safe. Capturing the effect of these dynamics as a matched disturbance, I design a host of controllers utilizing ISSf-CBFs and TISSf-CBFs, and demonstrate the ability of TISSf to produce improvements in theoretical guarantees and closed-loop performance over ISSf-CBF based controllers, including on the experimental system. In Section 4.5, I consider safety-critical control for discrete-time systems subject to stochastic disturbances. I review tools for discrete-time safety critical control and probabilistic tools for studying stochastic disturbances. I then propose a notion of finite-time safety that 222 draws inspiration from the set-expansion paradigm posed by ISSf, and show how discrete-time barrier functions certify this safety property. I conclude with studying how convexity properties of a discrete-time CBF play an important role in permitting the design of controllers that are robust to stochastic disturbances through convex optimization, and demonstrate such controllers on a collection of simulation environments. I think there are a number of meaningful directions of future work stemming from these results. In the context of ISSf and TISSf, TISSf makes significant strides in achieving theoretical guarantees that more accurately describe the closed-loop behavior of a system operating under a controller that is robust to disturbances. Still, I believe that there still remains the potential to more closely match theoretical safety guarantees with the closed-loop behavior that is actually observed, thereby ensuring that designs focused on pursuing quantitative theoretical guarantees don’t produce excessively conservative controllers. One particular observation is that both ISSf and TISSf rely on the ∞-norm of the disturbance. This contrasts the fact that disturbance signals impact the evolution of the state by being integrated over a period of time. If the disturbance is large but present for a small period of time, it will yield a large ∞-norm and have potentially little impact on the evolution of the system (this is why the essential supremum is used in the first place!). This is not a new observation in terms of studying the impact of disturbances on systems, which has been well characterized for linear systems [99] and codified for stability through the notion of L-stability [100, Chapter 5]. I believe that developing notions of ISSf that are based on Lp -norms may permit stronger theoretical guarantees for a system exposed to a specific class of disturbances. An another idea that would likely share connections with this line of thinking is that of integral Input-to-State Stability [254]. In the context of safety-critical control in the presence of stochastic disturbances, I also think there are a number of future work directions that would directly improve the results in Section 4.5. First, there is likely significant room for improving the quality of the bounds achieved through this analysis through the intelligent choice of CBFs, or synthesis of CBFs that optimize these bounds (similar to the work in [61]). Other approaches for improving the bound could look at making stronger claims on the higher-order moments of the disturbance distributions and making use of stronger tools than Ville’s 223 inequality. A potentially better avenue would consider the fact that the nominal controller knom is rarely attempting to make a system unsafe, such that the required CBF inequality constraint is rarely active. I hypothesize this may require studying the state distribution, similar to the line of work in [68]. Lastly, I think there is a natural opportunity to consider the impact of noisy measurements on safety-critical control through this framework. While the work in the next chapter makes some initial steps into measurement-robust safety-critical control, it takes a deterministic model of measurement error, and thus could be expanded to capture the fact that most measurement errors are better captured through a stochastic model. 224 Chapter 5 MEASUREMENT-ROBUST SAFETY-CRITICAL CONTROL In this chapter I will present results from my work developing methods for safety-critical control that are robust to measurement errors. Feedback control is predicated on the ability to observe some portion (or all) of the system state, which is then used as a state estimate in subsequent control computations. Forming a state estimated is generally done through a combination of direct measurements of states and observers such as Kalman filters which produce estimates of states that are not directly measured through an internal model of the system dynamics [255]. Direct measurements are almost always subject to some amount of noise, while observers can be subjected to errors in an internal model, and may need to use a simplified internal model to permit online computation (such as the linearization step that happens in an extended Kalman filter [255]). The recent development of vision-based measurement systems for robotic systems offers the promise of operating in highly unstructured environments, but also comes with its own challenges. In particular, vision-based measurement systems, and especially those built using learning methods, must convert a high-dimensional (and noisy) measurement (such as an image) down into a low-dimensional estimate of the system state (such as position, orientation, and velocities), which is an often imperfect process. Thus, the resulting state estimate is often not exactly equal to the true system state, introducing a mismatch between the state being used in the theory-driven control computations and the true system state. Consequently, this can not only lead to a failure to achieve theoretical guarantees, but poor closed-loop system behavior. It is worth noting that is an issue that will face the practical deployment of all of the methods developed in Chapters 2, 3, 4, and 6 of this thesis. Thus, a thorough characterization of robust methods for safety-critical control synthesis should consider the impacts of errors between a state estimate and the true system state. At the same time, building systems that are increasingly robust to different challenges comes with its own challenge of over-conservative closed-loop behavior. As studied in Chapter 4, if a controller is built to provide rigorous theoretical safety guarantees in the presence of potentially large disturbances, 225 and worst-case disturbances are not experienced, it can lead to unnecessarily poor performance. If a system must be robust to both worst-case disturbances, and worst-case measurement errors, it is reasonable to expect that this will only lead to greater degrees of conservativeness. In practice, the designer of a control system is often left performing the tedious task of tuning various robustness parameters to try and balance the various types of robustness until they get an acceptable level of closed-loop performance. This can be a time consuming task, as often the designer of the control system doesn’t have a complete understanding of how the various parameters interact to dictate overall robustness and closed-loop performance. One thing the control designer is often capable of easily providing feedback on is whether or not they think that the system is too conservative, or not safe enough. It is this observation that will motivate the contents of Section 5.4. My work in this chapter focuses on developing CBF-based control designs that are robust to measurement errors, as well as an algorithm for tuning robustness parameters of a CBF-based controller through preference-based learning (PBL). In Section 5.1 I discuss related work in the topics of safety-critical control techniques that are robust to measurement errors as well as the use of preference-based learning for tuning controller parameters. In Section 5.2 I will present the notion of a Measurement-Robust Control Barrier Function (MR-CBF). I begin by discussing how measurement error appears when trying to certify safety of a system. I will then present MR-CBFs and show how they can be used to synthesize controllers that are robust to measurement errors using convex optimization, resulting in a controller specified as a second-order cone program (SOCP). Following this, I show how this controller can be converted to a standard form SOCP and establish some local Lipschitz properties of a relaxed form of the controller. In Section 5.3 I will present the integration of MR-CBFs with backup set CBFs as formulated in [256]. I review backup set CBFs, present their integration with MR-CBFs, and demonstrate the ability to keep a Segway system using vision-based measurements safe in the presence of measurement errors in both simulation and experimentally. Lastly, in Section 5.4, I will present work on tuning robustness parameters in a CBFbased controller through the use of PBL. I will review PBL, frame the tuning of robustness parameters as a PBL problem, and demonstrate the proposed approach in simulation and experimentally. Key contributions of this work are described at the beginning of each respective section. 226 5.1 Related Work Measurement-Robust and Vision-Based Control Barrier Functions The consideration of measurement errors and their impact on CBF-based controllers has been a topic of recent interest. The work in [50] was the first to consider the impact of measurement errors by incorporating the covariance of an unscented Kalman filter into the CBF inequality as an additive term. In [66] measurements are modeled using stochastic differential equations, and a CBF is robustified against potential estimation error, while in [73] a bounded set of measurement errors is considered by requiring that a controller satisfy a restricted sub-tangentiality condition on the boundary of a set C that is to be kept forward invariant. The work in [74], [93] utilizes interval arithmetic to consider all possible evolutions of the value of a CBF in the presence of measurement errors, and chooses control inputs to require that all possible evolutions satisfy a safety constraint. In [257], measurement errors are treated as errors in the specification of safety via an uncertain CBF, and made subsequently robust. The work in [258] forms barriers based on locally sensed constraints drawn from online measurements. Additional work considering observers for forming state estimates has used the notion of ISSf explored in Section 2.7 to quantify the impact of observation errors on safety [259], used function approximation techniques to capture estimation error and synthesize robust controllers [260], and provided a framework for understanding safety with general observers that achieve ISS or bounded error properties [261]. Vision-based control in robotics has also seen significant development [262]– [264] (these reflect only a tiny fraction of the extensive body of recent work). A collection of this work has focused primarily on the problem of achieving theoretical stability or safety guarantees in the presence of imperfect perception systems. The work in [265] considers linear systems with imperfect perceptionbased measurements and achieves robust stability guarantees through the perspective of System Level Synthesis (SLS), while the work in [266] considers certainty-equivalent perception-based control for linear systems. The work in [267], [268] focuses on discrete-time system with perception-based measurements that are subject to adversarial attacks, and provides rigorous theoretical guarantees utilizing Lyapunov-based arguments. The use of CBFs with visionbased systems (not including my work and its extensions, discussed below) has considered producing barrier functions for LiDAR-based vision systems subject to sensor faults and attacks [269], or considered the use of Neural Radiance 227 Fields (NeRFs) in conjunction with CBFs [270]. My work in [184], presented in Section 5.2 proposes the notion of a MeasurementRobust Control Barrier Function (MR-CBF). This definition considers how error in a state estimate propagates through nonlinear control-affine dynamics, revealing specific structure on how measurement error reacts with the input to the system when determining the time derivative of a MR-CBF. This leads to a convex optimization-based controller specified as a second-order cone program (SOCP), which I evaluate the local Lipschitz continuity of. I demonstrate the proposed approach in simulation on a planar Segway system. In Section 5.3, which features my work in [23], I explore how MR-CBFs can be unified with backup set CBFs [256], leading to the first experimental demonstration of MRCBFs. Extensions of this work have looked at performing self-supervised learning using stereo vision and MR-CBFs [185] and end-to-end imitation learning with MR-CBFs [174], as well as ways in which perception systems can be improved to ensure feasibility of CBF-based inequalities [271]. Preference-Based Learning Preference-based learning (PBL) provides an approach for searching complex parameter spaces via subjective feedback, without an explicitly defined reward function [75]. The main advantage of online PBL is its ability to interactively infer a user’s latent utility function using only subjective feedback such as pairwise preferences and ordinal labels [272], [273]. It has seen application on a wide-range of problems including recommender systems [274], search engines [275], trajectory planning [276]–[278], and spinal cord stimulation [279]. A particular area of development relevant for control systems is safe PBL, which seeks to ensure the recommendations to users do not violate some prespecified safety criterion [135], [279], [280]. In the context of control systems, there has recently been significant effort in developing PBL approaches for tuning existing tools built from control theory. One such example is the selection of walking gaits preferred by a patient that is utilizing a lower-body exoskeleton, which has been explored extensively in [76]–[78]. These ideas were further extended to autonomous bipedal systems to tune walking gait parameters for the AMBER-3M planar biped explored in Section 3.7 [79], or used to tune the parameters of a (CLF-QP) controller for the AMBER-3M planar biped and Cassie 3D biped [80]. A toolbox and comprehensive survey of the use of PBL for robotic systems can be found in [281]. 228 My work in [23], presented in Section 5.4, draws inspiration from the ideas in [80] to tune parameters of a convex optimization-based controller. While the work in [80] focused specifically on learning the parameters of a (CLF-QP) focused on stabilizing a bipedal system, I will focus on tuning the parameters of a CBF-based controller that is robust to both disturbances and measurement errors. In particular, this controller will unify the reduced-order model tools for CBFs developed in [253] with ISSf-CBFs presented in Section 2.9 and MR-CBFs developed in Section 5.2 to design a robust safety-critical controller for a quadrupedal robotic system. This controller is initially extremely conservative, with almost negligible performance. PBL is deployed to adjust the parameters of this controller until a balance can be struck between robustness and performance in both simulation and experimentally. 5.2 Measurement-Robust Control Barrier Functions In this section I will present work on developing Measurement-Robust Control Barrier Functions (MR-CBFs) for safety-critical control in the presence of measurement errors. I begin by describing the setting of measurement error that I consider. Following this, I present novel definitions of MeasurementRobust Barrier Functions (MR-BFs), show how an MR-BF leads to safety in the presence of measurement errors. I then extend the notion of an MRBF to an MR-CBF, and demonstrate its application on a Segway system in simulation. Lastly, I show how the convex optimization problem defining an MR-CBF-based controller can be cast as an SOCP in standard form, and then establish local Lipschitz properties of the controller under a mild relaxation. The contributions of this section are as follows: • The definition of Measurement-Robust Control Barrier Functions (MRCBFs) as a tool for achieving safety-critical control in the presence of measurement errors through convex optimization. • A detailed analysis of the regularity properties of the resulting SOCPbased controllers synthesized using MR-CBFs. The text for this section is adapted from: S. Dean, A. J. Taylor, R. K. Cosner, B. Recht, and A. D. Ames, “Guaranteeing safety of learned perception modules via measurement-robust 229 control barrier functions,” in Proc. Conf. on Robotics Learning (CoRL), Cambridge, MA, USA, 2020. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, and writing of the article. Measurement Error In many applications, the state x is not directly available to the controller, but rather a corresponding measurement y ∈ Rk which satisfies the relationship: y = p(x), (5.1) for a function p : E → Rk that is assumed to be locally Lipschitz continuous on E. In this work I assume the relationship between the measurement and the true state is deterministic, such that the evolution of the system can be described by an ordinary differential equation rather than a stochastic differential equation. I further assume that there exists a function q : p(E) → E that is locally Lipschitz continuous on p(E) and for any x ∈ E with a corresponding measurement y ∈ p(E), satisfies q(y) = x. This assumption implies that the state x could be uniquely determined from its corresponding measurement y if the function q was known, but that is often not the case in many modern control applications (such as when using vision-based measurement systems). While the function p is often determined by the physical attributes of a system b : p(E) → and its sensors, an estimate of the function q, given by the function q E which is assumed to be locally Lipschitz continuous on p(E), is constructed such that given an x ∈ E with a corresponding measurement y ∈ p(E), an b ∈ E, can be determined as x b=q b(y). For notational estimate of the state, x b : E → p(E) × E simplicity, I define the measurement-estimate function v b(x) = (p(x), q b(p(x)) and v b(E) as the image of the set E under such that v the measurement-estimate function. b is constructed either via system and measurement models, The function q or from data using learning methods, and thus its accuracy in estimating q degrades with imperfections in sensor fabrication and integration, or imperfections in learning models and training data. Thus, given a state x ∈ E and a corresponding measurement y ∈ p(E), I assume that the corresponding state b is related to the true state by: estimate x b=q b(y) ≜ x + e(y), x (5.2) 230 Xb(x) x b x C b) X (y, x b). The set Xb(x) consists Figure 5.1. Visualization of the point-wise sets Xb(x) and X (y, x b) consists of of all possible estimates that may correspond to the state x, and the set X (y, x b). all the possible states for a given measurement-state estimate pair (y, x b. In for an unknown function e : p(E) → Rn that is defined implicitly via q practice, the function e can often be characterized via upper bounds on measurement model errors or via data-driven arguments for learning models. In particular, I assume that while e(y) is not known for a particular measurement y ∈ p(E), it is known that e(y) ∈ E(y) for a measurement-dependent, compact pointwise set E(y) ⊂ Rn . This leads to the definition of the following two pointwise sets: b = x + e} , Xb(x) ≜ {b x ∈ Rn | ∃ e ∈ E(p(x)) s.t. x b) ≜ {x ∈ Rn | ∃ e ∈ E(y) s.t. x b = x + e} . X (y, x (5.3) (5.4) The first of these two pointwise sets can be interpreted as all possible state b corresponding to a particular state x, restricted by the possible estimates x error dictated by the set E(p(x)). The second pointwise set consists of all potential states that may yield a measurement-state estimate pair. This set is directly computable given a measurement and state estimate, and will play an important role in control synthesis. These sets are visualized in Figure 5.1. Measurement-Robust Control Barrier Functions Consider a controller k : p(E) × E → Rm that is locally Lipschitz continuous b, rather on p(E) × E and depends on a measurement y and state estimate x than explicitly on the state x. This leads to the closed-loop system: b). ẋ = f (x) + g(x)k(y, x (5.5) I assume that for any initial condition x0 ∈ E, there exists an interval I(x0 ) = [0, tmax (x0 )) ⊆ R≥0 and a unique continuously differentiable solution φ : 231 I(x0 ) → E satisfying: b(p(φ(t)))), φ̇(t) = f (φ(t)) + g(φ(t))k(p(φ(t)), q t ∈ I(x0 ), (5.6) (5.7) φ(0) = x0 . Consider a function h : E → R that is continuously differentiable on E and its corresponding 0-superlevel set C. Observe that satisfying the BF condition in (2.57) would require that: ḣ(x) = ∂h b)) ≥ −α(h(x)), (x) (f (x) + g(x)k(y, x ∂x (5.8) b) ∈ for each x ∈ E with corresponding measurement-state estimate pair (y, x e b(E) for some α ∈ K . Importantly, while the controller k only has access to v the measurement-state estimate pair, the safety constraint depends on the true ∂h state in several ways, including the value of h and ∂x as well as the dynamics f and g. Thus, I will propose an alternative condition which depends on only b, and the set X (y, x b). Observe that to a measurement y, a state estimate x ensure safety with a BF, it is sufficient for the following condition to hold for b) ∈ v b(E): all (y, x ∂h b)) ≥ −α(h(x)). (x)(f (x) + g(x)k(y, x x∈X (y,b x) ∂x inf (5.9) This condition implies that the controller k renders the system safe for all possible states corresponding to a given measurement-state estimate pair. Verifying that this condition holds can be difficult for an arbitrary CBF, and it is not easily (or possibly) enforced in a convex-optimization based controller. To resolve these problems, I introduce the following definition: Definition 40 (Measurement-Robust Barrier Function (MR-BF)). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. The function h is a Measurement-Robust barrier function (MR-BF) for the closed-loop system (5.5) on C with parameter function b(E) → R2≥0 that is locally Lipschitz continuous on v b(E) if there exists (b, a) : v α ∈ Ke such that: ∂h b)) − (b(y, x b) + a(y, x b)∥k(y, x b)∥) ≥ −α(h(b (b x)(f (b x) + g(b x)k(y, x x)). (5.10) ∂x b) ∈ v b(E). for all (y, x 232 b) + The definition of a MR-BF introduces the non-positive term −(b(y, x b)∥k(y, x b)∥) to the BF condition (2.57), requiring that a stronger dea(y, x gree of safety be enforced compared to the typical BF. Furthermore, the norm b), of the input appears in this term, indicating that for large values of a(y, x large inputs will often violate this constraint. The advantage of this is that knowledge of the true state x can be removed from the BF condition and reb which placed with only knowledge of the measurement y and state estimate x are available to the controller k. This enables the following safety result: Theorem 36. Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. Assume1 the functions Lf h, Lg h, and α ◦ h : E → R are Lipschitz continuous on E with Lipschitz constants LLf h , LLg h , and Lα◦h , respectively. Further let a function ϵ : p(E) → R≥0 that is locally Lipschitz continuous on p(E) satisfy: max ∥e∥ ≤ ϵ(y) e∈E(y) (5.11) for all y ∈ p(E). If h is a MR-BF for (5.5) on C with parameter function (ϵ(y)(LLf h + Lα◦h ), ϵ(y)LLg h ), then the system (5.5) is safe with respect to C. Before proving this theorem, note that if h is an MR-BF for the closed-loop b(E) → R2≥0 such that system (5.5) on C with a parameter function (b, a) : v b) ≥ ϵ(y)(LLf h + Lα◦h ) and a(y, x b) ≥ ϵ(y)LLg h for all measurementb(y, x b) ∈ v b(E), then h is an MR-BF for the closed-loop state estimate pairs (y, x system (5.5) on C with the parameter function (ϵ(y)(LLf h + Lα◦h ), ϵ(y)LLg h ). Thus, even if tight Lipschitz constants of these functions are not known, overapproximations can be used and theoretical safety guarantees will be retained (at the expense of potential conservativeness). Proof. Define the function c : E × Rm → R as c(x, u) = Lf h(x) + Lg h(x)u + α(h(x)). (5.12) This proof will follow from the proof of Theorem 8 by showing that for any b) = v b(x): x ∈ E, with corresponding measurement-state estimate pair (y, x b)) ≥ 0. c(x, k(y, x 1 (5.13) The following results developed using MR-CBFs will focus on maintaining forward invariance of the set C, rather than an enlargement of C as was considered in Chapter 4. Due to this, the set that this assumption needs to hold over can be any arbitrarily small open set containing C, to which E is understood to be restricted to. This assumption is particularly reasonable if the set C is compact. 233 b) ∈ To show that (5.13) is true, consider a measurement-state estimate pair (y, x b(E). A sufficient condition for (5.13) to hold is given by: v inf x∈X (y,b x) b)) ≥ 0. c(x, k(y, x (5.14) b = x + e(x), we have: Recalling that x inf x∈X (y,b x) b)) = inf c(b b)), c(x, k(y, x x − e, k(y, x e∈E(y) (5.15) b)) + inf c(b b)) − c(b b)), = c(b x, k(y, x x − e, k(y, x x, k(y, x e∈E(y) (5.16) b)) − sup |c(b b)) − c(b b))|. ≥ c(b x, k(y, x x − e, k(y, x x, k(y, x e∈E(y) (5.17) The assumption on Lipschitz continuity of Lf h, Lg h, and α ◦ h on E enables the following bound: |c(x′ , u) − c(x, u)| ≤ |Lf h(x′ ) − Lf h(x)| + |Lg h(x′ )u − Lg h(x)u| + |α(h(x′ )) − α(h(x))|, (5.18) ≤ LLf h ∥x′ − x∥ + ∥Lg h(x′ ) − Lg h(x)∥ ∥u∥ + Lα◦h ∥x′ − x∥ , ≤ (LLf h + LLg h ∥u∥ + Lα◦h ) ∥x′ − x∥ . (5.19) (5.20) Therefore, using the definition of ϵ(y) we have: b)) − c(b b))| sup |c(b x − e, k(y, x x, k(y, x e∈E(y) b)∥ + Lα◦h ) ∥e∥ , (5.21) ≤ sup (LLf h + LLg h ∥k(y, x e∈E(y) b)∥ + Lα◦h )ϵ(y). ≤ (LLf h + LLg h ∥k(y, x (5.22) Thus: inf x∈X (y,b x) b)) ≥ c(b b)) − (LLf h + LLg h ∥k(y, x b)∥ + Lα◦h )ϵ(y). c(x, k(y, x x, k(y, x (5.23) By the MR-BF condition and the design of k we have that: b)) − ϵ(y)(LLf h + Lα◦h ) − ϵ(y)LLg h ∥k(y, x b)∥ ≥ 0, c(b x, k(y, x implying the condition (5.13). (5.24) 234 As with BFs and CBFs, the notion of an MR-BF extends to an MR-CBF: Definition 41 (Measurement-Robust Control Barrier Function (MR-CBF)). Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. The function h is a Measurement-Robust Control Barrier Function (MR-CBF) for the open-loop system (2.1) on C with parameter funcb(E) → R2≥0 that is locally Lipschitz continuous on v b(E) if there tion (b, a) : v e exists α ∈ K such that: sup u∈Rm ∂h b) + a(y, x b)∥u∥) > −α(h(b (b x)(f (b x) + g(b x)u − (b(y, x x)). ∂x (5.25) b) ∈ v b(E). for all (y, x This condition is equivalently stated as: b) =⇒ Lf h(b b). ∥Lg h(b x)∥ ≤ a(y, x x) > −α(h(b x)) + b(y, x (5.26) In contrast to the implication in (2.61), we must not only consider where Lg h(b x) = 0m , but also where it is smaller than some threshold. Intuitively, this captures the fact that if we are unsure of how the input will impact the time derivative of the MR-CBF (we don’t know the direction of Lg h(b x) sufficiently well), then large inputs can be highly detrimental, and we can’t take them to achieve safety. As ϵ(y) becomes smaller, the level of robustness required by an MR-CBF approaches that of a regular CBF for the same set C, and recovers the original CBF condition with no measurement error. Given a MR-CBF h for (2.1) on C with parameter function (b, a) and a corresponding α ∈ Ke , define the pointwise set: ( ) L h(b x ) + L h(b x )u f g b) ≜ u ∈ Rm KMR-CBF (y, x , (5.27) −(b(y) + a(y)∥u∥) ≥ −α(h(b x)) b) ∈ v b(E). Given this construction, we have the following result: for (y, x Theorem 37. Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. If h is an MR-CBF for the open-loop b(E) → R2≥0 , then the set system (2.1) on C with parameter function (b, a) : v b) is non-empty for all (y, x b) ∈ v b(E), and for any controller k KMR-CBF (y, x b(E) with k(y, x b) ∈ KMR-CBF (y, x b) for that is locally Lipschitz continuous on v b) ∈ v b(E), the function h is a MR-BF for the closed-loop system (5.5) all (y, x on C with parameter function (b, a). 235 One advantage of this approach for resolving the impact of measurement model error on safety is that the constraint in (5.27) remains convex. Given a nominal controller knom : E → Rm that is locally Lipschitz continuous on E, this constraint can then be directly integrated into an optimization based controller as follows: 1 b) = argmin ∥u − knom (b x)∥2 kMR-CBF (y, x u∈Rm 2 (MR-CBF-SOCP) b) + Lg h(b b)∥u∥ ≥ −α(h(b s.t. Lf h(b x) − b(y, x x)u − a(y, x x)). This optimization problem is a second-order cone program (SOCP), with an explicit conversion to standard form provided below. As this constraint is nonsmooth, existing methods for computing closed-form solutions via Lagrangian duality and assessing local Lipschitz continuity [51] are not applicable. Future work will consider methods from variational analysis to study the Lipschitz continuity of solutions to this problem [223]. In practice, a slack variable, δ, is often added to ensure constraint feasibility. This relaxation is penalized in the cost with a large coefficient c ∈ R>0 : 1 b) = argmin ∥u − knom (b kR-MR-CBF (y, x x)∥2 + cδ 2 m 2 u∈R (R-MR-CBF-SOCP) b) + Lg h(b b)∥u∥ ≥ −α(h(b s.t. Lf h(b x) − b(y, x x)u − a(y, x x)) − δ. While this relaxed controller does not necessarily enforce the desired safety constraint, if δ remains small the impact on safety can be understood through the notion of Projection-to-State Safety (PSSf) discussed in Section 3.6. Furthermore, this relaxation ensures that the resulting controller is locally Lipb(E), and with the assumptions on p and q b, is locally schitz continuous on v Lipschitz continuous with respect to the true state x on E, as shown below. Simulation Note that the contents of this subsection were primarily written by collaborators, edited by Andrew Taylor for their appearance in journal proceedings, and edited by Andrew Taylor for their appearance in this thesis. We now present simulation results using MR-CBFs on a simulated planar Segway platform. The Segway can be seen in the left panel of Figure 5.2, and uses the model in [27] as was done in Sections 3.3, 3.5, and 3.6. The nominal controller knom is a simple PD controller as was used in Section 3.5. The 236 θ˙y 1 φ(t), MR-CBF-SOCP φ(t), CBF-QP C 0 −1 −0.50 −0.25 0.00 0.25 θy − θy∗ 0.50 Figure 5.2. (Left) The Segway model used in simulation from the perspective of the fixed virtual camera used to estimate its state. (Right) Simulation results for worst-case measurement model uncertainty of ϵ = 0.2 subtracted from the true pitch angle θy when measured. A state trajectory generated using the the (CBF-QP) controller (red) and the (MR-CBF-SOCP) controller (blue) are shown as projections onto their pitch angle and pitch rate components. The set C is plotted in green. Given the same initial condition, the (MR-CBF-SOCP) controller ensured safety of the trajectory whereas the (CBF-QP) controller did not. simulation2 was written in a ROS based environment using a mixture of C++ and Python to mimic the code structure of the existing hardware platform [27]. The set C was defined as C = {x ∈ R4 | h1 (x) ≥ 0, and h2 (x) ≥ 0} with: h1 (x) = −θ̇ + αe (c − θ + θ⋆ ) h2 (x) = θ̇ + αe (c + θ − θ⋆ ), (5.28) where θ ∈ R is the pitch angle, θ̇ ∈ R is the pitch angle rate, θ⋆ is the pitch angle at equilibrium, and c, αe ∈ R>0 . The MR-CBF constraint in (5.25) was applied simultaneously to both functions in (5.28) and was then implemented using the ECOS SOCP solver [35]. The parameter function for each MRCBF was given by parameter function (ϵ(y)(LLf h + Lα◦h ), ϵ(y)LLg h ), with the Lipschitz constants in this constraint being estimated by sampling Lf h, Lg h and α ◦ h on a set of gridded values around the system’s equilibrium point by taking the maximum of the slopes between any two adjacent grid points. For measurement error, we assumed that direct measurements of the pitch angle θ were offset by a constant factor of ε > 0, such that θb = θ − ε. Implementing the (MR-CBF-SOCP) controller with ϵ(y) = ε for all y ∈ p(E) ensures safety for this measurement error. As a baseline comparison, a (CBF-QP) controller was implemented and applied using both functions in (5.28). The effect of this type of worst-case measurement model error in the Segway system with the (CBF-QP) and (MR-CBF-SOCP) controllers can be seen in Figure 5.2. 2 Simulation code at https://github.com/rkcosner/cyberpod_sim_ros.git. 237 Conversion to Second-Order Cone Program Standard Form I now show how the optimization problem specified by (MR-CBF-SOCP) can be written in standard SOCP form. Recalling the (MR-CBF-SOCP) controller, first, the constant term can be removed from the cost such that it becomes: 1 ∥u∥2 − knom (b x)⊤ u. Additionally, the constraint can be written in terms of a 2 second order cone: " # " # b) Lg h(b x) α(h(b x)) + Lf h(b x) − b(y, x q= u+ , q ∈ Qm , (5.29) b)Im a(y, x 0m where3 Qm ≜ {(q0 , q1 ) ∈ R × Rm | q0 ≥ ∥q1 ∥} is the second-order cone in Rm+1 . Furthermore, by adding the decision variable t ∈ R, the quadratic cost function can be converted to an equivalent linear cost, t − knom (b x)T u, with a rotated second order cone constraint, ∥u∥2 ≤ 2t, that can be converted to a standard second order cone constraint via a rotation matrix R ∈ R(m+2)×(m+2) [30]. Thus the additional constraint can be written as: h i⊤ r = R t 1 u⊤ , r ∈ Qm+1 . (5.30) Combining the two second order cone constraints, the problem is rewritten as: h ih i⊤ ⊤ ⊤ b) = argmin kMR-CBF (y, x (5.31) 1 0 −knom (b x) t 1 u (u,t,s)∈R3m+4   t   b)  1  + s = h(y, x b), s.t. G(y, x u s ∈ Qm+1 × Qm , where:   R1 0 R3:m+2   b) = −  0 G(y, x 0 Lg h(b x) , 0m 0m a(y)Im   R2   b) = α(h(b h(y, x b) . x)) + Lf h(b x) − b(y, x 0m (5.32) (5.33) Here Ri represents the ith column of R, and R3:m+2 represents the columns 3 through m+2. This equivalent formulation of (MR-CBF-SOCP) is in standard SOCP form as in [35]. 3 This is a slight abuse of notation from Section 3.8 for the sake of notational convenience. 238 Lipschitz Continuity of Second-Order Cone Program Controller I now show that the controller (R-MR-CBF-SOCP) is locally Lipschitz continuous on E in terms of the true state x. Recall this controller is given by: 1 b) = argmin ∥u − knom (b kR-MR-CBF (y, x x)∥2 + cδ 2 m 2 u∈R (R-MR-CBF-SOCP) b) + Lg h(b b)∥u∥ ≥ −α(h(b s.t. Lf h(b x) − b(y, x x)u − a(y, x x)) − δ, b=q b(p(x)), with p and q b locally Lipschitz continuous where y = p(x) and x on E. Thus if we prove that kR-MR-CBF is locally Lipschitz continuous on b(E) with respects to its arguments, the composition of kR-MR-CBF with the v measurement functions will be locally Lipschitz continuous on E. Note this optimization problem is always feasible, even if h is not an MR-CBF, due to the relaxation δ, and there is a unique minimizer as the cost is strictly convex. The constraint in this problem can be restated as a conic constraint: 1 ∥u − knom (b x)∥2 + cδ 2 (R-MR-CBF-SOCP) m 2 (u,δ)∈R ×R " # b) Lg h(b x)u + δ + α(h(b x)) + Lf h(b x) − b(y, x s.t. ∈ Qm . b)u a(y, x b) = kR-MR-CBF (y, x argmin To prove local Lipschitz continuity of this controller, I will make use of Theorem 4 in [282], stated below. To draw parallels with the notation of this work, I define the following: w ≜ (u, δ) ∈ Rm+1 , 1 C(y,bx) (w) ≜ ∥u − knom (b x)∥2 + cδ 2 , 2 " # b), Lg h(b x)u + δ + α(h(b x)) + Lf h(b x) − b(y, x G(y,bx) (w) ≜ , b)u a(y, x (5.34) (5.35) (5.36) The optimization problem can then be written as: b) = argmin C(y,bx) (w) kR-MR-CBF (y, x w∈Rm+1 (5.37) s.t. G(y,bx) (w) ∈ Qm . I note that G(y,bx) : Rm+1 → Rm+1 , with Rm+1 a Banach space, and that Qm ⊂ Rm+1 is a closed, convex cone with its vertex at the origin. Furthermore, C(y,bx) and G(y,bx) are twice differentiable with respect to w, and these derivatives are b) by local Lipschitz continuity locally Lipschitz continuous with respect to (y, x 239 of knom and (b, a) on their domains and Lipschitz continuity of Lf h, Lg h, α ◦ h on E (an assumption that appears in Theorem 36). Let H(y,bx) denote the Lagrangian: H(y,bx) (w, λ) = C(y,bx) (w) − ⟨λ, G(y,bx) (w)⟩, (5.38) where λ ∈ Rm+1 . The first-order necessary conditions associated with a solu⋆ ⋆ tion (w(y,b x) ) to (5.37) can be expressed as: x) , λ(y,b ∂H(y,bx) ⋆ (w(y,bx) , λ⋆(y,bx) ) = 0m+1 , ∂w ⋆ ⟨λ⋆(y,bx) , G(y,bx) (w(y,b x) )⟩ = 0, ⋆ G(y,bx) (w(y,b x) ) ∈ Qm , (5.39) (5.40) (5.41) ⋆ m+1 where w(y,b and λ⋆(y,bx) ∈ Qm . We also have that the following x) ∈ R coercivity condition is met:   ∂H(y,bx) ⋆ ⋆ (w(y,bx) , λ(y,bx) )(v2 − v1 ), v2 − v1 ≥ ∥v2 − v1 ∥2 , (5.42) ∂w for any v1 , v2 ∈ Rm+1 as: ∂H(y,bx) ⋆ (w(y,bx) , λ⋆(y,bx) ) = Im+1 , ∂w (5.43) b) ∈ v b(E). Lastly, I note that: for all measurement-state estimate pairs (y, x " # ∂G(y,bx) ⋆ Lg h(b x) 1 (w(y,bx) ) = . (5.44) ∂w b)Im 0m a(y, x b) ̸= 0 for all (y, x b) ∈ v b(E) (or that evUnder the assumption that a(y, x ery measurement has some amount of worst-case error), we have the map ∂G(y,b x) ⋆ m+1 (w(y,b to Rm+1 . Thus we have that all of the x) ) is surjective from R ∂w conditions of the following theorem are met [282]: ∂G (y,b x) ⋆ Theorem 38. If ∂w (w(y,b x) ) is surjective and the coercivity condition (5.42) holds, then there exists s ∈ R>0 such that (5.37) has a strict local mini⋆ ′ ⋆ b′ ) ∈ Bs ((y, x b)), and both w(y,b mizer w(y,b x) for each (y , x x) , and the associated ⋆ (unique) multiplier λ(y,bx) satisfying the first-order necessary condition (5.39), b′ ) ∈ Bs ((y, x b)). are Lipschitz continuous functions of (y′ , x b) were arbitrary, and are locally Lipschitz continuous functions of the As (y, x true state x, we have that kR-MR-CBF is locally Lipschitz with respect to x. 240 Conclusion In this section I have presented an approach for safety-critical control in the presence of measurement errors through the notion of Measurement-Robust Control Barrier Functions (MR-CBFs). I begin by introducing the notion of measurement error that I consider, and propose definitions of MR-BFs and MR-CBFs that allow controllers to be synthesized that provide rigorous safety guarantees in the presence of these measurement errors. I demonstrate the proposed approach in simulation on a planar Segway system that has an imperfect measurement of its pitch angle. Lastly, I show how the MR-CBF-based controller can be specified in standard SOCP form, and prove that a relaxed version of the controller is a locally Lipschitz continuous function of the underlying state of the system. 5.3 Integration with Backup-Set Methods In this section I will present work on unifying MR-CBFs with backup set CBF methods to produce performant yet robust safe behavior in the presence of measurement errors. I will begin by briefly reviewing backup set CBF methods, followed by integrating them with MR-CBFs. Following this, I will provide demonstrations of the proposed approach in both simulation and experimentally on a planar Segway system. Experimentally, the Segway system will utilize a vision-based measurement system for describing objects in its environment, providing the not only the first experimental demonstration of MR-CBFs, but also the first experimental demonstration of MR-CBFs using vision-based measurement systems. The contributions of this section are as follows: • The integration of backup set CBF methods with Measurement-Robust CBFs as a way of achieving both performant and safe behavior in the presence of measurement errors. • An experimental demonstration of MR-CBFs on a Segway platform using an onboard vision system. The text for this section is adapted from: R. K. Cosner, A. W. Singletary, A. J. Taylor, T. G. Molnar, K. L. Bouman, and A. D. Ames, “Measurement-robust control barrier func- 241 tions: certainty in safety with uncertainty in state,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Prague, Czech Republic, 2021, pp. 6286-6291. A. J. Taylor participated in the conception of the project, algorithm design, and writing of the article. Backup-Set Control Barrier Functions I now briefly review the topic of backup set CBFs methods, which a more detailed description of can be found in [256]. Let C ⊂ E be the 0-superlevel set of a function h that is continuously differentiable on E. To guarantee that a safe control input exists, one needs to ensure that the function h satisfies the CBF condition (2.60) for the open-loop system (2.1). For a given set C, fulfilling this requirement can be nontrivial and potentially impossible (h may not be a CBF on C for the open-loop system (2.1)). To this end, I will focus on finding a set CI ⊆ C which is control invariant: Definition 42 (Control Invariance). A set CI ⊆ E is control invariant for the open-loop system (2.1) if there exists a controller k : E → Rm that is locally Lipschitz continuous on E such that the set CI is forward invariant for the closed-loop system (2.2). A control invariant set CI is a set that can be kept forward invariant, and thus makes logical sense as a set we would wish to find a CBF for (we will never find one for a set that is not control invariant)4 . If a control invariant set CI can be found such that CI ⊆ C, then for initial conditions in CI , it is possible to keep the system state within the set C (as they can be kept in CI ). Thus, while C itself will not necessarily be forward invariant (considering initial conditions x0 ∈ C \ CI ), it will be possible to satisfy φ(t) ∈ C for all t ∈ I(x0 ) if x0 ∈ CI . While directly computing control invariant sets remains challenging in general, we may define one implicitly via a backup set [256]. Suppose that we desire to keep the system state within the set C, but C is not necessarily control invariant for the open-loop system (2.1). Additionally, suppose there exists a set CB ⊂ C, defined as the 0-superlevel set of a function hB : E → R that is continuously 4 Throughout this thesis, when taking a CBF h on a set C for the open-loop system (2.1), I am implicitly assuming that C is control invariant. In this way, backup-set CBF methods can offer a strong starting point for getting to a CBF that can later be robustified, as is done in this section. 242 differentiable on E and which is known a priori to be control invariant for the open-loop system (2.1) and can be rendered forward invariant for all time (I(x0 ) = R≥0 for all x0 ∈ CB ) by a known backup controller kB : E → Rm that is locally Lipschitz continuous on E. I refer to CB as the backup set. For simple backup controllers (such as linear controllers designed for the linearization of a system) it is possible to find analytical expressions for local regions of attraction to serve as backup sets. Alternatively, numerical tools such as SOS may be used to synthesize control invariant sets [283]. I extend the backup set to a larger control invariant set CI ⊂ E, satisfying CB ⊆ CI ⊆ C, by considering the backup trajectory over a finite and fixed time T ∈ R>0 as follows. Assume that for any x ∈ E there exists a unique solution φB : [0, T ] → E satisfying: φ̇B (τ ) = f (φ(τ )) + g(φ(τ ))kB (φ(τ )), τ ∈ [0, T ], φB (0) = x. (5.45) (5.46) The solution φB may be interpreted as the evolution of the system over the interval [0, T ] from a state, x, under the backup controller kB . I denote ϕB τ (x) ≜ φB (τ ) for the initial condition x and τ ∈ [0, T ]. Using this notation, I define the set CI ⊆ C as:  CI = x ∈ C B h(ϕB τ (x)) ≥ 0 ∀τ ∈ [0, T ] and hB (ϕT (x)) ≥ 0 . (5.47) The first inequality implies safety under the backup policy (ϕB τ (x) ∈ C for all τ ∈ [0, T ]), and the second inequality implies the backup trajectory reaches CB by time T (ϕB T (x) ∈ CB ). The set CI is thus control invariant as there exists at least one controller that is locally Lipschitz continuous on E, kB , which renders it forward invariant5 . While CI is not necessarily the largest control invariant subset of C due to its implicit dependence on CB (see viability kernel, [284]), backup-sets provide a computationally tractable method for finding an under-approximation of the largest control invariant set. This is a non-obvious fact. Note that for any state x ∈ CB , we have ϕB τ (x) ∈ CB for all τ ∈ [0, T ] by our assumption that kB renders CB forward invariant for all time. As CB ⊂ C, we have that both inequalities in (5.47) are satisfied, and thus CB ⊂ CI . Additionally, the first inequality defining CI requires that x ∈ CI =⇒ x ∈ C, such that CI ⊆ C. Now suppose ′ that x ∈ CI , and consider an x′ ∈ C such that x′ = ϕB τ ′ (x) for some τ ∈ [0, T ]. If CI is forward invariant using the backup controller kB , then we must have x′ ∈ CI . To see ′ ′ B ′ this, observe that ϕB τ (x ) ∈ C for all τ ∈ [0, T − τ ] and ϕT −τ ′ (x ) ∈ CB because x ∈ CI . Moreover, by the forward invariance of CB using the backup controller kB , we have that ′ ′ ′ ϕB τ (x ) ∈ CB ⊂ C for all τ ∈ [T − τ , T ], and thus x ∈ CI . 5 243 For notational simplicity, define the continuously differentiable functions hτ : E → R and hB : E → R as: hτ (x) ≜ h(ϕB τ (x)), (5.48) hB (x) ≜ hB (ϕB T (x)). Given these definitions, the CBF condition (2.60) can then be specified as:6 : sup h˙ τ (x, u) ≜ sup Lf hτ (x) + Lg hτ (x)u ≥ −α(hτ (x)), u∈Rm ∀τ ∈ [0, T ], u∈Rm (5.49) sup h˙ B (x, u) ≜ sup Lf hB (x) + Lg hB (x)u ≥ −α(hB (x)), u∈Rm (5.50) u∈Rm for7 α ∈ Ke . Any controller k : E → Rm that is locally Lipschitz continuous on E that satisfies both of the constraints (5.49) and (5.50) for all x ∈ E (importantly, not just the backup controller kB ) will render the closed-loop system (2.2) safe with respect to CI [13, p. 6]. I note that enforcing the constraint (5.49) is not necessarily tractable as it must hold for all τ ∈ [0, T ]. To resolve this, it can be reduced to a finite collection of more conservative constraints through constraint tightening. Given a nominal controller knom : E → Rm which is locally Lipschitz continuous on E, a controller which implements the finite number of tightened constraints, and thus renders (2.2) safe with respect to CI , is given by the following quadratic program-based controller: kBS (x) = argmin u∈Rm 1 ∥u − knom (x)∥2 2 (BS-QP) s.t. Lf hτj (x) + Lg hτj (x)u ≥ −α(hτj (x) − µ), Lf hB (x) + Lg hB (x)u ≥ −α(hB (x)), for all τj ∈ {0, ∆t , . . . , T }, where ∆t ∈ R>0 is a time-step such that T /∆t ∈ Z≥0 and µ ∈ R>0 satisfies: µ≥ 6 ∆t Lh sup ∥f (x) + g(x)kB (x)∥, 2 x∈E (5.51) ∂ϕB Taking the time derivative of the function hτ and hB requires evaluating ∂xτ (x), which is a first-order approximation of the change in the value of φB (τ ) if the initial condition x is changed. This evaluation requires integrating a sensitivity matrix as explained in [256, Section B.2]. Effort to make this computation easier through learning was proposed in [160]. 7 Note that α need not be the same between both constraints. Further, observe that just because kB keeps CI forward invariant, it will not necessarily satisfy these barrier constraints for a fixed α. In practice, this is usually overcome by parameterizing the function α with a linear constant (α(r) = kr for k ∈ R>0 ) and allowing the constant k to be a decision variable in the resulting optimization-based controller [256, Section 2.C]. 244 with Lh ∈ R>0 a Lipschitz constant8 for h on E [256, Theorem 1]. Backup Set & Measurement-Robust Control Barrier Functions In this section I will unify the backup set CBF method with MR-CBFs. I will assume that the functions Lf hτj , Lg hτj , Lf hB , Lg hB , hτj , hB , and α are Lipschitz continuous on E with known Lipschitz constants LLf hτ , LLg hτ , j j LLf hB , LLg hB , Lhτ , LhB , Lα ∈ R≥0 . Using the MR-CBF condition (5.25) the j finite set of constraints in the BS-QP become: b) + aτj (y, x b)∥u∥) ≥ −α(hτj (b Lf hτj (b x) + Lg hτj (b x)u − (bτj (y, x x) − µ), (5.52) b) + aB (y, x b)∥u∥) ≥ −α(hB (b Lf hB (b x) + Lg hB (b x)u − (bB (y, x x)), (5.53) with parameter functions: b) = (LLf hτ + Lα Lhτ )ϵ(y), bτj (y, x (5.54) b) = LLg hτ ϵ(y), aτj (y, x (5.55) b) = (LLf hB + Lα LhB )ϵ(y), bB (y, x (5.56) b) = LLg hB ϵ(y), aB (y, x (5.57) j j j for all τj ∈ {0, ∆t , . . . , T }, with ϵ : p(E) → R≥0 satisfying the relationship in (5.11) for the pointwise error set E(y). These constraints can be used to define the following set: b) = {u ∈ Rm | u satisfies (5.52) and (5.53)} . KBS-MR-CBF (y, x (5.58) These constructions enable the following definition: Definition 43 (Measurement-Robust Implicit Safe Set). The set CI ⊆ C ⊆ E defined as in (5.47) is a Measurement-Robust Implicit Safe Set (MRISS) for the open-loop system (2.1), error function ϵ : p(E) → R≥0 , and parameter b(E) → R≥0 if: functions a0 , b0 , . . . , aT , bT , aB , bB : v • the constant µ ∈ R≥0 satisfies (5.51), 8 As in Section 5.2, we are interested in keeping the system state inside C, and thus this constant and norm bound only needs to hold over some open set E that contains C, thus allowing us to restrict our attention to a potentially small set E containing C, especially if C is compact. 245 b) is non-empty for each measurement-state esti• The set KBS-MR-CBF (y, x b) ∈ v b(E). mate pair (y, x Next, using this definition, I show that the safety of such sets can be made robust to measurement model error. Theorem 39. Given a MRISS CI for the open-loop system (2.1), error function ϵ, and parameter functions a0 , b0 , . . . , aT , bT , aB , bB , if a controller k : b(E) → Rm is locally Lipschitz continuous on v b(E) and satisfies k(x) ∈ v b) for all (y, x b) ∈ v b(E), then the closed-loop system (5.5) is KBS-MR-CBF (y, x safe with with respect to CI . The proof of this follows the proof of Theorem 36 in Section 5.2 and is thus not provided. This result allows an alternative to the BS-QP controller which adds the measurement-robustness of MR-CBFs. The constraints (5.52) and (5.53) can be directly integrated into a Measurement-Robust Backup Set SecondOrder Cone Program controller MR-BS-SOCP as: b) = argmin k(y, x u∈Rm 1 ∥u − knom (b x)∥2 2 (MR-BS-SOCP) s.t. Lf hτj (b x) + Lg hτj (b x)u b) + aτj (y, x b)∥u∥) ≥ −α(hτj (b − (bτj (y, x x) − µ), x) + Lg hB (b x)u Lf hB (b b) + aB (y, x b)∥u∥) ≥ −α(hB (b − (bB (y, x x)), for all τj ∈ {0, ∆t , . . . , T }. Since this controller is a second-order cone program (SOCP), there exist a variety of solvers capable of implementing it including ECOS [35] and MOSEK [36]. Notably, the conservative nature of the method scales with the bound on the measurement-model error ϵ(y) and the MRBS-SOCP reduces to the BS-QP when ϵ(y) = 0 for all y ∈ p(E). I remark b) ∈ v b(E) can be ensured by that the feasibility of MR-BS-SOCP for all (y, x adding a slack variable to the optimization problem, the impact of which can be understood via the concept of PSSf in Section 3.6. Simulation & Experimental Results Note that the contents of this subsection were primarily written by collaborators, edited by Andrew Taylor for their appearance in journal proceedings, and edited by Andrew Taylor for their appearance in this thesis. 246 φ(t) φ(t) φ(t) φ(t) φ(t) φ(t) φ(t) Figure 5.3. Simulation results for a measurement model of x b = x − 0.4 [m] and constant desired velocity of 1 [m/s]. (Left) Trajectories generated using the (BS-QP) controller. Solid line represents the true state, dashed line shows the estimated state, and green region indicates the safe set C. The true trajectory fails to be safe and exits the safe set at t = 3 [s]. (Right) Trajectories generated using the (MR-BS-SOCP) controller. An additional robustness region is plotted in blue to indicate the set of of true states which the control input renders safe. Both the true and measured trajectories are safe demonstrating the robustness of the (MR-BS-SOCP) controller when compared to the (BS-QP) controller. We now demonstrate the efficacy of the proposed (MR-BS-SOCP) controller on Segway platform from Section 5.2 in both simulation and experimentally. The backup set method for generating control invariant sets is particularly relevant for this system due to its non-minimum phase dynamics. The set C was chosen to be the set of states with position less than 2 [m] from the origin, i.e. C = {x ∈ R4 | x ≤ 2} and h(x) = 2 − x. The backup controller was an LQR controller on the linearized system dynamics and the backup set was an estimate of the region of attraction of the LQR controller to the upright equilibrium state, given by a quadratic Lyapunov function. This set is then translated to match the current position of the Segway, while not allowing it to exceed the set boundary. The functions hτ , τ ∈ [0, T ] were converted into four CBFs hτj . Lastly, the Lipschitz constants for hτj were found explicitly by inspection of the Segway dynamics and the Lipschitz constants for hB were estimated by sampling the state space in simulation and taking the largest numerical gradient. The (MR-BS-SOCP) controller was first validated in simulation in a ROSbased environment9 . Measurement error was achieved by adding a constant error of −0.4 [m] to the true state, such that e(y) = −0.4 [m] for all y ∈ p(E). 9 Simulation code at github.com/rkcosner/mrcbf_IROS21.git. 247 The simple test scenario involved driving the Segway forward with a constant desired velocity of 1 [m/s]. As seen in Figure 5.3, the MR-BS-SOCP controller provided robustness to this error. Importantly, without measurementrobustness, the system would be unsafe due to error in the state estimate. The MR-BS-SOCP controller was then implemented on hardware. State estimates for ẋ, θ, and θ̇ were found using wheel incremental encoders and a VectorNav VN-100 Inertial Measurement Unit (IMU). The position estimate for x was obtained from an Intel RealSense T265 onboard camera running proprietary Visual Inertial Odometry (VIO) based SLAM. Onboard computation was performed by a Jetson TX2 which computes control inputs and relays them to the low-level motor controllers. The TX2 concurrently runs Linux with ROS, enabling external communication and logging, and the ERIKA3 real-time operating system, which enables real-time low-level communication and computation of the control input. As the (ẋ, θ, θ̇) state estimates provided by the encoders and IMU are highly accurate, the is focus on making the system robust to measurement error in its vision-based position estimate x b. An OptiTrack motion capture system was used in laboratory experiments to provide x estimates which are considered true. These closely matched the encoder position estimates for short trials, so the encoder x estimates were considered true in the outdoor experiments. This data was used to determine the error bound ϵ(y) that appears in the MR-CBF constraint when using the onboard camera. The function ϵ(y) = 0.4 for all y ∈ p(E) was chosen as an upper bound on the measurement error. The (MR-BS-SOCP) controller was implemented at the embedded level in the ERIKA3 operating system using the ECOS SOCP solver [35]. The nominal controller knom was a PD controller tracking user velocity inputs. The backup trajectory ϕB x) and its partial derivatives were approximated τ (b via Euler integration using a time step of ∆t = 5 [ms] and the time used to expand the backup set CB to CI was T = 1 [s]. The MR-BS-SOCP controller ran at 250 [Hz] with 5 decision variables, 4 linear constraints, and 6 second order cone constraints and saturated at ±20 [N·m]. To demonstrate the method, a simple scenario is executed on the Segway in which it is driven forward at a desired velocity of 1 [m/s]. This scenario is performed with both the BS-QP and MR-BS-SOCP controllers. The results of 248 φ(t) φ(t) φ(t) φ(t) φ(t) Figure 5.4. Experimental results using SLAM from the onboard Intel RealSense T265 and constant desired velocity of 1 m/s. The notation and color schemes are the same as in Figure (5.3). (Left) Trajectories generated using the (BS-QP) controller. The true trajectory exits the safe set at t = 6.7 [s]. The measurement error is plotted in blue. (Right) Trajectories generated using the (MR-BS-SOCP) controller. Both the true and measured trajectories are safe demonstrating the robustness of the (MR-BS-SOCP) controller compared to the (BS-QP) controller. Figure 5.5. Images from the experiment using the (MR-BS-SOCP) controller. The Segway is piloted towards a wall of yellow boxes and the controller ensures that it remains safe, i.e. that it does not crash into the boxes. (Top) Time lapse of the Segway trajectory. (Bottom) Camera images taken from the perspective of the Segway throughout the experiment. The images are displayed in chronological order from left to right. A video can be found at https://vimeo.com/520247516. these experiments can be found in Figure 5.4, and images from the experiment b can be seen in Figure 5.5. With the BS-QP controller the estimated state x remains safe, but the true state x becomes unsafe whereas with the MR-BSSOCP controller both the estimated and the true state are kept safe. This highlights the importance of providing robustness against measurement error, as achieved by Theorem 39. 249 Conclusion In conclusion, in this section I have presented work on integrating backup set CBF methods with MR-CBFs. I begin by reviewing backup set methods. Following this, I show how the modifications to CBF constraints that occur when forming MR-CBFs can be translated to CBFs synthesized using backup set methods. Following this, I demonstrate the proposed combination in simulation and experimentally on a Segway system using a vision system to estimate its position relative to obstacles. 5.4 Preference-Based Learning for Robust Controller Tuning In this section I present work on utilizing preference-based learning (PBL) to tune the robustness parameters of a CBF-based controller to achieve both robust safety and performant behavior. I begin by reviewing PBL, including an algorithm developed in this work (not my explicit contribution) for performing PBL in a safety-aware way. Next, I consider the setting of designing safety-critical controllers with reduced-order models and measurement error, producing a controller that unifies the concepts of ISSf-CBFs and MR-CBFs to provide a wide range of robustness properties. As worst-case robustness parameters for this controller can yield excessively conservative behavior, PBL is used to adjust the parameters according to a user’s feedback on performance and safety. Following this, I provide demonstrations of the proposed approach in both simulation and experimentally on a quadrupedal system using a visionbased measurement system for obstacle detection. The contributions of this section are as follows: • The utilization preference-based learning (PBL) as a tool for tuning the robustness parameters of a CBF-based controller to achieve both robust safety and performant behavior. • A rigorous unification of reduced-order CBFs methods with ISSf and MRCBFs to produce a convex optimization-based controller that provides robustness to a wide-range of challenges addressed in this thesis. The text for this section is adapted from: R. K. Cosner, M. Tucker, A. J. Taylor, et al., “Safety-aware preference based learning for safety-critical control,” in Proc. 4th Learning for Dy- 250 Figure 5.6. An overview of the Safety-Aware Preference-Based Learning design paradigm. Safety-Aware LineCoSpar is used to generate actions which are rolled out in experiments as parameters of the CBF-based safety filter to obtain user preferences and safety ordinal labels which are then used to update the user’s estimated utility and generate new actions. namics and Control (L4DC), vol. 168, Stanford, CA, USA, 2022, pp. 1020-1033. A. J. Taylor participated in the conception of the project, theoretical analysis, and writing of the article. Safety-Aware Preference-Based Learning Note that the contents of this subsection were primarily written by collaborators, edited by Andrew Taylor for their appearance in journal proceedings, and edited by Andrew Taylor for their appearance in this thesis. I now present Safety-Aware LineCoSpar, as outlined in Algorithm 4. I note that this algorithm is not explicitly a contribution of mine (Andrew Taylor) in this work, but introducing it will provide background information on PBL that will support the later developments in the context of tuning robust safetycritical controllers. This is a modification of the LineCoSpar algorithm [77], which iteratively selects actions to query user for subjective feedback and updates its belief of the user’s underlying utility function via Bayesian inference. Let a denote an action, such as a collection of l parameters used in a controller, that takes values in a finite search space A ⊂ Rl . We assume that each action a ∈ A has an unknown utility to the user, defined by a function r : A → R. These utilities are given by rA = [r(a1 ), . . . , r(a|A| )]⊤ ∈ R|A| . In each iteration, s ∈ Z>0 actions are sampled from A and executed. Then, the user is queried for two forms of feedback: pairwise preferences and ordinal labels, describing performance and safety, respectively, which is collected into a dataset D. Since collecting an exhaustive dataset to estimate the unknown utility rA is ex- 251 pensive for non-trivial action spaces, Bayesian optimization (BO), a sampling efficient paradigm, is used to identify the optimizer. In BO, rA is modeled as a Gaussian process with prior N (0|A| , Σpr ), where each element of the covariance |A|×|A| matrix Σpr ∈ S≻0 is computed as Σpr ij = k(ai , aj ) with a kernel function k : A × A → R and ai ∈ A denoting the ith action in A. The function k is selected to be the squared exponential kernel, yielding a prior given by the multivariate Gaussian:   1 1 ⊤ pr −1 P(rA ) = (5.59) exp − rA (Σ ) rA . (2π)|A|/2 |Σpr |1/2 2 Given a dataset D, the posterior is proportional to the likelihood and the prior by Bayes’ theorem, i.e., P(rA | D) ∝ P(D | rA )P(rA ). Denote the maximum a posteriori (MAP) estimate of the posterior by r̂A ∈ R|A| , which is defined as r̂A ≜ argmaxrA ∈R|A| P(rA | D), noting that r̂A is equivalent to the minimizer of S(rA ) = − ln(P(D | rA )) + 21 rTA (Σpr )−1 rA . As is common in BO, the posterior is modeled as a multivariate Gaussian centered at r̂A with |A|×|A| ∂2S −1 [75]. Additionally, the covariance ΣA ∈ S≻0 defined10 as ΣA = ( ∂r 2 (r̂A )) A tractability of calculating r̂A can be improved by reducing the action space A to a subset S ⊂ A, forming a partial characterization of the utilities denoted by P(rS | D) ≈ N (r̂S , ΣS ), with rS , r̂S ∈ R|S| . A pairwise preference is defined as a relation between two actions a1 , a2 ∈ A, where a1 ≻ a2 if action a1 is preferred to a2 . Since user preferences are expected to be corrupted by noise, individual pairwise preferences are modeled via a likelihood function:   r(a1 ) − r(a2 ) P(a1 ≻ a2 | r(a1 ), r(a2 )) = gp , (5.60) cp where gp : R → [0, 1] is any monotonically-increasing link function, and cp ∈ R>0 accounts for preference noise. The function gp is chosen to be the sigmoid function, i.e., gp (r) = 1/(1 + e−r ). Assuming conditional independence, the likelihood function for a collection of K ∈ Z>0 preferences, Dp , can be modeled as the product of each individual preference likelihood: P(Dp | r(a11 ), r(a12 ), · · · , r(aK2 )) = K Y P(ak1 ≻ ak2 | r(ak1 ), r(ak2 )), (5.61) k=1 10 This is known as the Laplace approximation of the distribution P(rA | D), i.e., P(rA | D) ≈ N (r̂A , ΣA ). 252 where ak1 , ak2 ∈ A are the preferred and non-preferred actions, respectively, in the k th preference. The action space is partitioned into “unsafe” and “safe” actions by leveraging the ordinal nature of these definitions (i.e., unsafe actions are always considered worse than safe actions). A user provides this feedback as an ordinal label, which assigns an action to a discrete ordered category such as “bad” and “good” [285]. While ordinal labels can be generalized to any number of ordinal categories [78], only two categories are used to represent “unsafe” and “safe”. In this case, the action space is decomposed into two disjoint sets, A = O1 ∪ O2 , with a ∈ O1 if r(a) < β and a ∈ O2 if r(a) ≥ β, with the ordinal threshold β ∈ R. As with preferences, it is assumed that ordinal label feedback is corrupted by noise and is modeled as:   β − r(a) , (5.62) P(a ∈ O1 | r(a)) = go co   β − r(a) P(a ∈ O2 | r(a)) = 1 − go , (5.63) co where go : R → [0, 1] is any monotonically-increasing link function and co quantifies the noise in the ordinal label feedback. Again, go is chosen to be the sigmoid function go (r) = 1/(1 + e−r ). Assuming conditional independence of ordinal label queries, the likelihood function for a collection of M ∈ Z>0 ordinal labels, Do , can be modeled as the product of individual ordinal likelihoods: P(Do | r(a1 ), · · · , r(ak )) = M Y
P ak ∈ Oo(k) | r(ak ) , (5.64) k=1 where ak ∈ A refers to the action corresponding to the k th ordinal label, o(k) ∈ {1, 2}. For the simulation and experimental results in this section, the hyperparameters cp , co , β are determined in advance. Lastly, assuming conditional independence of the feedback mechanisms, the combined likelihood function is calculated as the product of the individual likelihoods, P(D | r) = P(Dp | r)P(Do | r). In the first iteration (i = 1), s ∈ Z>0 actions are sampled randomly from A, (1) (s) recorded as the set of visited actions V1 = {a1 , . . . , a1 }, executed on the system, and the preferences and ordinal labels are collected into a dataset D1 . In each subsequent iteration (i > 1), s new actions are sampled using Thompson sampling, which is shown to have desirable regret minimization 253 Algorithm 4 Safety-Aware LineCoSpar (SA-LineCoSpar) Require: s uniform random actions V1 ⊂ A, corresponding feedback D1 , for i = 2, . . . , N do P(r̂A | D) ← Vi−1 ▷ Update posterior ∗ âi−1 ← argmaxa∈Vi−1 r̂Vi−1 (a) ▷ Find optimal action Li ← â∗i−1 ▷ Sample random intersecting line Si = Li ∪ Vi−1 ▷ Construct subspace P(r̂Si | D) ← Si ▷ Update posterior over Si rSi | D) ▷ Determine ROI SiROI ← P(b for j = 1, . . . , s do r(j) ∼ N (r̂Si , ΣSi ) ▷ Sample utility (j) (j) ai ← argmaxa∈SiROI r ▷ Sample action from utility sample end for (1) (s) Dp , Do ← {ai , . . . ai } ▷ Collect preferences and ordinal labels (s) (1) ▷ Aggregate Actions Vi ← Vi−1 ∪ {ai , . . . ai } Di ← Di−1 ∪ Dp ∪ Do ▷ Aggregate preferences and ordinal labels end for properties [286]. Ideally, Thompson sampling draws s samples from the posterior P(rA | Di−1 ), i.e r(j) ∼ P(rA | Di−1 ) for j ∈ {1, . . . , s}, and the action (j) ai ∈ A maximizing each r(j) is selected to execute on the system. These (1) (s) sampled actions {ai , . . . , ai } are concatenated with Vi−1 to produce Vi , executed on the system, and the resulting preferences and ordinal labels are concatenated with Di−1 to produce Di . However, since it is intractable to approximate P(rA | D) for high-dimensional action spaces, a dimensionalityreduction technique introduced in [77] that instead updates the posterior over a subset Si ⊂ A is utilized. Motivated by [287], a subset of actions is constructed as Si = Li ∪ Vi−1 , where Li ⊂ A is the collection of e ∈ Z>0 actions in A closest to a randomly drawn line ℓi ⊂ Rl . This line is drawn to intersect with the believed best action, computed as â∗i−1 = argmaxa∈Vi−1 r̂Vi−1 (a) where r̂Vi−1 is the MAP estimate of the posterior P(rVi−1 | Di ). See [77] for more details. It is important to avoid unsafe actions during sequential decision making in certain applications, such as learning robotic controllers on hardware, where low-reward actions might lead to physical damage of the platform. Safe exploration algorithms [135], [279], [280] considered the setting where actions below a prespecified safety threshold are catastrophic and must be avoided at 254 Figure 5.7. A comparison of Safety-Aware LineCoSpar and standard LineCoSpar on a synthetic utility function (drawn from the Gaussian prior) averaged over 50 runs with standard error shown by the shaded region. The safety-aware criteria reduces the number of sampled unsafe actions with a minimal effect on the prediction error, defined as |â∗i − a∗ | with â∗i ≜ argmaxa r̂Si and a∗ ≜ argmaxa r(a). all cost. As this work constructs controllers that account for safety, a more optimistic learning approach called safety-aware is adopted. In this case, actions labeled by a human as “unsafe” are not catastrophic but undesirable. Thus, the algorithm avoids these actions; whereas the safe exploration algorithms guarantee that no such actions are sampled which can be sometimes exceedingly conservative in settings like that considered in this work. To achieve this safety-awareness, the approach introduced in [78] is leveraged, which uses ordinal labels to identify a region of interest (ROI) in A. In this work, the ROI is defined to be the actions labeled as “safe”. In each iteration i an estimate an ROI within the set Si is formed as: SiROI = {a ∈ Si | r̂Si (a) + λσ Si (a) > β}, (5.65) where r̂Si (a) and σ Si (a) are the posterior mean and standard deviation, respectively, evaluated at the action a ∈ Si . The variable λ ∈ R determines how conservative the algorithm would be in estimating the safety region, as illustrated in Figure 5.7. We see that lower values of λ result in fewer unsafe actions being sampled, with only a slight effect on sample-efficiency. The restriction to SiROI is added to LineCoSpar by only considering actions in SiROI 255 during Thompson sampling. This algorithm is referred to as Safety-Aware LineCoSpar (SA-LineCoSpar), and is outlined in Algorithm 4. Preference-Based Learning for Robust Controller Tuning I now propose a design paradigm that leverages SA-LineCoSpar to select parameters for a CBF-based controller that achieves performance and safety for a multi-layered control system. Drawing inspiration from [253], I note that many real-life engineering systems have high-dimensional state spaces and complex dynamics. Hence control systems are often designed as a set of interconnected subsystems, such as a low-dimensional subsystem that provides reference signals capturing safe behavior and a high-dimensional subsystem that tracks these reference signals. In particular, let nξ , mξ ∈ Z>0 , let E ≜ Ex × Eξ ⊂ Rn × Rnξ be an open set, and consider the following cascaded open-loop system: ẋ = f (x) + g(x)κ(ξ), (5.66) ξ̇ = fξ (x, ξ) + gξ (x, ξ)u, (5.67) with state (x, ξ) ∈ E, control input u ∈ Rmξ , and functions f : Ex → Rn , g : Ex → Rn×m , κ : Eξ → Rm , fξ : E → Rnξ , and gξ : E → Rnξ ×mξ assumed to be locally Lipschitz continuous on their domains. I note that the input u from (2.1) was replaced by κ(ξ) in the top-level system (5.66). These dynamics may represent Euler-Lagrange systems such as robots, where x reflects base position, ξ captures base velocities and joint positions and velocities, and the input u reflects the torques applied to the joints. Control in this setting will be based on a measurement y = p(x, ξ) ∈ p(E) and b = q b(y) ∈ Ex (I will address ξ shortly) defined using the state estimate x b(x) = (y, x b). measurement model setting in Section 5.2, noting that v Given this cascaded system, I will seek to establish the Input-to-State Safety of a set C that can be described as the 0-superlevel set of a function h : Ex → R that is continuously differentiable on Ex and only depends on the states x and not the states ξ. For example, in the context of a robotic system, this assumption is justified if safety is described as keeping the base position of the robot away from obstacles. Second, I assume there exists a controller b(E) × Eξ × Rm → Rmξ and µ ∈ R≥0 such that for any continuous, π : v 256 bounded signal s : R≥0 → Rm , the closed-loop system: ẋ = f (x) + g(x)κ(ξ), (5.68) b, ξ, s(t)), ξ̇ = fξ (x, ξ) + gξ (x, ξ)π(y, x (5.69) satisfies the following implication: ∥κ(ξ 0 ) − s(0)∥ ≤ µ =⇒ ∥κ(φξ (t)) − s(t)∥ ≤ µ, t ∈ R≥0 , (5.70) for all initial conditions (x0 , ξ 0 ) ∈ E, where φξ is the part of the corresponding solution (φx , φξ ) from the initial condition (x0 , ξ 0 ). This assumption reflects that a separate controller may be designed11 for the high-dimensional dynamics to track well-behaved reference signals synthesized via the low-dimensional b(E) → model. In particular, if a continuous measurement-based controller k : v Rm is designed for the low-dimensional system (2.1) and ∥κ(ξ 0 ) − k(b v(x0 ))∥ ≤ µ, then we have that the controller π ensures ∥κ(φξ (t)) − k(b v(φx (t)))∥ ≤ µ for t ∈ R≥0 . With this assumption in mind, we may explicitly define the signal s and study the ISSf behavior of the closed-loop system with matched disturbances: b) + d(t)), ẋ = f (x) + g(x)(k(y, x (5.71) b, ξ, k(y, x b)), ξ̇ = fξ (x, ξ) + gξ (x, ξ)π(y, x (5.72) where similar to Sections 3.4 and 3.6, the disturbance signal d : R≥0 → Rm satisfies the relationship d(t) = κ(φξ (t))−k(b v(φx (t))) for any initial condition (x0 , ξ 0 ) ∈ E, and thus must satisfy ∥d∥∞ ≤ µ. I now combine the robustness properties of MR-CBFs and ISSf-CBFs to account for measurement uncertainty and the disturbance, d, allowing us to make robust safety guarantees for the full closed-system (5.72). This is formalized in the following theorem: Theorem 40. Let C ⊂ Ex be the 0-superlevel set of a function h : Ex → R that is continuously differentiable on Ex . Suppose that the functions Lf h, Lg h, ∥Lg h∥2 , and α ◦ h are Lipschitz continuous on their domains, and assume that 11 The controller π is assumed to have perfect measurements of ξ. Note that it yields bounded tracking error (by a particular value of µ, the implication is not for all values of µ) for φξ . This bound tracking can be seen as an abstraction that captures the effect of b as well as implicit measurement error in the value of ξ being explicit measurement error in x passed to π. 257 ∥b x − x∥ ≤ ϵ for some ϵ ∈ R≥0 and all x ∈ Ex . Then there exists a, b ∈ R≥0 such that if h satisfies: sup Lf h(b x) + Lg h(b x)ν − ϕ∥Lg h(b x)∥2 − b − a∥ν∥ > −α(h(b x)), ν∈Rm (5.73) b ∈ q b(p(E)) and some a, b ∈ R≥0 satisfying a ≥ a and b ≥ b, then for all x there exists a controller k : Ex → Rm such that (5.72) is ISSf with respect to C with γ(d) = −α−1 (−d2 /(4ϕ)). Before proving this result, I note that given a nominal controller knom : Ex → Rm that is locally Lipschitz continuous on Ex , the constraint in (5.73) can be incorporated into a convex second-order cone program-based controller kTR-SOCP : Ex → Rm referred to as a Tunable Robustified Second-Order Cone Program (TR-SOCP) controller with tunable parameters α, ϕ, a, and b: (TR-SOCP) kTR-SOCP (b x) = argmin ∥ν − knom (b x)∥2 ν∈Rm s.t. Lf h(b x) + Lg h(b x)ν − ϕ∥Lg h(b x)∥2 − b − a∥ν∥ ≥ −αh(b x). Here a linear extended class-K∞ function with coefficient α ∈ R>0 is used to set up the use of PBL, but is not necessary for the following proof. Proof. I will show that the controller kTR-SOCP : Ex → Rm using values of a and b that are greater than a particular pair (a, b) implies satisfaction of the ISSf-BF constraint for matched disturbances (2.95). For this I choose: b = ϵ(LLf h + Lα◦h + Lϕ∥Lg h∥2 ), (5.74) a = ϵLLg h , (5.75) where L(·) indicates the Lipschitz coefficient of the subscripted function on its domain. Define the function c : Ex × Rm → R such that: c(x, ν) ≜ Lf h(x) + Lg h(x)ν − ϕ∥Lg h(x)∥2 + α(h(x)). (5.76) Using this definition we have for all ν ∈ Rm that: (5.77) c(x, ν) = c(b x, ν) + c(x, ν) − c(b x, ν), ≥ c(b x, ν) − ϵ(LLf h + Lα◦h + Lϕ∥Lg h∥2 ) − ϵLLg h ∥ν∥, | {z } | {z } (5.78) ≥ c(b x, ν) − b − a∥ν∥. (5.79) b a 258 Above I added zero in (5.77) and used the Lipschitz coefficients and the worstcase measurement error ϵ to achieve the bound in (5.78). Thus we have that: c(x, kTR-SOCP (b x)) ≥ c(b x, kTR-SOCP (b x)) − b − a∥kTR-SOCP (b x)∥ ≥ 0, (5.80) and thus we may follow the proof of Theorem 29 to show that C is ISSf with γ(d) = −α−1 (−d2 /(4ϕ)) where α−1 ∈ Ke . The parameter selection process of the TR-SOCP controller is particularly important, since the parameters a and b guaranteed to exist by Theorem 40 are worst-case approximations of the uncertainty generated using Lipschitz constants. Such approximations often lead to undesired conservatism and may render the system incapable of performing its goal (as seen in Figure 5.8). Thus, as illustrated in Figure 5.6, SA-LineCoSpar is used to identify userpreferred parameters of the TR-SOCP controller. This relaxes the worst-case over-approximation to experimentally realize performant and safe behavior. This design paradigm relies on the tunable construction of the TR-SOCP controller, allowing the definition of actions for SA-LineCoSpar as a = (α, ϕ, a, b). I note the construction of the TR-SOCP controller assures that unsafe actions are not necessarily catastrophic, as any α, ϕ, a, b > 0 endows the system with a non-zero degree of robustness to disturbances and measurement error. This allows using a safety-aware approach where unsafe actions are considered undesirable as opposed to a more conservative safety-critical approach to learning where unsafe actions are considered catastrophic. Simulation & Experimental Results Note that the contents of this subsection were primarily written by collaborators, edited by Andrew Taylor for their appearance in journal proceedings, and edited by Andrew Taylor for their appearance in this thesis. The proposed design paradigm is applied to a perception-based obstacle avoidance task with a Unitree A1 quadrupedal robot as seen in Figure 5.6 in simulation and on hardware for both indoor and outdoor environments. The action space A and hyperparameters of PBL are defined in Table 5.1. A unicycle 259 model is used as the low-order model (5.66):         " # ẋ 0 cos(ψ) 0         v  = + +d(t) ẏ 0 sin(ψ) 0  ,        ω  ψ̇ 0 0 1 |{z} | {z } |{z} | {z } ν ẋ f (x) (5.81) g(x) where (x, y) is the planar position of the robot, ψ is the yaw angle, v is planar velocity, and ω is the yaw rate. The nominal controller knom is given by: " # K v dg + C   , knom (x) = (5.82) (yg −y) −Kω sin(ψ) − dg where (xg , yg ) is the goal position of the robot, dg = ∥(xg − x, yg − y)∥ is the distance to the goal, and Kv , Kω , C ∈ R>0 . Obstacle avoidance is encoded via the 0-superlevel set of the function: (5.83) h(x) = dobs − robs − ζ cos(ψ − θ), where (xobs , yobs ) is the location of an obstacle, dobs = ∥(xobs − x, yobs − y)∥ and θ = arctan((yobs − y)/(xobs − x)) are the distance and angle from the obstacle, robs is the sum of the radii of the obstacle and robot, and ζ > 0 determines the effect of the heading angle on safety. The TR-SOCP controller with the nominal controller knom from (5.82) is used to drive the system. In practice, infeasibility of the (TR-SOCP) controller was assigned an ordinal label of unsafe and the inputs were saturated such that v ∈ [−0.2, 0.3] [m/s] and ω ∈ [−0.4, 0.4] [rad/s]. The velocity command v is computed at 20 [Hz] and error introduced by this sampling scheme is captured by the tracking error d(t). Tracking of v is performed by an inverse dynamics quadratic program Hyperparameter λ β Value −0.5 0 Name α ϕ b a Min. 0.5 0 0 0 Max. 5 1 1 0.05 ∆ 0.5 0.1 0.1 0.005 Table 5.1. The safety-aware hyperparameters, and action space bounds (minimum and maximum) with discretizations ∆. 260 Figure 5.8. Actions sampled during simulation in 30 iterations with 3 new actions in each iteration. The preferred action, â30 = (3, 0.6, 0.5, 0.015), is shown in black and white. A conservative action, a = (2, 0.5, 0.0651, 0.485), is indicated by the black circle, where a and b were determined by estimating the Lipschitz coefficients present in the proof of Theorem 40. The conservative action fails to progress whereas Safety-Aware LineCoSpar provides an action which successfully navigates between obstacles. (ID-QP) walking controller designed using the concepts in [251], which realizes a stable walking gait for (5.67) at 1 [kHz]. The quadruped is simulated executing the proposed controller with parameters provided by SA-LineCoSpar, and the resulting trajectories and the position of the obstacles12 are shown in Figure 5.8. Thirty iterations were ran, with 3 new actions sampled in each iteration (s = 3), to obtain user preferences and ordinal labels in between each set of actions. To simulate perception error, the measurements of the obstacles were shifted by −0.1 m in the y-direction. The parameters found with SA-LineCoSpar allow the robot to navigate between obstacles. For comparison, a conservative action is also shown, which is safe but fails to progress towards the goal. SA-LineCoSpar eliminates this conservatism with only minor safety violations and determines a parameter set which is both safe and performant. After simulation, learning was performed with hardware experiments in a lab12 The simulation uses two CBF constraints, one for each obstacle, and both are enforced in the (TR-SOCP) controller. 261 Figure 5.9. Seven additional iterations of 3 actions executed indoors. The preferred action, â∗37 = (4, 0.6, 0.4, 0), successfully traverses between the obstacles. oratory setting for 7 additional iterations until the user was satisfied with the experimental behavior. The robot and obstacle positions were estimated using Intel RealSense T265 and D415 cameras to perform SLAM and segmentation. Centroids of segmented clusters in the occupancy map were used as the measured obstacle positions. The true robot and obstacle positions were obtained for comparison using an OptiTrack motion capture system. The results of these experiments is seen in Figure 5.9. Afterwards, three iterations were conducted outdoors on grass until the user was satisfied with the experimental behavior. The resulting best trajectory can be seen in Figure 5.10. The preferred action was tested on other obstacle arrangements to confirm its generalizability. Conclusion In conclusion, in this section I have presented work on using PBL to tune the robustness parameters of CBF-based controllers. I begin by reviewing PBL, and present an algorithm (not my explicit contribution) for safety-aware PBL. Following this, I unify reduced-order CBF methods with ISSf-CBFs and MRCBFs to produced a second-order cone program-based controller that provides a wide range of robustness properties. Following this, I demonstrate the use of PBL to tune the parameters of this controller for a quadrupedal system until safety and performance are adequately balanced. 262 Figure 5.10. The preferred action, â∗40 = (5, 0.1, 0.4, 0.02), after simulation, indoor experiments, and 3 additional iterations of 3 actions in an outdoor environment is shown alongside views from the onboard camera. Video can be found at https://youtu.be/QEuwRDTG7TE 5.5 Conclusion In this chapter I have presented a collection of results on designing safetycritical controllers that are robust to measurement errors. In Section 5.2 I look at how measurement errors can arise in a control system and how they present challenges for standard CBF-based design techniques. This leads to the definition of Measurement-Robust Barrier Function (MR-BFs), which certify safety in the presence of measurement errors. These are then extended to Measurement-Robust Control Barrier Functions (MR-CBFs) which can be incorporated to controllers specified via convex second-order cone programs, one of which I explore local Lipschitz continuity properties of. Next, in Section 5.3 I unify MR-CBFs with backup set CBF methods. I begin by reviewing backup set CBF methods, and then show how they can be modified to incorporate MRCBF constraints, providing a tool for specifying measurement-robust control invariant sets. This leads to the first demonstration of an MR-CBF-based controller on an experimental Segway system using a vision-based measurement system for object detection. Importantly, this result reveals that more complex convex optimization problems can be efficiently solved to the point they can be deployed in real-time robotic applications. Lastly, in Section 5.4, I consider using preference-based learning (PBL) for tuning the parameters of a robust CBF-based controller. I provide a review of PBL, and then consider the setting of safety-critical control with a reduced-order model and measurement errors. By unifying the tools of ISSf-CBFs and MR-CBFs, I provide a parameterized robust CBF-based controller that address the reduced-order and full-order model gap utilizing ISSf guarantees that hold in the presence of measurement errors. As this robust controller using worst-case parameter estimates can be excessively conservative, PBL is deployed to adjust parameters 263 according to user feedback on performance and safety, producing a safetycritical controller that is robust to a wide set of real-world challenges while remaining performant. There are a number of potential directions for future work stemming from the contributions in this chapter. One direction would be exploring MR-CBFs in different settings of measurement error than the one considered in Section 5.2. In particular, it is often not possible to determine the entire state x from a single measurement, but rather, a time history of measurements are used b using a model-based observer, such as in [50], to produce a state estimate x [261]. Thus, quantifying the error in an observer for a nonlinear system and integrating that with MR-CBFs may expand their capabilities to a wider range of systems. Furthermore, direct measurements are almost always corrupted by noise, and observers often provide probabilistic state estimates, suggesting that extending MR-CBF guarantees to the stochastic measurement setting would more accurately describe real-world systems. The work explored in Section 4.5 is a potential path forward for this task, but will need to address how probabilistic properties of state estimates are propagated through MR-CBFbased controllers while ideally preserving convexity. Another direction would continue to explore how PBL can be further integrated into the tuning process of robust controllers. The significant body of work [76]–[80], [281] and the results in Section 5.4 suggests that PBL can successfully capture the underlying preferences of a control system designer to produce desirable closed-loop behavior. This raises the question of how can problems in control be set up parametrically so that PBL can be deployed using user feedback. One example may include parameterizing a reduced-order model for a full-order system and querying feedback from the control designer on the ability of the full-order system to track the behaviors produced by the reduced-order model (seeking to produce the constant µ in Section 5.4). A second example could include parameterizing the model error sets in Section 3.8 instead of constructing them from data, and querying control designer feedback on the performance and robustness of the controller, similar to the results in Section 5.4. 264 Chapter 6 SAMPLED-DATA & EVENT-TRIGGERED CONTROL In this chapter I will present results from my work developing methods for safety-critical control in the presence of input sampling. Very early feedback control systems, such as governors [288], employed mechanical means for implementing feedback loops, such that the implementation of a controller was intrinsically a continuous process that is accurately captured by the closed-loop model (2.2). The advent of modern compute resources has led control system design to dramatically depart from this type of implementation. Now, the majority of autonomous systems feature a host of computers sampling measurements from various analog and digital sensors, performing numericallyintensive control computations, and sending commands to actuators. Importantly, this process is not instantaneous. Rather, there is time required to sample sensors, time required to perform control computations, and time required to send an input command to actuators. During these collective times, the actuators typically continue to perform the previously commanded input, such that inputs are applied with a sample-and-hold implementation. This fundamentally differs1 from the modeling paradigm that CLFs and CBFs utilize in providing rigorous guarantees on stability and safety, and thus, can compromise said guarantees (including the ones developed in Chapters 2-5 of this thesis). Thus, it is critical that a framework for robust safety-critical control address the effects of sampling and ensure that theoretically rigorous stability and safety guarantees can be made. There are different paradigms for considering the effect of sampling on control systems, including sampled-data control [289], event-triggered control [95], [96], and time-delays in control [290]. Of these, my results will focus on sampled-data control and event-triggered control. The perhaps most natural setting for describing sample-and-hold input implementations is sampled-data control. In this setting, inputs are held constant across a fixed period of time, 1 I note that classical linear system theory focuses on the notion of bandwidth to describe the underlying frequency at which things are being performed in the control-loop. The same issues with limited bandwidth for linear systems are bound to occur for nonlinear systems as well, they are just not as easily mathematically quantified. 265 referred to as a sample period, and updated at sequential sample times. This is an accurate description for many cyber-physical systems, especially those running a real-time operating systems (RTOS), in which the control-loop is ran at a fixed sample rate. Within the sampled-data paradigm, approaches can generally be broken into three categories [289]. The first category focuses on emulation, or how a controller designed for a continuous-time system model can be modified in implementation to account for the fact it will be sampled. The second category, and the direction my work will take, is discrete-time design, which uses a discrete-time model of the sampled-data dynamics to design controllers. The last category, focuses on design that is both continuous and discrete-time in nature, but there have been few results in this direction given the complexity of the problem. In contrast, event-triggered control does not assume a fixed sample period, but assumes that the control system can monitor2 for events while holding the control input constant, and when an event occurs, update the input. While this is perhaps not as natural in the setting of robotic systems, it provides a powerful tool for efficient utilization of resources when there is a cost to changing the input, such as context switching in compute tasks [291], or firing thrusters on a spacecraft [292]. The challenge in developing rigorous event-triggered control paradigms is to ensure that the trigger-law defining events leads to some minimum amount of time between events, more commonly known as a minimum interevent time (MIET). My work in this chapter focuses on CLF and CBF-based control designs for sampled-data systems using approximate discrete-time models, as well as developing a method for safety-critical event-triggered control through ISSfCBFs. In Section 6.1 I will review related work in the area of sampled-data control with CLFs and CBFs as well as safety-critial event-triggered control. Following this, in Section 6.2 I will provide some empirical studies of the effects of sampling on the (CLF-QP) controller for different systems, highlighting its fragility as motivation for my results. I will then provide a mathematical description of sampled-data control and event-triggered control. Next, in Section 6.3, I will present work on sampled-data stabilization using CLFs. I will first review some important structural results regarding feedback linearization and sampled-data control which will underlie the subsequent theory. Then, I will 2 For most problems in which event-triggered control is of interest, events can be monitored significantly more quickly than they occur, such that event monitoring is abstracted to be in continuous-time. That being said, a complete addressing of the problem would consider the fact that event monitoring is also a process that involves sampling. 266 review approximate discrete-time models and practical stability as a goal when designing stabilizing controllers for sampled-data systems. This will lead to the main result of this work, which is a constructive approach for extending feedback linearizability of continuous-time dynamics to a set of approximate discrete-time dynamics that allows the synthesis of practically stabilizing controllers through convex optimization. I will conclude by revisiting the simulation results from Section 6.2 with this control paradigm and demonstrate a significant improvement in closed-loop stability properties. In Section 6.4 I will develop a framework for safety-critical control of sampleddata systems with CBFs and approximate discrete-time models. I will begin by considering a more general class of approximate discrete-time models that allow working with CBFs that do not possess a relative degree of one. Next, I will define practical safety and Sampled-Data Barrier Functions (SD-BFs) and Sampled-Data Control Barrier Functions (SD-CBFs), and show how SDBFs can be used to certify practical safety of a sampled-data system. I will conclude by exploring how convexity properties of a SD-CBF lead to a convex optimization-based controller that I demonstrate in simulation. Lastly, in Section 6.5, I will consider the problem of safety-critical event-triggered control using ISSf-CBFs. I will briefly review event-triggered control for stability as in [96] to motivate the approach for event-triggered control for safety. Then, I will consider how errors introduced by an event-triggered control scheme can be captured by ISSf-BFs when considering safety. I will consider a trigger-law similar to that proposed for stability in [96], and show that it fails to produce a MIET through a counterexample, highlighting the importance of how a system behaves near the boundary of a set that is to be kept forward invariant. Following this, I will modify the trigger-law slightly to achieve safetycritical event-triggered control while ensuring the existence of a MIET, which I demonstrate in simulation. Key contributions of this work are described at the beginning of each respective section. 6.1 Related Work Sampled-Data Stabilization A key overview and starting point for exploring sampled-data control for nonlinear systems is the review in [289]. My review of the literature will focus in particular on discrete-time design with approximate models, its relationship with feedback linearizability, and methods that have used CLFs. A key 267 challenge in discrete-time design for sampled-data nonlinear systems is that it is difficult to produce a computationally tractable representation of the exact discrete-time dynamics of the sampled-data system. Finding an analytic expression requires solving a general nonlinear differential equation (known to be quite a difficult task), and numerically integrating the differential equations poses computational challenges in synthesis that have been prohibitive (until perhaps recently with modern compute resources, such as with the backup set CBF methods in Section 5.3). The idea to instead use approximations of the discrete-time sampled-data dynamics was proposed early in the pursuit of controllers for nonlinear sampled-data systems [293]–[295] A key development in this direction was the sequence of work in [81], [82], [296], which built a framework which identified properties of both approximation schemes and controllers that ensure a property known as practical stability. This framework permitted simple approximations such as first-order Euler methods (which I will frequently use in this chapter), which greatly simplified the synthesis problem. This led to a wealth of work extending control techniques such as backstepping [83], [84], Lyapunov redesign [85], and finite-horizon optimizationbased schemes (such as Model Predictive Control (MPC)) [86]–[88]. One of the observations of this work was that sampled-data designs with approximate discrete-time models often significantly outperform their counterparts designed with continuous-time models, and even successfully operate at low sample rates [94]. This will be something that I observe in Sections 6.2 and 6.3 with my proposed method. I also note that the notion of practical stability was extended to corresponding ideas of ISS [297] and integral ISS [298]. Given the constructive relationship between feedback linearization and CLFs established in Chapter 2, CLF-based controllers developed for sampled-data nonlinear systems should account for the interactions between feedback linearization and sampling. The relationship between feedback linearization and sampled-data control was a deeply studied topic in the early formulation of controllers for nonlinear sampled-data systems [293], [294], [299]–[302]. A key result from this work appeared in [293]. In particular, this work identified that feedback linearizability and sampling do not commute. In particular, there were three critical observations. • Even if the continuous-time dynamics of a system are feedback linearizable, the exact discrete-time sampled dynamics are not necessarily feed- 268 back linearizable. • The feedback linearizability of approximate discrete-time dynamics for feedback linearizable continuous-time dynamics can depend on the order of the approximation used. • The feedback linearizability of approximate discrete-time dynamics for feedback linearizable continuous-time dynamics can depend on the coordinates in which the continuous-time dynamics are expressed before being approximately discretized. The second and third point of this preceding list will be important in Section 6.3, as producing CLF-based controllers will rely on the order of the approximate discrete-time dynamics considered, as well as what coordinates the discretization is performed in. I note that zero-dynamics that arise due to sampling and higher-order approximations have been studied [303]–[305], but the effects of sampling on the stability of zero-dynamics for a continuous-time system that is feedback linearizable have yet to be studied (I will consider them in Section 6.3). Lastly, sampled-data control with CLFs has been explored in [306], [307], but it did not make use of approximate discrete-time models or convex optimization as I will do in my work. My work in [115], presented in Section 6.3, will translate the pipeline from feedback linearization to the optimization-based (CLF-QP) controller in Chapter 2 to the sampled-data setting by integrating with the sampled-data stabilization framework through approximate discrete-time models proposed in [81], thereby yielding a constructive method for optimization-based stabilizing control for sampled-data systems. To do this, I will first review important tools from [81] regarding approximate discrete-time models and practical stability, and show that properties of these tools are preserved when considering coordinate transformations needed to preserve feedback linearizability between continuous-time and approximate discrete-time models. I will then show that a CLF synthesized for a feedback linearizable continuous-time system still satisfies the necessary properties to permit CLF-based control synthesis for the approximate discrete-time dynamics. Given a controller from this process, I will then show that if the zero-dynamics of the continuous-time feedback linearizable system are locally exponentially stable, that stability is preserved with sampling. Lastly, I will use the CLF to propose a novel optimization- 269 based controller specified as a convex quadratically constrained quadratic program (QCQP). I note that a convex QCQP is a more complex class of convex optimization problem than a QP, but is a sub-class of the class of second-order cone programs (SOCPs) seen throughout this thesis. Lastly, I demonstrate the significant improvement in closed-loop behavior of this controller over deploying a continuous-time (CLF-QP) controller with a sampled implementation on the problem settings considered in Section 6.2. Safety-Critical Sampled-Data Control The development of safety-critical controllers for sampled-data systems is a significantly more nascent field than the development of stabilizing controllers, in large part due only to the recent development of CBFs in [11], [62]. The first work looking at the effect of sampled-and-hold inputs through the perspective of barrier functions [308], which considered only the setting of a double integrator. Subsequently, a wide collection of work was developed for safety-critical control of sampled-data systems [19], [73], [74], [89]–[93]. Much of this work takes an emulation approach, where an additive correction term is factored in to the CBF inequality [19], [73], [90]–[93]. This correction term often relies on Lipschitz constants of the system dynamics and bounds on the control inputs, which can be difficult to produce, and over-approximations will often lead to conservative behavior that can only be resolved with exceptionally high sample rates, as studied in [90]. The work in [74] takes a computationally intensive approach to reduce conservatism by propagating sensitivity functions, which may be difficult for high-dimensional systems. Notably, none of this work considers using discrete-time designs with approximate discrete-time models. My work in [309], presented in Section 6.4, is the first to consider safety-critical control using CBFs and discrete-time design with approximate discrete-time models for nonlinear sampled-data systems. In particular, it will draw inspiration from the framework in [81] and my preceding work in Section 6.3. I will begin by establishing that the class of higher-order Runge-Kutta approximation schemes satisfy the necessary consistency properties to provide rigorous safety-critical control guarantees. Then, by considering the notions of practical stability and corresponding Lyapunov functions in [81] and discrete-time CBFs in [234], I will produce a notion of practical safety and corresponding Sampled-Data barrier functions (SD-BFs) that certify practical safety. I will then extend this to a definition of a Sampled-Data Control Barrier Function 270 (SD-CBF). To ensure that SD-CBFs can be used in convex optimization-based control synthesis, I will establish a result relating convexity properties of an SD-CBF with the order of a Runge-Kutta approximate discrete-time model to ensure that the inequality constraint that leads to practical safety is convex with respect to the input, enabling its use in convex optimization-based controllers. Lastly, in simulation I will demonstrate the efficacy of this method in ensuring practical safety of nonlinear sampled-data systems, including the ability to operate at low sample rates without being conservative. Event-Triggered Safety-Critical Control Event-triggered control provides a framework that allows the prescription of, in a principled way, when certain resources (such as actuators, sensors, access to a communication network with neighboring agents, or even a human) should be utilized in order to guarantee the satisfaction of said control objectives efficiently, The first exploration into advantages of event-triggered control was the work in [310], [311], which was later expanded to a collection of results focused on stabilization [96], [312]–[314]. Following this, a number of frameworks and generalizations have been proposed to extended the capabilities and improve the performance of event-triggered control schemes [95], [97], [315], [316]. A key observation in the event-triggered control literature is the need to ensure the existence of a minimum interevent time (MIET) [97], [317], [318], thereby preventing the accumulation of events in “Zeno-like” like behavior. One notable investigation into output-based event-triggered control observed challenges in ensuring an MIET if the entire system state did not converge to an equilibrium point [319], which is a challenge that will translate into my work on safety-critical event-triggered control. Early work in safety-critical event-triggered control considered using BarrierLyapunov functions, which unify both stability and safety into one function, and enforce forward invariance by requiring stability [320]. The first worked that looked at using BFs and CBFs in the context of event-triggered control were focused on improving the stabilizing properties of a controller, rather than enforcing the forward invariance of a particular set in the state space [89], [98]. Taking inspiration from the approach for event-triggered stabilization using ISS in [96], my work in [321], presented in Section 6.5, was the first to explicitly explore safety-critical event-triggered control (without requiring stabilization) through ISSf. My work shows that naïvely extending the ap- 271 proach in [96] does not lead to a MIET using a counterexample that does not converge to an equilibrium point, similarly to the issues seen in [319]. I instead propose an alternative triggering scheme that ensures that a system remains safe while possessing an MIET, which I both prove and show in simulation. There have been follow-on to this work that has considered relaxing assumptions on boundedness of dynamics (made locally in my work, but a strong assumption to take globally) [322], safety-critical event-triggered control with model error [323], and greedy approaches to maximize event separation [324]. Similarly, there have been a number of applications of the ideas in this work to different settings, including spacecraft orbit adjustment [292], connected automated vehicles [325], and a Stefan partial differential equation system with actuator dynamics [326]. 6.2 Sampling Paradigms In this section I will present some motivating simulation results highlighting the fragility of the (CLF-QP) controller to sampling on a set of systems, and then provide a mathematical description of the sampled-data and eventtriggered sampling schemes. I will first consider the inverted pendulum system: θ̈ = 1 g sin(θ) + u, ℓ mℓ2 (6.1) with angle θ, angle rate θ̇, gravity g, length ℓ, and mass m. The point xe = 02 is an unforced equilibrium point of this system, and the system is full-state feedback linearizable to this point. Thus, a CLF V can be synthesized as in Section 2.4. In this example, I will use a matrix Q2 = 0.99Q, where Q is used to find the matrix P via the CTLE. I will deploy the (CLF-QP) controller on this system with a sample-and-hold implementation at a variety of different sample rates, as seen in Figure 6.1. Observe that at 10 [Hz], the controller is not stabilizing. At 20 [Hz], the controller does stabilize the system, but it displays undesirable properties where the control input periodically drops to 0 before reactivating. At 50 [Hz], this controller is stabilizing and the input profile does not display undesirable behavior. This aligns with the mentality of “just run it fast enough and it will work” that controllers designed with continuous-time models are often implemented with. Now let us consider the more complicated system of a double inverted pendu- 272 10 Hz CLF-QP (State) 1.5 1.0 0.5 0.0 0.5 10 Hz CLF-QP (Input) 0.0 0.5 0 10 20 t (sec) 30 (rad) (rad / sec) 40 50 20 Hz CLF-QP (State) 1.0 1.0 0 10 20 t (sec) 30 40 50 40 50 40 50 20 Hz CLF-QP (Input) 0.00 0.25 0.5 0.50 0.0 0 10 20 t (sec) 30 (rad) (rad / sec) 40 50 50 Hz CLF-QP (State) 1.0 0.75 1.00 0 10 20 t (sec) 30 50 Hz CLF-QP (Input) 0.00 0.25 0.5 0.50 0.0 0 10 20 t (sec) 30 (rad) (rad / sec) 40 50 0.75 1.00 0 10 20 t (sec) 30 Figure 6.1. Results using sample-and-hold implementation of (CLF-QP) controller on inverted pendulum at various sample rates. lum, seen in Figure 6.2, with the dynamics: #" # " (m1 + m2 )ℓ21 + m2 ℓ22 + 2m2 ℓ1 ℓ2 cos(θ2 ) m2 ℓ22 + m2 ℓ1 ℓ2 cos(θ2 ) θ̈1 θ̈2 m2 ℓ22 m2 ℓ22 + m2 ℓ1 ℓ2 cos(θ2 ) " #" # 0 −m2 ℓ1 ℓ2 (2θ̇1 + θ̇2 ) sin(θ2 ) θ̇1 + 1 m ℓ ℓ (2θ̇1 + θ̇2 ) sin(θ2 ) − 12 m2 ℓ1 ℓ2 θ̇1 sin(θ2 ) θ̇2 2 2 1 2 " # " #" # (m1 + m2 )ℓ1 sin(θ1 ) + m2 ℓ2 sin(θ1 + θ2 ) 1 0 u1 +g = . (6.2) m2 ℓ2 sin(θ1 + θ2 ) 0 1 u2 This system has an unforced equilibrium point xe = 04 to which it is fullstate feedback linearizable. This can be used to synthesize a CLF (using Q2 = 0.99Q) and a corresponding (CLF-QP) controller. The results of deploying this controller on the system with a sample-and-hold implementation at various frequencies can be seen in Figure 6.3. Observe that this controller fails to stabilize the system at sample rates of 10 [Hz], 100 [Hz], and even at 1000 [Hz], which is likely beyond a reasonable sample rate that can be achieved on lightweight compute platforms. Moreover, the system is wholly unstable 273 u2 θ2 `2 m2 θ1 m1 u1 `1 Figure 6.2. Double Inverted Pendulum Schematic and displays very frequency high oscillations in the input that would not be possible with real actuators (such as motors that have inertia). Lastly, I will consider a planar quadrotor system, as seen in Figure 6.4. This system has dynamics given by: ft sin(θ), (6.3) m ft z̈ = −g + cos(θ), (6.4) m 1 θ̈ = τ, (6.5) J with horizontal position x, vertical position z, horizontal velocity ẋ, vertical velocity ż, roll angle θ, roll angle rate θ̇, thrust force ft , torque τ , mass m, inertia J, and gravity g. If this system is dynamically extended by treating ft and f˙t as states and the inputs to the system as f¨t and τ , the system is full-state feedback linearizable [327] to the unforced equilibrium point xe = h i⊤ 0 0 ze 0 0 0 mg 0 for any ze ∈ R>0 . This permits the constructive synthesis of a CLF. Once again synthesizing a CLF with Q2 = 0.99Q, the results of using the (CLF-QP) controller deployed with a sample-and-hold implementation can be see in Figure 6.5 Observe that at both 10 [Hz] and 100 [Hz] the controller fails to stabilize the system. At 500 [Hz] the system is stabilized, but the controller displays large, high frequency oscillations that would likely be undesirable (or impossible) on a real-world hardware platform. ẍ = These simulation results suggest that not only does a sample-and-hold implementation compromise theoretical stability guarantees, but the degree of the failures also indicate that the sample rates needed to justify the mentality of 274 10 Hz CLF-QP (State) 10.0 10 Hz CLF-QP (Input) 2 7.5 5.0 1 2.5 0.0 0 2.5 1 1 (rad) 2 (rad) 5.0 1 (rad / sec) 7.5 2 (rad / sec) 0 2 4 t (sec) 6 8 10 100 Hz CLF-QP (State) 0 4 t (sec) 6 u1 (N m) u2 (N m) 8 10 100 Hz CLF-QP (Input) 1 0 2 1 2 0 3 1 (rad) 2 (rad) 2 1 (rad / sec) 2 (rad / sec) 0 2 4 t (sec) 6 8 4 5 10 0 2 1000 Hz CLF-QP (State) 3.0 4 t (sec) 6 u1 (N m) u2 (N m) 8 10 1000 Hz CLF-QP (Input) 2.5 1 2.0 0 1.5 1 1.0 2 0.5 0.0 1 (rad) 2 (rad) 3 2 (rad / sec) 4 1 (rad / sec) 0.5 1.0 2 2 4 4 2 0 2 4 t (sec) 6 8 10 0 2 4 t (sec) 6 u1 (N m) u2 (N m) 8 10 Figure 6.3. Results using sample-and-hold implementation of (CLF-QP) controller on double inverted pendulum at various sample rates. “just run it fast enough and it will work” for these continuous-time CLF-based controllers may not be obtainable on real-world hardware platforms. In Section 6.3 I will revisit each of these examples using a controller using discrete-time approximations, and demonstrate the ability of said control approach to work at every one of the frequencies presented in these results (including 10 [Hz]). 275 z ft x θ τ m, J Figure 6.4. Planar Quadrotor Schematic This supports the idea that addressing how a controller is implemented in its design is vital for achieving desirable behavior. Sampled-Data Sampling I now describe the sampled-data control paradigm for describing systems with input sampling. Consider the open-loop system (2.1), and consider an open subset Z ⊆ E × Rm and its projection onto the state space: X ≜ {x ∈ E | ∃ u ∈ Rm s.t. (x, u) ∈ Z} ⊆ E. (6.6) I will assume that there exists a Tmax ∈ R>0 such that for every state-input pair (x, u) ∈ Z, there exists a unique continuously differentiable solution φ : [0, Tmax ] → E satisfying: φ̇(t) = f (φ(t)) + g(φ(t))u, (6.7) φ(0) = x, (6.8) for all t ∈ (0, Tmax ). Given a sample period T ∈ (0, Tmax ], a controller k : X → Rm is T -admissible if for any x ∈ X , the state-input pair (x, k(x)) satisfies (x, k(x)) ∈ Z and the corresponding solution φ satisfies φ(t) ∈ X for all t ∈ [0, T ]. This requirement on T -admissible controllers will ensure that in the sampled-data context, the closed-loop system is forward complete and its evolution may be described by iterative solutions to (6.7) and (6.8). Though verifying T -admissibility of a controller may be intractable, assuming that a controller is T -admissible and renders the set X invariant is relatively weak as X is defined to ensure the continued existence of solutions rather than reflecting a task-specific set. The preceding construction of solutions and admissible controllers describes the sampled-data control setting, in which inputs are applied to the system 276 10 Hz CLF-QP (State) 10 Hz CLF-QP (Input) 20 0 10 10 20 0 10 20 30 0 x (m) z (m) (rad) x (m / sec) z (m / sec) (rad / sec) f (N) f (N / sec) 5 30 40 50 60 10 t (sec) 15 20 0 100 Hz CLF-QP (State) 15 20 5.0 10 2.5 8 0.0 6 4 2 0 2 0 x (m) z (m) (rad) x (m / sec) z (m / sec) (rad / sec) f (N) f (N / sec) 5 2.5 5.0 7.5 10.0 10 t (sec) 15 20 500 Hz CLF-QP (State) 10 12.5 0.3 6 0.2 4 2 0 2 0 x (m) z (m) (rad) x (m / sec) z (m / sec) (rad / sec) f (N) f (N / sec) 5 0 f (N / sec2) (N m) 5 10 t (sec) 15 20 500 Hz CLF-QP (Input) 0.4 8 4 10 t (sec) 100 Hz CLF-QP (Input) 12 4 f (N / sec2) (N m) 5 0.1 0.0 0.1 0.2 10 t (sec) 15 20 0 f (N / sec2) (N m) 5 10 t (sec) 15 20 Figure 6.5. Results using sample-and-hold implementation of (CLF-QP) controller on planar quadrotor at various sample rates. with a zero-order hold over a sample period. More precisely, the set of possible sample periods is given by I = (0, Tmax ]. Given a sample period T ∈ I and a T -admissible controller k : X → Rm , we have that for any initial condition x0 ∈ X , there exists a unique solution φSD : R≥0 → X that is piecewise 277 continuously differentiable on R≥0 and satisfies: φ̇SD (t) = f (φSD (t)) + g(φSD (t))k(φSD (tk )), ∀t ∈ [tk , tk+1 ) (6.9) (6.10) φSD (0) = x0 , with sample times satisfying tk+1 − tk = T for all k ∈ Z≥0 . Given an x0 ∈ X , I will denote the solution at sample times as xk ≜ φSD (tk ) for k ∈ Z≥0 . An alternative way of representing the evolution of the system over a sample period is given by the exact map FeT : Z → Rn : FeT (x, u) = x + Z T [f (φ(τ )) + g(φ(τ ))u] dτ, (6.11) 0 for all state-input pairs (x, u) ∈ Z. For all T ∈ I such that kT is T -admissible and any initial condition x0 ∈ X , the solution φSD satisfies the recursion: xk+1 = FeT (xk , kT (xk )) ∈ X , (6.12) for all k ∈ Z≥0 . Closed-form expressions for exact maps are rarely obtainable, suggesting the use of approximations in control synthesis, which I will consider in Sections 6.3 and 6.4. I call {kT : X → Rm | T ∈ I} a family of admissible controllers if there is a T ∗ ∈ I such that for each T ∈ (0, T ∗ ), kT is T admissible. This enables the following definition: Definition 44 (Exact Family). I define the exact family of maps {FeT | T ∈ I}, and for a family of admissible controllers {kT | T ∈ I}, I define the exact family of controller-map pairs {(kT , FeT ) | T ∈ I}. Lastly, if xe ∈ X is an unforced or forced equilibrium point of the open-loop system (2.1) for an equilibrium input ue ∈ Rm such that (xe , ue ) ∈ Z, then xe is said3 to be an unforced or forced equilibrium point of the exact family of maps {FeT | T ∈ I}, noting that: FeT (xe , ue ) = xe . 3 (6.13) Throughout this work I will require that a point be an unforced or forced equilibrium point of an underlying continuous-time open-loop system to define it as an unforced or forced equilibrium point of a discrete-time family of maps. Notably, just satisfying a condition (6.13) will not define an equilibrium point for a discrete-time family of maps, as it does not say anything about equilibrium behavior in the inter-sample behavior. Rather, relationships like (6.13) are a consequence of the definition I take. 278 Given a family of admissible controllers {kT | T ∈ I}, the point xe is said to be an equilibrium point of the exact family of controller-map pairs {(kT , FeT ) | T ∈ I} if there exists a T ∗ ∈ I such that: (6.14) kT (xe ) = ue , for all T ∈ (0, T ∗ ). Event-Triggered Sampling I now describe the event-triggered control paradigm for systems with input sampling. I will assume that for every state-input pair4 (x, u) ∈ Rn ×Rm , there exists a unique continuously differentiable solution φ : R≥0 → Rn satisfying: φ̇(t) = f (φ(t)) + g(φ(t))u, (6.15) φ(0) = x, (6.16) for all t ∈ R≥0 . The assumption that solutions exist for all time may appear strong, but note that the control input is fixed, and thus this is mainly a requirement that the functions f and g won’t lead to finite escape times. Importantly, it will later permit arbitrarily large spacing between events, which is a desirable property in event-triggered control. The implementation of a controller k : Rn → Rm is done by sampling the state at sample times tk for k ∈ Z≥0 with t0 = 0 and tk ≤ tk+1 , evaluating the controller on the corresponding state x(tk ), and holding the input constant until the next sample-time. This leads to an event-triggered closed-loop system: ẋ(t) = f (x(t)) + g(x(t))k(x(tk )) ∀t ∈ [tk , tk+1 ), (6.17) where the argument t is used for x to denote that it continues to evolve continuously in time while the input k(x(tk )) is held constant. As opposed to the sampled-data sampling paradigm, the sample times at which the controller is updated are determined by a state-dependent execution rule or trigger-law: tk+1 = min{t ≥ tk | ρ(x(t), tk ) = 0}, (6.18) for a function ρ : Rn × R≥0 → R. Given a trigger-law, for any initial condition x0 ∈ Rn , there exists an interval I(x0 ) ≜ [0, tmax (x0 )), sample times tk ∈ R≥0 I work with Rn instead of an open set E ⊆ Rn with this topic for simplicity. This is not needed, but otherwise requires some additional labor maintaining function domains that is not particularly illuminating. 4 279 with t0 = 0 and tk ≤ tk+1 < tmax (x0 ) for all k ∈ Z≥0 , and a function φET : I(x0 ) → Rn that is piecewise continuously differentiable on I(x0 ) and satisfies: φ̇ET (t) = f (φET (t)) + g(φET (t))k(φET (tk )), φET (0) = x0 . ∀t ∈ [tk , tk+1 ) (6.19) (6.20) There are two interesting behaviors to observe regarding this paradigm. First, it maybe possible for tk+1 to be undefined (tk+1 = tmax = ∞) for all k greater than some k ∗ ∈ Z≥0 if ρ(φET (t)) never equals 0 for t ∈ [tk , ∞). This is actually a desirable probably in the event-triggered paradigm, as it means the control input to the system does not need to be changed (with switching assumed to have some undesirable cost). The other, more pressing behavior is if tk+1 can become arbitrarily close to tk as k → ∞, such that there is no minimum interevent time (MIET). This is described as Zeno behavior5 , and is undesirable as it requires the controller essentially be implemented continuously in time, which not only defeats the purpose of an event-triggered implementation, but is likely not possible on real-world hardware platforms. 6.3 Sampled-Data Stabilization In this section I will present work on stabilizing sampled-data control through feedback linearization and CLF-based controllers specified via convex optimization problems. I will begin by extending the sampled-data sampling paradigm in Section 6.2 to account for the coordinate transformations that occur with feedback linearization. Next I will consider approximate discretetime models, with a focus on the Euler approximation. I review an important property for relating the accuracy of approximate discrete-time models known as one-step consistency, and show how regularity properties on the dynamics and controller lead to one-step consistency. Next, I will review the notion of practical stability and asymptotic stability by equi-Lipschitz Lyapunov functions, which allows for establishing stability results for sampled-data systems based on approximate discrete-time models. Following this, in the first main contribution of this work, I show that a feedback linearizable continuous5 What I will show in the counterexample in Section 6.5 is Zeno-like behavior. Zeno behavior requires that the tk converge to a finite tmax (x0 ) as k → ∞. Instead, I will not require tk converge to a limit, but I will show that for any ϵ ∈ R>0 , there exists some k ∈ Z≥0 such that |tk+1 − tk P | < ϵ, which P is equally undesirable. The distinction is best ∞ ∞ understood by the two series k=1 k1 and k=1 k12 . The terms in each series converge to 0, but the first series does not converge (the behavior I achieve) while the second series converges (Zeno behavior). 280 time system with locally exponentially stable zero-dynamics can be rendered practically stable with a sample-data implementation of the continuous-time feedback linearizing controller. After this, I will present the second main contribution of this work by extending the results from feedback linearization to produce CLF-based controllers specified via convex optimization for sampleddata systems. I conclude by demonstrating the proposed control approach on the simulation environments in Section 6.2, demonstrating significant improvements over continuous-time counterparts. The contributions of this section are as follows: • Adaptation of the framework for stabilizing sampled-data control with approximate discrete-time models [81] to support the coordinate transformations and local stability results that are common with feedback linearization and zero-dynamics, leading to a full system stability result for a sampled-data system using feedback linearization. • A method for synthesizing CLF-based controllers specified as convex optimization problems for sampled-data systems that greatly outperforms continuous-time counterparts in simulation. The text for this section is adapted from: A. J. Taylor, V. D. Dorobantu, Y. Yue, P.Tabuada, and A. D. Ames, “Sampled-data stabilization with control lyapunov functions via quadratically constrained quadratic programs,” IEEE Control Sys. Let., vol.6, pp.680–685, 2022. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. Feedback Linearization for Sampled-Data Control I will begin by expanding the sampled-data sampling paradigm in Section 6.2 to account for the coordinate transformation that accompanies feedback linearization. First, define the following reachable set using the solution satisfying (6.7) and (6.8):  D ≜ x ∈ E | ∃(x0 , u0 ) ∈ Z, t ∈ [0, Tmax ] s.t. x = φ(t) , (6.21) 281 noting that X ⊆ D. This set can be interpreted as the region of the state space which can be reached by the continuous-time solution φ within Tmax time from an initial condition x0 ∈ X with an input u0 that satisfies (x0 , u0 ) ∈ Z. Let xe ∈ X be an unforced or forced equilibrium point of the open-loop system (2.1) such that (xe , ue ) ∈ Z, and recalling the definition of feedback linearization in Section 2.4, assume that the open-loop system (2.1) is locally feedback linearizable with respect to the equilibrium point xe with an open set D that satisfies D ⊆ D. This additional assumption that D ⊆ D is made simply so that the diffeomorphism Φ is well defined on all of the states that can be encountered when evolving from an initial condition x0 ∈ X with an input u0 satisfying (x0 , u0 ) ∈ Z, and may require considering a restricted set Z that restricts X to achieve D ⊆ D. As shown in [293], feedback linearizability of a continuous-time system does not guarantee feedback linearizability of the resulting discrete-time sampleddata system, even when using approximate discrete-time models. In particular, this property may be lost due to a change of coordinates. The preservation of this property motivates studying the evolution of the normal-form open-loop system in the sampled-data context. To this end, define the set:  Zξ = (ξ, u) ∈ Φ(X ) × Rm | (Φ−1 (ξ), u) ∈ Z , (6.22) noting that by assumption, for every state-input pair (ξ 0 , u0 ) ∈ Zξ , there exists a unique solution ψ : [0, Tmax ] → Φ(D) satisfying: ψ̇(t) = fξ (ψ(t)) + gξ (ψ(t))u0 , (6.23) ψ(0) = ξ 0 , (6.24) for all t ∈ (0, Tmax ). For T ∈ (0, Tmax ], a controller k : Φ(X ) → Rm is a T -admissible controller if the corresponding controller k′ : X → Rm given by k′ (x) = k(Φ(x)) for all x ∈ X is T -admissible. A controller kaux : Φ(X ) → Rk is a T -admissible auxiliary controller if kξfbl : Φ(X ) → Rm given by: kξfbl (ξ) = kfbl (Φ−1 (ξ), kaux (ξ)), (6.25) is a T -admissible controller. Because we are working with two different coordinate systems (state coordinates and normal-form coordinates), the evolution of the system over a sample 282 period can be described by both the exact state discrete map Fe,x T : Z → D and exact normal discrete map Fe,ξ T : Zξ → Φ(D): Z T e,x [f (φ(τ )) + g(φ(τ ))u0 ] dτ, (6.26) FT (x0 , u0 ) = x0 + 0 Z T e,ξ [fξ (ψ(τ )) + gξ (ψ(τ ))u0 ] dτ, (6.27) FT (ξ 0 , u0 ) = ξ 0 + 0 for all state-input pairs (x0 , u0 ) ∈ Z and all normal state-input pairs (ξ 0 , u0 ) ∈ Zξ . The exact maps are related by: e,x −1 Fe,ξ T (ξ 0 , u0 ) = Φ(FT (Φ (ξ 0 ), u0 )), (6.28) for all normal state-input pairs (ξ 0 , u0 ) ∈ Zξ . While an equivalence between the exact state discrete map and exact normal discrete map is achieved via the diffeomorphism Φ, it is useful to define both maps as the notion of stability I later consider for sampled-data systems is defined for a particular exact map. I call {kT : Φ(X ) → Rm | T ∈ I} a family of admissible controllers if there is an T ∗ ∈ I such that for each T ∈ (0, T ∗ ), kT is T -admissible. This enables the following definition: Definition 45 (Exact Families). I define the exact state family of maps {Fe,x | T ∈ I} and exact normal family of maps {Fe,ξ T T | T ∈ I}, and for m a family of admissible controllers {kT : Φ(X ) → R | T ∈ I}, I define the exact state family of controller-map pairs {(kT ◦ Φ, Fe,x T ) | T ∈ I} and exact normal family of controller-map pairs {(kT , Fe,ξ T ) | T ∈ I}. For all T ∈ I such that kT is T -admissible, the recursion: ξ k+1 = Fe,ξ T (ξ k , kT (ξ k )) ∈ Φ(X ), (6.29) is well-defined for all ξ 0 ∈ Φ(X ) and k ∈ Z≥0 . As before, if xe ∈ X is an unforced or forced equilibrium point of the open-loop system (2.1) for an equilibrium input ue ∈ Rm such that (xe , ue ) ∈ Z, it is said to be an unforced or forced equilibrium point of the exact state family of maps {Fe,x T | T ∈ I}. The relationship in (2.25) implies that: fξ (0n ) + gξ (0n )ue = 0n , (6.30) such that xe is an unforced or forced equilibrium point of the exact state family of maps {Fe,x T | T ∈ I} if and only if 0n is an unforced or forced equilibrium 283 point of the exact normal family of maps {Fe,ξ T | T ∈ I}, implying that: Fe,ξ T (0n , ue ) = 0n , (6.31) for all T ∈ I. Moreover, given a family of admissible controllers {kT : Φ(X ) → Rm | T ∈ I}, (2.25) also implies that the point xe is an equilibrium point of the exact state family of controller-map pairs {(kT ◦ Φ, Fe,x T ) | T ∈ I}, such that kT (Φ(xe )) = ue for all T ∈ I, if and only if the point 0n is an equilibrium point of the exact normal family of controller-map pairs {(kT , Fe,ξ T ) | T ∈ I} as kT (0n ) = ue for all T ∈ I. This is an important point as it suggests that relating stability results for the exact normal family of controller-map pairs to the exact state family of controller-map pairs will be well-posed. Approximate Discrete-Time Models In practice, closed-form expressions for the exact maps are rarely obtainable, suggesting the use of approximations in the control synthesis process. While there are many approaches to approximating this map, I will use the following approximation of the exact normal map: Definition 46 (Euler Approximation Family). For every sample period T ∈ I, n define the map Fa,ξ T : Zξ → R as: Fa,ξ T (ξ 0 , u0 ) = ξ 0 + T (fξ (ξ 0 ) + gξ (ξ 0 )u0 ), (6.32) for all (ξ 0 , u0 ) ∈ Zξ , yielding the Euler approximation family of maps {Fa,ξ T |T ∈ m I}. For a family of admissible controllers {kT : Φ(X ) → R | T ∈ I}, I define the corresponding Euler approximation family of controller-map pairs is {(kT , Fa,ξ T ) | T ∈ I}. The motivation behind this particular approximation will be preserving the strict feedback nature of the normal form [84], which I will utilize later when considering control synthesis. Observe that by (6.30), 0n is an unforced or forced equilibrium point of the exact normal family of maps {Fe,ξ T | T ∈ I} if and only if: Fa,ξ (6.33) T (0n , ue ) = 0n . Also note that if a family of admissible controllers {kT : Φ(X ) → Rm | T ∈ I} satisfies kT (0n ) = ue for all T ∈ (0, T ∗ ) for some T ∗ ∈ I, then: Fa,ξ T (0n , kT (0n )) = 0n . (6.34) 284 This is an important point as it suggests that designing for stability with the Euler approximation family of controller-map pairs in an effort to stabilize the exact normal family of controller-map pairs will be well-posed. a,z γ n−γ For T ∈ I, we can also define Fa,η such T : Z ξ → R and FT : Φ(X ) → R that for all (ξ, u) = ((η, z), u) ∈ Zξ : " # " # a,η F (ξ, u) η + T (f (ξ) + g (ξ)u) η η T Fa,ξ = . (6.35) T (ξ, u) = a,z FT (ξ) z + T ω(ξ) First, observe that using kξfbl yields: ξ Fa,η T (ξ, kfbl (ξ)) = (Iγ + T A)η + T Bkaux (ξ), (6.36) such that the Euler approximation of the output dynamics appear as a linear discrete-time system. Moreover, it is straightforward to show that (A, B) being a controllable pair implies that (Iγ +T A, T B) is a discrete-time controllable pair for all T ∈ I, such that the Euler approximation of the output dynamics can be rendered a stable discrete-time system through an appropriate choice of kaux . I note that if the approximation was taken using the state coordinates instead of the normal coordinates, this structure would not necessarily appear. This is an instance of sampling and feedback linearization failing to commute, but it also suggests that if the coordinate transformation associated with feedback linearization is performed before sampling, it is possible to retain the desirable control properties of feedback linearizability. Second, observe that there may be a T ∈ I such that the controller kT is T -admissible but the recursion ξ k+1 = Fa,ξ T (ξ k , kT (ξ k )) is not well-defined for all ξ 0 ∈ Φ(X ) and k ∈ Z≥0 . This is due to this map enabling ξ k ∈ / Φ(X ) for some k > 0. While my results do not need this recursion to be well-defined, this can be achieved by extending the domains of fξ , gξ , and kT to Rn . The following definition characterizes how accurately an approximate map captures the exact map: Definition 47 (One-Step Consistency). A family of controller-map pairs, {(kT , FT ) : T ∈ I}, is one-step consistent with the exact normal family of controller-map pairs {(kT , FTe,ξ ) | T ∈ I} if, for each compact set K ⊆ Φ(X ), there exist a function ρ ∈ K∞ and T ∗ ∈ I such that for all ξ ∈ K and T ∈ (0, T ∗ ), we have: ∥Fe,ξ T (ξ, kT (ξ)) − FT (ξ, kT (ξ))∥ ≤ T ρ(T ). (6.37) 285 The next lemma establishes that the Euler approximation family of controllermap pairs is one-step consistent with the exact normal family of controller-map pairs. I note that this result is based on [81, Lemma 1], which establishes the one-step consistency of the Euler approximation family of controller-map pairs with an exact family of controller-map pairs. The proof of my lemma will only provide conditions for which I show the sufficient conditions of [81, Lemma 1] are met. The conclusion of one-step consistency in my lemma can then be established by directly following the proof of [81, Lemma 1]. In this sense, this lemma should be understood as at most a minor contribution of my work, but important for understanding my later results. I also note that in Section 6.4 I will provide a complete proof of one-step consistency for a larger class of approximate discrete-time maps that includes the Euler approximation family which will follow the steps I have omitted here. Lemma 6. Suppose fξ and gξ are locally Lipschitz continuous on Φ(X ). Consider a family of admissible controllers {kT : Φ(X ) → Rm | T ∈ I} and suppose that for any compact set K ⊂ Φ(X ) there exist T ∗ ∈ I and a bound M ∈ R>0 such that for every sample time T ∈ (0, T ∗ ), the controller kT is bounded by M on K. Then the Euler approximation family of controller-map pairs {(kT , Fa,ξ T ) | T ∈ I} is one-step consistent with the exact normal family of controller-map pairs {(kT , Fe,ξ T ) | T ∈ I}. are assumed to be locally Lipschitz continuous I note that since f , g, Φ, and ∂Φ ∂x on at least D and X ⊆ D ⊆ D, the first condition of Lemma 6 is met. Proof. Consider a compact set K ⊂ Φ(X ) and corresponding T ∗ ∈ I and M ∈ R>0 , and fix a sample period T ∈ (0, T ∗ ). By assumption, kT is bounded on K, and since fξ and gξ are continuous, fξ and gξ are also bounded on K, implying there exists a bound M ′ ∈ R>0 with: ∥fξ (ξ 2 ) + gξ (ξ 2 )kT (ξ 1 )∥ ≤ M ′ , (6.38) for all normal states ξ 1 , ξ 2 ∈ K. As fξ and gξ are locally Lipschitz continuous on the compact set K, it follows that fξ and gξ are Lipschitz continuous on 286 K. Therefore: ∥fξ (ξ 2 ) + gξ (ξ 2 )kT (ξ 1 ) − (fξ (ξ 1 ) + g(ξ 1 )kT (ξ 1 ))∥ ≤ ∥fξ (ξ 2 ) − fξ (ξ 1 )∥ + ∥gξ (ξ 2 ) − gξ (ξ 1 )∥∥kT (ξ 1 )∥, (6.39) ≤ (Lfξ + Lgξ M )∥ξ 2 − ξ 1 ∥, (6.40) ≜ ρ(∥ξ 2 − ξ 1 ∥), (6.41) for all states ξ 1 , ξ 2 ∈ K, where Lfξ , Lgξ ∈ R>0 are Lipschitz constants for fξ and gξ , respectively, on K, and ρ ∈ K∞ satisfies ρ(r) = (Lfξ + Lgξ M )r for all r ∈ R≥0 . The remainder of the proof directly follows the proof of Lemma 1 in [81] by substituting (using the notation of [81]) X = N (K, ϵ) ⊂ Φ(X ), with proper containment implied for some ϵ ∈ R>0 as Φ(X ) is open, and substituting T1∗ = min{T ∗ , ϵ/M ′ }. Practical Stability Before defining the notion of practical stability which will be the objective of stabilizing controller design for sampled-data systems, I recall the following definition [100]: Definition 48 (Class-KL Function). Let a ∈ R>0 . A continuous function β : [0, a) × R≥0 is said to be a class-KL function (β ∈ KL) if for all s ∈ R≥0 the function β(·, s) : [0, a) → R≥0 is a class-K function, and for all r ∈ [0, a) and s1 , s2 ∈ R≥0 satisfying s1 < s2 , the function β(r, ·) : R≥0 → R≥0 satisfies β(r, s2 ) < β(r, s1 ) and lims→∞ β(r, s) = 0. A class-KL function is said to be a class-KL∞ function (β ∈ KL∞ ) if a = ∞. This allows the following definition of practical stability, defined for both the exact state discrete map and the exact normal discrete map: Definition 49 (Practical Stability). Let ζ e be an equilibrium point of a family of controller-map pairs {(kT , FT ) | T ∈ I}, let β ∈ KL∞ , and let N ⊆ Rn be an open set with ζ e ∈ N . The family of controller-map pairs {(kT , FT ) | T ∈ I} is (β, N )-practically stable with respect to ζ e if for each R ∈ R>0 , there exists a T ∗ ∈ I such that for each sample period T ∈ (0, T ∗ ), initial state ζ 0 ∈ N , and number of steps k ∈ Z≥0 , the recursion ζ k+1 = FT (ζ k , kT (ζ k )) is well-defined and: ∥ζ k − ζ e ∥ ≤ β(∥ζ 0 − ζ e ∥, kT ) + R. (6.42) 287 This definition requires that the state of the system converge to a neighborhood of the equilibrium point ζ e (standing in for both xe and 0n ) that can be made arbitrarily small (smaller values of R) by requiring potentially higher sample rates (a smaller value of T ∗ ). The following lemma relates the practical stability of the exact normal family and the exact state family of controller-map pairs. Importantly, it justifies considering the sampled normal form dynamics, which can be feedback linearized, rather than the sampled state dynamics that may not be feedback linearizable. I note that this lemma is a contribution of my work. Lemma 7. Suppose that for any compact sets K, K ′ ⊂ Rn , Φ and Φ−1 are Lipschitz continuous on K ′ ∩X and K ∩Φ(X ), respectively. If the exact normal family of controller-map pairs {(kT , Fe,ξ T ) | T ∈ I} is (β, N )-practically stable with respect to 0n for an open set N ⊆ Φ(X ) with 0n ∈ N , then there exist β ′ ∈ KL∞ and a bounded open set N ′ ⊆ X with xe ∈ N ′ such that the ′ ′ exact state family of controller-map pairs {(kT ◦ Φ, Fe,x T ) | T ∈ I} is (β , N )practically stable with respect to xe . Proof. Let N ′ ⊆ Φ−1 (N ) be a bounded open set satisfying cl(N ′ ) ⊆ X and xe ∈ N ′ , noting that such an N ′ exists as X is open and xe ∈ Φ−1 (N ) ⊆ X . As cl(N ′ ) is compact and Φ is a homeomorphism (because it is a diffeomorphism) between X and Φ(X ), we have Φ(cl(N ′ )) ⊆ Φ(X ) is compact and satisfies Φ(cl(N ′ )) = cl(Φ(N ′ )). Fix R ∈ R>0 and define the corresponding radii r, r′ ∈ R>0 as: r = max ξ0 ∈cl(Φ(N ′ )) β(∥ξ 0 ∥, 0) + R, r′ = max ′ ∥x0 ∥, x0 ∈cl(N ) (6.43) (6.44) and let Br , Br′ ⊂ Rn be closed norm-balls centered at of radius r centered at 0n and r′ centered at xe , respectively. By assumption, Φ−1 and Φ are Lipschitz continuous on Φ(X )∩Br and X ∩Br′ with Lipschitz constants LΦ−1 , LΦ ∈ R>0 , respectively. Given an arbitrary R′ ∈ R>0 , pick R < min{R, R′ /LΦ−1 }. Let x0 ∈ N ′ and let ξ 0 = Φ(x0 ) ∈ Φ(N ′ ) ⊆ N . By the (β, N )-practical stability of the exact normal family of controller-map pairs, corresponding to R, there exists a T ∗ ∈ I such that for any sample period T ∈ (0, T ∗ ), the recursion ξ k+1 = Fe,ξ T (ξ k , kT (ξ k )) ∈ Φ(X ) satisfies ∥ξ k ∥ ≤ β(∥ξ 0 ∥, kT ) + R, and implies ξ k ∈ Φ(X ) ∩ Br for all k ∈ Z≥0 . Letting xk+1 = Fe,x T (xk , kT (Φ(xk ))) for all 288 k ∈ Z≥0 , note that xk = Φ−1 (ξ k ). It follows that for all k ∈ Z≥0 : ∥xk − xe ∥ = ∥Φ−1 (ξ k ) − Φ−1 (0n )∥, (6.45) ≤ LΦ−1 ∥ξ k − 0n ∥, (6.46) ≤ LΦ−1 (β(∥ξ 0 ∥, kT ) + R), (6.47) = LΦ−1 (β(∥Φ(x0 ) − Φ(xe )∥, kT ) + R), (6.48) ≤ LΦ−1 (β(LΦ ∥x0 − xe ∥, kT ) + R), (6.49) < β ′ (∥x0 − xe ∥, kT ) + R′ , (6.50) where β ′ (r, s) = LΦ−1 β(LΦ r, s) for all r, s ∈ R≥0 is a class-KL∞ function. Note that the first inequality results from the fact we have taken R < R, such that ξ k ∈ Φ(X ) ∩ Br for all k ∈ Z≥0 , and thus the Lipschitz bound can be used, while the second inequality follows from the way r′ is defined and the fact x0 ∈ N ′ . Therefore, the exact state family of controller-map pairs is (β ′ , N ′ )-practically stable. Certifying practical stability will come through the following class of Lyapunov functions defined in [81]: Definition 50 (Asymptotic Stability by Equi-Lipschitz Lyapunov Functions). A family of controller-map pairs {(kT , FT ) | T ∈ I} is asymptotically stable by equi-Lipschitz Lyapunov functions if for some open set N ⊆ Φ(X ) with 0n ∈ N and any compact set K ⊂ N , there exist T ∗ ∈ I, comparison functions α1 , α2 ∈ K∞ and α3 ∈ K, a family {VT : Rn → R≥0 | T ∈ (0, T ∗ )}, and a Lipschitz constant M ∈ R>0 such that: α1 (∥ξ 1 ∥) ≤ VT (ξ 1 ) ≤ α2 (∥ξ 1 ∥), (6.51) VT (FT (ξ 2 , kT (ξ 2 ))) − VT (ξ 2 ) ≤ −T α3 (∥ξ 2 ∥), (6.52) |VT (ξ 3 ) − VT (ξ 4 )| ≤ M ∥ξ 3 − ξ 4 ∥, (6.53) for all ξ 1 ∈ Rn , normal states ξ 2 ∈ N and ξ 3 , ξ 4 ∈ K, and sample times T ∈ (0, T ∗ ). The first condition is the standard positive-definite condition seen with Lyapunov functions. The second condition is a discrete-time decrement condition on the value of the Lyapunov function that scales with the sample-rate T . Observe that (6.51) and (6.52) require that: FT (0n , kT (0n )) = 0n . (6.54) 289 The last condition requires that the function VT be Lipschitz continuous on any compact set K ⊆ N . Notably, the functions α1 , α2 , α3 and the constant M must hold uniformly for all T ∈ (0, T ∗ ), leading to the name equi-Lipschitz. These functions link one-step consistency to practical stability in the next lemma. I note that this lemma is a modification of [81, Theorem 2], which is the main result of that work and provides a global practical stability result. My modification focuses on making the technically necessary adjustments that allow for local stability results, as I will be later studying the local exponential stability of zero-dynamics. A full proof of this lemma would require significant repetition of the steps in [81], and thus my “proof” should be seen as a list of instructions that can be followed to modify the proof steps of [81, Theorem 2] rather than a full proof. Again, in this sense, this lemma should be seen as at most a minor contribution of my work, but it is important for justifying my later results. Lemma 8. If the corresponding Euler approximation family of controller-map pairs {(kT , Fa,ξ T ) | T ∈ I} is asymptotically stable by equi-Lipschitz Lyapunov functions, then there exist β ∈ KL∞ and a bounded open set U ⊆ Φ(X ) with 0n ∈ U such that the exact normal family of controller-map pairs {(kT , Fe,ξ T ) | T ∈ I} is (β, N )-practically stable for any open set N ⊆ U with 0n ∈ N . The importance of this lemma is that it states that controllers designed using the Euler approximation family of maps can endow the corresponding exact normal family of controller-map pairs with practical stability guarantees, and using Lemma 7, endow the exact state family of controller-map pairs with practical stability guarantees. Proof. Consider the open set N ⊆ Φ(X ) and the functions α1 , α2 ∈ K∞ and α3 ∈ K as specified by Definition 50. Let K ⊂ N be a compact set with 0n ∈ int(K). By one-step consistency and asymptotic stability by equiLipschitz Lyapunov functions, there exist a ρ ∈ K∞ , a Lipschitz constant M ∈ R≥0 , and an T0∗ ∈ I such that for all T ∈ (0, T0∗ ), (6.37), (6.51), (6.52), and (6.53) hold for all ξ, ξ 1 , . . . , ξ 4 ∈ K. There exists a radius R ∈ R>0 such that the closed norm-ball of radius R centered at 0n is contained in K. I modify the claim of [81, Equation 37] for the local setting in this work as follows: 290 Claim 1. For any d, D ∈ R>0 with D ≤ α2−1 (α1 ( R2 )) and d ≤ 2α2 (R), there exists an T ∗ ∈ (0, T0∗ ) such that for every ξ ∈ Φ(X ) and T ∈ (0, T ∗ ), if ∥ξ∥ ≤ D and max {VT (Fe,ξ T (ξ, kT (ξ))), VT (ξ)} ≥ d, then: T VT (Fe,ξ T (ξ, kT (ξ))) − VT (ξ) ≤ − α3 (∥ξ∥). 2 (6.55) In the language of [81], the restrictions on d and D imply ∆ ≤ R, and the proof follows by replacing the sets X and X1 with K, the constant M with the Lipschitz constant M given above, and the constants T1∗ and T2∗ with T0∗ . Letting U ⊂ K be the open ball of radius α2−1 (α1 (α2−1 (α1 ( R2 )))), the modified claim may be used to prove the existence of a β ∈ KL∞ such that the exact ′ normal family of controller-map pairs {(kT , Fe,ξ T ) | T ∈ I} is (β, N )-practically stable for any open set N ′ ⊆ U containing the origin by following the proof of Theorem 2 in [81]. Sampled-Data Stabilization In now present my main result establishing feedback linearization as a method for practically stabilizing sampled-data nonlinear systems. The first result builds on [328] to make a claim on the stabilizability of the output dynamics: Lemma 9. Let the open-loop system (2.1) be feedback linearizable to an unforced or forced equilibrium point xe ∈ X , consider K ∈ Rk×γ such that Acl ≜ A − BK is Hurwitz, and let Pη ∈ Sγ≻0 solve the CTLE: A⊤ cl Pη + Pη Acl = −Qη , (6.56) for some Q ∈ Sγ≻0 . Define the function Vη : Rγ → R≥0 as Vη (η) = η ⊤ Pη η for all η ∈ Rγ . For any Qη ∈ Sγ≻0 such that Qη ≺ Q and c ∈ (0, 1), there exists Tη∗ ∈ I such that for any η 0 ∈ Rγ , ξ = (η, z) ∈ Φ(X ) \ {0n }, and T ∈ (0, Tη∗ ), there exists an input u ∈ Rm such that: λmin (Pη )∥η 0 ∥2 ≤ Vη (η 0 ) ≤ λmax (Pη )∥η 0 ∥2 , Vη (Fa,η T (ξ, u)) − Vη (η) < −T cλmin (Qη )∥η∥2 . (6.57) (6.58) This result states that if the continuous-time open-loop system (2.1) is feedback linearizable to an equilibrium point xe and we pick a stabilizing linear gain K for the output dynamics, we can solve the CTLE for the output portion of the corresponding closed-loop normal-form dynamics to produce a type of 291 discrete-time CLF for the output portion of the Euler approximation family of maps. While I have yet to show that the equi-Lipschitz property is met (I will shortly), note that Vη satisfies the necessary uniformity properties in the sample period T with respect to α1 , α2 and α3 required by equi-Lipschitz Lyapunov functions. Proof. The bounds in (6.57) follow from the definition of Vη . Define the auxiliary controller kaux ((η, z)) = −Kη for all (η, z) ∈ Φ(X ). For the controller kξfbl defined in (6.25), we have: ξ Vη (Fa,η T (ξ, kfbl (ξ))) − Vη (η) = Vη ((Iγ + T Acl )η) − Vη (η), (6.59) ⊤ = T η ⊤ (A⊤ cl Pη + Pη Acl + T Acl Pη Acl )η (6.60) 2 < −T (λmin (Qη ) − T λmax (A⊤ cl Pη Acl ))∥η∥ , (6.61) for all ξ = (η, z) ∈ Φ(X ) \ {0n } and T ∈ I. Picking Tη∗ ∈ I with: Tη∗ ≤ (1 − c)λmin (Qη )/λmax (A⊤ cl Pη Acl ), (6.62) implies that for all T ∈ (0, Tη∗ ] and ξ ∈ Φ(X ) \ {0n }, the input kξfbl (ξ) satisfies (6.58). I note that the preceding result did not include 0n , as the strict inequality can not be satisfied there. Observe that using the controller kξfbl with kaux as ξ defined in the proof, (2.33) implies that Fa,η T (0n , kfbl (0n )) = 0γ . Furthermore, if xe is an unforced or forced equilibrium point of the open-loop system (2.1), the assumption that the dynamics of z do not depend on the input implies that ξ ω(0n ) = 0n−γ . Thus, with this controller, we have that Fa,ξ T (0n , kfbl (0n )) = 0n . The function Vη is said to be a discrete-time CLF for the output portion of the Euler approximation family of maps for T ∈ (0, Tη∗ ]. For each T ∈ (0, Tη∗ ), define UT : Φ(X ) → P(Rm ) as: ) ( (ξ, u) ∈ Z and ξ , UT (ξ) = u ∈ Rm 2 Vη (Fa,η T (ξ, u)) − Vη (η) ≤ −T cλmin (Qη )∥η∥ (6.63) for all ξ ∈ Φ(X ), noting that it if kaux as in the proof of Lemma 9 is T admissible for all T ∈ (0, Tη∗ ], then this set is non-empty for all T ∈ (0, Tη∗ ] 292 and ξ ∈ Φ(X ), as kξfbl (ξ) ∈ UT (ξ). The next result connects these functions to continuous-time stability of the zero-dynamics, implying the conditions of Lemma 8 are met for controllers other than feedback linearizing controllers: Theorem 41. Let Vη and Tη∗ be defined as in Lemma 9, and the zero-dynamics system governed by the differential equation: (6.64) ż = ω((0γ , z)), for zero-coordinate signal z is locally exponentially stable with respect to 0n−γ . Let {kT : Φ(X ) → Rm | T ∈ I} be a family of admissible controllers satisfying kT (ξ) ∈ UT (ξ) for all T ∈ (0, Tη∗ ] and ξ ∈ Φ(X ). Then the Euler approximation family of controller-map pairs {(kT , Fa,ξ T ) | T ∈ I} is asymptotically stable by equi-Lipschitz Lyapunov functions. Proof. The local exponential stability of (6.64) implies that for any Qz ∈ Sn≻0 and d ∈ (0, 1), there exist an open neighborhood of the origin N ⊆ Rn−γ , a Tz∗ ∈ I, a Pz ∈ Sn≻0 , and a quadratic Lyapunov function Vz : Rn−γ → R≥0 defined as Vz (z) = z⊤ Pz z for all z ∈ Rn−γ and satisfying: λmin (Pz )∥z0 ∥2 ≤ Vz (z0 ) ≤ λmax (Pz )∥z0 ∥2 , (6.65) 2 Vz (Fa,z T ((0γ , z))) − Vz (z) ≤ −T dλmin (Qz )∥z∥ , (6.66) for all z0 ∈ Rn−γ , z ∈ N , and T ∈ (0, Tz∗ ). Construction of Vz , N , and Tz∗ first follows the proof steps of [100, Theorem 4.7] with the linearization of ω(0γ , ·) at 0n−γ to produce a suitable neighborhood N and then follows the proof steps of Lemma 9 to produce Tz∗ . Let σ ∈ R>0 be a coefficient to be specified later. Let Vη , c, and Tη∗ be defined as in Lemma 4, and define the composite Lyapunov function V : Rn → R≥0 as: (6.67) V (ξ) = σVη (η) + Vz (z), for all ξ = (η, z) ∈ Rn . First, note that (6.57) and (6.65) imply: min {σλmin (Pη ), λmin (Pz )}∥ξ∥2 ≤ V (ξ) ≤ max {σλmax (Pη ), λmax (Pz )} ∥ξ∥2 , | {z } ≜µ (6.68) for all ξ ∈ Rn , implying (6.51) is met. Second, note that: ∂V (ξ) ∂ξ ≤ 2 (σλmax (Pη )∥η∥ + λmax (Pz )∥z∥) , ≤ 2(µ∥ξ∥ + µ∥ξ∥) = 4µ∥ξ∥, (6.69) 293 for all ξ = (η, z) ∈ Φ(X ), implying that for any compact set K ⊂ Φ(X ), (6.53) is met as we have:   |V (ξ 1 ) − V (ξ 2 )| ≤ 4µ max ∥ξ∥ ∥ξ 1 − ξ 2 ∥, (6.70) ξ∈K for all ξ 1 , ξ 2 ∈ K. Third, define a bounded open set Nξ ⊂ Rn with 0n ∈ Nξ and closure cl(Nξ ) ⊂ Φ(X )∩(Rγ ×N ), let Lω ∈ R>0 be a Lipschitz constant of ω on Nξ , and let T1∗ = min {Tη∗ , Tz∗ }. Note that Lω exists as ω is continuously differentiable on Φ(X ) ⊆ Φ(D), and cl(Nξ ) is a compact set contained in Φ(X ). For all ξ = (η, z) ∈ Nξ and T ∈ (0, T1∗ ), we have: V (Fa,ξ T (ξ, kT (ξ))) − V (ξ) a,z = σ(Vη (Fa,η T (ξ, kT (ξ))) − Vη (η)) + Vz (FT (ξ)) − Vz (z), (6.71) ≤ −σT cλmin (Qη )∥η∥2 + Vz (Fa,z T ((0γ , z))) − Vz (z) a,z + Vz (Fa,z T ((η, z))) − Vz (FT ((0γ , z))), (6.72) ≤ −σT cλmin (Qη )∥η∥2 − T dλmin (Qz )∥z∥2 + 2T z⊤ Pz (ω((η, z)) − ω((0γ , z))) + T 2 (ω((η, z))⊤ Pz ω((η, z)) − ω((0γ , z))⊤ Pz ω((0γ , z))), (6.73) ≤ −σT cλmin (Qη )∥η∥2 − T dλmin (Qz )∥z∥2 + 2T λmax (Pz )Lω ∥η∥∥z∥ + T 2 λmax (Pz )Lω ∥ξ∥2 , " #⊤ " #" # ∥η∥ θη (σ, T ) −θ× ∥η∥ = −T , ∥z∥ −θ× θz (T ) ∥z∥ | {z } (6.74) (6.75) ≜Θσ (T ) where θη (σ, T ) = σcλmin (Qη ) − T λmax (Pz )Lω , θ× = λmax (Pz )Lω , and θz (T ) = dλmin (Qz ) − T λmax (Pz )Lω . Observe that in going from (6.73) to (6.74) I dropped the last term (as it is negative), and used the Lipschitz property of ω to produce the term ∥ξ∥2 , which can be decoupled into the diagonal terms of the matrix Θσ (T ). Pick T2∗ ∈ (0, T1∗ ] such that T2∗ < dλmin (Qz )/θ× , implying θz (T ) > 0 for all T ∈ (0, T2∗ ), and note that θz (T2 ) ≥ θz (T1 ) for all T1 , T2 ∈ (0, T2∗ ) such that T2 < T1 . Choosing σ such that: 2 σ > (θ× /θz (T2∗ ) + T2∗ θ× )/(cλmin (Qη )), (6.76) then implies that Θσ (T ) ∈ Sn≻0 for all T ∈ [0, T2∗ ]. The composition λmin ◦ Θσ is continuous and R>0 -valued for all T ∈ [0, T2∗ ] as Θσ is an affine function, 294 and thus attains its minimum on the compact interval [0, T2∗ ], such that: α ≜ ′min ∗ λmin (Θσ (T ′ )) > 0. (6.77) 2 V (Fa,ξ T (ξ, kT (ξ))) − V (ξ) ≤ −T λmin (Θσ (T ))∥ξ∥ , (6.78) T ∈[0,T2 ] Therefore: (6.79) ≤ −T α∥ξ∥2 , for all ξ ∈ Nξ and T ∈ (0, T2∗ ]. As the controller kξfbl with kaux defined in the proof of Lemma 9 does not explicitly depend on the sample-period T , if (ξ, kξfbl (ξ)) ∈ Zξ for all ξ ∈ Φ(X ) and kξfbl is T0 -admissible for some T0 ∈ I, then it is T -admissible for all T ∈ (0, T0 ) (as (ξ, kξfbl (ξ)) ∈ Zξ for all T ∈ (0, T0 )). Consequently, this result implies that if such a T0 exists, the exact normal family of controllermap pairs {(kξfbl , Fe,ξ T ) | T ∈ I} is (β, N )-practically stable with respect to 0n for some β ∈ KL∞ and open set N ⊆ Φ(X ) with 0n ∈ N . In this way, I have shown that a feedback linearizing controller for a continuous-time system will render the corresponding sampled-data system practically stable, without modification of the feedback linearizing controller. Optimization-Based Control Synthesis The existence of the feedback linearizing controller ensures the function Vη is also a local exponential CLF for the continuous-time output dynamics and equilibrium point 0γ . For a sample period T ∈ I, continuous-time design : Φ(X ) → Rm specified as: yields the standard (CLF-QP) controller kCLF T kCLF (ξ) = argmin ∥u∥2 T u∈Rm (CLF-QP) s.t. Lfη Vη (ξ) + Lgη Vη (ξ)u ≤ −λmin (Qη )∥η∥2 , for all ξ = (η, z) ∈ Φ(X ). As shown in Section 6.2, this controller often displays degradation in performance with sampling, motivating the specification of a sampled-data controller. For T ∈ (0, Tη∗ ], using the Euler approximate SD−CLF model Fa,η : Φ(X ) → Rm specified by the T , consider a controller kT following convex quadratically constrained quadratic program (QCQP): kSD-CLF (ξ) = argmin ∥u∥2 T u∈Rm (CLF-QCQP) 2 s.t. Vη (Fa,η T (ξ, u)) − Vη (η) ≤ −T cλmin (Qη )∥η∥ , (6.80) 295 for all ξ = (η, z) ∈ Φ(X ). To see that this controller is a convex QCQP, we can expand the constraint to reveal the alternative form: (ξ) = argmin ∥u∥2 kSD-CLF T u∈Rm (CLF-QCQP) s.t. u⊤ ΛT (ξ)u + 2λT (ξ)⊤ u + lT (ξ) ≤ 0, m for all ξ = (η, z) ∈ Φ(X ), where ΛT : Φ(X ) → Sm ⪰0 , λT : Φ(X ) → R , and lT : Φ(X ) → R are defined with Pη , Qη , and c from Lemma 9: ΛT (ξ) = T gη (ξ)⊤ Pη gη (ξ), (6.81) λT (ξ) = gη (ξ)⊤ Pη (η + T fη (ξ)), (6.82) lT (ξ) = fη (ξ)⊤ Pη (2η + T fη (ξ)) + cλmin (Qη )∥η∥2 , (6.83) for all ξ = (η, z) ∈ Φ(X ). For any ξ ∈ Φ(X ), the input kξfbl (ξ) is in the feasible set of the corresponding optimization problem, and as the feasible set is closed and the ∥kξfbl (ξ)∥2 -sublevel set of the continuous objective function is compact, there exists a minimizer in this sublevel set. Since the objective function is strictly convex and the feasible set is convex, this minimizer is unique and can be found in polynomial time [35]. For each T ∈ (Tη∗ , Tmax ], define kSD−CLF : Φ(X ) → Rm arbitrarily. If {kSD−CLF | T ∈ I} is a family T T SD−CLF of admissible controllers, then the exact family {(kT , Fe,ξ T ) | T ∈ I} is (β, N )-practically stable for some β ∈ KL∞ and open set N ⊆ Φ(X ) by Theorem 41. This follows as the feasibility of the feedback linearizing control input implies the family {(kSD−CLF , Fa,ξ T T ) | T ∈ I} is asymptotically stable by the same equi-Lipschitz Lyapunov functions as the family {(kξfbl , Fa,ξ T ) | h ∈ I}. Simulation Results To illustrate the advantage of sampled-data design, I will first consider the following system with exponentially stable zero-dynamics: η̇1 = η2 , (6.84) η̇2 = 10 sin(η1 ) + u, (6.85) ż = η12 − z, (6.86) where (η1 , η2 ), z, and u denote i zero-coordinate, and control sigh the√output, nal, respectively. For K = 1/2 3/2 , Qη = I2 , c = 0.5, T = 0.2, and initial condition (1, 0, 1), the (CLF-QP) fails to stabilize the system, while the (CLF-QCQP) stabilizes the system as seen in Figure 6.6. 296 η1 CLF-QP 2.00 z η2 1.00 0.00 0.0 2.5 5.0 7.5 10.0 CLF-QCQP 1.00 12.5 15.0 η1 0.75 17.5 20.0 z η2 0.50 0.25 0.00 −0.25 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 t Figure 6.6. With inputs applied via a zero-order hold, the (CLF-QP) controller does not stabilize the system (6.84) (Top), while the (CLF-QCQP) does (Bottom). Simulation code listed at https://bit.ly/CLF-QCQP. Next, I deploy the proposed (CLF-QCQP) controller on the inverted pendulum, double inverted pendulum, and planar quadrotor simulation environments considered in Section 6.2, with the results seen in Figures 6.7, 6.8, and 6.9, respectively. We see that the proposed controller stabilizes each system at each sample rate, including the exceptionally low sample rate of 10 [Hz]. Moreover, the input does not display high frequency oscillations for any system at any frequency. I do note the large input jump that occurs with the double inverted pendulum near the beginning of the simulation, which suggests the controller is quickly pushing the system into a region of the state space from which it continues to produce smooth inputs. These simulation results show that addressing sampling in the control design can lead to significantly better performance, and my results in this section highlight that there is a rigorous theoretical path towards doing so. Conclusion In this section I presented my work on stabilizing sampled-data control through feedback linearization and CLF-based controllers specified via convex optimization. I begin by reviewing the stabilizing sampled-data control framework with approximate discrete-time models in [81] and providing necessary adjustments to support local stability results and the coordinate transformations that accompany feedback linearization. I then demonstrate how a continuous-time 297 10 Hz SD-CLF-QCQP (State) (rad) (rad / sec) 1.0 0.5 10 Hz SD-CLF-QCQP (Input) 0.00 0.25 0.50 0.75 0.0 0 10 20 t (sec) 30 40 50 20 Hz SD-CLF-QCQP (State) (rad) (rad / sec) 1.0 0.5 1.00 0 10 20 t (sec) 30 40 50 20 Hz SD-CLF-QCQP (Input) 0.00 0.25 0.50 0.75 0.0 0 10 20 t (sec) 30 40 50 50 Hz SD-CLF-QCQP (State) (rad) (rad / sec) 1.0 0.5 1.00 0 10 20 t (sec) 30 40 50 50 Hz SD-CLF-QCQP (Input) 0.00 0.25 0.50 0.75 0.0 0 10 20 t (sec) 30 40 50 1.00 0 10 20 t (sec) 30 40 50 Figure 6.7. Simulation results for the inverted pendulum system using the (CLF-QCQP) controller. system that is feedback linearizable with locally exponentially stable zerodynamics can be rendered practically stable using the continuous-time feedback linearization controller without modification. I then use this result to develop CLF-based controllers specified via convex quadratically constrained quadratic programs (QCQP), a more complex class of convex optimization problems than the standard QPs. I conclude by demonstrating this control approach in simulation on the systems in Section 6.2, demonstrating a significant improvement over the standard continuous-time (CLF-QP) controller. 6.4 Sampled-Data Safety In this section I will present work on sampled-data safety-critical control through approximate discrete-time models. Building off of the work in Section 6.3, I will propose a framework for achieving theoretically meaningful safety guarantees for sampled-data systems through controllers that are specified via convex optimization problems. I will begin by considering the class of RungeKutta discrete-time approximations, which include the Euler approximation used in Section 6.3, but also permit working with CBFs that have a rela- 298 10 Hz SD-CLF-QCQP (State) 1.0 0.5 1 1 (rad) 2 (rad) 1 (rad / sec) 0.0 2 (rad / sec) 0 2 4 t (sec) 6 8 0.5 2 (rad / sec) 2 4 t (sec) 6 8 0 4 t (sec) 6 100 Hz SD-CLF-QCQP (Input) 2 3 10 0 1000 Hz SD-CLF-QCQP (State) 1.0 2 u1 (N m) u2 (N m) 8 10 1 1 (rad / sec) 0 3 0 1 (rad) 2 (rad) 0.0 2 10 100 Hz SD-CLF-QCQP (State) 1.0 10 Hz SD-CLF-QCQP (Input) 0 2 4 t (sec) 6 u1 (N m) u2 (N m) 8 10 1000 Hz SD-CLF-QCQP (Input) 0 0.5 1 (rad) 2 (rad) 1 2 (rad / sec) 3 1 (rad / sec) 0.0 0 2 4 t (sec) 6 8 10 2 0 2 4 t (sec) 6 u1 (N m) u2 (N m) 8 10 Figure 6.8. Simulation results for the double inverted pendulum system using the (CLF-QCQP) controller. tive degree higher than one. I will show that these approximations satisfy a one-step consistency property, suggesting that designing controllers with these approximations can endow the exact family of controller-map pairs with safety guarantees. Next, I will present the notion of practical safety as a theoretical safety property for sampled-data systems. I subsequently define a notion of sampled-data barrier functions (SD-BFs) and sampled-data Control Barrier Functions (SD-CBFs) that satisfy properties uniformly in the sample rate. In the largest contribution of this work, I provide a rigorous proof that a family of SD-BFs for a Runge-Kutta approximation family of controller-map pairs leads to practical safety of the exact family of controller-map pairs. Then, I consider a common class of nonlinear systems (with integrator structure), and show that given a SD-CBF satisfying a convexity property, the order of a Runge-Kutta approximation can be chosen to ensure that the SD-CBF inequality constraint is convex, and thus can be incorporated into a convex optimization-based controller. I conclude by demonstrating the proposed control strategy in simulation. 299 10 Hz SD-CLF-QCQP (State) 10 10 Hz SD-CLF-QCQP (Input) 8 0.4 6 0.2 4 2 0 2 4 0 x (m) z (m) (rad) x (m / sec) z (m / sec) (rad / sec) f (N) f (N / sec) 5 0.0 0.2 10 t (sec) 15 20 0 100 Hz SD-CLF-QCQP (State) 10 f (N / sec2) (N m) 5 10 t (sec) 15 20 100 Hz SD-CLF-QCQP (Input) 0.6 8 0.4 6 4 2 0 2 4 0 x (m) z (m) (rad) x (m / sec) z (m / sec) (rad / sec) f (N) f (N / sec) 5 0.2 0.0 0.2 10 t (sec) 15 20 0.4 0 500 Hz SD-CLF-QCQP (State) 10 f (N / sec2) (N m) 5 10 t (sec) 15 20 500 Hz SD-CLF-QCQP (Input) 0.6 8 0.4 6 4 2 0 2 4 0 x (m) z (m) (rad) x (m / sec) z (m / sec) (rad / sec) f (N) f (N / sec) 5 0.2 0.0 0.2 10 t (sec) 15 20 0.4 0 f (N / sec2) (N m) 5 10 t (sec) 15 20 Figure 6.9. Simulation results for the planar quadrotor system using the (CLF-QCQP) controller. The contributions of this section are as follows: • A complete framework for safety-critical of sampled-data systems through approximate discrete-time models, including a characterization of onestep consistency of Runge-Kutta approximate discrete-time models, a 300 definition of practical safety, sampled-data barrier functions, and sampleddata Control Barrier Functions, and a rigorous statement relating control designs for approximate discrete-time models to practical safety of the exact family of controller-map pairs. • A characterization of the relationship between system dynamics, SDCBF convexity properties, and Runge-Kutta approximation order that yields controllers that can be specified via convex optimization problems, which I demonstrate in simulation. The text for this section is adapted from: A. J. Taylor, V. D. Dorobantu, R. K. Cosner, Y. Yue, and A. D. Ames, “Safety of sampled-data systems with control barrier functions via approximate discrete time models,” in Proc. IEEE 61st Conf. on Decision and Control (CDC), Cancún, Mexico, 2022, pp.7127–7134. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. Approximate Discrete-Time Models I will begin by considering a larger class of discrete-time approximations that include the Euler approximation used in Section 6.3. The purpose for considering higher-order approximations is that CBFs are often not relative degree one (motivating the use of extended CBFs as was done in Section 3.7), where the CLFs considered in Section 6.3 are typically relative degree one. The importance of this is that the input u may not show up in a first-order approximation of the evolution of a function with relative degree greater than one6 . Later in this section I will show how these higher-order approximations can ensure the input appears when studying the evolution of a CBF, and provide simulation examples where higher-order approximations are necessary. Definition 51 (Runge-Kutta Approximation Family). Let p ∈ Z>0 . The Runge-Kutta approximation family of maps {Fa,p | T ∈ I} is defined such T 6 This is relatively easy to see for a double integrator with a CBF h(x) = x1 . Using an Euler approximation returns h(Fa,1 T (x, u)) = x1 + T x2 , such that the input does not appear in the value h at the next time step. 301 n that for every sample period T ∈ I, the function Fa,p T : Z → R is defined recursively as: Fa,p T (x, u) = x + T zi = x + T p X i=1 i−1 X bi (f (zi ) + g(zi )u), (6.87) ai,j (f (zj ) + g(zj )u), (6.88) j=1 P for all pairs (x, u) ∈ Z, with z1 = x. Here, b1 , . . . , bp ∈ R≥0 satisfy pi= bi = 1 and ai,j ∈ R for each i ∈ {1, . . . p} and j ∈ {1, . . . , i − 1}. For a family of admissible controllers {kT | T ∈ I}, I define the Runge-Kutta approximation family of controller-map pairs {(kT , Fa,p T ) | T ∈ I}. I will once again make use of one-step consistency, which I define slightly differently (the introduction of the set A) from the definition in Section 6.3: Definition 52 (One-Step Consistency). A family of controller-map pairs {(kT , FT ) : T ∈ I} is one-step consistent with the exact family of controller-map pairs {(kT , FeT ) | T ∈ I} over a set A ⊆ X if there exist ρ ∈ K∞ and T ∗ ∈ I such that for all x ∈ A and T ∈ (0, T ∗ ), we have: ∥FeT (x, kT (x)) − FT (x, kT (x))∥ ≤ T ρ(T ). (6.89) Before establishing a relationship between a Runge-Kutta approximation family and one-step consistency, I state the following lemma I will use throughout this work: Lemma 10. For any compact set K ⊂ X , there is an ε ∈ R>0 such that K ⊕ B ε ⊂ X and K ⊕ B ε is compact, where B ε is the closed norm-ball of radius ε and ⊕ is the Minkowski sum. Proof. As X is open, for every x ∈ K, there is a corresponding open ball centered at x with radius δx ∈ R>0 that is contained in X . Let Bx ⊂ X be the open ball centered at x of radius δx /2. Consider the collection {Bx : x ∈ K}; this is an open cover for the compact set K, so some finite collection Bx1 , . . . , BxN for some x1 , . . . , xN ∈ K, respectively, also covers K [249, Theorem 3.3.8]. Let δ = mini δxi , and consider any z ∈ K ⊕ B δ/4 . There is some x ∈ K such that ∥z − x∥ ≤ δ/4 and some i ∈ {1, . . . , N } such that ∥x − xi ∥ < δxi /2. Thus, ∥z − xi ∥ < δ/4 + δxi /2 < δxi , so z ∈ X . As z was 302 arbitrary, K ⊕ B δ/4 ⊆ X , so pick ε ≤ δ/4. The set K ⊕ B ε is compact as K × B ε is compact and (x, y) 7→ x + y is continuous. I now provide the first result of this work showing how properties of the dynamics and a family of controllers can be used to establish one-step consistency of the Runge-Kutta approximation family with the exact family of controllermap pairs: Theorem 42. Let K ⊂ X be compact, consider a family of admissible controllers {kT | T ∈ I}, and suppose there exists T1 ∈ I and a bound MK ∈ R≥0 such that for every sample period T ∈ (0, T1 ), the controller kT is bounded in norm by MK over K. Then for any p ∈ Z>0 , the Runge-Kutta approximation family of controller-map pairs {(kT , Fa,p T ) | T ∈ I} is one-step consistent with the family of controller-map pairs {(kT , FeT ) | T ∈ I} over K. Proof. Consider a compact set K ⊂ X and corresponding T1 ∈ I and MK ∈ R>0 , and fix a sample period T ∈ (0, T1 ). By Lemma 10, there exists an ε ∈ R>0 such that the compact set N = K ⊕ B ε satisfies N ⊂ X . By assumption, kT is bounded on K, and f and g are bounded on N by continuity, implying there exists an M ∈ R>0 such that: ∥f (z) + g(z)kT (y)∥ ≤ M, (6.90) for all y ∈ K and z ∈ N . As f and g are locally Lipschitz continuous over X , they are Lipschitz continuous over the compact set N . Therefore: ∥f (z) + g(z)kT (y) − (f (y) + g(y)kT (y))∥ (6.91) ≤ ∥f (z) − f (y)∥ + ∥g(z) − g(y)∥∥kT (y)∥, (6.92) ≤ (Lf + Lg MK )∥z − y∥, (6.93) = ρ(∥z − y∥), (6.94) for all y ∈ K and z ∈ N , with Lf , Lg ∈ R>0 Lipschitz constants of f and g on N , respectively, and ρ ∈ K∞ satisfying ρ(r) = (Lf + Lg MK )r for all r ∈ R≥0 . 303 Let x ∈ K. Then: FeT (x, kT (x)) − Fa,p T (x, kT (x)) Z T p X bi (f (zi ) + g(zi )kT (x)) [f (φ(t)) + g(φ(t))kT (x)] dt − T = 0 (6.95) (6.96) i=1 Z T [f (φ(t)) + g(φ(t))kT (x) − (f (x) + g(x)kT (x))]dt = 0 +T p X bi [f (x) + g(x)kT (x) − (f (zi ) + g(zi )kT (x))], (6.97) i=1 Pp where I use the fact i=1 bi = 1. To bound the first term in (6.97), let T2 ∈ (0, T1 ) satisfy T2 < ε/M . By continuity of the solution φ, if φ(t0 ) ̸∈ N for any t0 ∈ I, then there is a minimal t∗ ∈ (0, t0 ) such that ∥φ(t) − x∥ < ε (φ(t) ∈ N ) for all t ∈ [0, t∗ ) and ∥φ(t∗ ) − x∥ = ε. We have: Z t ∥φ(t) − x∥ ≤ ∥f (φ(s)) + g(φ(s))kT (x)∥ ds ≤ M t, (6.98) 0 for all t ∈ [0, t∗ ]. Since ε = ∥φ(t∗ ) − x∥ ≤ M t∗ , we know that t∗ ≥ ε/M > T2 . Thus if T ∈ (0, T2 ), then: ∥φ(t) − x∥ ≤ M t ≤ M T < M T2 < ε, (6.99) for all t ∈ [0, T ], implying φ(t) ∈ N for all t ∈ [0, T ]. To bound the second term in (6.97), I show by induction that if T is sufficiently small, then zi ∈ N for all i ∈ {1, . . . , p}. First, since z1 = x, we have z1 ∈ N . Next, for i ∈ {1, . . . , p}, suppose zj ∈ N for all j ∈ {1, . . . , i − 1}. Considering the definition of zi in (6.88) and the bound (6.90): ∥zi − x∥ ≤ T i−1 X |ai,j |∥f (zj ) + g(zj )kT (x)∥, (6.100) j=1 ≤ MT i−1 X j=1 |ai,j | ≤ M T (p − 1) max |aj,k | ≜ LT. j,k (6.101) Let T ∗ ∈ (0, T2 ) satisfy T ∗ < ε/L. Then for T ∈ (0, T ∗ ), we have ∥zi − x∥ < ε, or zi ∈ N . Since this choice of T ∗ does not depend on i, we can conclude by induction that if T ∈ (0, T ∗ ), then zi ∈ N for all i ∈ {1, . . . , p}. I have shown that if T ∈ (0, T ∗ ), then φ(t) ∈ N for all t ∈ [0, T ], and zi ∈ N for 304 i ∈ {1, . . . , p}. Thus, using the bound (6.94) in (6.97), we have that: ∥FeT (x, kT (x)) − Fa,p T (x, kT (x))∥, Z T p X bi ρ(∥zi − x∥), ρ(∥φ(t) − x∥) dt + T ≤ 0 ≤ T ρ(M T ) + T (6.102) (6.103) i=1 p X bi ρ(LT ), (6.104) i=1 ≤ T ρ̃(T ), (6.105) and ρ̃ ∈ K∞ satisfies ρ̃(r) = ρ(M r) + ρ(Lr) for all r ∈ R≥0 . Practical Safety & Sampled-Data Control Barrier Functions I now develop a notion of practical safety for sampled-data systems, define Sampled-Data Control Barrier Functions (SD-CBFs) as a tool for safety-critical sampled-data control synthesis, and highlight how a function with 0 as a regular value satisfies a specific property required by SD-CBFs. I begin with the following definition relating the evolution of a sampled-data system and a set: Definition 53 (Discrete-Time Forward Invariance). A set C ⊆ X is forward invariant for a controller-map pair (k, F) if for every x0 ∈ C and number of steps k ∈ Z≥0 , the recursion xk+1 = F(xk , k(xk )) is well-defined and satisfies xk ∈ C. This definition of forward invariance requires that the system state be contained in the set C only at sample times, which is aligned with the notion of practical stability for sampled-data systems presented in [81] and used in Section 6.3. This differs from the standard definition of forward invariance used in the existing sampled-data safety literature, which additionally requires that the solution remain in the set C between sample times, i.e. φ(t) ∈ C for t ∈ [tk , tk+1 ]. As seen in this literature, requiring inter-sample safety typically requires selecting control actions that meet a robustified continuous-time CBF time derivative condition. This robust condition typically depends on parameters of the system that are difficult to estimate, and using over-approximations may produce very conservative behavior [90]. Reducing this conservativeness usually amounts to operating at exceedingly high sample rates, which may not be practical, and which may excite unmodeled features of the system dynamics. Moreover, in practice, inter-sample safety violations at high sample rates can be inconsequential (and may not even be detectable). 305 As we are focused on designing controllers using an approximate family of maps, the following definition will describe the safety properties of the exact family of controller-map pairs when design uses approximations: Definition 54 (Practical Safety). The family of controller-map pairs {(kT , FT ) | T ∈ I} is practically safe with respect to a set C ⊆ X if for each R ∈ R>0 , there exists a T ∗ ∈ I such that for each sample period T ∈ (0, T ∗ ), there is a corresponding set CT ⊆ X that is forward invariant for the controllermap pair (kT , FT ) and satisfies C ⊆ CT ⊆ C ⊕ B R . This definition is posed to mirror that of practical stability for sampled-data systems proposed in [81] and used in Section 6.3. In particular, the burden of proof lies with small values of R. If R′ ≥ R and CT is a forward invariant subset of C ⊕B R , then it is automatically a forward invariant subset of C ⊕B R′ . Before defining Sampled-Data Control Barrier Functions, for a non-empty set C ⊆ X , denote dC (x) = inf y∈C ∥y − x∥ for all x ∈ X . I now define SampledData Barrier Functions and Sampled-Data Control Barrier Functions: Definition 55 (Sampled-Data Barrier Function Candidate). Consider a set C ⊆ X . A collection of functions {hT : X → R | T ∈ I} is a family of Sampled-Data Barrier Function Candidates on C if there exist T ∗ ∈ I, a function α ∈ Ke , a radius ε ∈ R>0 , and a Lipschitz constant M ∈ R>0 such that: hT (x1 ) > 0, hT (x2 ) = 0, hT (x3 ) < 0, (6.106) T α(hT (x4 )) ≤ hT (x4 ), (6.107) |hT (x5 ) − hT (x6 )| ≤ M ∥x5 − x6 ∥, (6.108) for all states x1 ∈ Int(C), x2 ∈ ∂C, x3 ∈ X \ C, x4 ∈ C, x5 , x6 ∈ X ∩ (C ⊕ B ε ), and sample periods T ∈ (0, T ∗ ). Additionally, I require that for each η ∈ R>0 there exists a δ ∈ R>0 such that7 : dC (x) > η =⇒ hT (x) < −δ, (6.109) for all x ∈ X ∩ (C ⊕ B ε ) and T ∈ (0, T ∗ ). Definition 56 (Sampled-Data Control Barrier Functions (SD-CBFs)). A family of Sampled-Data Barrier Function Candidates {hT | T ∈ I} is a family of 7 See Theorem 43 for how this property relates to regular values. 306 Sampled-Data Control Barrier Functions (SD-CBFs) on C for the family of maps {FT | T ∈ I} if for each state x ∈ X and sample time T ∈ (0, T ∗ ), there exists a corresponding input u ∈ Rm such that (x, u) ∈ Z and: hT (Fh (x, u)) − hT (x) > −T α(hT (x)). (6.110) Definition 57 (Sampled-Data Barrier Functions (SD-BFs)). Given a family of admissible controllers {kT | T ∈ I}, a family of Sampled-Data Barrier Function Candidates {hT | T ∈ I} is a family of Sampled-Data Barrier Functions (SD-BFs) on C for the family of controller-map pairs {(kT , FT ) | T ∈ I} if: hT (FT (x, kT (x))) − hT (x) ≥ −T α(hT (x)), (6.111) for all states x ∈ X and sample times T ∈ (0, T ∗ ). I note that the conditions in (6.106) and (6.107) are standard conditions required by barrier functions for discrete-time systems [234]. The inequalities in (6.106) imply that for each T ∈ (0, T ∗ ), C is the 0-superlevel set of hT . The inequality in (6.107) places a requirement on the SD-BF decrement through (6.111) that implies that for each T ∈ (0, T ∗ ), C is forward invariant for (kh , Fh ). The condition in (6.108) requires the SD-BF to be Lipschitz continuous over a slightly larger set than C with a Lipschitz constant that is uniform in the sample period, and will be used to relate exact and approximate families of controller-map pairs through one-step consistency. The implication in (6.109) resembles a coercivity condition, requiring the SD-BF value to decrease locally outside of the set C in a way that is uniform in the sample period. This property will be critical for producing forward invariant sets contained in C ⊕ B R for arbitrarily small values of R. The distinction between (6.110) and (6.111) is that the former condition states the possibility of safe control synthesis for an open-loop system, while the latter applies as a certificate for a closed-loop system. These properties are illustrated in Figure 6.10. To more clearly understand the nature of the coercivity condition (6.109), the following result makes a connection with regular values that are common with continuous-time CBFs: Theorem 43. If h : X → R is twice continuously differentiable with a compact 0-superlevel set C and 0 as a regular value, then there is an ε ∈ R>0 such that 307 Figure 6.10. Visualizing the properties (6.106)-(6.109) of SD-BF candidates. The dark green region represents the lower bound T α(hT (x4 )) and hT (x6 ) cannot be in the red region due to the Lipschitz bound. each η ∈ R>0 corresponds to a δ ∈ R>0 satisfying: dC (x) > η =⇒ h(x) < −δ, (6.112) for all states x ∈ X ∩ (C ⊕ B ε ). Proof. The boundary ∂C is compact as a closed subset of the compact set C. Thus, σ ≜ minx∈∂C ∥∇h(x)∥ is strictly positive since 0 is a regular value. By Lemma 10, there is an ε′ ∈ R>0 with C ⊕ B ε′ ⊂ X and C ⊕ B ε′ compact. Consider a state x ∈ C ⊕ B ε′ with x ̸∈ C. There exists a y ∈ ∂C such dC (x) = ∥y − x∥ > 0. Since h has 0 as a regular value, by [329, Proposition 1.1.9] we have that: ∇h(y) = −∥∇h(y)∥ x−y , ∥x − y∥ (6.113) that is ∇h(y) is anti-parallel to x−y. As B ε′ is convex, (1−λ)y+λx ∈ C ⊕B ε′ for all λ ∈ [0, 1], and Lagrange’s Remainder Formula [249, Theorem 6.6.3] implies that for some λ∗ ∈ [0, 1], ξ ≜ (1 − λ∗ )y + λ∗ x satisfies: 1 h(x) = h(y) + (x − y)⊤ ∇h(y) + (x − y)⊤ ∇2 h(ξ)(x − y), 2 1 = −∥∇h(y)∥∥x − y∥ + (x − y)⊤ ∇2 h(ξ)(x − y). 2 (6.114) (6.115) Since C ⊕ B ε′ is compact, there is an upper bound µ ∈ R≥0 such that: max ∥∇2 h(z)∥ = µ, z∈C⊕B ε′ (6.116) and thus we have: h(x) ≤ −(σ − µ ∥x − y∥)∥x − y∥. 2 (6.117) 308 If ∥x − y∥ ≤ σ/µ, then: σ σ h(x) ≤ − ∥x − y∥ = − dC (x). 2 2 (6.118) I pick ε ∈ R>0 such that ε ≤ min {ε′ , σ/µ}, and for any η ∈ R>0 , I pick δ ∈ R>0 such that δ < ση/2. Practical Safety via Sampled-Data Control Barrier Functions In this section I present the main contribution of this work by establishing how a family of SD-BFs for an approximate family of controller-map pairs can be used to ensure the practical safety of the exact family of controller-map pairs via one-step consistency: Theorem 44. Consider a set C ⊆ X and a family of admissible controllers {kT | T ∈ I}. Suppose that: 1. There exists a family of Sampled-Data Barrier Functions on C for a family of controller-map pairs {(kT , FT ) | T ∈ I}. 2. There exists an ε′ ∈ R>0 such that the family of controller-map pairs {(kT , FT ) | T ∈ I} is one-step consistent with the exact family of controllermap pairs {(kT , FeT ) | T ∈ I} over the set X ∩ (C ⊕ B ε′ ). Then the exact family of controller-map pairs {(kT , FeT ) | T ∈ I} is practically safe with respect to C. Proof. Let T1∗ , α, ε, and M be defined as in Definition 55. By assumption, there exists a T2∗ ∈ I and ρ ∈ K∞ such that (6.89) holds for all x ∈ X ∩(C⊕B ε′ ) and T ∈ (0, T2∗ ). As the family of controllers is assumed admissible, there is an T3∗ ∈ I such that kT is T -admissible for each T ∈ (0, T3∗ ). Let R ∈ R>0 , and pick R′ ∈ R>0 such that R′ ≤ min{ε, ε′ , R}. By (6.109), there exist δ, ∆ ∈ R>0 such that: dC (x) > R′ /2 =⇒ hT (x) < −δ, (6.119) dC (x) > δ/(2M ) =⇒ hT (x) < −∆, (6.120) for all x ∈ X ∩(C ⊕B ε ) and T ∈ (0, T1∗ ). Fix T ∈ I with T < min {T1∗ , T2∗ , T3∗ }. For any c ∈ R, I denote the c-superlevel set of hT as: Ωc,T = {x ∈ X | hT (x) ≥ c}. (6.121) 309 Case 1 Case 2 Case 3 Figure 6.11. A visual representation of the main sets and three cases discussed in the proof of Theorem 44. For any state x ∈ Ω−δ,T , we have dC (x) ≤ R′ /2, and thus C ⊆ Ω−δ,T ⊆ X ∩ (C ⊕ B R′ /2 ) ⊆ C ⊕ B R . I will prove that for small enough T , the set Ω−δ,T is forward invariant for the controller-map pair (kT , FeT ). I denote three cases, as seen in Figure 6.11, considering a state x ∈ X such that either: 1. x ∈ C, 2. x ∈ Ω−δ,T \ C and dC (x) ≤ δ/(2M ), 3. x ∈ Ω−δ,T \ C and dC (x) > δ/(2M ). Case 1: Suppose x ∈ C. From (6.111) and (6.107), we have: hT (FT (x, kT (x))) − hT (x) ≥ −T α(hT (x)) ≥ −hT (x), (6.122) so hT (FT (x, kT (x))) ≥ 0, or FT (x, kT (x)) ∈ C. By one-step consistency, we have: ∥FeT (x, kT (x)) − FT (x, kT (x))∥ ≤ T ρ(T ), (6.123) so if T ρ(T ) ≤ ε, then FeT (x, kT (x)) ∈ X ∩ (C ⊕ B ε ). Thus: |hT (FeT (x, kT (x))) − hT (FT (x, kT (x)))| ≤ M T ρ(T ), (6.124) and if M T ρ(T ) ≤ δ as well, then: hT (FeT (x, kT (x))) ≥ hT (FT (x, kT (x))) − M T ρ(h) ≥ −δ, (6.125) 310 giving us FeT (x, kT (x)) ∈ Ω−δ,T . The analysis of this case gives us the requirement T ρ(T ) ≤ min{ε, δ/M }. Before continuing to cases 2 and 3, I establish some additional properties. First, note that the superlevel sets have the containment property Ω−δ/2,T ⊆ / C, there is Ω−δ,T . Next, for any η ∈ R>0 and any x ∈ X ∩ (C ⊕ B ε ) with x ∈ a state y ∈ C such that ∥x − y∥ < dC (x) + η. Therefore: hT (x) ≥ hT (y) − M ∥x − y∥ ≥ −M dC (x) − M η, (6.126) since hT (y) ≥ 0. Since η can be chosen arbitrarily small, we have hT (x) ≥ −M dC (x). If dC (x) ≤ δ/(2M ), then hT (x) ≥ −δ/2, so X ∩ (C ⊕ B δ/(2M ) ) ⊆ Ω−δ/2,T ⊆ Ω−δ,T . Next, consider x ∈ Ω−δ,T \ C. Since x ∈ / C, meaning hT (x) < 0 and thus α(hT (x)) < 0, we have from (6.111) that: hT (FT (x, kT (x))) ≥ hT (x) − T α(hT (x)) > −δ. (6.127) Thus FT (x, kT (x)) ∈ Ω−δ,T ⊆ X ∩ (C ⊕ B R′ /2 ) so we can apply one-step consistency to achieve: ∥FeT (x, kT (x)) − FT (x, kT (x))∥ ≤ T ρ(T ). (6.128) If T ρ(T ) ≤ R′ /2, then FeT (x, kT (x)) ∈ X ∩ (C ⊕ B R′ ), in which case the Lipschitz property of hT yields the bound: |hT (FeT (x, kT (x))) − hT (FT (x, kT (x)))| ≤ M T ρ(T ). (6.129) Note that because R′ /2 < ε, the requirement from Case 1 can be replaced by T ρ(T ) ≤ min {R′ /2, δ/M }. Case 2: Suppose x ∈ Ω−δ,T \ C and dC (x) ≤ δ/(2M ). Since x ̸∈ C and X ∩ (C ⊕ B δ/(2M ) ) ⊆ Ω−δ/2,T , we have −δ/2 ≤ hT (x) < 0. Therefore: hT (FT (x, kT (x))) ≥ hT (x) − T α(hT (x)) ≥ −δ/2, (6.130) so FT (x, kT (x)) ∈ Ω−δ/2,T . By adding and subtracting hT (FeT (x, kT (x))) and using (6.129), we have: hT (FeT (x, kT (x))) ≥ −M T ρ(T ) − δ/2, (6.131) 311 when T ρ(T ) ≤ R′ /2. If M T ρ(T ) ≤ δ/2 as well, then hT (FeT (x, kT (x))) ≥ −δ, or FeT (x, kT (x)) ∈ Ω−δ,T . Thus I update the requirements to be T ρ(T ) ≤ min {R′ /2, δ/(2M )}. Case 3: Suppose x ∈ Ω−δ,T \ C and dC (x) > δ/(2M ). From (6.120), we have: hT (FT (x, kT (x))) − hT (x) > −T α(−∆). (6.132) Adding and subtracting hT (FeT (x, kT (x))) and (6.129) yield: hT (FeT (x, kT (x))) > hT (x) − M T ρ(T ) − T α(−∆), (6.133) = hT (x) − T (M ρ(T ) + α(−∆)), (6.134) when T ρ(T ) ≤ R′ /2. If M ρ(T ) ≤ −α(−∆) as well, then hT (FeT (x, kT (x))) > hT (x) ≥ −δ, or FeT (x, kT (x)) ∈ Ω−δ,T . To conclude, if both: 1. T < min {T1∗ , T2∗ , T3∗ , ρ−1 (−α(−∆)/M )}, 2. T ρ(T ) ≤ min {R′ /2, δ/(2M )}, then the set CT ≜ Ω−δ,T ⊆ C ⊕ B R is forward invariant for the exact controllermap pair (kT , FeT ), and thus the exact family of controller-map pairs {(kT , FeT ) | T ∈ I} is practically safe with respect to C. Optimization-Based Controller Synthesis I will now explore convexity of the CBF decrement condition, and define an optimization-based controller via an SD-CBF for achieving practical safety. The following result establishes how for a system with a block integrator structure, a Runge-Kutta approximation family of maps of the appropriate order can preserve a convexity property of a family {hT | T ∈ I}: Theorem 45. Consider ℓ, γ, q ∈ Z≥>0 such that n = ℓγ and q ≤ γ. Suppose the system dynamics have the form:     0ℓ×ℓ Iℓ 0ℓ     .. .. ..     . . .   ,  ẋ =  x + (6.135)    0ℓ×ℓ Iℓ  0ℓ    0ℓ×ℓ fγ (x) + gγ (x)u | {z } | {z } A r(x,u) 312 where fγ : E → Rℓ and gγ : E → Rℓ×m . For each T ∈ I, consider a function hT : X → R, and suppose there exists a function h̃T : (Rℓ )q → R satisfying: hT (x) = h̃T (ζ 1 , . . . , ζ q ), (6.136) for all x = (ζ 1 , . . . , ζ γ ) ∈ X . If the function h̃T is concave with respect to its last argument and p = γ − q + 1, then for α ∈ Ke , the function ϕT : Z → R defined as: ϕT (x, u) = −hT (Fa,p T (x, u)) + hT (x) − T α(hT (x)), (6.137) is convex in its second argument. Proof. For all (x, u) ∈ Z, denote: a,p a,p Fa,p T (x, u) = ((F1 )T (x, u), . . . , (Fγ )T (x, u)), (6.138) ℓ where (Fi )a,p T : Z → R for all i ∈ {1, . . . , γ}. For (x, u) ∈ Z, the block vector r(x, u) can be nonzero only in the last (γth) block. Noting the block chainof-integrators structure of A, for any degree d ∈ {0, . . . , γ − 1}, Ad r(x, u) can be nonzero only in the (γ − d)th block, and for a degree d polynomial ρd , ρd (A)r(x, u) can be nonzero only in the last d + 1 blocks (that is, blocks γ − d through γ). Consider a state-input pair (x, u) ∈ Z. We have: Fa,p T (x, u) = x + T zi = x + T p X i=1 i−1 X bi (Azi + r(zi , u)), (6.139) ai,j (Azj + r(zj , u)), (6.140) j=1 with z1 = x. By induction, for any i ∈ {1, . . . , p}, I show: zi = ρi,i−1 (A)x + i−1 X σi,i−j−1 (A)r(zj , u), (6.141) j=1 where ρi,i−1 is a degree i − 1 polynomial, and for j ∈ {1, . . . , i − 1}, σi,i−j−1 is a degree i − j − 1 polynomial. Indeed, z1 = In · x, and assuming (6.141) holds 313 for 0, . . . , i − 1, substituting (6.141) into (6.140) yields the following: degree i−1 z zi = }| { ! degree j i−1 i−1 z }| { X X I+T ai,j Aρj,j−1 (A) x + T ai,j r(zj , u) j=1 j=1 {z | } ≜ρi,i−1 (A) +T i−1 X k−1 X (6.142) ai,k Aσk,k−j−1 (A)r(zj , u), k=1 j=1 which we may further manipulate to obtain: zi − ρi,i−1 (A)x = i−1 X T ai,j + j=1 k=j+1 | ≜ i−1 X i−1 X ! ai,k Aσk,k−j−1 (A) r(zj , u), {z } | degree k−j {z } (6.143) degree i−j−1 (6.144) σi,i−j−1 (A)r(zj , u), j=1 establishing (6.141) holds for i. Substituting the expression (6.141) into (6.139) and following a similar sequence of steps, we find a degree p polynomial ρ̃p , and for each i ∈ {1, . . . , p}, a degree p − i polynomial σ̃p−i such that: Fa,p T (x, u) = ρ̃p (A)x + p X σ̃p−i (A)r(zi , u). (6.145) i=1 For i ∈ {1, . . . , p}, the term σ̃p−i (A)r(zi , u) can be nonzero only in blocks γ − (p − i) = q + i − 1 through γ. The highest-order polynomial multiplying the block vectors r(z1 , u), . . . , r(zp , u) is σ̃p−1 = σ̃γ−q . Therefore, the functions a,p (F1 )a,p h , . . . , (Fq−1 )T are independent of their second argument (they depend only on state). Moreover, (Fq )a,p T (x, u) depends on the block vector r(z1 , u) = r(x, u), which depends on u affinely, and does not depend on the block vectors r(z2 , u), . . . , r(zp , u), which may depend on u nonlinearly. The composition hT ◦ Fa,p T : Z → R satisfies: a,p a,p hT (Fa,p T (x, u)) = h̃T ((F1 )T (x, u), · · · , (Fq )T (x, u)), (6.146) for all (x, u) ∈ Z. The composition of concave and affine functions is concave, so hT ◦ Fa,p T is concave in its second argument, and ϕT in (6.137) is convex in its second argument. 314 The following result highlights how we may synthesize a family of controllers that achieve practical safety through optimization using only a Runge-Kutta approximation family of maps and not requiring knowledge of the exact family of maps: Theorem 46. Let {hT | T ∈ I} be a family of SD-CBFs on C for a RungeKutta approximation family of map {Fa,p T | T ∈ I} such that the set: (6.147) FT (x) = {u ∈ Rm | (x, u) ∈ Z and ϕT (x, u) ≤ 0}, is closed and convex for each T ∈ I and x ∈ X . Consider a set of controllers {kT | T ∈ I} satisfying: kT (x) = argmin u∈Rm 1 ∥u − knom (x)∥2 2 (SD-CBF-OP) s.t. hT (Fa,p T (x, u)) − hT (x) ≥ −T α(hT (x)), for each x ∈ X and T ∈ (0, T ∗ ), where knom : X → Rm is a nominal controller. If {kT | T ∈ I} is a family of admissible controllers, then {hT | T ∈ I} is a family of Sampled-Data Barrier Functions on C for the Runge-Kutta approxmation family of controller-map pairs {(kT , Fa,p T ) | T ∈ I}. Proof. Consider T ∈ (0, T ∗ ) and x ∈ X . As hT is a SD-CBF on C, there is a u′ ∈ Rm such that (x, u′ ) ∈ Z and: ′ hT (Fa,p T (x, u )) − hT (x) ≥ −T α(hT (x)), (6.148) implying that u′ ∈ FT (x). Thus the optimization problem in (SD-CBF-OP) is feasible. Define the compact, convex set:  A = u ∈ Rm | ∥u − knom (x)∥2 ≤ ∥u′ − knom (x)∥2 . (6.149) Note that u′ ∈ A. As FT (x) is closed and convex, A ∩ FT (x) is compact, convex, and non-empty. As the cost is continuous and strictly convex with respect to u, there is a unique minimizer u∗ ∈ A ∩ FT (x). We have ∥u∗ − knom (x)∥2 ≤ ∥u′ − knom (x)∥2 < ∥u − knom (x)∥2 for all u ∈ FT (x) \ A, implying u∗ is the unique minimizer in FT (x) (the argmin is well-defined). Thus: hT (Fa,p T (x, kT (x))) − hT (x) ≥ −T α(hT (x)), (6.150) and as x and T were arbitrary, we have that {hT | T ∈ I} is a family of SDBFs on C for the Runge-Kutta approximation family of controller-map pairs {(kT , Fa,p T ) | T ∈ I}. 315 Figure 6.12. The single (left) and double (right) inverted pendulums and their sets C. The inverted pendulum sets C are, from left to right, a configuration ellipsoid, a halfspace, and a Lyapunov sublevel set. The far right image shows the set C for the double inverted pendulum where (θ1 , θ2 ) ∈ R2 are constrained to an ellipse and no constraint is placed on (θ̇1 , θ̇2 ) ∈ R2 (represented by vertical fibers). Simulation Results I now evaluate the proposed (SD-CBF-OP) on the inverted pendulum and double inverted pendulum in Section 6.2. With state vector x = (q, q̇) ∈ Rn , the dynamics of these systems can be expressed in the form (6.135), where ℓ = m and γ = 2. For the inverted pendulum, I use sets C with the form of a Lyapunov sublevel (hT (x) = 1−x⊤ Px with P ∈ S2≻0 ), a configuration ellipsoid (hT (θ) = 1 − θ2 ), and a halfspace (hT (θ) = θ + 0.1). For the double inverted pendulum I enforce safety of a configuration ellipsoid (hT (q) = 1 − ∥q∥2 ). These sets are visualized in Figure 6.12. I use Runge-Kutta approximations with p = 1 (forward Euler) for the Lyapunov sublevel set and p = 2 (midpoint rule) for the other settings. Controllers of the form (SD-CBF-OP) are employed with identity comparison functions; for the Lyapunov sublevel set, knom is a feedback linearizing controller with auxiliary PD control (proportional gain 1, derivative gain 2), and for the other settings, knom is a zero (constant) controller. With 11 sample periods spaced logarithmically (over [0.05, 0.5] and [0.01, 0.1] seconds for the single and double inverted pendulums, respectively) and initial conditions sampled from each set C, the sampled-data closed-loop systems are simulated for 10 seconds. For the inverted pendulum, 500 initial states are sampled uniformly from the Lyapunov sublevel set, and 41 × 41 grids of initial states cover [−1, 1] × [−5, 5] for the configuration ellipsoid and [−0.1, 1] × [−5, 5] for the halfspace. For the double inverted pendulum, 500 initial states are uniformly sampled with configurations in the unit Euclidean ball in R2 and velocities in [−1, 1]2 . The worst-case distances from the safe sets are reported as a function of sample period in Figure 6.13. These distances decrease for sufficiently small sample periods, as predicted by the definition of practical safety. 316 Figure 6.13. The maximum distance from the safe set C (lower is better) achieved during trials vs. the the sampling frequency. The simulations and animations be found at https: //bit.ly/CBF-OP and https://vimeo.com/690803272. (Top) The inverted pendulum for 3 different safe sets. (Bottom) The double inverted pendulum. Conclusion In this section I have presented my work on sampled-data safety-critical control through approximate discrete-time models. I show how Runge-Kutta approximation models satisfy important consistency properties, and define the concepts of practical safety, sampled-data barrier functions, and sampled-data Control Barrier Functions. In the main contribution of this work, I show how a family of sampled-data barrier functions for a Runge-Kutta approximation family of controller-map pairs implies the exact family of controller-map pairs is practically safe, permitting theoretical guarantees for the exact discretetime systems using controllers designed with approximations. I then show how for a common class of nonlinear systems, the convexity properties of a SD-CBF can be preserved with a Runge-Kutta approximation family of a specific order, allow safety-critical controllers using approximate discrete-time models to be specified as convex optimization problems. Lastly I demonstrate these controllers in a variety of simulation settings, highlighting that worstcase safety violations decrease with increasing sample rate as predicted by practical safety. 6.5 Event-Triggered Safety In this section I will present work on safety-critical event-triggered control. I will begin by reviewing event-triggered control for stabilization using ISS-LFs as presented in [96]. Drawing inspiration from this, I will propose a direct 317 translation of the ideas in this paper to the safety-critical setting using ISSfBFs. I will then demonstrate that this direct translation does not lead to a minimum interevent time (MIET) through a rigorous counterexample. I resolve the issues shown with this counter example through a concept known as the strong event-triggered ISSf barrier property, which requires a more stringent degree of safe behavior on the boundary of a set that is to be kept forward invariant. Lastly, I demonstrate that this trigger law ensures both safety and an MIET in simulation on the system used in the counterexample. The contributions of this section are as follows: • The first framework for safety-critical event-triggered control that takes inspiration from [96] by utilizing the tools of ISSf-BFs. • A rigorous counterexample highlighting challenges not seen when performing event-triggered stabilization, and a proposed method for resolving these challenges. The text for this section is adapted from: A. J. Taylor, P. Ong, J. Cortés, and A. D. Ames, “Safety-critical event triggered control via input-to-state safe barrier functions,” IEEE Control Sys. Let., vol. 5, no. 3, pp. 749–754, 2021. A. J. Taylor participated in the conception of the project, algorithm design and theoretical analysis, simulation code implementation, and writing of the article. Event-Triggered Stability I will now review the approach for event-triggered stabilization as presented in [96], which will inspire how I develop my proposed approach for safety-critical event-triggered control. Let k : Rn → Rm be a controller, and recall the event-triggered closed-loop system: ẋ(t) = f (x(t)) + g(x(t))k(x(tk )) ∀t ∈ [tk , tk+1 ). (6.151) I define the sampling error e : [tk , tk+1 ) → Rn as: e(t) = x(tk ) − x(t) ∀t ∈ [tk , tk+1 ), (6.152) 318 noting that this in conjunction with (6.151) implies: ẋ(t) = −ė(t) = f (x(t)) + g(x(t))k(x(tk )) ∀t ∈ [tk , tk+1 ). (6.153) This can alternatively be written as: ẋ(t) = f (x(t)) + g(x(t))k(x(t) + e(t)) ∀t ∈ [tk , tk+1 ). (6.154) Let xe ∈ Rn be an equilibrium point of the continuous-time closed-loop system: ẋ = f (x) + g(x)k(x). (6.155) I will define an ISS-LF in the event-triggered setting as follows: Definition 58 (Event-Triggered ISS-LF (ET-ISS-LF)). Let xe ∈ Rn be an equilibrium point of the continuous-time closed-loop system (6.155). A function V : Rn → R≥0 that is continuously differentiable on Rn is said to be an exponential Event-Triggered Input-to-Stable Lyapunov function (ET-ISS-LF) for the continuous-time closed-loop system (6.155) and equilibrium point xe if there exist constants k1 , k2 , k3 , a, and σ ∈ K∞ such that: k1 ∥x − xe ∥a ≤ V (x) ≤ k2 ∥x − xe ∥a , V̇ (x, e) ≜ Lf V (x) + Lg V (x)k(x + e) ≤ −k3 ∥x − xe ∥a + σ(∥e∥), (6.156) (6.157) for all x, e ∈ Rn . Suppose that we have an ET-ISS-LF for an equilibrium point xe of the continuoustime closed-loop system (6.155). If the trigger-law is defined to enforce: σ(∥e(t)∥) ≤ ck3 ∥x(t) − xe ∥a , (6.158) for c ∈ (0, 1), then the ET-ISS-LF condition (6.157) implies that: Lf V (x(t)) + Lg V (x(t))k(x(t) + e(t)) ≤ (c − 1)k3 ∥x(t) − xe ∥a , (6.159) for all t ∈ [tk , tk+1 ) and k ∈ Z≥0 , such that V is an exponential Lyapunov function for the the event-triggered closed-loop system (6.151) and equilibrium point xe and the event-triggered closed-loop system (6.151) is exponentially stable with respect to the equilibrium point xe . In this way, a controller that is designed to be robust to sampling error (such that (6.157) is met) can be used with an event-triggered implementation and recover the desired exponential 319 stability property at the expense of convergence rate ((c−1)k3 < k3 in (6.159)). The inequality in (6.158) can be enforced by defining the trigger-law as: tk+1 = min {t ≥ tk | σ(∥e(t)∥) = ck3 ∥x(t) − xe ∥a } . (6.160) As is typical in event-triggered control formulations, it is critical to show that such a trigger-law does not lead to the control being updated at arbitrarily close time instances [315], or that the interevent times {tk+1 − tk }k∈Z≥0 are lower bounded by a positive constant τ ∈ R>0 , referred to as the minimum interevent time (MIET). The results of [96] ensure that a MIET exists under the trigger-law (6.159) under the assumption that the functions f , g, and k are Lipschitz continuous on any compact set K ⊂ Rn and σ is Lipschitz continuous on any compact set K ⊂ R≥0 , noting that if these function are locally Lipschitz continuous on their domains, then they are Lipschitz continuous on any compact set contained in their domain. Counterexample I will now begin to set up an approach for safety-critical event-triggered control. In attempting to mirror the approach for stabilization, I will reveal a challenge that occurs with a naïve trigger-law near the boundary of a set C that is to be kept safe. I will explore resolving this in the subsequent section. I will define an ISSf-BF in the event-triggered context as follows: Definition 59 (Event-Triggered ISSf-BF (ET-ISSf-BF)). Let C ⊂ Rn be the 0-superlevel set of a function h : Rn → R that is continuously differentiable on Rn . The function h is said to be an Event-Triggered Input-to-State Safe barrier function (ET-ISSf-BF) for the continuous-time closed-loop system (6.155) on e C if there exist α ∈ K∞ and ι ∈ K∞ such that: ḣ(x, e) ≜ Lf h(x) + Lg h(x)k(x + e) ≥ −α(h(x)) − ι(∥e∥), (6.161) for all x, e ∈ Rn . Given the similarity of the ET-ISS-LF constraint (6.157) and the ET-ISSf-BF constraint (6.161), it is natural to propose a trigger-law that enforces: ι(∥e(t)∥) ≤ cα(h(x(t))), (6.162) for some c ∈ R>0 , implying: Lf h(x(t)) + Lg h(x(t))k(x(t) + e(t))) ≥ −(1 + c)α(h(x(t))), (6.163) 320 for all t ∈ [tk , tk+1 ) and k ∈ Z≥0 , such that h is a barrier function for the event-triggered closed-loop system (6.151) on C, and thus the event-triggered closed-loop system is safe with respect to the set C. This can be interpreted as conserving actuation resources at the expense of allowing the system to more quickly approach the boundary of the set C from the inside. It is important to note that inside C it is possible to satisfy (6.162) and thus enforce safety, but it is impossible to satisfy (6.162) outside C as α(h(x(t))) < 0 if x(t) ∈ / C. This type of behavior does not arise in the context of event-triggered stabilization, where convergence is to a point. One way to solve this issue is to instead define the trigger-law to enforce: ι(∥e(t)∥) ≤ c|α(h(x(t)))|, (6.164) for c ∈ (0, 1), which enforces (6.163) if x(t) ∈ C and enforces: Lf h(x(t)) + Lg h(x(t))k(x(t) + e(t)) ≥ −(1 − c)α(h(x(t))), (6.165) if x(t) ∈ / C. In this formulation, the system is not only allowed to more quickly approach the boundary, but is also not required to converge to the set as quickly when outside of the set. This is a generalization of event-triggered stabilization to a set. Even with this solution, it is not guaranteed that this trigger law will have an MIET. Although ruling out Zeno behavior is not required to guarantee safety, unlike stabilization (unique solutions must exist for all time), it is important to have an MIET in term of implementation of the controller [97]. The key difference between stability and safety leading to the failure of an MIET to exist for a safe event-triggered controller lies in how the system dynamics must behave close to an equilibrium point compared to how they can behave close to the boundary of the set C. In stabilization, the continuity of the dynamics requires that they vanish as the equilibrium is approached, leading to the error dynamics in (6.153) vanishing. In safety, the dynamics close to the boundary of the safe set need not vanish as the boundary is approached, such that the error dynamics in (6.153) need not vanish. I provide the following counterexample to illustrate how this difference can lead to a MIET failing to exist for the trigger-law design (6.164). Consider the following system: " # " # " # x2 x1 d x1 = + u, dt x2 −x1 x2 | {z } | {z } f (x) g(x) (6.166) 321 for which we wish to ensure the safety of the set C, given by the 0-superlevel set of the continuously differentiable function h(x) = 1 − x21 − x22 = 1 − ∥x∥2 . The time derivative of this function along solutions to (6.166) is given by ḣ(x, u) = −2(x21 + x22 )u, for which the controller k(x) = 12 (1 − x21 − x22 ) = 12 h(x) yields ḣ(x) = −(x21 + x22 )h(x) ≥ −h(x) for all x ∈ R2 , which implies h is a BF for the resulting closed-loop system on the set C. In an event-triggered context, the closed-loop system is given by: " # k(x(tk )) 1 ẋ(t) = x(t) ∀t ∈ [tk , tk+1 ). −1 k(x(tk )) (6.167) This leads to the time derivative of h along solutions to (6.167) being given by: ḣ(x(t), e(t)) = −∥x(t)∥2 h(x(tk )) = −∥x(t)∥2 h(x(t) + e(t)), (6.168) for t ∈ [tk , tk+1 ), where e(t) = x(tk ) − x(t). To see that h is in fact an ETISSf-BF, note that its time derivative can be bounded as follows: ḣ(x(t), e(t)) = −∥x(t)∥2 h(x(t) + e(t)), = −∥x(t)∥2 (1 − ∥x(t)∥2 − 2x(t)⊤ e(t) − ∥e(t)∥2 ), ≥ −∥x(t)∥2 (1 − ∥x(t)∥2 ) − 2∥x(t)∥3 ∥e(t)∥, ≥ −(1 − ∥x(t)∥2 ) − 2∥x(t)∥3 ∥e(t)∥, ≥ −h(x(t)) − 2r3 ∥e(t)∥, for all r ∈ R≥0 such that r ≥ 1 and x(t) ∈ R2 such that ∥x(t)∥ ≤ r. Note that for a fixed value of r, this restricts the claim to a neighborhood of the compact set C. The value of r can be chosen arbitrarily large to produce an arbitrarily large neighborhood, but this detail is not critical for the subsequent results regarding the non-existence of an MIET. Given that h is an ET-ISSfBF on some domain containing the unit circle (choose r > 1), the trigger-law enforcing (6.164) is given by: tk+1 = min{t ≥ tk | 2r3 ∥e(t)∥ = c|h(x(t))|}, with c ∈ (0, 1). This will guarantee that C is safe as:  −(1 + c)h(x(t)), ∥x(t)∥ ≤ 1, ḣ(x(t), e(t)) ≥ −(1 − c)h(x(t)), 1 < ∥x(t)∥ ≤ r. (6.169) (6.170) 322 Despite the fact f , g, k, α (α(r) = r), and ι ((ι(r) = 2r3 ) are Lipschitz continuous on compacts, which was sufficient in the case of stabilization, the following result shows the trigger-law fails to provide an MIET: Lemma 11. The system (6.167) with the trigger-law defined as in (6.169) does not possess an MIET. Proof. To show that the interevent times {tk+1 −tk }k∈Z≥0 are not lower bounded, I will proceed via contradiction. In particular, let us assume that there exists an MIET τ ∈ R>0 such that tk+1 − tk ≥ τ for all k ∈ Z≥0 . If the state xk = x(tk ) at event time tk is used as an initial condition, the solution to the event-triggered closed-loop system (6.167) is: #  " cos(∆tk ) sin(∆tk ) h(xk )∆tk xk , (6.171) x(t) = exp 2 − sin(∆tk ) cos(∆tk ) = Mk (∆tk )xk , (6.172) for t ∈ [tk , tk+1 ) with ∆tk = t − tk . Denote ωk = h(xk )∆tk and observe that the norm of the error is lower bounded by a function monotonically increasing in time: ∥e(t)∥ = ∥(I2 − Mk (∆tk ))xk ∥, (6.173) r ω   k = exp (ωk ) − 2 exp cos(∆tk ) + 1 ∥xk ∥, (6.174) 2 r  ω  k + 1 ∥xk ∥, (6.175) ≥ exp (ωk ) − 2 exp 2 r ω  2 k = exp − 1 ∥xk ∥, (6.176)  ω 2 k (6.177) ≥ exp − 1 ∥xk ∥. 2 This lower bound on the error grows unbounded in time. This implies that no matter the state in Int(C) that an event occurs, another event must occur at some time in the future (or the bound in (6.169) will be violated as h is upper bounded on C). Thus, for all T > 0, there exists an event time tk > T . Next, I show that limt→∞ h(x(t)) = 0. Note that: h(x(t)) = 1 − ∥x(t)∥2 = 1 − exp (h(xk )∆tk )∥xk ∥2 , (6.178) with time derivative: ḣ(x(t)) = −h(xk ) exp (h(xk )∆tk )∥xk ∥2 , (6.179) 323 for t ∈ [tk , tk+1 ). Within the safe set we have that ḣ(x(t)) ≤ 0, such that h(x(t)) is monotonically decreasing in time. The safety of C implies h(x(t)) is lower bounded by 0, and thus we can conclude that limt→∞ h(x(t)) exists by the Monotone Convergence Theorem [249, Theorem 2.4.2] Assume that this limit is some value 0 < a < 1. For any δ ∈ R>0 , there exists T > 0 such that for t > T , h(x(t)) < a + δ. Since there are an infinite number of events, we deduce there exists tk > T such that h(x(tk )) < a + δ. As h(x(t)) is monotonically decreasing, it also follows h(x(t)) ≥ a for all t. This implies: ḣ(x(t)) ≤ −a exp (a∆tk )(1 − (a + δ)) ≤ −a + a2 + aδ < 0. (6.180) for t ≥ tk where δ can be chosen small enough to enforce the strict inequality with 0 as a2 < a. Thus, between two events we have: h(x(tk+1 )) ≤ h(x(tk )) + τ (−a + a2 + aδ), (6.181) where τ is the assumed MIET. Choosing δ < τ (a − a2 )/(1 + τ a) in conjunction with h(x(tk )) < a + δ implies h(x(tk+1 )) < a, contradicting the assumption that a ̸= 0 (and maintaining the assumption on the existence of τ ). To complete the proof, note e(tk ) = 0n and take the second-order (one-sided) Taylor expansion of ∥e(t)∥2 at t = tk :
∥e(t)∥2 = ė(tk )⊤ ė(tk ) (t − tk )2 + O((t − tk )3 ) (6.182) = (1 + k(x(tk )))∥x(tk )∥2 (t − tk )2 + O((t − tk )3 ) (6.183) ≥ ∥x(tk )∥2 (t − tk )2 − c3 (t − tk )3 , (6.184) with c3 ∈ R>0 . The first term in the inequality follows from k(x(tk )) ≥ 0 for x(tk ) ∈ C. To understand the second term, note that for any t ∈ [tk , tk+1 ), Lagrange’s Remainder Formula [249, Theorem 6.6.3] implies there exists a t′ ∈ (tk , t) such that:
1 d3 ∥e(t)∥2 = ė(tk )⊤ ė(tk ) (t − tk )2 + ∥e(t′ )∥2 (t − tk )3 . 6 dt3 (6.185) 3 d 2 As dt is a continuous function of x(t) and x(tk ) (can be shown by 3 ∥e(t)∥ expanding it), both of which remain in the compact set C, we must have that d3 ∥e(t)∥2 is bounded by a constant, denoted by 6c3 ∈ R≥0 , for all t ∈ R≥0 , dt3 including t = t′ . This yields the desired bound. At trigger time tk , let h(x(tk )) = ϵk for ϵk ∈ R>0 . Note that for arbitrarily small ϵ ∈ R>0 , there exists a trigger time tk such that ϵk < ϵ due to the 324 existence of infinite triggers and convergence of h(x(t)) to 0. This implies ∥x(tk )∥2 = 1 − ϵk . Let n ∈ Z≥0 be such that c13 < nτ and define t⋆k =   k tk + n1 1−ϵ , noting t⋆k < tk + τ . It follows from the Taylor expansion that: c3 (1 − ϵk )3 n − 1 ⋆ 2 ∥e(tk )∥ ≥ . c23 n3 (6.186) As ϵk can be chosen arbitrarily small, I choose it such that: (1 − ϵk )3 ≥ c2 n 3 ϵ2 , 4r6 (n − 1) k which indicates that: 2r3 ∥e(t⋆k )∥ ≥ c|h(x(tk ))| ≥ c|h(x(t⋆k ))|, as h(x(t)) is monotonically decreasing. As t⋆k < tk + τ , this contradicts that τ is the MIET. Figure 6.14 shows the number of events as a function of time and distance from the barrier. The blue curves in Figure 6.15 correspond to the interevent times. In the proof of Lemma 11, we see the fact that the dynamics of the eventtriggered closed-loop system (6.167) are not required to vanish on the boundary d of the set C. This leads to the derivative of the measurement error, dt ∥e(tk )∥, being uniformly lower bounded at event times tk , which, together with the convergence of h to 0, as seen in Figure 6.14, leads to arbitrarily small interevent times. In particular, the dynamics evolve tangentially to the boundary of the set C, leading to growing measurement error while moving arbitrarily close to the 0-level set of h. As the original controller may have additional objectives beyond safety (such as stabilization), it is desirable that the event-triggered implementation not completely eliminate tangential motion near the boundary that may be necessary to achieve the other objectives. To accommodate this, I will introduce a trigger-law that limits dynamic evolution tangential to the boundary of the safe set. Event-Triggered Safety & Simulation Results I now propose an alternative trigger-law that ensures an MIET exists. To resolve the issues in the preceding example, I introduce the following definition: 325 0.4 0.3 2 0.2 10-3 1 0.1 0 6.7684 6.7686 6.7688 6.769 0 0 1 2 3 4 5 6 7 Figure 6.14. Simulation results for the system (6.167) using the trigger law (6.169). Even as the boundary of the safe set is approached (h(φET (t)) → 0), the growth rate of the error does not diminish, leading to arbitrarily small interevent times. Definition 60 (Strong ET-ISSf Barrier Property). An ET-ISSf-BF h for the continuous-time closed-loop system (6.155) satisfies the strong ET-ISSf barrier property if there exists a d ∈ R>0 such that: Lf h(x) + Lg h(x)k(x + e)) ≥ −α(h(x)) + d − ι(∥e∥), (6.187) for all x, e ∈ Rn . This property introduces a positive constant, d, into the ET-ISSf-BF condition (6.161). In the presence of zero measurement error, this enforces that the state dynamics must lie in the interior of the tangent cone [104] when on the d |h(x(tk ))| will be greater boundary of the safe set C. It also enforces that dt d than a positive constant as we approach the boundary, similarly to dt ∥e(tk )∥. I now show this property is sufficient to design a trigger-law that ensures safety with a MIET. Theorem 47. Let h be an ET-ISSf-BF for the continuous-time closed-loop system (6.155) on a set C ⊂ Rn defined as the 0-superlevel set of a function h e that is continuously differentiable on Rn , with corresponding functions α ∈ K∞ e and ι ∈ K∞ . Let β ∈ K∞ and c ∈ (0, 1]. If the following assumptions hold: 1. h satisfies the strong ET-ISSf barrier property for a constant d ∈ R>0 , 2. ι is Lipschitz continuous with Lipschitz constant Lι , 3. there exists F ∈ R>0 , such that for all x, e ∈ Rn : ∥f (x) + g(x)k(x + e))∥ ≤ F, (6.188) 326 4. β(r) ≥ α(r) for all r ∈ R, then the trigger-law:  tk+1 = min t ≥ tk | ι(∥e(t)∥) = β(h(x(t))) − α(h(x(t))) + cd , (6.189) deployed recursively enforces: ḣ(x(t), e(t)) = Lf h(x(t)) + Lg h(x(t))k(x(t) + e(t)) ≥ −β(h(x(t))), (6.190) thus rendering the set C safe. Furthermore, there exists a MIET given by: tk+1 − tk ≥ τ ≜ cd , Lι F ∀k ∈ Z≥0 . (6.191) Before proving the result, I make a few observations regarding its assumptions. Assumption 3 on the boundedness of the dynamics need not hold over the entire state space for safety, but can hold for (x, e) ∈ C × Rn . Furthermore, if C is compact, the trigger-law enforces the existence of a compact set K ⊂ Rn such that e(t) ∈ K for all t ∈ R≥0 . Thus, the continuity of f , g, and k on Rn would imply the assumption is satisfied on C × K, which also would be sufficient for the result to hold. Assumption 4 ensures the right-hand side of the equality in the trigger will always be positive. The function β can be viewed as a tuning function which can raise interevent times (but not the MIET) at the expense of less “braking” within the safe set and convergence outside of the safe set. One choice is β = α, in which case interevent times are lowered for more braking and faster convergence. Lastly, I note this triggerlaw can be used in conjunction with (6.160) to jointly achieve event-triggered stabilization and safety if there exists an ET-ISS-LF for the continuous-time closed-loop system (6.155). Proof. To see the set C is rendered safe, observe that: ḣ(x(t), e(t)) = Lf h(x(t)) + Lg h(x(t))k(x(t) + e(t)), (6.192) ≥ −α(h(x(t))) + d − ι(∥e(t)∥), (6.193) ≥ −α(h(x(t))) + d − (β(h(x(t))) − α(h(x(t))) + cd), (6.194) = −β(h(x(t))) + (1 − c)d (6.195) ≥ −β(h(x(t))), (6.196) 327 for all t ∈ [tk , tk+1 ) and k ∈ Z≥0 , implying that h is a barrier function for the event-triggered closed-loop system (6.151) on C, such that the event-triggered closed-loop system (6.151) is safe with respect to C. To see the interevent times are lower bounded, observe that: Z t (−f (x(τ )) − g(x(τ ))k(x(τ ) + e(τ )))dτ , (6.197) ∥e(t)∥ = e(tk ) + tk Z t = (−f (x(τ )) − g(x(τ ))k(x(τ ) + e(τ )))dτ , (6.198) tk Z t ≤ F dτ. (6.199) tk This inequality together with the trigger law (6.189) yields: tk+1 ≥ min {t ≥ tk | Lι ∥e(t)∥ = cd} , ≥ min {t ≥ tk | Lι F (t − tk ) = cd} cd = + tk , Lι F (6.200) (6.201) (6.202) ensuring the desired result. In the case that an ET-ISSf-BF h does not satisfy the strong barrier property, an auxiliary ET-ISSf-BF, hb , satisfying the strong ISSf barrier property can be synthesized via h at the expense of keeping a larger set safe: Theorem 48. Let h be an ET-ISSf-BF for the continuous-time closed-loop system (6.155) on a set C ⊂ Rn defined as the 0-superlevel set of a function h : Rn → R that is continuously differentiable on Rn , with corresponding e functions α ∈ K∞ and ι ∈ K∞ . Then the function hb : Rn → R defined as hb (x) = h(x) + b, with b ∈ R>0 , is an ET-ISSf-BF satisfying the strong ET-ISSf barrier property on the set Cb ⊂ Rn defined as: Cb ≜ {x ∈ Rn | hb (x) ≥ 0}. (6.203) Proof. Observe that: Lf hb (x) + Lg hb (x)k(x + e) = Lf h(x) + Lg h(x)k(x + e), ≥ −α(h(x)) − ι(∥e∥), (6.204) (6.205) ≥ −α(hb (x) − b) + α(−b) − α(−b) − ι(∥e∥), (6.206) = −αb (hb (x)) + db − ι(∥e∥), (6.207) 328 e for all x, e ∈ Rn , where αb ∈ K∞ is defined as αb (r) = α(r − b) − α(−b) and db = −α(−b) > 0. Thus an ET-ISSf-BF can be used with the trigger law (6.189) by enlarging the safe set an arbitrarily small amount, captured by the following result: Corollary 2. If h is an ET-ISSf-BF for (6.155) on the set C satisfying Assumptions (2-4) of Theorem 47, then hb is an ET-ISSf-BF for (6.155) on the set Cb satisfying Assumptions (1-4) of Theorem 47 such that the corresponding trigger-law renders the event-triggered closed-loop system (6.151) safe with respect to Cb , and yields an MIET. This is effectively an instance of Input-to-State Safety, in which case the original set C defined via h becomes an ISSf safe set. We note that the larger the set is made (via a larger choice of b), the larger the MIET will be. This effectively highlights a trade-off that arises in the context of safety but not in stabilization: allowing motion near the boundary of a safe set requires additional relaxations to achieve the additional desirable property of an MIET. To verify the ability of this trigger to keep the system safe and have a MIET, I simulate the event-triggered closed-loop system (6.167) using both the naïve trigger law (6.169) and the corrected trigger-law (6.189). The results of these simulations can be seen in Figure 6.15. Observe that although both systems are kept safe, the trigger-law not using the strong ISSf barrier property has interevent times that approach 0. Conclusion In this section I have presented work on safety-critical event-triggered control. I begin by reviewing the formulation of event-triggered stabilization through ISS-LFs as originally presented in [96]. In attempting to translate these results to the safety-critical setting through the use of ISSf-BFs, I reveal a challenge that occurs due to motion of the system along the boundary of a set C that is to be kept forward invariant. I rigorously show that this challenge can prevent the existence of an MIET through a counterexample. I conclude by proposing the notion of the strong ET-ISSf barrier property, which limits tangential motion on the boundary of the set C and allows for a trigger-law that possesses an MIET, as I show in simulation. 329 1 0.8 0.5 0.6 0 0.4 -0.5 0.2 -1 -1 0 -0.5 0 0.5 1 0 2 4 6 8 10 10-1 10-2 10-3 100 101 102 103 Figure 6.15. Simulation results demonstrating safety achieved with an event-triggered controller. (Top, Left) State trajectories for both triggers remain within the safe set for the length of the simulation. (Top, Right) The value of the ISSf-BF h remains above zero for the length of the simulation, corresponding to the system remaining safe. (Right) The interevent times of the two trigger laws. The interevent times of the trigger law (6.169) decreases towards 0 as predicted by Lemma 11 while the trigger law (6.189) satisfies the theoretical bound. 6.6 Conclusion In this chapter I have presented a collection of results on designing safetycritical controllers that are robust to input sampling. In Section 6.2 I look at a few simulation examples that demonstrate the fragility of the (CLF-QP) controller to sampling, even at high sample rates. Following this, I provide mathematical descriptions of the sampled-data and event-triggered sampling paradigms. Next, in Section 6.3 I present my first set of contributions in the form of a framework for using feedback linearization and CLF-based controllers specified via convex optimization problems for sampled-data systems. I augment the framework of sampled-data control through approximate discretetime models presented in [81] to support local stability results and the coordinate transformations that accompany feedback linearization. Given this setup, 330 I show that a continuous-time feedback linearizable system with locally exponentially stable zero dynamics can be practically stabilized at sufficiently high sample rates using a sample-and-hold implementation of the continuous-time feedback linearizing controller without modification. As was reviewed in Section 2.4, I then use feedback linearization to produce a discrete-time CLF which I can incorporate into a CLF-based controller specified as a convex quadratically constrained quadratic program (QCQP). This controller is shown to greatly outperform a sample-and-hold implementation of the continuous-time (CLF-QP) controller in the simulation environments in Section 6.2. In Section 6.4, I present my second set of contributions in the form of a framework for safety-critical sampled-data control through Control Barrier Functions and approximate discrete-time models. I consider a larger class of approximate discrete-time models that support working with CBFs that have a relative degree greater than one, and show how these models satisfy certain consistency properties. Following this, I present definitions of practical safety, sampleddata barrier functions, and sampled-data Control Barrier Functions, and establish how a family of sampled-data barrier functions for a family of approximate discrete-time models implies the exact discrete-time system is practically safe. Following this, I consider a large class of nonlinear systems for which I can show that convexity properties of a sampled-data CBF can be preserved through the use of a Runge-Kutta approximation of the appropriate order, leading to controller that can be specified as convex optimization problems. I demonstrate these controllers in simulation, showing that safety violations decrease with increased sample rate as described by practical safety. Lastly, in Section 6.5 I present my third set of contributions in the form of a framework for safety-critical event-triggered control through ISSf-BFs. I first review the framework for event-triggered stabilization through ISS-LFs presented in [96]. I then consider a naïve transfer of this framework to the safety-critical setting, and show that the importance requirement of a minimum interevent time fails to hold through a rigorous counterexample. Building off the challenges seen in this counterexample, I present a modification that leads to event-triggered controllers that ensure safety and a minimum interevent time, which I show in simulation. There are several directions for future work building off the results presented in this chapter. The controllers that were produced through CLFs and CBFs with convex optimization in Sections 6.3 and 6.4, respectively, are more complex 331 forms of convex optimization problems than QPs. In particular, the CLF-based controllers are QCQPs, and the CBF-based controllers depend on the safe set specification, returning QPs for half-space CBFs, QCQPs for ellipsoidal CBFs, SOCPs for norm CBFs, and more general convex programs for CBFs that utilize other convex functions. While the sampled-data control paradigm doesn’t require controllers be locally Lipschitz continuous, or even continuous (because they are not evaluated continuously in time in the differential equation, they are held constant), these types of regularity properties are often desirable in practically implementing these controllers. Thus, it would be worthwhile to characterize the regularity properties of these controllers similar to the way that regularity properties of the (CLF-QP) and (CBF-QP) controllers have been characterized in [51]. Doing so may not be able to rely on the sort of tools used in [51], and may require a more analysis-driven approach using tools from variational analysis [223], [224]. Another direction would consider the approximate discrete-time models used. My work uses the relatively simple Euler and Runge-Kutta approximations, in large part due to their wide existing use, but also because they can be shown to preserve the feedback linearization and convexity properties ideal for control synthesis. I first note that my theory did not require any thing about the coefficients ai,j in the Runge-Kutta approximation. Many methods (of the sameorder) for numerical approximation fall under the umbrella of Runge-Kutta approximations with different values for these coefficients, and can be shown to demonstrate different desirable properties for specific problem structures. Thus, a comparative study, both theoretically and empirically, evaluating controller performance using different values of the coefficients associated with common numerical approximation schemes may be a worthwhile direction of exploration. Second, other approximation schemes outside of Runge-Kutta approximations could be explored, with the goal of obtaining higher approximation accuracy or conforming more closely to underlying structure in the dynamics. An important focus of this study would be on what sorts of approximations preserve convexity in a way that allows for efficient downstream control synthesis, as was considered in my work. Lastly, I think there are interesting avenues to consider event-triggered paradigms. As noted in [95], the widespread adoption and use of event-triggered control schemes has been lacking. It is my belief that the adoption of such methods in 332 robotics has not happened because the cost for switching inputs to a robot at a torque level are fairly negligible, and it is generally best to run a controller as fast as possible on a given set of hardware. Instead, I think that the use of event-triggered ideas in robotics need to focus on tasks that are costly to perform, and for which it is desirable not to have to perform many times. One such example could be event-triggered learning, where new data is incorporated into learning models only after it has been deemed to contain enough information to warrant the expensive learning process. In fact, recent work has even begun to explore this question [330]. I believe it would be worthwhile to develop an understanding of the stability and safety of these types of dynamic processes through tools from event-triggered control as explored in this thesis. 333 Chapter 7 CONCLUSION Motivated by the tragic failures seen with the ongoing proliferation of autonomous systems in modern society, this thesis has focused on making progress towards a complete theory of robust safety-critical. Built upon the foundational tools of Control Lyapunov Functions (CLFs) and Control Barrier Functions (CBFs), my contributions in this thesis constitute a collection of methods for resolving the challenges of model error, disturbances, measurement error, and input sampling seen in real-world applications. In each chapter of this thesis, I show the consequences for safety in failing to address these challenges in control design through simulation or experimental demonstration. Equipped with an understanding of these failures, I develop methods that provide theoretically rigorous guarantees of robust safety. A hallmark of my contributions is the focus on developing controllers that can be specified via convex optimization problems, enabling their computationally efficient implementation on real-world platforms, as demonstrated in the collection of experimental results presented in this thesis. In summary, the contributions of this thesis are as follows: • Learning Frameworks: In Sections 3.3-3.8 I present a series of three episodic learning frameworks for resolving model error as it directly impacts stability or safety as encoded by CLFs and CBFs, respectively. In particular, these frameworks address the fact that only model error that affects the time derivative of a CLF or CBF needs to be characterized to achieve a control objective. The frameworks in Sections 3.3 and 3.5 take an episodic learning approach built on supervised learning with a focus on learning models that are affine in the control input, permitting their direct integration into existing convex optimization-based controllers. Accompanying these frameworks in Sections 3.4 and 3.6 are a rigorous characterization of the impacts of residual learning error on stability and safety through the notions of Projection-to-State Stability (PSS) and Projection-to-State Safety (PSSf). In Section 3.7 I explore some of the challenges with these frameworks on high-dimensional system, and utilize 334 a learning model that instead learns a projected disturbance associated with a particular controller, allowing iterative improvement in achieving safety objectives. Lastly, in Section 3.8, I draw upon the lessons collected in developing the preceding frameworks to present a data-driven control method through robust convex optimization. Importantly, this method capitalizes on the fact that it is not necessary to completely characterize model error to choose robust control inputs. • Adaptive Safety-Critical Control: In Section 3.9 I present the first instance of an adaptive Control Barrier Function for safety-critical control in the presence of parametric model error. Building upon the tool of adaptive Control Lyapunov Functions in [47], I develop an understanding of the challenges in achieving forward invariance as opposed to stability with adaptive control techniques, and present a notion of aCBFs that resolve these challenges. • Tunable Input-to-State Safety: In Section 4.2 I highlight some of the challenges in balancing performance and robustness to disturbances with controllers designed with Input-to-State Safe Control Barrier Functions as presented in [58]. In Sections 4.3 and 4.4 I greatly improve upon this balance of performance and robustness through the the notion of Tunable Input-to-State Safety, which allows prioritizing robustness to disturbances near the boundary of a set that is to be kept forward invariant, and relaxing robustness requirements deep within the set. This methodology is experimentally demonstrated on an autonomous semitrailer truck, showcasing it on a challenging real-world problem. • Stochastic Safety-Critical Control: In Section 4.5 I present a framework for safety-critical control of discrete-time systems with stochastic disturbances. This work focuses on using the inherent robustness properties of CBFs to produce ISSf-like finite-time safety guarantees. Additionally, it studies how the the interaction between convexity properties of CBFs and the distributions of stochastic disturbances must be accounted for in producing theoretically rigorous safety-critical controllers specified as convex optimization problems. • Measurement-Robust Control Barrier Functions: In Sections 5.2 I explore how model error impacts safety-critical control, and develop 335 the notion of Measurement-Robust Control Barrier Functions as a tool for measurement-robust control synthesis through convex optimization. In Section 5.3 MR-CBFs are integrated with backup-set CBF methods to produced a measurement-robust controller specified as a second-order cone program (SOCP) that is demonstrated experimentally on a Segway platform. Importantly, this result indicates that the controllers specified via convex optimization problems that are more complex than QPs (such as QCQPs and SOCPs) in this thesis are efficient enough to be deployed in real-time on real-world robotic systems. • Preference Based Learning: In Section 5.4 I explore how preferencebased learning can be used to tune the robustness parameters of a CBFbased controller, which is the first instance of using preference-based learning with CBFs. While my particular contributions in this work focus on the control theoretic aspects of safety guarantees, it is an important result as it presents a methodology that may be deployed with many of the methods presented in this thesis to produce controllers that are both robust and performant. • Sampled-Data Control: In Section 6.2 I highlight the fragility of the continuous-time (CLF-QP) controller to a sample-and-hold implementation for a number of systems in simulation. I resolve this issue in Section 6.3 by presenting a framework for stabilizing sampled-data control through approximate discrete-time models. Building upon the framework in [81], I show how continuous-time feedback linearizable systems can be practically stabilized through a sample-hold implementation of a feedback linearizing controller. Furthermore, I extend the pipeline from feedback linearization to CLFs to this setting, producing discrete-time CLFs which can be incorporated into convex-optimization based controllers specified as quadratically constrained quadratic programs (QCQPs). I demonstrate the significant improvement in performance of these controllers over their continuous-time counterparts with a sampled-data implementation in simulation. In Section 6.4 I develop a framework for safety-critical sampled-data control through CBFs. I propose notions of practical safety and sampled-data CBFs, and rigorously connect designs using approximate discrete-time models to the practically safety of the exact discrete-time sampled-data dynamics of a system. Lastly, I 336 show how convexity properties of a SD-CBF can be utilized with an appropriately chosen approximate discrete-time model to produce convex optimization-based controllers. • Event-Triggered Safety-Critical Control: In Section 6.5 I provide a framework for event-triggered safety-critical control. Drawing inspiration from the work in [96] that studies stabilizing event-triggered control through ISS-LFs, my approach studies event-triggered safety through the perspective of ISSf-BFs. I show that that directly transferring the approach in [96] is sufficient to ensure safety, but will not guarantee a minimum interevent time. To resolve this, I propose the strong ISSf-BF property and show how it can be used to guarantee both safety and a minimum interevent time. Together, these contributions constitute a significant advance in the theory of robust safety-critical control, and unified by the use of CLFs and CBFs, provide a cohesive approach for addressing the vital challenges faced by realworld systems. 7.1 Infeasible Optimization Problems Throughout this thesis there has been a focus on developing controllers specified via convex optimization problems. In implementing these controllers, a relevant question that practitioners may face is what to do if the optimization problem defining these controllers returns that there is no feasible solution (or does not return a result at all). This is certainly an important question for developing the ability to trace faults in complex systems. I wish to note that in some sense, the ability to entirely eliminate the possibility of infeasibility while maintaining rigorous theoretical stability and safety guarantees is tied directly to whether the assumptions a control designer makes actually hold. If exceptional events can happen (as they are want to do in the real world) that violate these assumptions, there is always some potential for infeasibility or failure to achieve stability and safety. The following discussion seeks to highlight common reasons for infeasibility, and discuss practical methods that can be used to improve upon feasibility. A practitioner will hopefully find this helpful in identifying why their controller went infeasible, and provide a spectrum of increasingly involved options to consider in trying to improve upon their design. 337 Numerical Issues First, the cost functions in these controllers tend to be fairly well-behaved, and issues with infeasibility (either certifiable infeasibility, or due to the solver not returning any result) are almost always due to the constraints. One annoying, but fairly mild, source of infeasibility is numerical conditioning, which typically leads to the solver not returning. If we recall the CLF condition: Lf V (x) + Lg V (x)u ≤ −k3 ∥x − xe ∥2 , (7.1) with a = 2, we note that the term Lg V (x) multiplies the decision variable u in the optimization problem. At the equilibrium point xe , we will often have that Lf V (x) and Lg V (x) will converge to 0 and 0m , respectively, linearly with the value of ∥x − xe ∥, while the right hand-side converges to 0 quadratically in ∥x − xe ∥. In this case, the vanishing of the term multiplying the decision variable can be rectified by rescaling the problem by dividing by ∥x − xe ∥, noting that the quadratic term will remain small. In fact, most off-the-shelf optimization tools will perform this sort of rescaling for a user. But such rescaling away from xe is not so straightforward. In particular, if we have a state x∗ ̸= xe such that Lg V (x∗ ) = 0m , then for a state x close to x∗ , dividing each term of the constraint by ∥Lg V (x)∥ will amplify the terms Lf V (x) and −k3 ∥x − xe ∥2 greatly, thereby still leading to a poorly scaled problem. Many off-the-shelf optimization tools will struggle with this problem, but it reflects important geometric properties of the interaction between the underlying system dynamics and CLF. Instead, observe that if we have a CLF, then in a neighborhood of x∗ , we have that: Lf V (x) ≤ −k3 ∥x − xe ∥2 + ϵ, (7.2) for some small value of ϵ ≈ 0. This states that by taking no input (or using knom ), we are almost satisfying the inequality we need for local exponential stability. Thus, rather than solve the optimization problem, it may be prudent to set a tolerance on the value of ∥Lg V (x)∥ under which no input (or knom ) is used. The price of this is a small degradation in the convergence rate of stability, or ISS-like behavior. The same sort of issue arises with CBFs when Lg h(x∗ ) = 0m , and can be similarly addressed by using knom directly at the expense of ISSf behavior. This type of case-based solution is an effective tool for many of the CLF and CBF-based methods throughout this thesis, and generally leads to insignificant degradation in stability and safety guarantees. 338 Infeasible Constraints Another source of infeasibility (that often produces certifiable infeasibility) is when a function V or h is not actually the purported type of CLF or CBF it is claimed to be when it is used as a constraint in the optimization problem. Why does this happen? One example is if the CLF or CBF is simply specified for the system without using a constructive technique like feedback linearization as shown in Chapter 2. In this case, the CLF or CBF is being used as a heuristic design primitive, but not necessarily as a rigorous control theoretic tool. As shown by many of the simulation and experimental results in this thesis, this is often an okay approach (though I will discuss alternatives if it is not). Another example is if the CLF or CBF has numerical models (like learning models in Chapter 3) integrated into the constraint. These numerical models often do not display exactly the right analytical behavior at the “pain points” (such as when Lg V (x) or Lg h(x) approach 0m ), and can lead to infeasibility in the constraint. Note that this sort of problem can occur even if the CLF or CBF is in fact a CLF or CBF for the true dynamics of the system, as discussed in Section 3.3. An additional example is when a CLF or CBF for a system (one that actually is a CLF or CBF) is modified with robustness terms. Examples include using ISS-CLFs or ISSfCBFs with unmatched disturbances (this is why matched disturbances are a nicer problem to work with), adding model error sets into the CLF or CBF constraint as in Section 3.8 (where I explicitly address feasibility), and MRCBFs in Section 5.2. The robustness modifications in these formulations may lead to infeasibility, especially if large robustness parameters are used. This is not unintuitive, as requiring robustness to an increasingly large quantity of model inaccuracies will at some point become impossible in most control formulations. Lastly, and perhaps most importantly, even if a CLF or CBF meets the necessary infimum and supremum conditions used throughout this thesis, nearly all real-world systems face limitations to their inputs, such as input bounds. It is generally not possible to take any input in Rm . In this case, if these input constraints are enforced in the optimization with the CLF or CBF constraint, infeasibility may occur. If instead input saturation is applied after solving the optimization problem, it will not only be unclear if theoretical stability or safety properties will be preserved, but it will also be unclear if a potential feasible solution satisfying input bounds did in fact exist. 339 Relaxations What are our options in the face of these potential causes of infeasibility? There are three main directions, all of which I believe are currently areas of research interest. The first is simply to relax the optimization problems. Such a relaxation was employed in the (CLF-CBF-QP) controller, and used throughout many of the experimental results in this thesis. This approach can guarantee the “feasibility” of the optimization problem in the face of not having a valid CLF or CBF, poorly behaved numerical models, excessive robustness, and input constraints. Of course, it is not a silver bullet, and will compromise the theoretical stability and safety guarantees of the controller. If the relaxation is kept small (one is only a small distance away from being feasible) over a region of the interest in the state space, one could employ the ideas of PSS and PSSf explored in Sections 3.4 and 3.6, respectively, to state what the degradation of the guarantees will be. But there are perhaps more interesting and principled approaches for relaxation, which suggest the potential for further research into the topic. Examples include the formulation in [51, QP Problem −γm Version], which uses a particular structure of the relaxation term to ensure that local exponential stability guarantees are preserved for equilibrium points that remain inside the interior of a set C that is to be kept forward invariant. An other idea is to add additional decision variables to the controller that allow the strength of stability or safety requirements to be modulated online to achieve feasibility. Examples would be making the parameter k3 in a CLF constraint a decision variable that is constrained to remain above some small positive value, or parameterizing the function α ∈ Ke in a CBF constraint to be a linear function with a coefficient that is a decision variable constrained above some small positive value, as was done in [256, Section 2.C]. This type of scheme relaxes the convergence rate of local exponential stability, or allows a system to more rapidly approach the boundary of a set that is to be kept forward invariant, but can improve the feasibility of the controller while still giving some form of stability and safety guarantees. Constructing CLFs and CBFs The second direction is the offline construction of CLFs and CBFs that satisfy the necessary requirements arising from system dynamics, safety specifications, robustness considerations, and input constraints. In this way, one can reduce the number of assumptions made on the satisfaction of these requirements by 340 explicitly addressing them. Examples would include tools such as sums-ofsquares (SOS) programming which has been used for computing CLFs [331] and CBFs [332], [333], solving the Hamilton-Jacobi-Bellman (HJB) equations to produce control invariant sets that yield CLFs [334] and CBFs [229], [335], or constructing CLFs and CBFs through data-driven methods [176], [177]. A significant benefit of these approaches is the potential to utilize large computational capabilities to produce solutions meeting all of the above requirements simultaneously in a way that can be verified before deployment on a safetycritical system. Of course, if these requirements are insufficient for the actual control task at hand, infeasibility may still occur, and these offline computations may need to be performed again with adjusted requirements. One limit of this approach is the well-known “curse of dimensionality”, with many of these methods being limited to single-digit state and input dimensions and offline computations. For a high-dimensional system, a prudent course of action may be to reduce the system to a collection of smaller, interconnected subsystems, and deploy the computational techniques with additional requirements to compensate for any issues that arise due to using an interconnected system modeling perspective. Multi-Rate Control Drawing some degree of inspiration from this previous comment, the third direction is to offload some of the requirements that a CLF or CBF-based controller needs to satisfy to a higher-level in the autonomy stack. For instance, one may require that the equilibrium state or trajectory that a CLF controller is stabilizing to be designed (online) to ensure that the CLF controller can accurately track the trajectory while meeting robustness requirements and input constraints. This was the focus on my work in [336], which modified the constraints of a low-frequency model predictive controller (MPC) producing desired trajectories to ensure the resulting trajectory could be tracked by a (CLF-QP) controller. Other recent work has also begun to explore this sort of multi-rate control paradigm, where high-level and mid-level tasks effectively reduce the requirements that need to be satisfied by low-level CLF and CBF-based controllers [337]–[339]. A particular interesting observation in this direction occurred in my work in [22]. By fusing CBF constraints into both a low-level controller and a mid-level model predictive controller producing trajectories, it was possible to achieve consistently safer behavior than merely 341 using CBFs only at the low-level, using CBFs only at the mid-level, and using hard state constraints at the mid-level and CBFs at the low-level. This suggests that truly robust safety is not a problem that can, nor should, be achieved at one level in the autonomy stack. Instead, feasibility at all levels of the autonomy stack is best achieved by allocating some portion of safety requirements to each level in a principled manner. I believe that this will be a critical direction of future research over the next 20 years as efforts continue to transfer theoretically rigorous control systems into complex real world applications. 7.2 Future Work At the conclusion of each chapter I have provided directions for future work related to achieving robustness to each respective challenge. One future direction that stems from the results of this thesis is the construction of a safety-critical control framework that is simultaneously robust to each of the presented challenges. In reality, real-world systems will face model error, disturbances, measurement error, and input sampling all at once. As each chapter of this thesis has shown, neglecting any one of these challenges can lead to significant violations of safety requirements. I believe that capturing the results of this thesis into a singular framework will require addressing two challenges: • A theme of my approaches is taking the (CLF-QP) and (CBF-QP) controllers and making them robust by producing controllers specified by more complex convex optimization problems than QPs, and in particular QCQPs and SOCPs. This can be observed in the data-driven control framework in Section 3.8, the measurement-robust control in 5.2, and the sampled-data controllers in Sections 6.3 and 6.4. The question that must be addressed is if it is possible to integrate multiple forms of robustness without compromising the convexity of the control synthesis process. Throughout this thesis structural assumptions such as Lipschitz continuity or approximate models are used in a way that permits convex synthesis. Answering this question will rely on finding the right set of structural assumptions and approximations that ensure multiple forms of robustness while maintaining convexity. It is my belief that the most fruitful starting point for addressing this question will follow along the lines of the robust optimization used in the data-driven control 342 framework in Section 3.8 in conjunction with the discrete-time approximations used in Sections 6.3 and 6.4. In particular, if model error, disturbances, and measurement error can be cast as the model error sets used in Section 3.8, it may be possible to consider satisfying CLF and CBF requirements for all discrete-time approximations permitted by these error sets. It is likely this will yield more complex convex optimization problems, such as semidefinite programs. As the capabilities of computing resources become increasingly powerful, the potential to deploy these complex controllers on real-world systems may become possible. • In Section 5.4 both disturbance robustness using ISSf and measurement robustness using MR-CBFs are integrated into the same controller. The purpose of that work was in addressing the fact that having both forms of robustness led to extremely conservative behavior that was not desirable. As more forms of robustness are brought into the same controller, it is likely that conservativeness will continue to increase, rendering such a controller pointless. Thus, how to tune these robust controllers to attain performance and robustness is an important question. Preference-based learning was shown to be a powerful tool for achieving this, but there may be opportunities to achieve this through other data-driven techniques. In particular, if data can be used to identify robustness that is not necessary (because the model error, disturbances, or measurement error being robustified against isn’t actually present), then the characterization of error being used in this controller can be improved until a desirable level of performance is achieved. In conclusion, I believe there many important questions remaining in building safety-critical controllers. I think that the most important focus of this work is continuing to integrate with advancing computational capabilities, and utilizing real experimental data to balance conservativeness and performance. 343 BIBLIOGRAPHY [1] M. Chui, K. George, J. Manyika, and M. Miremadi, “Human + machine: A new era of automation in manufacturing,” 2017. [Online]. Available: https://www.mckinsey.com/capabilities/operations/ourinsights/human- plus- machine- a- new- era- of- automation- inmanufacturing. [2] J. Diechmann, E. Ebel, K. Heineke, R. Heuss, M. Kellner, and F. Steiner, “Autonomous driving’s future: Convenient and connected,” 2023. [Online]. Available: https://www.mckinsey.com/industries/ automotive-and-assembly/our-insights/autonomous-drivingsfuture-convenient-and-connected#/. [3] M. McFarland, “Tesla ‘full self-driving’ triggered an eight-car crash, a driver tells police,” 2022. [Online]. Available: https://www.cnn.com/ 2022/12/21/business/tesla-fsd-8-car-crash/index.html. [4] A. Adler, “Feds close probe into tusimple autonomous truck crash,” 2023. [Online]. Available: https : / / www . freightwaves . com / news / feds-close-probe-into-tusimple-autonomous-truck-crash. [5] A. J. Hawkins, “Tesla’s autopilot was engaged when model 3 crashed into truck, report states,” 2019. [Online]. Available: https : / / www . theverge . com / 2019 / 5 / 16 / 18627766 / tesla - autopilot - fatal crash-delray-florida-ntsb-model-3. [6] L. Wamsley, “Backup driver of autonomous uber suv charged with negligent homicide in arizona,” 2022. [Online]. Available: https://www. npr.org/2020/09/16/913530100/backup-driver-of-autonomousuber-suv-charged-with-negligent-homicide-in-arizona. [7] T. Krisher and S. Dazio, “Felony charges are 1st in a fatal crash involving autopilot,” 2022. [Online]. Available: https : / / apnews . com / article / tesla - autopilot - fatal - crash - charges 91b4a0341e07244f3f03051b5c2462ae. [8] Z. Artstein, “Stabilization with relaxed controls,” Nonlinear Analysis: Theory, Methods & App., vol. 7, no. 11, pp. 1163–1173, 1983. doi: 10.1016/0362-546X(83)90049-4. [9] E. D. Sontag, “A ‘universal’ construction of artstein’s theorem on nonlinear stabilization,” Sys. & Control Let., vol. 13, no. 2, pp. 117–123, 1989. doi: 10.1016/0167-6911(89)90028-5. [10] P. Wieland and F. Allgöwer, “Constructive safety using control barrier functions,” IFAC Proc. Vol., vol. 40, no. 12, pp. 462–467, 2007. doi: 10.3182/20070822-3-ZA-2920.00076. 344 [11] A. Ames, J. Grizzle, and P. Tabuada, “Control barrier function based quadratic programs with application to adaptive cruise control,” in Proc. IEEE 53rd Conf. on Decision and Control (CDC), Los Angeles, CA, USA, 2014, pp. 6271–6278. doi: 10.1109/CDC.2014.7040372. [12] X. Xu, T. Waters, D. Pickem, et al., “Realizing simultaneous lane keeping and adaptive speed regulation on accessible mobile robot testbeds,” in Proc. 1st IEEE Conf. on Control Tech. and App. (CCTA), Kohala Coast, HI, USA, 2017, pp. 1769–1775. doi: 10 . 1109 / CCTA . 2017 . 8062713. [13] T. Gurriet, M. Mote, A. D. Ames, and E. Feron, “An online approach to active set invariance,” in Proc. IEEE 57th Conf. on Decision and Control (CDC), Miami Beach, FL, USA, 2018, pp. 3592–3599. doi: 10.1109/CDC.2018.8619139. [14] R. Grandia, A. J. Taylor, A. Singletary, M. Hutter, and A. D. Ames, “Nonlinear model predictive control of robotic systems with control lyapunov functions,” in Proc. Robotics: Science and Sys. XVI (RSS), Bend, OR, USA, 2020. doi: 10.15607/RSS.2020.XVI.098. [15] L. Wang, A. D. Ames, and M. Egerstedt, “Safety barrier certificates for collisions-free multirobot systems,” IEEE Trans. on Robotics, vol. 33, no. 3, pp. 661–674, 2017. doi: 10.1109/TRO.2017.2659727. [16] L. Wang, E. A. Theodorou, and M. Egerstedt, “Safe learning of quadrotor dynamics using barrier certificates,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Brisbane, QLD, Australia, 2018, pp. 2460–2465. doi: 10.1109/ICRA.2018.8460471. [17] S. M. Khansari-Zadeh and A. Billard, “Learning control lyapunov function to ensure stability of dynamical system-based robot reaching motions,” Robotics and Autonomous Sys., vol. 62, no. 6, 2014. doi: 10. 1016/j.robot.2014.03.001. [18] A. Singletary, W. Guffey, T. G. Molnar, R. Sinnet, and A. D. Ames, “Safety-critical manipulation for collision-free food preparation,” IEEE Robotics and Automation Let., vol. 7, no. 4, pp. 10 954–10 961, 2022. doi: 10.1109/LRA.2022.3192634. [19] W. S. Cortez, D. Oetomo, C. Manzie, and P. Choong, “Control barrier functions for mechanical systems: Theory and application to robotic grasping,” IEEE Trans. on Control Sys. Tech., vol. 29, no. 2, pp. 530– 545, 2019. doi: 10.1109/TCST.2019.2952317. [20] S. Farzan, V. Azimi, A.-P. Hu, and J. Rogers, “Adaptive control of wire-borne underactuated brachiating robots using control lyapunov and barrier functions,” IEEE Trans. on Control Sys. Tech., vol. 30, no. 6, pp. 2598–2614, 2022. doi: 10.1109/TCST.2022.3160058. 345 [21] A. Alan, A. J. Taylor, C. R. He, A. D. Ames, and G. Orosz, “Control barrier functions and input-to-state safety with application to automated vehicles,” arXiv preprint arXiv:2206.03568, 2022. [Online]. Available: https://arxiv.org/abs/2206.03568. [22] R. Grandia, A. J. Taylor, A. D. Ames, and M. Hutter, “Multi-layered safety for legged robots via control barrier functions and model predictive control,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Xi’an, China, 2021, pp. 8352–8358. doi: 10.1109/ICRA48506. 2021.9561510. [23] R. K. Cosner, M. Tucker, A. J. Taylor, et al., “Safety-aware preferencebased learning for safety-critical control,” in Proc. 4th Learning for Dynamics and Control (L4DC), vol. 168, Stanford, CA, USA, 2022, pp. 1020–1033. [Online]. Available: https://proceedings.mlr.press/ v168/cosner22a.html. [24] K. Galloway, K. Sreenath, A. D. Ames, and J. W. Grizzle, “Torque saturation in bipedal robotic walking through control lyapunov functionbased quadratic programs,” IEEE Access, vol. 3, pp. 323–332, 2015. doi: 10.1109/ACCESS.2015.2419630. [25] N. Csomay-Shanklin, R. K. Cosner, M. Dai, A. J. Taylor, and A. D. Ames, “Episodic learning for safe bipedal locomotion with control barrier functions and projection-to-state safety,” in Proc. 3rd Learning for Dynamics and Control (L4DC), vol. 144, Zürich, Switzerland, 2021, pp. 1041–1053. [Online]. Available: http://proceedings.mlr.press/ v144/csomay-shanklin21a/csomay-shanklin21a.pdf. [26] A. D. Ames and M. Powell, “Towards the unification of locomotion and manipulation through control lyapunov functions and quadratic programs,” in Control of Cyber-Physical Sys. Springer, 2013, pp. 219– 240. doi: 10.1007/978-3-319-01159-2_12. [27] T. Gurriet, A. Singletary, J. Reher, L. Ciarletta, E. Feron, and A. Ames, “Towards a framework for realizable safety critical control through active set invariance,” in Proc. ACM/IEEE 9th Int. Conf. on CyberPhysical Sys. (ICCPS), Porto, Portugal, 2018, pp. 98–106. doi: 10 . 1109/ICCPS.2018.00018. [28] R. T. Rockafellar, “Lagrange multipliers and optimality,” SIAM Review, vol. 35, no. 2, pp. 183–238, 1993. doi: 10.1137/103504. [29] S. Boyd and L. Vandenberghe, Convex optimization. Cambridge University Press, 2004, isbn: 9780521833783. [30] G. C. Calafiore and L. El Ghaoui, Optimization models. Cambridge university press, 2014, isbn: 978-1-107-05087-7. [31] Y. Nesterov and A. Nemirovskii, Interior-point polynomial algorithms in convex programming. SIAM, 1994. doi: 10.1137/1.9781611970791. 346 [32] Y. E. Nesterov and M. J. Todd, “Self-scaled barriers and interior-point methods for convex programming,” INFORMS Mathematics of Operations research, vol. 22, no. 1, pp. 1–42, 1997. doi: 10.1287/moor.22. 1.1. [33] S. Diamond and S. Boyd, “Cvxpy: A python-embedded modeling language for convex optimization,” J. of Machine Learning Research, vol. 17, no. 1, pp. 2909–2913, 2016. [Online]. Available: https://www. jmlr.org/papers/volume17/15-408/15-408.pdf. [34] J. Lofberg, “Yalmip: A toolbox for modeling and optimization in matlab,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Taipei, Taiwan, 2004, pp. 284–289. doi: 10 . 1109 / CACSD . 2004 . 1393890. [35] A. Domahidi, E. Chu, and S. Boyd, “Ecos: An socp solver for embedded systems,” in Proc. IEEE European Control Conf. (ECC), Zürich, Switzerland, 2013, pp. 3071–3076. doi: 10.23919/ECC.2013.6669541. [36] M. ApS, Introducing the mosek optimization suite 10.0.40, 2019. [Online]. Available: https://docs.mosek.com/latest/intro/index. html. [37] Gurobi Optimization, LLC, Gurobi Optimizer Reference Manual, 2023. [Online]. Available: https://www.gurobi.com. [38] J. Umlauft, L. Pöhler, and S. Hirche, “An uncertainty-based control lyapunov approach for control-affine systems modeled by gaussian process,” IEEE Control Sys. Let., vol. 2, no. 3, pp. 483–488, 2018. doi: 10.1109/LCSYS.2018.2841961. [39] H. Ravanbakhsh and S. Sankaranarayanan, “Learning lyapunov (potential) functions from counterexamples and demonstrations,” Cambridge, MA, USA, 2017. doi: 10.15607/RSS.2017.XIII.049. [40] T. Westenbroek, F. Castaneda, A. Agrawal, S. Sastry, and K. Sreenath, “Lyapunov design for robust and efficient robotic reinforcement learning,” arXiv preprint arXiv:2208.06721, 2022. doi: 10.48550/arXiv. 2208.06721. [41] A. Anand, K. Seel, V. Gjærum, A. Håkansson, H. Robinson, and A. Saad, “Safe learning for control using control lyapunov functions and control barrier functions: A review,” Procedia Computer Science, vol. 192, pp. 3987–3997, 2021. doi: 10.1016/j.procs.2021.09.173. [42] P. Jagtap, G. J. Pappas, and M. Zamani, “Control barrier functions for unknown nonlinear systems using gaussian processes,” in Proc. IEEE 59th Conf. on Decision and Control (CDC), Jeju, South Korea, 2020, pp. 3699–3704. doi: 10.1109/CDC42340.2020.9303847. 347 [43] M. J. Khojasteh, V. Dhiman, M. Franceschetti, and N. Atanasov, “Probabilistic safety constraints for learned high relative degree system dynamics,” in Proc. 2nd Learning for Dynamics and Control (L4DC), vol. 120, Berkeley, CA, USA, 2020, pp. 781–792. [Online]. Available: https://proceedings.mlr.press/v120/khojasteh20a.html. [44] R. Cheng, G. Orosz, R. M. Murray, and J. W. Burdick, “End-to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks,” in Proc. AAAI Conf. on Artificial Intelligence, vol. 33, 2019, pp. 3387–3395. doi: 10.1609/aaai.v33i01.33013387. [45] C. Dawson, S. Gao, and C. Fan, “Safe control with learned certificates: A survey of neural lyapunov, barrier, and contraction methods for robotics and control,” IEEE Trans. on Robotics, pp. 1–19, 2023. doi: 10.1109/TRO.2022.3232542. [46] F. Castañeda, J. J. Choi, B. Zhang, C. J. Tomlin, and K. Sreenath, “Gaussian process-based min-norm stabilizing controller for controlaffine systems with uncertain input effects and dynamics,” in Proc. IEEE American Control Conf. (ACC), New Orleans, LA, USA, 2021, pp. 3683–3690. doi: 10.23919/ACC50511.2021.9483420. [47] M. Krstić and P. V. Kokotović, “Control lyapunov functions for adaptive nonlinear stabilization,” Sys. & Control Let., vol. 26, no. 1, pp. 17– 23, 1995. doi: 10.1016/0167-6911(94)00107-7. [48] Z. Li and M. Krstić, “Optimal design of adaptive tracking controllers for non-linear systems,” Automatica, vol. 33, no. 8, pp. 1459–1473, 1997. doi: 10.1016/S0005-1098(97)00072-1. [49] J. Moore and R. Tedrake, “Adaptive control design for underactuated systems using sums-of-squares optimization,” in Proc. IEEE American Control Conf. (ACC), Portland, OR, USA, 2014, pp. 721–728. doi: 10.1109/ACC.2014.6859508. [50] R. Takano and M. Yamakita, “Robust constrained stabilization control using control lyapunov and control barrier function in the presence of measurement noises,” in Proc. 2nd IEEE Conf. on Control Tech. and App. (CCTA), Copenhagen, Denmark, 2018, pp. 300–305. doi: 10.1109/CCTA.2018.8511637. [51] M. Jankovic, “Robust control barrier functions for constrained stabilization of nonlinear systems,” Automatica, vol. 96, pp. 359–367, 2018. doi: 10.1016/j.automatica.2018.07.004. [52] Y. Emam, P. Glotfelter, and M. Egerstedt, “Robust barrier functions for a fully autonomous, remotely accessible swarm-robotics testbed,” in Proc. IEEE 58th Conf. on Decision and Control (CDC), Nice, France, 2019, pp. 3984–3990. doi: 10.1109/CDC40024.2019.9029886. 348 [53] K. Garg and D. Panagou, “Robust control barrier and control lyapunov functions with fixed-time convergence guarantees,” in Proc. IEEE American Control Conf. (ACC), New Orleans, LA, USA, 2021, pp. 2292–2297. doi: 10.23919/ACC50511.2021.9482751. [54] X. Tan, W. S. Cortez, and D. V. Dimarogonas, “High-order barrier functions: Robustness, safety, and performance-critical control,” IEEE Trans. on Automatic Control, vol. 67, no. 6, pp. 3021–3028, 2021. doi: 10.1109/TAC.2021.3089639. [55] J. Buch, S.-C. Liao, and P. Seiler, “Robust control barrier functions with sector-bounded uncertainties,” IEEE Control Sys. Let., vol. 6, pp. 1994– 1999, 2021. doi: 10.1109/LCSYS.2021.3136653. [56] E. D. Sontag, “Smooth stabilization implies coprime factorization,” IEEE Trans. on Automatic Control, vol. 34, no. 4, pp. 435–443, 1989. doi: 10.1109/9.28018. [57] E. D. Sontag, “Input to state stability: Basic concepts and results,” in Nonlinear and Optimal Control Theory, Springer, 2008, pp. 163–220. doi: 10.1007/978-3-540-77653-6_3. [58] S. Kolathaya and A. D. Ames, “Input-to-state safety with control barrier functions,” IEEE Control Sys. Let., vol. 3, no. 1, pp. 108–113, 2018. doi: 10.1109/LCSYS.2018.2853698. [59] H. J. Kushner, “Stochastic stability and control,” Brown Univ Providence RI, Tech. Rep., 1967. [60] S. Prajna, A. Jadbabaie, and G. J. Pappas, “Stochastic safety verification using barrier certificates,” in Proc. IEEE 43rd Conf. on Decision and Control (CDC), Nassau, Bahamas, 2004, pp. 929–934. doi: 10.1109/CDC.2004.1428804. [61] J. Steinhardt and R. Tedrake, “Finite-time regional verification of stochastic non-linear systems,” The Int. J. of Robotics Research, vol. 31, no. 7, pp. 901–923, 2012. doi: 10.1177/0278364912444146. [62] A. D. Ames, X. Xu, J. W. Grizzle, and P. Tabuada, “Control barrier function based quadratic programs for safety critical systems,” IEEE Trans. on Automatic Control, vol. 62, no. 8, pp. 3861–3876, 2017. doi: 10.1109/TAC.2016.2638961. [63] C. Santoyo, M. Dutreix, and S. Coogan, “Verification and control for finite-time safety of stochastic systems via barrier functions,” in Proc. IEEE Conf. on Control Tech. and App. (CCTA), Hong Kong, China, 2019, pp. 712–717. doi: 10.1109/CCTA.2019.8920407. 349 [64] Z. Wang, H. Jing, C. Kurniawan, A. Chern, and Y. Nakahira, “Myopically verifiable probabilistic certificate for long-term safety,” in Proc. IEEE American Control Conf. (ACC), Atlanta, GA, USA, 2022, pp. 4894–4900. doi: 10.23919/ACC53348.2022.9867675. [65] A. Lavaei, S. Soudjani, and E. Frazzoli, “Safety barrier certificates for stochastic hybrid systems,” in Proc. IEEE American Control Conf. (ACC), Atlanta, GA, USA, 2022, pp. 880–885. doi: 10 . 23919 / ACC53348.2022.9867754. [66] A. Clark, “Control barrier functions for complete and incomplete information stochastic systems,” in Proc. IEEE American Control Conf. (ACC), Philadelphia, PA, USA, 2019, pp. 2928–2935. doi: 10.23919/ ACC.2019.8814901. [67] S. Yaghoubi, K. Majd, G. Fainekos, T. Yamaguchi, D. Prokhorov, and B. Hoxha, “Risk-bounded control using stochastic barrier functions,” IEEE Control Sys. Let., vol. 5, no. 5, pp. 1831–1836, 2020. doi: 10. 1109/LCSYS.2020.3043287. [68] A. Chern, X. Wang, A. Iyer, and Y. Nakahira, “Safe control in the presence of stochastic uncertainties,” in Proc. IEEE 60th Conf. on Decision and Control (CDC), Austin, TX, USA, 2021, pp. 6640–6645. doi: 10.1109/CDC45484.2021.9683542. [69] C. Santoyo, M. Dutreix, and S. Coogan, “A barrier function approach to finite-time stochastic system verification and control,” Automatica, vol. 125, p. 109 439, 2021. doi: 10.1016/j.automatica.2020.109439. [70] C. Wang, M. Bahreinian, and R. Tron, “Chance constraint robust control with control barrier functions,” in Proc. IEEE American Control Conf. (ACC), New Orleans, LA, USA, 2021, pp. 2315–2322. doi: 10. 23919/ACC50511.2021.9482973. [71] C. Wang, Y. Meng, S. L. Smith, and J. Liu, “Safety-critical control of stochastic systems using stochastic control barrier functions,” in Proc. IEEE 60th Conf. on Decision and Control (CDC), Austin, TX, USA, 2021, pp. 5924–5931. doi: 10.1109/CDC45484.2021.9683349. [72] M. Ahmadi, X. Xiong, and A. D. Ames, “Risk-averse control via cvar barrier functions: Application to bipedal robot locomotion,” IEEE Control Sys. Let., vol. 6, pp. 878–883, 2021. doi: 10.1109/LCSYS.2021. 3086854. [73] T. Gurriet, P. Nilsson, A. Singletary, and A. D. Ames, “Realizable set invariance conditions for cyber-physical systems,” in Proc. IEEE American Control Conf. (ACC), Philadelphia, PA, USA, 2019, pp. 3642–3649. doi: 10.23919/ACC.2019.8815332. 350 [74] A. Singletary, Y. Chen, and A. D. Ames, “Control barrier functions for sampled-data systems with input delays,” in Proc. IEEE 59th Conf. on Decision and Control (CDC), Jeju, South Korea, 2020, pp. 804–809. doi: 10.1109/CDC42340.2020.9304281. [75] W. Chu and Z. Ghahramani, “Preference learning with gaussian processes,” in Proc. Int. Conf. on Machine Learning (ICML), 2005, pp. 137–144. doi: 10.1145/1102351.1102369. [76] M. Tucker, E. Novoseller, C. Kann, et al., “Preference-based learning for exoskeleton gait optimization,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Paris, France, 2020, pp. 2351–2357. doi: 10. 1109/ICRA40945.2020.9196661. [77] M. Tucker, M. Cheng, E. Novoseller, et al., “Human preferencebased learning for high-dimensional optimization of exoskeleton walking gaits,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Las Vegas, NV, USA, 2020, pp. 3423–3430. doi: 10 . 1109 / IROS45743.2020.9341416. [78] K. Li, M. Tucker, E. Bıyık, et al., “Roial: Region of interest active learning for characterizing exoskeleton gait preference landscapes,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Xi’an, China, 2021, pp. 3212–3218. doi: 10.1109/ICRA48506.2021.9560840. [79] M. Tucker, N. Csomay-Shanklin, W.-L. Ma, and A. D. Ames, “Preference-based learning for user-guided hzd gait generation on bipedal walking robots,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Xi’an, China, 2021, pp. 2804–2810. doi: 10.1109/ ICRA48506.2021.9561515. [80] N. Csomay-Shanklin, M. Tucker, M. Dai, J. Reher, and A. D. Ames, “Learning controller gains on bipedal walking robots via user preferences,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Philadelphia, PA, USA, 2022, pp. 10 405–10 411. doi: 10 . 1109 / ICRA46639.2022.9811541. [81] D. Nešić, A. R. Teel, and P. V. Kokotović, “Sufficient conditions for stabilization of sampled-data nonlinear systems via discrete-time approximations,” Sys. & Control Let., vol. 38, no. 4-5, pp. 259–270, 1999. doi: 10.1016/S0167-6911(99)00073-0. [82] D. Nešić and A. R. Teel, “A framework for stabilization of nonlinear sampled-data systems based on their approximate discrete-time models,” IEEE Trans. on Automatic Control, vol. 49, no. 7, pp. 1103–1122, 2004. doi: 10.1109/TAC.2004.831175. [83] D. Nešić and A. R. Teel, “Backstepping on the euler approximate model for stabilization of sampled-data nonlinear systems,” in Proc. IEEE 40th 351 Conf. on Decision and Control (CDC), vol. 2, Orlando, FL, USA, 2001, pp. 1737–1742. doi: 10.1109/CDC.2001.981153. [84] D. Nešić and A. R. Teel, “Stabilization of sampled-data nonlinear systems via backstepping on their euler approximate model,” Automatica, vol. 42, no. 10, pp. 1801–1808, 2006. doi: 10.1016/j.automatica. 2006.05.015. [85] D. Nešić and L. Grüne, “Lyapunov-based continuous-time nonlinear controller redesign for sampled-data implementation,” Automatica, vol. 41, no. 7, pp. 1143–1156, 2005. doi: 10 . 1016 / j . automatica . 2005.03.001. [86] L. Grüne and D. Nešić, “Optimization-based stabilization of sampleddata nonlinear systems via their approximate discrete-time models,” SIAM J. on Control and Optimization, vol. 42, no. 1, pp. 98–122, 2003. doi: 10.1137/S036301290240258X. [87] L. Grüne, D. Nešić, and J. Pannek, “Model predictive sampled-data redesign for nonlinear systems,” in Proc. IEEE 44th Conf. on Decision and Control (CDC), Seville, Spain, 2005, pp. 36–41. doi: 10 . 1109 / CDC.2005.1582127. [88] L. Grüne, D. Nešić, and J. Pannek, “Model predictive control for nonlinear sampled-data systems,” in Assessment and future directions of nonlinear model predictive control, Springer, 2007, pp. 105–113. doi: 10.1007/978-3-540-72699-9_8. [89] G. Yang, C. Belta, and R. Tron, “Self-triggered control for safety critical systems using control barrier functions,” in Proc. IEEE American Control Conf. (ACC), Philadelphia, PA, USA, 2019, pp. 4454–4459. doi: 10.23919/ACC.2019.8814657. [90] J. Breeden, K. Garg, and D. Panagou, “Control barrier functions in sampled-data systems,” IEEE Control Sys. Let., vol. 6, pp. 367–372, 2021. doi: 10.1109/LCSYS.2021.3076127. [91] L. Niu, H. Zhang, and A. Clark, “Safety-critical control synthesis for unknown sampled-data systems via control barrier functions,” in Proc. IEEE 60th Conf. on Decision and Control (CDC), Austin, TX, USA, 2021, pp. 6806–6813. doi: 10.1109/CDC45484.2021.9683019. [92] J. Usevitch and D. Panagou, “Adversarial resilience for sampled-data systems using control barrier function methods,” in Proc. IEEE American Control Conf. (ACC), New Orleans, LA, USA, 2021, pp. 758–763. doi: 10.23919/ACC50511.2021.9482659. [93] Y. Zhang, S. Walters, and X. Xu, “Control barrier function meets interval analysis: Safety-critical control with measurement and actuation uncertainties,” in Proc. IEEE American Control Conf. (ACC), Atlanta, 352 GA, USA, 2022, pp. 3814–3819. doi: 10 . 23919 / ACC53348 . 2022 . 9867681. [94] D. S. Laila and D. Nešić, “Changing supply rates for input–output to state stable discrete-time nonlinear systems with applications,” Automatica, vol. 39, no. 5, pp. 821–835, 2003. doi: 10 . 1016 / S0005 1098(03)00055-4. [95] W. P. M. H. Heemels, K. H. Johansson, and P. Tabuada, “An introduction to event-triggered and self-triggered control,” in Proc. IEEE 51st Conf. on Decision and Control (CDC), Maui, HI, USA, 2012, pp. 3270– 3285. doi: 10.1109/CDC.2012.6425820. [96] P. Tabuada, “Event-triggered real-time scheduling of stabilizing control tasks,” IEEE Trans. on Automatic Control, vol. 52, no. 9, pp. 1680– 1685, 2007. doi: 10.1109/TAC.2007.904277. [97] D. P. Borgers and W. P. M. H. Heemels, “Event-separation properties of event-triggered control systems,” IEEE Trans. on Automatic Control, vol. 59, no. 10, pp. 2644–2656, 2014. doi: 10.1109/TAC.2014.2325272. [98] P. Ong and J. Cortés, “Event-triggered control design with performance barrier,” in Proc. IEEE 57th Conf. on Decision and Control (CDC), Miami Beach, FL, 2018, pp. 951–956. doi: 10.1109/CDC.2018.8619524. [99] J. C. Doyle, B. A. Francis, and A. R. Tannenbaum, Feedback control theory. Courier Corporation, 2013, isbn: 9780486469331. [100] H. K. Khalil and J. W. Grizzle, Nonlinear Systems. Upper Saddle River, NJ: Prentice Hall, 2002, vol. 3, isbn: 9780130673893. [101] L. Perko, Differential equations and dynamical systems. Springer Science & Business Media, 2013, vol. 7, isbn: 9780387951164. [102] W. Rudin, Principles of mathematical analysis. McGraw-hill New York, 1964, vol. 3, isbn: 9780070542358. [103] R. I. Leine, “The historical development of classical stability concepts: Lagrange, poisson and lyapunov stability,” Nonlinear Dynamics, vol. 59, no. 1, pp. 173–182, 2010. doi: 10.1007/s11071-009-9530-z. [104] F. Blanchini and S. Miani, Set-theoretic methods in control. Springer, 2008, isbn: 9780817632557. [105] T. Yoshizawa, Stability theory by Liapunov’s second method. Mathematical Society of Japan, 1966, vol. 9. [106] R. A. Freeman and P. V. Kokotović, “Inverse optimality in robust stabilization,” SIAM J. on Control and Optimization, vol. 34, no. 4, pp. 1365–1391, 1996. doi: 10.1137/S0363012993258732. 353 [107] A. D. Ames, K. Galloway, K. Sreenath, and J. W. Grizzle, “Rapidly exponentially stabilizing control lyapunov functions and hybrid zero dynamics,” IEEE Trans. on Automatic Control, vol. 59, no. 4, pp. 876– 891, 2014. doi: 10.1109/TAC.2014.2299335. [108] S. Kolathaya and S. Veer, “Pd based robust quadratic programs for robotic systems,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Macau, China, 2019, pp. 6834–6841. doi: 10.1109/ IROS40897.2019.8968214. [109] R. Freeman and P. V. Kokotovic, Robust nonlinear control design: state-space and Lyapunov techniques. Birkhauser Basel, 1996, isbn: 9780817647582. [110] A. Nemirovski, “Interior point polynomial time methods in convex programming,” Lecture notes, 1996. [Online]. Available: https://www2. isye.gatech.edu/~nemirovs/Lect_IPM.pdf. [111] A. Isidori, Nonlinear control systems. Springer-Verlag London, 1995, vol. 3, isbn: 9783540199168. [112] S. S. Sastry, Nonlinear Systems: Analysis, Stability and Control. Springer New York, NY, 1999, isbn: 9780387985138. [113] J.-J. E. Slotine and K. J. Hedrick, “Robust input-output feedback linearization,” Int. J. of Control, vol. 57, no. 5, pp. 1133–1139, 1993. doi: 10.1080/00207179308934435. [114] A. Isidori, “The zero dynamics of a nonlinear system: From the origin to the latest progresses of a long successful story,” European J. of Control, vol. 19, no. 5, pp. 369–378, 2013. doi: 10.1016/j.ejcon.2013.05.014. [115] A. J. Taylor, V. D. Dorobantu, Y. Yue, P. Tabuada, and A. D. Ames, “Sampled-data stabilization with control lyapunov functions via quadratically constrained quadratic programs,” IEEE Control Sys. Let., vol. 6, pp. 680–685, 2022. doi: 10.1109/LCSYS.2021.3085172. [116] A. Hereid, C. M. Hubicki, E. A. Cousineau, J. W. Hurst, and A. D. Ames, “Hybrid zero dynamics based multiple shooting optimization with applications to robotic walking,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Seattle, WA, USA, 2015, pp. 5734– 5740. doi: 10.1109/ICRA.2015.7140002. [117] C.-T. Chen, Linear System Theory and Design. New York, NY: Oxford University Press, Inc., 1999, isbn: 9780195117776. [118] M. Jankovic, “Control barrier functions for constrained control of linear systems with input delay,” in Proc. IEEE American Control Conf. (ACC), Milwaukee, WI, USA, 2018, pp. 3316–3321. doi: 10.23919/ ACC.2018.8430747. 354 [119] A. V. Fiacco and G. P. McCormick, Nonlinear programming: sequential unconstrained minimization techniques. SIAM, 1990. doi: 10 . 1093 / comjnl/12.3.207-a. [120] S. Prajna, “Barrier certificates for nonlinear model validation,” Automatica, vol. 42, no. 1, pp. 117–126, 2006. doi: 10.1016/j.automatica. 2005.08.007. [121] A. D. Ames, S. Coogan, M. Egerstedt, G. Notomista, K. Sreenath, and P. Tabuada, “Control barrier functions: Theory and applications,” in Proc. IEEE European Control Conf. (ECC), Naples, Italy, 2019, pp. 3420–3431. doi: 10.23919/ECC.2019.8796030. [122] C. M. Kellett, “A compendium of comparison function results,” Mathematics of Control, Signals, and Sys., vol. 26, no. 3, pp. 339–374, 2014. doi: 10.1007/s00498-014-0128-8. [123] M. Nagumo, “Über die lage der integralkurven gewöhnlicher differentialgleichungen,” Proc. of the Physico-Mathematical Society of Japan. 3rd Series, vol. 24, pp. 551–559, 1942. doi: 10.11429/ppmsj1919.24. 0_551. [124] R. Konda, A. D. Ames, and S. Coogan, “Characterizing safety: Minimal control barrier functions from scalar comparison systems,” IEEE Control Sys. Let., vol. 5, no. 2, pp. 523–528, 2020. doi: 10.1109/LCSYS. 2020.3003887. [125] E. D. Sontag and Y. Wang, “On characterizations of the input-to-state stability property,” Sys. & Control Let., vol. 24, no. 5, pp. 351–359, 1995. doi: 10.1016/0167-6911(94)00050-6. [126] S. Kolathaya, J. Reher, A. Hereid, and A. D. Ames, “Input to state stabilizing control lyapunov functions for robust bipedal robotic locomotion,” in Proc. IEEE American Control Conf. (ACC), Milwaukee, WI, USA, 2018, pp. 2224–2230. doi: 10.23919/ACC.2018.8430946. [127] E. D. Sontag and Y. Wang, “On characterizations of input-to-state stability with respect to compact sets,” in Nonlinear Control Sys. Design, Elsevier, 1995, pp. 203–208. doi: 10.1016/B978-0-08-0423715.50039-3. [128] K. J. Aström and R. M. Murray, Feedback systems: an introduction for scientists and engineers. Princeton University Press, 2010, isbn: 9780691135762. [129] L. Ljung, System identification. Springer, 1998. doi: 10.1007/978-14612-1768-8_11. [130] M. Krstić, I. Kanellakopoulos, and P. V. Kokotović, Nonlinear and adaptive control design. Wiley New York, 1995, isbn: 978-0-471-127321. 355 [131] S. Schaal and C. G. Atkeson, “Learning control in robotics,” IEEE Robotics & Automation Mag., vol. 17, no. 2, pp. 20–29, 2010. doi: 10.1109/MRA.2010.936957. [132] J. Kober, J. A. Bagnell, and J. Peters, “Reinforcement learning in robotics: A survey,” The Int. J. of Robotics Research, vol. 32, no. 11, pp. 1238–1274, 2013. doi: 10.1177/0278364913495721. [133] L. Brunke, M. Greeff, A. W. Hall, et al., “Safe learning in robotics: From learning-based control to safe reinforcement learning,” Ann. Review of Control, Robotics, and Autonomous Sys., vol. 5, pp. 411–444, 2022. doi: 10.1146/annurev-control-042920-020211. [134] T. Beckers, D. Kulić, and S. Hirche, “Stable gaussian process based tracking control of euler–lagrange systems,” Automatica, vol. 103, pp. 390–397, 2019. doi: 10.1016/j.automatica.2019.01.023. [135] F. Berkenkamp, A. P. Schoellig, and A. Krause, “Safe controller optimization for quadrotors with gaussian processes,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Stockholm, Sweden, 2016, pp. 491–496. doi: 10.1109/ICRA.2016.7487170. [136] F. Berkenkamp and A. P. Schoellig, “Safe and robust learning control with gaussian processes,” in Proc. IEEE European Control Conf. (ECC), Linz, Austria, 2015, pp. 2496–2501. doi: 10.1109/ECC.2015. 7330913. [137] G. Shi, X. Shi, M. O’Connell, et al., “Neural lander: Stable drone landing control using learned dynamics,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Montréal, QC, Canada, 2019, pp. 9784–9790. doi: 10.1109/ICRA.2019.8794351. [138] T. P. Lillicrap, J. J. Hunt, A. Pritzel, et al., “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015. [Online]. Available: https://arxiv.org/abs/1509.02971. [139] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017. [Online]. Available: https://arxiv.org/abs/1707.06347. [140] Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel, “Benchmarking deep reinforcement learning for continuous control,” in Proc. Int. Conf. on Machine Learning (ICML), New York City, NY, USA, 2016, pp. 1329–1338. [Online]. Available: https://proceedings.mlr. press/v48/duan16.html. [141] J. Tan, T. Zhang, E. Coumans, et al., “Sim-to-real: Learning agile locomotion for quadruped robots,” in Proc. Robotics: Science and Sys. XIV (RSS), Pittsburgh, PA, USA, 2018. doi: 10.15607/RSS.2018. XIV.010. 356 [142] H. M. Le, A. Kang, Y. Yue, and P. Carr, “Smooth imitation learning for online sequence prediction,” in Proc. Int. Conf. on Machine Learning (ICML), 2016, pp. 680–688. [Online]. Available: https : / / proceedings.mlr.press/v48/le16.html. [143] F. Berkenkamp, R. Moriconi, A. P. Schoellig, and A. Krause, “Safe learning of regions of attraction for uncertain, nonlinear systems with gaussian processes,” in Proc. IEEE 55th Conf. on Decision and Control (CDC), Las Vegas, NV, USA, 2016, pp. 4661–4666. doi: 10.1109/CDC. 2016.7798979. [144] S. M. Richards, F. Berkenkamp, and A. Krause, “The lyapunov neural network: Adaptive stability certification for safe learning of dynamic systems,” in Proc. Conf. on Robotic Learning (CoRL), Z urich, Switzerland, 2018. [Online]. Available: https://proceedings.mlr.press/ v87/richards18a.html. [145] F. Berkenkamp, M. Turchetta, A. Schoellig, and A. Krause, “Safe model-based reinforcement learning with stability guarantees,” in Proc. Advances in Neural Information Processing Sys. (NeurIPS), vol. 30, Long Beach, CA, USA, 2017, pp. 908–918. [Online]. Available: https : / / proceedings . neurips . cc / paper / 2017 / hash / 766ebcd59621e305170616ba3d3dac32-Abstract.html. [146] Y. Chow, O. Nachum, E. Duenez-Guzman, and M. Ghavamzadeh, “A lyapunov-based approach to safe reinforcement learning,” in Proc. Advances in Neural Information Processing Sys. (NeurIPS), vol. 31, Montréal, QC, Canada, 2018, pp. 8092–8101. [Online]. Available: https : / / proceedings . neurips . cc / paper / 2018 / hash / 4fe5149039b52765bde64beb9f674940-Abstract.html. [147] M. Gallieri, S. S. M. Salehian, N. E. Toklu, et al., “Safe interactive model-based learning,” arXiv preprint arXiv:1911.06556, 2019. doi: 10.48550/arXiv.1911.06556. [148] Y.-C. Chang, N. Roohi, and S. Gao, “Neural lyapunov control,” Proc. Advances in Neural Information Processing Systems (NeurIPS), vol. 32, 2019. [Online]. Available: https://proceedings.neurips.cc/paper/ 2019/hash/2647c1dba23bc0e0f9cdf75339e120d2-Abstract.html. [149] A. J. Taylor, V. D. Dorobantu, H. M. Le, Y. Yue, and A. D. Ames, “Episodic learning with control lyapunov functions for uncertain robotic systems,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Macau, China, 2019, pp. 6878–6884. doi: 10 . 1109 / IROS40897.2019.8967820. [150] A. J. Taylor, V. D. Dorobantu, M. Krishnamoorthy, H. M. Le, Y. Yue, and A. D. Ames, “A control lyapunov perspective on episodic learning via projection to state stability,” in Proc. IEEE 58th Conf. on Decision 357 and Control (CDC), Nice, France, 2019, pp. 1448–1455. doi: 10.1109/ CDC40024.2019.9029226. [151] J. Choi, F. Castañeda, C. Tomlin, and K. Sreenath, “Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions,” in Proc. Robotics: Science and Sys. XVI (RSS), Corvalis, OR, USA, 2020. doi: 10.15607/RSS.2020.XVI.088. [152] A. J. Taylor, V. D. Dorobantu, S. Dean, B. Recht, Y. Yue, and A. D. Ames, “Towards robust data-driven control synthesis for nonlinear systems with actuation uncertainty,” in Proc. IEEE 60th Conf. on Decision and Control (CDC), Austin, TX, USA, 2021, pp. 6469–6476. doi: 10.1109/CDC45484.2021.9683511. [153] K. P. Wabersich, A. J. Taylor, J. J. Choi, et al., “Data-driven safety filters: Hamilton-jacobi reachability, control barrier functions, and predictive methods for uncertain systems,” 2023. [Online]. Available: https: / / andrewjamestaylor . github . io / assets / paper _ materials / wabersich2022data _ driven _ safety _ filters _ hamilton _ jacobi _ reachability _ control _ barrier _ functions _ and _ predictive _ methods_for_uncertain_systems/preprint.pdf. [154] D. D. Fan, J. Nguyen, R. Thakker, N. Alatur, A.-a. Agha-mohammadi, and E. A. Theodorou, “Bayesian learning-based adaptive control for safety critical systems,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Paris, France, 2020, pp. 4093–4099. doi: 10.1109/ ICRA40945.2020.9196709. [155] V. Dhiman, M. J. Khojasteh, M. Franceschetti, and N. Atanasov, “Control barriers in bayesian learning of system dynamics,” IEEE Trans. on Automatic Control, vol. 68, pp. 214–229, 1 2023. doi: 10.1109/TAC. 2021.3137059. [156] Y. Emam, P. Glotfelter, S. Wilson, G. Notomista, and M. Egerstedt, “Data-driven robust barrier functions for safe, long-term operation,” IEEE Trans. on Robotics, vol. 38, no. 3, pp. 1671–1685, 2021. doi: 10.1109/TRO.2021.3118965. [157] Y. Hu, J. Fu, and G. Wen, “Decentralized robust collision avoidance for cooperative multirobot systems: A gaussian process-based control barrier function approach,” IEEE Trans. on Control of Network Sys., pp. 1–12, 2022. doi: 10.1109/TCNS.2022.3203928. [158] A. Lederer, A. Begzadić, N. Das, and S. Hirche, “Safe learning-based control of elastic joint robots via control barrier functions,” arXiv preprint arXiv:2212.00478, 2022. doi: 10.48550/arXiv.2212.00478. 358 [159] V. Zinage and E. Bakolas, “Neural koopman control barrier functions for safety-critical control of unknown nonlinear systems,” arXiv preprint arXiv:2209.07685, 2022. doi: 10.48550/arXiv.2209.07685. [160] C. Folkestad, Y. Chen, A. D. Ames, and J. W. Burdick, “Data-driven safety-critical control: Synthesizing control barrier functions with koopman operators,” IEEE Control Sys. Let., vol. 5, no. 6, pp. 2012–2017, 2020. doi: 10.1109/LCSYS.2020.3046159. [161] A. J. Taylor, A. Singletary, Y. Yue, and A. D. Ames, “Learning for safety-critical control with control barrier functions,” in Proc. 2nd Learning for Dynamics and Control (L4DC), vol. 120, Berekeley, CA, USA, 2020, pp. 708–717. [Online]. Available: https://proceedings. mlr.press/v120/taylor20a.html. [162] A. J. Taylor, A. Singletary, Y. Yue, and A. D. Ames, “A control barrier perspective on episodic learning via projection-to-state safety,” IEEE Control Sys. Let., vol. 5, no. 3, pp. 1019–1024, 2021. doi: 10.1109/ LCSYS.2020.3009082. [163] E. Munoz, D. Kalaria, Q. Lin, and J. M. Dolan, “Online adaptive compensation for model uncertainty using extreme learning machine-based control barrier functions,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Kyoto, Japan, 2022, pp. 10 959–10 966. doi: 10.1109/IROS47612.2022.9981680. [164] F. Castañeda, J. J. Choi, B. Zhang, C. J. Tomlin, and K. Sreenath, “Pointwise feasibility of gaussian process-based safety-critical control under model uncertainty,” in Proc. IEEE 60th Conf. on Decision and Control (CDC), Austin, TX, USA, 2021, pp. 6762–6769. doi: 10.1109/ CDC45484.2021.9683743. [165] F. Castañeda, J. J. Choi, W. Jung, B. Zhang, C. J. Tomlin, and K. Sreenath, “Probabilistic safe online learning with control barrier functions,” arXiv preprint arXiv:2208.10733, 2022. doi: 10.48550/arXiv. 2208.10733. [166] M. H. Cohen, C. Belta, and R. Tron, “Robust control barrier functions for nonlinear control systems with uncertainty: A duality-based approach,” in Proc. IEEE 61st Conf. on Decision and Control (CDC), Cancún, Mexico, 2022, pp. 174–179. doi: 10.1109/CDC51059.2022. 9992667. [167] Z. Jin, M. Khajenejad, and S. Z. Yong, “Robust data-driven control barrier functions for unknown continuous control affine systems,” IEEE Control Sys. Let., vol. 7, pp. 1309–1314, 2023. doi: 10.1109/LCSYS. 2023.3235958. 359 [168] M. Ohnishi, L. Wang, G. Notomista, and M. Egerstedt, “Barriercertified adaptive reinforcement learning with applications to brushbot navigation,” IEEE Trans. on Robotics, vol. 35, no. 5, pp. 1186–1205, 2019. doi: 10.1109/TRO.2019.2920206. [169] S. Liu, L. Liu, and Z. Yu, “Safe reinforcement learning for affine nonlinear systems with state constraints and input saturation using control barrier functions,” Neurocomputing, vol. 518, pp. 562–576, 2023. doi: 10.1016/j.neucom.2022.11.006. [170] Q. Zhao, Y. Zhang, and X. Li, “Safe reinforcement learning for dynamical systems using barrier certificates,” Connection Science, vol. 34, no. 1, pp. 2822–2844, 2022. doi: 10.1080/09540091.2022.2151567. [171] M. H. Cohen and C. Belta, “Approximate optimal control for safetycritical systems with control barrier functions,” in Proc. IEEE 59th Conf. on Decision and Control (CDC), 2020, pp. 2062–2067. doi: 10. 1109/CDC42340.2020.9303896. [172] Z. Cai, H. Cao, W. Lu, L. Zhang, and H. Xiong, “Safe multi-agent reinforcement learning through decentralized multiple control barrier functions,” arXiv preprint arXiv:2103.12553, 2021. doi: 10 . 48550 / arXiv.2103.12553. [173] Y. Yang, L. Chen, and M. Gombolay, “Safe inverse reinforcement learning via control barrier function,” arXiv preprint arXiv:2212.02753, 2022. doi: 10.48550/arXiv.2212.02753. [174] R. K. Cosner, Y. Yue, and A. D. Ames, “End-to-end imitation learning with safety guarantees using control barrier functions,” in Proc. IEEE 61st Conf. on Decision and Control (CDC), Cancún, Mexico, 2022, pp. 5316–5322. doi: 10.1109/CDC51059.2022.9993193. [175] M. Saveriano and D. Lee, “Learning barrier functions for constrained motion planning with dynamical systems,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Macau, China, 2019, pp. 112–119. doi: 10.1109/IROS40897.2019.8967981. [176] A. Robey, H. Hu, L. Lindemann, et al., “Learning control barrier functions from expert demonstrations,” in Proc. IEEE 59th Conf. on Decision and Control (CDC), Jeju, South Korea, 2020, pp. 3717–3724. doi: 10.1109/CDC42340.2020.9303785. [177] L. Lindemann, H. Hu, A. Robey, et al., “Learning hybrid control barrier functions from data,” in Proc. Conf. on Robot Learning (CoRL), London, UK, 2021, pp. 1351–1370. [Online]. Available: https : / / proceedings.mlr.press/v155/lindemann21a.html. 360 [178] K. Long, C. Qian, J. Cortés, and N. Atanasov, “Learning barrier functions with memory for robust safe navigation,” IEEE Robotics and Automation Let., vol. 6, no. 3, pp. 4931–4938, 2021. doi: 10.1109/LRA. 2021.3070250. [179] E. Scukins and P. Ögren, “Using reinforcement learning to create control barrier functions for explicit risk mitigation in adversarial environments,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), IEEE, 2021. doi: 10.1109/ICRA48506.2021.9561853. [180] A. S. Lafmejani, S. Berman, and G. Fainekos, “Nmpc-lbf: Nonlinear mpc with learned barrier function for decentralized safe navigation of multiple robots in unknown environments,” arXiv preprint arXiv:2208.07974, 2022. [181] Z. Liang and J. K. C. Lo, “An iterative method to learn a linear control barrier function,” arXiv preprint arXiv:2211.09854, 2022. doi: 10. 48550/arXiv.2211.09854. [182] S. McIlvanna, N. N. Minh, Y. Sun, M. Van, and W. Naeem, “Reinforcement learning-enhanced control barrier functions for robot manipulators,” arXiv preprint arXiv:2211.11391, 2022. doi: 10.48550/arXiv. 2211.11391. [183] H. Parwana and D. Panagou, “Foresee: Model-based reinforcement learning using unscented transform with application to tuning of control barrier functions,” arXiv preprint arXiv:2209.12644, 2022. doi: 10.48550/arXiv.2209.12644. [184] S. Dean, A. J. Taylor, R. K. Cosner, B. Recht, and A. D. Ames, “Guaranteeing safety of learned perception modules via measurementrobust control barrier functions,” in Proc. Conf. on Robotics Learning (CoRL), Cambridge, MA, USA, 2020. [Online]. Available: https: //proceedings.mlr.press/v155/dean21a/dean21a.pdf. [185] R. K. Cosner, I. D. J. Rodriguez, T. G. Molnar, et al., “Self-supervised online learning for safety-critical control using stereo vision,” in Proc. Int. Conf. on Robotics and Automation (ICRA), Philadelphia, PA, USA, 2022, pp. 11 487–11 493. doi: 10 . 1109 / ICRA46639 . 2022 . 9812183. [186] I. D. J. Rodriguez, N. Csomay-Shanklin, Y. Yue, and A. D. Ames, “Neural gaits: Learning bipedal locomotion via control barrier functions and zero dynamics policies,” in Proc. 4th Learning for Dynamics and Control (L4DC), vol. 168, Stanford, CA, USA, 2022, pp. 1060– 1072. [Online]. Available: https://proceedings.mlr.press/v168/ rodriguez22a/rodriguez22a.pdf. 361 [187] Z. Qin, T.-W. Weng, and S. Gao, “Quantifying safety of learning-based self-driving control using almost-barrier functions,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Kyoto, Japan, 2022, pp. 12 903–12 910. doi: 10.1109/IROS47612.2022.9982058. [188] A. Didier, R. C. Jacobs, J. Sieber, K. P. Wabersich, and M. N. Zeilinger, “Approximate predictive control barrier functions using neural networks: A computationally cheap and permissive safety filter,” arXiv preprint arXiv:2211.15104, 2022. doi: 10.48550/arXiv.2211.15104. [189] M. Krstić and P. V. Kokotović, “Modular approach to adaptive nonlinear stabilization,” Automatica, vol. 32, no. 4, pp. 625–629, 1996. doi: 10.1016/0005-1098(95)00179-4. [190] A. J. Taylor and A. D. Ames, “Adaptive safety with control barrier functions,” in Proc. IEEE American Control Conf. (ACC), Denver, CO, USA, 2020, pp. 1399–1405. doi: 10.23919/ACC45564.2020.9147463. [191] B. T. Lopez, J. E. Slotine, and J. P. How, “Robust adaptive control barrier functions: An adaptive and data-driven approach to safety,” IEEE Control Sys. Let., vol. 5, no. 3, pp. 1031–1036, 2021. doi: 10. 1109/LCSYS.2020.3005923. [192] G. Chowdhary and E. Johnson, “Concurrent learning for convergence in adaptive control without persistency of excitation,” in Proc. IEEE 49th Conf. on Decision and Control (CDC), Atlanta, GA, USA, 2010, pp. 3674–3679. doi: 10.1109/CDC.2010.5717148. [193] M. Black, E. Arabi, and D. Panagou, “A fixed-time stable adaptation law for safety-critical control under parametric uncertainty,” in Proc. IEEE European Control Conf. (ECC), Delft, Netherlands, 2021, pp. 1328–1333. doi: 10.23919/ECC54610.2021.9655080. [194] M. Maghenem, A. J. Taylor, A. D. Ames, and R. G. Sanfelice, “Adaptive safety using control barrier functions and hybrid adaptation,” in Proc. IEEE American Control Conf. (ACC), New Orleans, LA, USA, 2021, pp. 2418–2423. doi: 10.23919/ACC50511.2021.9482871. [195] A. Isaly, O. S. Patil, R. G. Sanfelice, and W. E. Dixon, “Adaptive safety with multiple barrier functions using integral concurrent learning,” in Proc. IEEE American Control Conf. (ACC), New Orleans, LA, USA, 2021, pp. 3719–3724. doi: 10.23919/ACC50511.2021.9483208. [196] M. H. Cohen and C. Belta, “High order robust adaptive control barrier functions and exponentially stabilizing adaptive control lyapunov functions,” in Proc. IEEE American Control Conf. (ACC), Atlanta, GA, USA, 2022, pp. 2233–2238. doi: 10.23919/ACC53348.2022.9867633. [197] B. T. Lopez and J.-J. Slotine, “Unmatched control barrier functions: Certainty equivalence adaptive safety,” arXiv preprint arXiv:2207.13873, 2022. doi: 10.48550/arXiv.2207.13873. 362 [198] S. Wang, B. Lyu, S. Wen, K. Shi, S. Zhu, and T. Huang, “Robust adaptive safety-critical control for unknown systems with finite-time elementwise parameter estimation,” IEEE Trans. on Sys., Man, and Cybernetics: Sys., vol. 53, no. 3, pp. 1607–1617, 2023. doi: 10.1109/ TSMC.2022.3203176. [199] M. Cohen and C. Belta, “Modular adaptive safety-critical control,” arXiv preprint arXiv:2303.04241, 2023. doi: 10.48550/arXiv.2303. 04241. [200] Y. Wang and X. Xu, “Adaptive safety-critical control for a class of nonlinear systems with parametric uncertainties: A control barrier function approach,” arXiv preprint arXiv:2302.08601, 2023. doi: 10 . 48550 / arXiv.2302.08601. [201] J. F. Fisac, A. K. Akametalu, M. N. Zeilinger, S. Kaynama, J. Gillula, and C. J. Tomlin, “A general safety framework for learning-based control in uncertain robotic systems,” IEEE Trans. on Automatic Control, vol. 64, no. 7, pp. 2737–2752, 2018. doi: 10.1109/TAC.2018.2876389. [202] L. Zheng, R. Yang, J. Pan, H. Cheng, and H. Hu, “Learning-based safety-stability-driven control for safety-critical systems under model uncertainties,” in Proc. IEEE Int. Conf. on Wireless Communications and Signal Processing (WCSP), Nanjing, China, 2020, pp. 1112–1118. doi: 10.1109/WCSP49889.2020.9299833. [203] S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” in Proc. 14th Int. Conf. on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, FL, USA, 2011, pp. 627–635. [Online]. Available: https: //proceedings.mlr.press/v15/ross11a.html. [204] L. Györfi, M. Kohler, A. Krzyzak, and H. Walk, A distribution-free theory of nonparametric regression. Springer Science & Business Media, 2006, vol. 1. doi: 10.1007/b97848. [205] D. Ernst, P. Geurts, and L. Wehenkel, “Tree-based batch mode reinforcement learning,” J. of Machine Learning Research, vol. 6, no. Apr, pp. 503–556, 2005. [Online]. Available: https : / / www . jmlr . org / papers/volume6/ernst05a/ernst05a.pdf. [206] K. S. Narendra and A. M. Annaswamy, “Persistent excitation in adaptive systems,” Int. J. of Control, vol. 45, no. 1, pp. 127–160, 1987. doi: 10.1080/00207178708933715. [207] M. A. Patterson and A. V. Rao, “Gpops-ii: A matlab software for solving multiple-phase optimal control problems using hp-adaptive gaussian quadrature collocation methods and sparse nonlinear programming,” ACM Trans. on Mathematical Software (TOMS), vol. 41, no. 1, pp. 1– 37, 2014. doi: 10.1145/2558904. 363 [208] F. Chollet et al., Keras, https://keras.io, 2015. [209] A. Singletary, P. Nilsson, T. Gurriet, and A. D. Ames, “Online active safety for robotic manipulators,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Macau, China, 2019, pp. 173–178. doi: 10.1109/IROS40897.2019.8968231. [210] E. R. Westervelt, J. W. Grizzle, and D. E. Koditschek, “Hybrid zero dynamics of planar biped walkers,” IEEE Trans. on Automatic Control, vol. 48, no. 1, pp. 42–56, 2003. doi: 10.1109/TAC.2002.806653. [211] E. Ambrose, W.-L. Ma, C. Hubicki, and A. D. Ames, “Toward benchmarking locomotion economy across design configurations on the modular robot: Amber-3m,” in Proc. 1st IEEE Conf. on Control Tech. and App. (CCTA), Kohala Coast, HI, USA, 2017, pp. 1270–1276. doi: 10. 1109/CCTA.2017.8062633. [212] W.-L. Ma, H.-H. Zhao, S. Kolathaya, and A. D. Ames, “Human-inspired walking via unified pd and impedance control,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Hong Kong, China, 2014, pp. 5088–5094. doi: 10.1109/ICRA.2014.6907605. [213] A. Hereid, C. M. Hubicki, E. A. Cousineau, and A. D. Ames, “Dynamic humanoid locomotion: A scalable formulation for hzd gait optimization,” IEEE Trans. on Robotics, vol. 34, no. 2, pp. 370–387, 2018. doi: 10.1109/TRO.2017.2783371. [214] R. Goebel, R. G. Sanfelice, and A. R. Teel, Hybrid dynamical systems. Princeton University Press, 2012, isbn: 9780691153896. [215] E. R. Westervelt, J. W. Grizzle, C. Chevallereau, J. H. Choi, and B. Morris, Feedback Control of Dynamic Bipedal Robot Locomotion. Taylor & Francis/CRC Press, 2007, isbn: 1-42005-372-8. [216] Q. Nguyen and K. Sreenath, “Exponential control barrier functions for enforcing high relative-degree safety-critical constraints,” in Proc. IEEE American Control Conf. (ACC), Boston, MA, USA, 2016, pp. 322–328. doi: 10.1109/ACC.2016.7524935. [217] J. Hwangbo, J. Lee, and M. Hutter, “Per-contact iteration method for solving contact dynamics,” IEEE Robotics and Automation Let., vol. 3, no. 2, pp. 895–902, 2018. doi: 10.1109/LRA.2018.2792536. [218] R. Dimitrova and R. Majumdar, “Deductive control synthesis for alternating-time logics,” in Proc. IEEE 14th Int. Conf. on Embedded Software (EMSOFT), Uttar Pradesh, India, 2014, pp. 1–10. doi: 10. 1145/2656045.2656054. 364 [219] N. Boffi, S. Tu, N. Matni, J.-J. Slotine, and V. Sindhwani, “Learning stability certificates from data,” in Proc. Conf. on Robot Learning (CoRL), Cambridge, MA, USA, 2020, pp. 1341–1350. [Online]. Available: https://proceedings.mlr.press/v155/boffi21a.html. [220] A. Ben-Tal, L. El Ghaoui, and A. Nemirovski, Robust optimization. Princeton University Press, 2009, isbn: 978-0-691-14368-2. [221] X. Xu, P. Tabuada, J. W. Grizzle, and A. D. Ames, “Robustness of control barrier functions for safety critical control,” IFAC-PapersOnLine, vol. 48, no. 27, pp. 54–61, 2015. doi: 10.1016/j.ifacol.2015.11.152. [222] B. J. Morris, M. J. Powell, and A. D. Ames, “Continuity and smoothness properties of nonlinear optimization-based feedback controllers,” in Proc. IEEE 54th Conf. on Decision and Control (CDC), Osaka, Japan, 2015, pp. 151–158. doi: 10.1109/CDC.2015.7402101. [223] B. S. Mordukhovich, J. V. Outrata, and M. E. Sarabi, “Full stability of locally optimal solutions in second-order cone programs,” SIAM J. on Optimization, vol. 24, no. 4, pp. 1581–1613, 2014. doi: 10.1137/ 130928637. [224] B. S. Mordukhovich, Variational analysis and applications. Springer, 2018, vol. 30. doi: 10.1007/978-3-319-92775-6. [225] A. Lederer, A. J. O. Conejo, K. Maier, W. Xiao, and S. Hirche, “Realtime regression with dividing local gaussian processes,” arXiv preprint arXiv:2006.09446, 2020. doi: 10.48550/arXiv.2006.09446. [226] C. E. Rasmussen, C. K. I. Williams, et al., Gaussian processes for machine learning. Springer, 2006, vol. 1, pp. 63–71, isbn: 3-540-23122-6. [227] A. Jain, P. Sarda, and J. R. Haritsa, “Providing diversity in k-nearest neighbor query results,” in Advances in Knowledge Discovery and Data Mining: 8th Pacific-Asia Conf. (PAKDD), Springer, Sydney, Australia, 2004, pp. 404–413. doi: 10.1007/978-3-540-24775-3_49. [228] A. Alan, A. J. Taylor, C. R. He, G. Orosz, and A. D. Ames, “Safe controller synthesis with tunable input-to-state safe control barrier functions,” IEEE Control Sys. Let., vol. 6, pp. 908–913, 2022. doi: 10 . 1109/LCSYS.2021.3087443. [229] J. J. Choi, D. Lee, K. Sreenath, C. J. Tomlin, and S. L. Herbert, “Robust control barrier–value functions for safety-critical control,” in Proc. IEEE 60th Conf. on Decision and Control (CDC), Austin, TX, USA, 2021, pp. 6814–6821. doi: 10.1109/CDC45484.2021.9683085. [230] E. Daş and R. M. Murray, “Robust safe control synthesis with disturbance observer-based control barrier functions,” in Proc. IEEE 61st Conf. on Decision and Control (CDC), Cancún, Mexico, 2022, pp. 5566–5573. doi: 10.1109/CDC51059.2022.9993032. 365 [231] A. Alan, T. G. Molnar, E. Daş, A. D. Ames, and G. Orosz, “Disturbance observers for robust safety-critical control with control barrier functions,” IEEE Control Sys. Let., vol. 7, pp. 1123–1128, 2022. doi: 10.1109/LCSYS.2022.3232059. [232] P. Seiler, M. Jankovic, and E. Hellstrom, “Control barrier functions with unmodeled input dynamics using integral quadratic constraints,” IEEE Control Sys. Let., vol. 6, pp. 1664–1669, 2021. doi: 10.1109/ LCSYS.2021.3130782. [233] L. C. Evans, An introduction to stochastic differential equations. American Mathematical Soc., 2012, vol. 82, isbn: 978-1-4704-1054-4. [234] A. Agrawal and K. Sreenath, “Discrete control barrier functions for safety-critical control of discrete systems with application to bipedal robot navigation,” in Proc. Robotics: Science and Sys. XIII (RSS), Cambridge, MA, USA, vol. 13, 2017. doi: 10.15607/RSS.2017.XIII. 073. [235] R. K. Cosner, P. Culbertson, A. J. Taylor, and A. D. Ames, “Robust safety under stochastic uncertainty with discrete-time control barrier functions,” arXiv preprint arXiv:2302.07469, 2023. doi: 10 . 48550 / arXiv.2302.07469. [236] C. R. He and G. Orosz, “Safety guaranteed connected cruise control,” in Proc. IEEE 21st Int. Conf. on Intelligent Transportation Sys. (ITSC), Maui, Hawaii, USA, 2018, pp. 549–554. doi: 10 . 1109 / ITSC . 2018 . 8569979. [237] L. Zhang and G. Orosz, “Motif-based design for connected vehicle systems in presence of heterogeneous connectivity structures and time delays,” IEEE Trans. on Intelligent Transportation Sys., vol. 17, no. 6, pp. 1638–1651, 2016. doi: 10.1109/TITS.2015.2509782. [238] J. I. Ge, S. S. Avedisov, C. R. He, W. B. Qin, M. Sadeghpour, and G. Orosz, “Experimental validation of connected automated vehicle design among human-driven vehicles,” Transportation Research Part C: Emerging Tech., vol. 91, pp. 335–352, 2018. doi: 10 . 1016 / j . trc . 2018.04.005. [239] International-Trucks, Prostar+ truck. [Online]. Available: https : / / www.internationaltrucks.com/trucks/prostar. [240] Commsignia, V2x onboard unit. [Online]. Available: https : / / www . commsignia.com/products/obu/. [241] Speedgoat, Mobile real-time target machine. [Online]. Available: https: //www.speedgoat.com/products- services/real- time- targetmachines/mobile. 366 [242] D. R. Agrawal, H. Parwana, R. K. Cosner, U. Rosolia, A. D. Ames, and D. Panagou, “A constructive method for designing safe multirate controllers for differentially-flat systems,” IEEE Control Systems Let., vol. 6, pp. 2138–2143, 2021. doi: 10.1109/LCSYS.2021.3136465. [243] M. Ahmadi, A. Singletary, J. W. Burdick, and A. D. Ames, “Safe policy synthesis in multi-agent pomdps via discrete-time barrier functions,” in Proc. IEEE 58th Conf. on Decision and Control (CDC), Nice, France, 2019, pp. 4797–4803. doi: 10.1109/CDC40024.2019.9030241. [244] J. Zeng, B. Zhang, and K. Sreenath, “Safety-critical model predictive control with discrete-time control barrier function,” in Proc. IEEE American Control Conf. (ACC), New Orleans, LA, USA, 2021, pp. 3882–3889. doi: 10.23919/ACC50511.2021.9483029. [245] G. Grimmett and D. Stirzaker, Probability and random processes. Oxford university press, 2020, isbn: 978-0-19-884760-1. [246] J. Ville, “Etude critique de la notion de collectif,” 1939. [Online]. Available: http://www.numdam.org/item/THESE_1939__218__1_0.pdf. [247] J. G. Liao and A. Berg, “Sharpening jensen’s inequality,” The American Statistician, vol. 73, no. 3, pp. 278–281, 2019. doi: 10.1080/00031305. 2017.1419145. [248] R. A. Becker, “The variance drain and jensen’s inequality,” 2012. [Online]. Available: https://caepr.indiana.edu/RePEc/inu/caeprp/ CAEPR2012-004.pdf. [249] S. Abbott, Understanding analysis. Springer, 2001, vol. 2, isbn: 978-14939-2711-1. [250] K. Shebrawi and H. Albadawi, “Trace inequalities for matrices,” Bul. of the Australian Mathematical Society, vol. 87, no. 1, pp. 139–148, 2013. doi: 10.1017/S0004972712000627. [251] J. Buchli, M. Kalakrishnan, M. Mistry, P. Pastor, and S. Schaal, “Compliant quadruped locomotion over rough terrain,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), St. Louis, MO, USA, 2009, pp. 814–820. doi: 10.1109/IROS.2009.5354681. [252] W. Ubellacker, N. Csomay-Shanklin, T. G. Molnar, and A. D. Ames, “Verifying safe transitions between dynamic motion primitives on legged robots,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Prague, Czech Republic, 2021, pp. 8477–8484. doi: 10.1109/IROS51168.2021.9636537. [253] T. G. Molnar, R. K. Cosner, A. W. Singletary, W. Ubellacker, and A. D. Ames, “Model-free safety-critical control for robotic systems,” IEEE Robotics and Automation Let., vol. 7, no. 2, pp. 944–951, 2021. doi: 10.1109/LRA.2021.3135569. 367 [254] D. Angeli, E. D. Sontag, and Y. Wang, “A characterization of integral input-to-state stability,” IEEE Trans. on Automatic Control, vol. 45, no. 6, pp. 1082–1097, 2000. doi: 10.1109/9.863594. [255] J. L. Crassidis and J. L. Junkins, Optimal estimation of dynamic systems. CRC press, 2011, isbn: 978-1-4398-3985-0. [256] T. Gurriet, M. Mote, A. Singletary, P. Nilsson, E. Feron, and A. D. Ames, “A scalable safety critical control framework for nonlinear systems,” IEEE Access, vol. 8, pp. 187 249–187 275, 2020. doi: 10.1109/ ACCESS.2020.3025248. [257] K. Long, V. Dhiman, M. Leok, J. Cortés, and N. Atanasov, “Safe control synthesis with uncertain dynamics and constraints,” IEEE Robotics and Automation Let., vol. 7, no. 3, pp. 7295–7302, 2022. doi: 10.1109/LRA. 2022.3182544. [258] Z. Li and N. Atanasov, “Governor-parameterized barrier function for safe output tracking with locally sensed constraints,” Automatica, vol. 152, p. 110 996, 2023. doi: 10.1016/j.automatica.2023.110996. [259] I. Tezuka and H. Nakamura, “Time-varying obstacle avoidance by using high-gain observer and input-to-state constraint safe control barrier function,” IFAC-PapersOnLine, vol. 53, no. 5, pp. 391–396, 2020. doi: 10.1016/j.ifacol.2021.04.192. [260] Y. Wang and X. Xu, “Observer-based control barrier functions for safety critical systems,” in Proc. IEEE American Control Conf. (ACC), Atlanta, GA, USA, 2022, pp. 709–714. doi: 10.23919/ACC53348.2022. 9867262. [261] D. R. Agrawal and D. Panagou, “Safe and robust observer-controller synthesis using control barrier functions,” IEEE Control Sys. Let., vol. 7, pp. 127–132, 2022. doi: 10.1109/LCSYS.2022.3185142. [262] F. Codevilla, M. Müller, A. López, V. Koltun, and A. Dosovitskiy, “Endto-end driving via conditional imitation learning,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Brisbane, QLD, Australia, 2018, pp. 4693–4700. doi: 10.1109/ICRA.2018.8460487. [263] A. Lambert, A. Shaban, A. Raj, Z. Liu, and B. Boots, “Deep forward and inverse perceptual models for tracking and prediction,” in Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), Brisbane, QLD, Australia, 2018, pp. 675–682. doi: 10.1109/ICRA.2018.8461050. [264] S. Tang, V. Wüest, and V. Kumar, “Aggressive flight with suspended payloads using vision-based control,” IEEE Robotics and Automation Let., vol. 3, no. 2, pp. 1152–1159, 2018. doi: 10 . 1109 / LRA . 2018 . 2793305. 368 [265] S. Dean, N. Matni, B. Recht, and V. Ye, “Robust guarantees for perception-based control,” in Proc. 2nd Learning for Dynamics and Control (L4DC), Berkeley, CA, USA, 2020, pp. 350–360. [Online]. Available: http://proceedings.mlr.press/v120/dean20a.html. [266] S. Dean and B. Recht, “Certainty equivalent perception-based control,” in Proc. 3rd Learning for Dynamics and Control (L4DC), vol. 144, Zürich, Switzerland, 2021, pp. 399–411. [Online]. Available: http:// proceedings.mlr.press/v144/dean21a.html. [267] A. Khazraei, H. Pfister, and M. Pajic, “Resiliency of perceptionbased controllers against attacks,” in Proc. 4th Learning for Dynamics and Control (L4DC), PMLR, Stanford, CA, USA, 2022, pp. 713– 725. [Online]. Available: https : / / proceedings . mlr . press / v168 / khazraei22a.html. [268] A. Khazraei and M. Pfister H.and Pajic, “Attacks on perception-based control systems: Modeling and fundamental limits,” arXiv preprint arXiv:2206.07150, 2022. doi: 10.48550/arXiv.2206.07150. [269] H. Zhang, S. Cheng, L. Niu, and A. Clark, “Barrier certificate based safe control for lidar-based systems under sensor faults and attacks,” in Proc. IEEE 61st Conf. on Decision and Control (CDC), Cancún, Mexico, 2022, pp. 2256–2263. doi: 10.1109/CDC51059.2022.9992432. [270] M. Tong, C. Dawson, and C. Fan, “Enforcing safety for vision-based controllers via control barrier functions and neural radiance fields,” arXiv preprint arXiv:2209.12266, 2022. doi: 10.48550/arXiv.2209. 12266. [271] D. Sun, N. Musavi, G. Dullerud, S. Shakkottai, and S. Mitra, “Learning certifiably robust controllers using fragile perception,” arXiv preprint arXiv:2209.11328, 2022. doi: 10.48550/arXiv.2209.11328. [272] Y. Yue, J. Broder, R. Kleinberg, and T. Joachims, “The k-armed dueling bandits problem,” J. of Computer and Sys. Sciences, vol. 78, no. 5, pp. 1538–1556, 2012. doi: 10.1016/j.jcss.2011.12.028. [273] P. Shivaswamy and T. Joachims, “Online structured prediction via coactive learning,” arXiv preprint arXiv:1205.4213, 2012. doi: 10.48550/ arXiv.1205.4213. [274] M. De Gemmis, L. Iaquinta, P. Lops, C. Musto, F. Narducci, and G. Semeraro, “Preference learning in recommender systems,” Preference Learning, vol. 41, pp. 41–55, 2009. [Online]. Available: https : //citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi= e91613a8d5a48b698e1a4f3f5bb73c78fc22348e#page=45. 369 [275] K. Raman, T. Joachims, P. Shivaswamy, and T. Schnabel, “Stable coactive learning via perturbation,” in Proc. Int. Conf. on Machine Learning (ICML), PMLR, 2013, pp. 837–845. [Online]. Available: https : //proceedings.mlr.press/v28/raman13.html. [276] A. Jain, S. Sharma, T. Joachims, and A. Saxena, “Learning preferences for manipulation tasks from online coactive feedback,” The Int. J. of Robotics Research, vol. 34, no. 10, pp. 1296–1313, 2015. doi: 10.1177/ 0278364915581193. [277] D. Sadigh, A. Dragan, S. Sastry, and S. Seshia, “Active preferencebased learning of reward functions,” in Proc. of Robotics: Science and Sys. XIII (RSS), Cambridge, MA, USA, 2017. doi: 10.15607/RSS. 2017.XIII.053. [278] E. Biyik, N. Huynh, M. Kochenderfer, and D. Sadigh, “Active Preference-Based Gaussian Process Regression for Reward Learning,” in Proc. of Robotics: Science and Sys. XVI (RSS), Corvalis, OR, USA, 2020. doi: 10.15607/RSS.2020.XVI.041. [279] Y. Sui, V. Zhuang, J. W. Burdick, and Y. Yue, “Stagewise safe bayesian optimization with gaussian processes,” in Proc. Int. Conf. on Machine Learning (ICML), 2018, pp. 4781–4789. [Online]. Available: https:// proceedings.mlr.press/v80/sui18a.html. [280] Y. Sui, A. Gotovos, J. Burdick, and A. Krause, “Safe exploration for optimization with gaussian processes,” in Proc. Int. Conf. on Machine Learning (ICML), 2015, pp. 997–1005. [Online]. Available: https:// proceedings.mlr.press/v37/sui15.html. [281] M. Tucker, K. Li, Y. Yue, and A. D. Ames, “Polar: Preference optimization and learning algorithms for robotics,” arXiv preprint arXiv:2208.04404, 2022. doi: 10.48550/arXiv.2208.04404. [282] A. L. Dontchev and W. W. Hager, “Lipschitzian stability in nonlinear control and optimization,” SIAM J. on Control and Optimization, vol. 31, no. 3, pp. 569–603, 1993. doi: 10.1137/0331026. [283] W. Tan and A. Packard, “Stability region analysis using sum of squares programming,” in Proc. IEEE American Control Conf. (ACC), Minneapolis, MN, USA, 2006, pp. 2297–2302. doi: 10.1109/ACC.2006. 1656562. [284] J.-P. Aubin, A. M. Bayen, and P. Saint-Pierre, Viability theory: new directions. Springer Science & Business Media, 2011, isbn: 978-3-64216683-9. [285] W. Chu, Z. Ghahramani, and C. K. I. Williams, “Gaussian processes for ordinal regression.,” J. of machine learning research, vol. 6, no. 7, pp. 1019–1041, 2005. [Online]. Available: https : / / www . jmlr . org / papers/volume6/chu05a/chu05a.pdf. 370 [286] O. Chapelle and L. Li, “An empirical evaluation of thompson sampling,” Proc. Advances in Neural Information Processing Systems (NeurIPS), vol. 24, 2011. [Online]. Available: https : / / proceedings . neurips . cc / paper _ files / paper / 2011 / hash / e53a0a2978c28872a4505bdb51db06dc-Abstract.html. [287] J. Kirschner, M. Mutny, N. Hiller, R. Ischebeck, and A. Krause, “Adaptive and safe bayesian optimization in high dimensions via one-dimensional subspaces,” in Proc. Int. Conf. on Machine Learning (ICML), 2019, pp. 3429–3438. [Online]. Available: http : / / proceedings.mlr.press/v97/kirschner19a.html. [288] J. C. Maxwell, “I. on governors,” Proc. of the Royal Society of London, no. 16, pp. 270–283, 1868. doi: 10.1098/rspl.1867.0055. [289] S. Monaco and D. Normand-Cyrot, “Advanced tools for nonlinear sampled-data systems’ analysis and control,” in Proc. IEEE European Control Conf. (ECC), 2007, pp. 1155–1158. doi: 10.23919/ECC.2007. 7069061. [290] M. Di Ferdinando and P. Pepe, “Robustification of sample-and-hold stabilizers for control-affine time-delay systems,” Automatica, vol. 83, pp. 141–154, 2017. doi: 10.1016/j.automatica.2017.06.029. [291] C. Li, C. Ding, and K. Shen, “Quantifying the cost of context switch,” in Proc. Work. on Experimental Computer Science (ExpCS), San Diego, CA, USA, 2007, pp. 1–4. doi: 10.1145/1281700.1281702. [292] P. Ong, G. Bahati, and A. D. Ames, “Stability and safety through event-triggered intermittent control with application to spacecraft orbit stabilization,” in Proc. IEEE 61st Conf. on Decision and Control (CDC), Cancún, Mexico, 2022, pp. 453–460. doi: 10.1109/CDC51059. 2022.9992757. [293] J. W. Grizzle and P. V. Kokotovic, “Feedback linearization of sampleddata systems,” IEEE Trans. on Automatic Control, vol. 33, no. 9, pp. 857–859, 1988. doi: 10.1109/9.1316. [294] S. Monaco, D. Normand-Cyrot, and S. Stornelli, “On the linearizing feedback in nonlinear sampled data control schemes,” in Proc. IEEE 25th Conf. on Decision and Control (CDC), Athens, Greece, 1986, pp. 2056–2060. doi: 10.1109/CDC.1986.267399. [295] I. M. Mareels, H. Penfold, and R. J. Evans, “Controlling nonlinear timevarying systems via euler approximations,” Automatica, vol. 28, no. 4, pp. 681–696, 1992. [296] D. Nešić, A. Teel, and P. Kokotovic, “On the design of a controller based on the discrete-time approximation of the nonlinear plant model,” in IEEE American Control Conf. (ACC), vol. 5, San Diego, CA, USA, 1999, pp. 3474–3478. doi: 10.1109/ACC.1999.782411. 371 [297] D. Nešić and D. S. Laila, “A note on input-to-state stabilization for nonlinear sampled-data systems,” IEEE Trans. on Automatic Control, vol. 47, no. 7, pp. 1153–1158, 2002. doi: 10.1109/TAC.2002.800663. [298] D. Nešić and D. Angeli, “Integral versions of iss for sampled-data nonlinear systems via their approximate discrete-time models,” IEEE Trans. on Automatic Control, vol. 47, no. 12, pp. 2033–2037, 2002. doi: 10. 1109/TAC.2002.805681. [299] J. W. Grizzle, “Feedback linearization of discrete-time systems,” in Analysis and Optimization of Systems, A. Bensoussan and J. L. Lions, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 1986, pp. 273–281, isbn: 978-3-540-39856-1. [300] B. Jakubczyk and E. D. Sontag, “The effect of sampling on feedback linearization,” in Proc. IEEE 26th Conf. on Decision and Control (CDC), vol. 26, Los Angeles, CA, USA, 1987, pp. 1374–1379. doi: 10.1109/ CDC.1987.272636. [301] A. Arapostathis, B. Jakubczyk, H.-G. Lee, S. I. Marcus, and E. D. Sontag, “The effect of sampling on linear equivalence and feedback linearization,” Sys. & Control Let., vol. 13, no. 5, pp. 373–381, 1989. doi: 10.1016/0167-6911(89)90103-5. [302] J.-P. Barbot, S. Monaco, and D. Normand-Cyrot, “A sampled normal form for feedback linearization,” Mathematics of Control, Signals and Sys., vol. 9, no. 2, pp. 162–188, 1996. doi: 10.1007/BF01211752. [303] S. Monaco and D. Normand-Cyrot, “Zero dynamics of sampled nonlinear systems,” Sys. & Control Let., vol. 11, no. 3, pp. 229–234, 1988. doi: 10.1016/0167-6911(88)90063-1. [304] J. I. Yuz and G. C. Goodwin, “On sampled-data models for nonlinear systems,” IEEE Trans. on Automatic Control, vol. 50, no. 10, pp. 1477– 1489, 2005. doi: 10.1109/TAC.2005.856640. [305] M. Nishi, M. Ishitobi, and S. Kunimatsu, “Nonlinear sampled-data models and zero dynamics,” in IEEE Int. Conf. on Networking, Sensing and Control (ICNSC), Okayama, Japan, 2009, pp. 373–378. doi: 10.1109/ICNSC.2009.4919304. [306] P. Pepe, “Robustification of nonlinear stabilizers in the sample-andhold sense,” J. of The Franklin Institute, vol. 352, no. 10, pp. 4107– 4128, 2015. doi: 10.1016/j.jfranklin.2015.06.007. [307] M. Di Ferdinando, P. Pepe, and S. Di Gennaro, “A new approach to the design of sampled–data dynamic output feedback stabilizers,” IEEE Trans. on Automatic Control, vol. 67, no. 2, pp. 1038–1045, 2021. doi: 10.1109/TAC.2021.3062345. 372 [308] A. Ghaffari, I. Abel, D. Ricketts, S. Lerner, and M. Krstić, “Safety verification using barrier certificates with application to double integrator with input saturation and zero-order hold,” in Proc. IEEE American Control Conf. (ACC), Milwaukee, WI, USA, 2018, pp. 4664–4669. doi: 10.23919/ACC.2018.8430929. [309] A. J. Taylor, V. D. Dorobantu, R. K. Cosner, Y. Yue, and A. D. Ames, “Safety of sampled-data systems with control barrier functions via approximate discrete time models,” in Proc. IEEE 61st Conf. on Decision and Control (CDC), Cancún, Mexico, 2022, pp. 7127–7134. doi: 10.1109/CDC51059.2022.9993226. [310] K. J. Åström and B. Bernhardsson, “Comparison of periodic and event based sampling for first-order stochastic systems,” IFAC Proc. Vol., vol. 32, no. 2, pp. 5006–5011, 1999. doi: 10.1016/S1474- 6670(17) 56852-4. [311] K.-E. Åarzén, “A simple event-based pid controller,” IFAC Proc. Vol., vol. 32, no. 2, pp. 8687–8692, 1999. doi: 10.1016/S1474- 6670(17) 57482-0. [312] J. K. Yook, D. M. Tilbury, and N. R. Soparkar, “Trading computation for bandwidth: Reducing communication in distributed control systems using state estimators,” IEEE Trans. on Control Sys. Tech., vol. 10, no. 4, pp. 503–518, 2002. doi: 10.1109/TCST.2002.1014671. [313] W. P. M. H. Heemels, J. H. Sandee, and P. P. J. Van Den Bosch, “Analysis of event-driven controllers for linear systems,” Int. J. of Control, vol. 81, no. 4, pp. 571–590, 2008. doi: 10.1080/00207170701506919. [314] T. Henningsson, E. Johannesson, and A. Cervin, “Sporadic event-based control of first-order linear stochastic systems,” Automatica, vol. 44, no. 11, pp. 2890–2895, 2008. doi: 10.1016/j.automatica.2008.03. 026. [315] R. Postoyan, P. Tabuada, D. Nešić, and A. Anta, “A framework for the event-triggered stabilization of nonlinear systems,” IEEE Trans. on Automatic Control, vol. 60, no. 4, pp. 982–996, 2014. doi: 10.1109/ TAC.2014.2363603. [316] C. Nowzari, E. Garcia, and J. Cortés, “Event-triggered communication and control of networked systems for multi-agent consensus,” Automatica, vol. 105, pp. 1–27, 2019. doi: 10.1016/j.automatica.2019.03. 009. [317] J. Berneburg and C. Nowzari, “Distributed dynamic event-triggered coordination with a designable minimum inter-event time,” in Proc. IEEE American Control Conf. (ACC), Philadelphia, PA, USA, 2019, pp. 1424–1429. doi: 10.23919/ACC.2019.8815227. 373 [318] R. Postoyan, R. Sanfelice, and W. P. M. H. Heemels, “Inter-event times analysis for planar linear event-triggered controlled systems,” in Proc. IEEE 58th Conf. on Decision and Control (CDC), Nice, France, 2019, pp. 1662–1667. doi: 10.1109/CDC40024.2019.9028888. [319] M. C. F. Donkers and W. P. M. H. Heemels, “Output-based eventtriggered control with guaranteed L∞ -gain and improved eventtriggering,” in Proc. IEEE 49th Conf. on Decision and Control (CDC), Atlanta, GA, USA, 2010, pp. 3246–3251. doi: 10 . 1109 / CDC . 2010 . 5718032. [320] K. P. Tee, S. S. Ge, and E. H. Tay, “Barrier lyapunov functions for the control of output-constrained nonlinear systems,” Automatica, vol. 45, no. 4, pp. 918–927, 2009. doi: 10.1016/j.automatica.2008.11.017. [321] A. J. Taylor, P. Ong, J. Cortés, and A. D. Ames, “Safety-critical event triggered control via input-to-state safe barrier functions,” IEEE Control Sys. Let., vol. 5, no. 3, pp. 749–754, 2021. doi: 10.1109/LCSYS. 2020.3005101. [322] L. Long and J. Wang, “Safety-critical dynamic event-triggered control of nonlinear systems,” Sys. & Control Let., vol. 162, pp. 1–9, 2022. doi: 10.1016/j.sysconle.2022.105176. [323] W. Xiao, C. Belta, and C. G. Cassandras, “Event-triggered safetycritical control for systems with unknown dynamics,” in Proc. IEEE 60th Conf. on Decision and Control (CDC), Austin, TX, USA, 2021, pp. 540–545. doi: 10.1109/CDC45484.2021.9682792. [324] M. Kishida, “Greedy synthesis of event-and self-triggered controls with control lyapunov-barrier function,” arXiv preprint arXiv:2302.12435, 2023. doi: 10.48550/arXiv.2302.12435. [325] E. Sabouni, C. G. Cassandras, W. Xiao, and N. Meskin, “Optimal control of connected automated vehicles with event/self-triggered control barrier functions,” arXiv preprint arXiv:2209.13053, 2022. doi: 10 . 48550/arXiv.2209.13053. [326] S. Koga, C. Demir, and M. Krstic, “Event-triggered safe stabilizing boundary control for the stefan pde system with actuator dynamics,” arXiv preprint arXiv:2210.01454, 2022. doi: 10.48550/arXiv.2210. 01454. [327] S. A. Al-Hiddabi, “Quadrotor control using feedback linearization with dynamic extension,” in IEEE 6th Int. Sym. on Mechatronics and its App. (ISMA), Sharjah, United Arab Emirates, 2009, pp. 1–3. doi: 10. 1109/ISMA.2009.5164788. [328] P. Tabuada and L. Fraile, “Data-driven stabilization of siso feedback linearizable systems,” arXiv preprint arXiv:2003.14240, 2020. doi: 10. 48550/arXiv.2003.14240. 374 [329] F. H. Clarke, Y. S. Ledyaev, R. J. Stern, and P. R. Wolenski, Nonsmooth analysis and control theory. Springer Science & Business Media, 2008, vol. 178, isbn: 0-387-98336-8. [330] F. Solowjow and S. Trimpe, “Event-triggered learning,” Automatica, vol. 117, pp. 1–10, 2020. doi: 10.1016/j.automatica.2020.109009. [331] W. Tan and A. Packard, “Searching for control lyapunov functions using sums of squares programming,” sibi, vol. 1, no. 1, [Online]. Available: https://bcci.me.berkeley.edu/downloads/papers/weehong_3. pdf. [332] L. Wang, D. Han, and M. Egerstedt, “Permissive barrier certificates for safe stabilization using sum-of-squares,” in Proc. IEEE American Control Conf. (ACC), Milwaukee, WI, USA, 2018, pp. 585–590. doi: 10.23919/ACC.2018.8431617. [333] A. Clark, “Verification and synthesis of control barrier functions,” in Proc. IEEE 60th Conf. on Decision and Control (CDC), Austin, TX, USA, 2021, pp. 6105–6112. doi: 10.1109/CDC45484.2021.9683520. [334] Z. Gong, M. Zhao, T. Bewley, and S. Herbert, “Constructing control lyapunov-value functions using hamilton-jacobi reachability analysis,” IEEE Control Sys. Let., vol. 7, pp. 925–930, 2022. doi: 10.1109/LCSYS. 2022.3228728. [335] S. Tonkens and S. Herbert, “Refining control barrier functions through hamilton-jacobi reachability,” in Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Sys. (IROS), Kyoto, Japan, 2022, pp. 13 355–13 362. doi: 10.1109/IROS47612.2022.9982203. [336] N. Csomay-Shanklin, A. J. Taylor, U. Rosolia, and A. D. Ames, “Multirate planning and control of uncertain nonlinear systems: Model predictive control and control lyapunov functions,” in Proc. IEEE 61st Conf. on Decision and Control (CDC), Cancún, Mexico, 2022, pp. 3732–3739. doi: 10.1109/CDC51059.2022.9992902. [337] S. L. Herbert, M. Chen, S. Han, S. Bansal, J. F. Fisac, and C. J. Tomlin, “Fastrack: A modular framework for fast and guaranteed safe motion planning,” in Proc. IEEE 56th Conf. on Decision and Control (CDC), Melbourne, VIC, Australia, 2017, pp. 1517–1522. doi: 10.1109/CDC. 2017.8263867. [338] U. Rosolia and A. D. Ames, “Multi-rate control design leveraging control barrier functions and model predictive control policies,” IEEE Control Sys. Let., vol. 5, no. 3, pp. 1007–1012, 2021. doi: 10.1109/LCSYS. 2020.3008326. 375 [339] U. Rosolia, A. Singletary, and A. D. Ames, “Unified multirate control: From low-level actuation to high-level planning,” IEEE Trans. on Automatic Control, vol. 67, no. 12, pp. 6627–6640, 2022. doi: 10.1109/ TAC.2022.3184664. 376 Appendix A UNMATCHED TUNABLE INPUT-TO-STATE SAFETY PROOF In this appendix I will provide a proof of the unmatched disturbance case of Theorem 29 for completeness. Recall Theorem 29 as it appears in Section 4.3. Theorem 29. Let C ⊂ E be the 0-superlevel set of a function h : E → R that is continuously differentiable on E. If h is a TISSf-BF for the closed-loop system with unmatched disturbances (2.70) (matched disturbances (2.74)) on C, then the closed-loop system with unmatched disturbances (2.70) (matched disturbances (2.74)) is TISSf with respect to the set C with a function γT : R × R≥0 → R≥0 defined as:   ϵ(r)d2 −1 γT (r, d) ≜ −α . (A.1) − 4 Proof. Observe that by assumption α−1 and ϵ are continuously differentiable on R, such that the function γT (·, b) is continuously differentiable for any e b ∈ R≥0 . Similarly, as ϵ(r) > 0 for all r ∈ R and α−1 ∈ K∞ , the function e γT (a, ·) ∈ K∞ for any a ∈ R as required. Our next goal is to verify that for all d ∈ R≥0 , the set Cd,T defined in (4.12) is forward invariant up to d. Consider a value of d ∈ R≥0 and consider a disturbance signal d : R≥0 → Rn that is piecewise continuous on R≥0 and satisfies ∥d∥∞ ≤ d. As h is a TISSf-BF for the closed-loop system with unmatched disturbances (2.74), we have that: ∂h (x)d(t), ∂x 2 1 ∂h ∂h ≥ −α(h(x)) + (x) + (x)d(t). ϵ(h(x)) ∂x ∂x ḣ(x, t) ≜ Lf h(x) + Lg h(x)k(x) + (A.2) (A.3) Noting that: ∂h ∂h ∂h (x)d(t) ≥ − (x) ∥d∥∞ ≥ − (x) d, ∂x ∂x ∂x (A.4) for all x ∈ E and t ∈ R≥0 , and ϵ(h(x)) > 0 for all x ∈ E, adding and 2 subtracting ϵ(h(x))d , and completing the squares yields: 4 ϵ(h(x))d2 ḣ(x, t) ≥ −α(h(x)) − . 4 (A.5) 377 Next, observe that: ∂γT d2 ∂α−1 (r, d) = ∂r 4 ∂r  ϵ(r)d2 − 4  ∂ϵ (r) ≥ 0, ∂r (A.6) for all r ∈ R. Taking the time derivative of the function hT defined in (4.11) yields:   ∂γT (h(x), d) ḣ(x, t). (A.7) ḣT (x, d, t) = 1 + ∂r noting that d is a fixed constant in this proof. By (A.6), we have that: 1+ ∂γT (h(x), d) > 0, ∂r for all x ∈ E. Substituting (A.5) into (A.7), we obtain:    ∂γT ϵ(h(x))d2 ḣT (x, d, t) ≥ 1 + (h(x), d) −α(h(x)) − . ∂r 4 (A.8) (A.9) Next, consider a state x ∈ ∂Cd,T , such that hT (x, d) = 0, for which (4.11) and (A.1) imply: ϵ(h(x))d2 = 0, (A.10) −α(h(x)) − 4 yielding: ḣT (x, d, t) ≥ 0, (A.11) for all t ∈ R≥0 . In the case that d = 0, such that hT (x, 0) = h(x), we have that ∂h e hT (x, 0) = 0 implies ∂x (x) ̸= 0n . Considering d > 0, because γ(a, ·) ∈ K∞ for all a ∈ R, when hT (x, d) = 0, we have −α(h(x)) > 0. Thus, the inequality in ∂h (4.13) requires that ∂x (x) ̸= 0n for x ∈ ∂Cd,T . Finally, we have:   ∂h ∂hT ∂γT (x, d) = 1 + (h(x), d) (x) ̸= 0n , (A.12) ∂x ∂r ∂x {z } | >0 using (A.8). Therefore, Nagumo’s theorem [104], [123] implies the set Cd,T is forward invariant as hT (x, d) = 0 implies ḣT (x, d, t) ≥ 0 for all t ∈ R≥0 , and ∂hT (x, d) ̸= 0n . ∂x