ABSTRACT: The presentation will focus on two distinct topics involving the application of learning techniques to analysis of dynamical systems. First: we present an algorithm and a tool for statistical model checking (SMC) of continuous state space Markov chains initialized to a prescribed set of states. This model checking problem requires maximization of probabilities of sets of executions over all choices of initial states. We observe that it can be formulated as an Xarmed bandit problem, and therefore, can be solved using hierarchical optimistic optimization. We propose a new algorithm (HOO-MB) and provide a regret bound on its sample efficiency which relies on the smoothness and the near-optimality dimension of the objective function as well as the sampling batch size. The batch size parameter enables us to strike a balance between the sample efficiency and the memory usage of the algorithm. Our experiments, using the tool HooVer, suggest that the approach scales to realistic-sized problems and is often more sample-efficient compared to other existing tools. Second: we present recent results on the global convergence of policy gradient methods for quadratic optimal control of discrete-time Markovian jump linear systems (MJLS); switching is a common feature in systems that are comprised of interacting software and physical processes, and MJLS are models in which discrete states evolve according to a finite Markov chain and continuous states evolve according to linear dynamics specified by these discrete states. We study the optimization landscape of direct policy optimization for MJLS. Despite the non-convexity of the resultant problem, we are able to identify several useful properties such as coercivity, gradient dominance, and smoothness. Based on these properties, we demonstrate global convergence of three types of flows: the Gauss-Newton flow, the natural gradient flow, and the gradient flow. Then we discretize these flows as the Gauss-Newton method, the natural policy gradient method, and the policy gradient method, and prove that all three methods converge to the optimal state feedback controller for MJLS at a linear rate if initialized at a controller which stabilizes the closed-loop dynamics in the mean square sense. Finally, numerical examples are presented to support our theory. This work brings new insights for understanding the performance of policy gradient methods on the Markovian jump linear quadratic control problem. Also presented will be the HoTDeC multivehicle testbed, which consists of indoor airborne and ground-based vehicles.

BIOSKETCH: Geir E. Dullerud is the W. Grafton and Lillian B. Wilkins Professor in Mechanical Engineering at the University of Illinois at Urbana-Champaign, where he is the Founding Director of the Illinois Center for Autonomy. He is also a member of the Coordinated Science Laboratory, and is an Affiliate Professor of both Computer Science, and Electrical and Computer Engineering. He has held visiting positions in Electrical Engineering KTH, Stockholm (2013), and Aeronautics and Astronautics, Stanford University (2005-2006). Earlier he was on faculty in Applied Mathematics at the University of Waterloo (1996-1998), after being a Research Fellow at the California Institute of Technology (1994- 1995), in the Control and Dynamical Systems Department. He holds a PhD in Engineering from Cambridge University. He has published two books: “A Course in Robust Control Theory”, Texts in Applied Mathematics, Springer, and “Control of Uncertain Sampled-data Systems”, Birkhauser. His areas of current research interest include autonomy and cooperative robotics, convex optimization in control, cyber-physical system security, stochastic simulation, and hybrid dynamical systems. In 1999 he received the CAREER Award from the National Science Foundation, and in 2005 the Xerox Faculty Research Award at UIUC. In 2018 he was awarded the UIUC Engineering Council Award for Excellence in Advising. He is a Fellow of both IEEE (2008) and ASME (2011). He was the General Chair of the recent IFAC workshop Distributed Estimation and Control in Networked Systems (NECSYS2019).