ABSTRACT: Solutions of optimization problems, including policy optimization in reinforcement learning, typically rely upon some variant of gradient descent. There has been much recent work in the machine learning, control, and optimization communities applying the Polyak-Łojasiewicz Inequality (PŁI) to such problems in order to establish an exponential rate of convergence  (a.k.a. “linear convergence” in the local-iteration language of numerical analysis) of loss functions to their minima under the gradient flow. Often, as is the case of policy iteration for the continuous-time LQR problem, this rate vanishes for large initial conditions, resulting in a mixed globally linear / locally exponential behavior. This is in sharp contrast with the discrete-time LQR problem, where there is global exponential convergence. That gap between CT and DT behaviors motivates the search for various generalized PŁI-like conditions, and this talk will address that topic. Moreover, these generalizations are key to understanding the transient and asymptotic effects of errors in the estimation of the gradient, errors which might arise from adversarial attacks, wrong evaluation by an oracle, early stopping of a simulation, inaccurate and very approximate digital twins, stochastic computations (algorithm “reproducibility”), or learning by sampling from limited data. We will describe an “input to state stability” (ISS) analysis of this issue. We will also discuss convergence and PŁI-like properties of “linear feedforward neural networks” in feedback control. (Joint work with A.C.B. de Oliveira, L. Cui, Z.P. Jiang, and M. Siami).

BIOSKETCH: Eduardo D. Sontag received his Licenciado in Mathematics at the University of Buenos Aires (1972) and a Ph.D. in Mathematics (1977) under Rudolf E. Kalman at the University of Florida. From 1977 to 2017, he was at Rutgers University, where he was a Distinguished Professor of Mathematics and a Member of the Graduate Faculty of the Departments of Computer Science and of Electrical and Computer Engineering and the Cancer Institute of NJ. He directed the undergraduate Biomathematics Interdisciplinary Major and the Center for Quantitative Biology, and was Graduate Director at the Institute for Quantitative Biomedicine. In January 2018, Dr. Sontag became a University Distinguished Professor in the Departments of Electrical and Computer Engineering and of BioEngineering at Northeastern University, where he is also affiliated with the Mathematics and the Chemical Engineering departments. Since 2006, he has been a Research Affiliate at the Laboratory for Information and Decision Systems, MIT, and since 2018 he has been a Faculty Member in the Program in Therapeutic Science at Harvard Medical School. His major current research interests lie in several areas of control and dynamical systems theory, systems molecular biology, cancer and immunology, machine learning, and computational biology. Sontag was awarded the Reid Prize in Mathematics in 2001, the 2002 Hendrik W. Bode Lecture Prize and the 2011 Control Systems Field Award from the IEEE, the 2022 Richard E. Bellman Control Heritage Award, the 2023 IFAC Triennial Award on Nonlinear Control, the 2002 Board of Trustees Award for Excellence in Research from Rutgers, and the 2005 Teacher/Scholar Award from Rutgers. Sontag is a Fellow of IEEE, AMS, SIAM, and IFAC. In 2024 he was inducted into the American Academy of Arts and Sciences, and in 2025 he was elected into the National Academy of Sciences.