ABSTRACT: Real-world, large-scale, vision-based multi-agent autonomy demands the ability to
efficiently sense, plan, and act under uncertainties. Although vision-based data is a rich and
relatively easily acquired source of information, perceptual uncertainties and constraints—such as
limited fields-of-view in planning, as well as onboard computational and communication limits—
necessitate careful consideration at the algorithmic level. In this talk, we present strategies to
account for vision-based constraints at the control, planning, and localization levels. First, we
introduce a strategy to enable computationally efficient learning of vision-based neural networks
for control using Imitation Learning. Second, we showcase PUMA, an Imitation Learning-based
uncertainty- and perception-aware multi-agent trajectory planner that accounts for uncertainty
arising from state estimation drift. Last, we present a SLAM approach that leverages local graphs of
detected objects/landmarks to build a local and global map. We present evaluations on a variety of
real and simulated aerial vehicles, including a novel, insect-scale soft aerial robot.

BIO: Jonathan P. How is the Richard C. Maclaurin Professor of Aeronautics and Astronautics at the
Massachusetts Institute of Technology. He received a B.A.Sc. from the University of Toronto in
1987, and his S.M. and Ph.D. from MIT in 1990 and 1993, respectively. Prior to joining MIT in 2000,
he was an assistant professor at Stanford University. He was the editor-in-chief of the IEEE Control
Systems Magazine (2015-19) and was elected to the Board of Governors of the IEEE Control System
Society in 2019. His research focuses on robust planning and learning under uncertainty with an
emphasis on multiagent systems. He is a Fellow of IEEE and AIAA and was elected to the National
Academy of Engineering in 2021.