ABSTRACT: The human skin is capable of sensing various types of forces with high resolution and accuracy. The development of an artificial sense of touch for robots needs to address these properties, while retaining scalability to large surface areas of arbitrary shape. This talk describes the development of a soft, vision-based tactile sensor that fully exploits the high resolution of modern cameras, while at the same time offering ease of manufacture. In order to overcome the complexity of interpreting raw tactile data, a machine learning strategy is presented to map from the tactile images to the distribution of (pressure and shear) forces applied to the sensor. The strategy is based on data that are entirely generated via finite element simulations and retains high real-time accuracy on real-world data, while generalizing across different sensors and contact objects. Additionally, the talk presents an application of the tactile sensing technique to dexterous manipulation. Reinforcement learning is employed to achieve highly dynamic motions with control policies entirely trained on a faster-than-real-time simulator and successfully deployed to a real-world robotic system without further adaptation.

BIOSKETCH: Carlo Sferrazza is a PhD candidate at the Institute for Dynamic Systems and Control, ETH Zurich, under the supervision of Prof. Dr. Raffaello D’Andrea. His main research interests include the design and development of vision-based, data-driven tactile sensors, and the applications of such sensors to robot control and dexterous manipulation. He received the B.Sc. degree in Automation Engineering from Politecnico di Milano in 2014, the B.Eng. in Electronics and Information Engineering from Tongji University, China, and the M.Sc. degree in Robotics, Systems and Control from ETH Zurich in 2016. He was a recipient of the Best Paper Award at the 2020 IEEE International Conference on Soft Robotics, and he received the 2017 ETEL Award for his master’s thesis on model predictive control of an unmanned aerial vehicle. He has been repeatedly involved in communicating his research to the general public, and has been a presenter at the 2019 WORLD.MINDS Annual Symposium and at TEDxZurich 2020.