Visuo-Haptic Object Perception for Robots: An Overview

Object perception capabilities of humans are impressive, something that becomes even more evident when trying to develop solutions with similar proficiency for robots. Taking inspiration from how humans use vision and haptics for object perception and related tasks, this article summarises the current state of multimodal object perception for robotic applications. It covers aspects of biological inspiration, sensor technologies, data sets, and sensory data processing for object recognition and grasping. Firstly, the biological basis of multimodal object perception is outlined. Then the sensing technologies and data collection strategies are discussed. Next, an introduction to the main computational aspects is presented, highlighting a few representative articles for each main application area, including object recognition, transfer learning, and object manipulation and grasping. Finally, informed by the current advancements in each area, this article outlines promising new research directions.