Interactive Learning of Physical Object Properties Through Robot Manipulation and Database of Object Measurements

This work presents a framework for automatically extracting physical object properties, such as material composition, mass, volume, and stiffness, through robot manipulation and a database of object measurements. The framework involves exploratory action selection to maximize learning about objects on a table. A Bayesian network models conditional dependencies between object properties, incorporating prior probability distributions and uncertainty associated with measurement actions. The algorithm selects optimal exploratory actions based on expected information gain and updates object properties through Bayesian inference. Experimental evaluation demonstrates effective action selection compared to a baseline and correct termination of the experiments if there is nothing more to be learned. The algorithm proved to behave intelligently when presented with trick objects with material properties in conflict with their appearance. The robot pipeline integrates with a logging module and an online database of objects, containing over 24,000 measurements of 63 objects with different grippers. All code and data are publicly available, facilitating automatic digitization of objects and their physical properties through exploratory manipulations.

@article{kruzliak2025_2404.07344,
  title={ Interactive Learning of Physical Object Properties Through Robot Manipulation and Database of Object Measurements },
  author={ Andrej Kruzliak and Jiri Hartvich and Shubhan P. Patni and Lukas Rustler and Jan Kristof Behrens and Fares J. Abu-Dakka and Krystian Mikolajczyk and Ville Kyrki and Matej Hoffmann },
  journal={arXiv preprint arXiv:2404.07344},
  year={ 2025 }
}