A Vision-Inertial Fusion Based Approach for Dynamic Grasping and Motion Planning of a Table Tennis Ball Collection Robot

This paper presents a novel vision-inertial fusion methodology to significantly enhance the dynamic grasping and motion planning performance of an autonomous table tennis ball collection robot. Operating in highly dynamic and unstructured environments necessitates robust spatial perception and rapid, accurate trajectory prediction. To address this challenge, our system deeply integrates data from stereo cameras and Inertial Measurement Units (IMUs) within a tightly-coupled Visual-Inertial Simultaneous Localization and Mapping (VI-SLAM) framework. This integration ensures robust egomotion estimation and consistent environmental mapping, even during rapid agent movements. For precise ball state estimation under varying spin and bounce conditions, we employ an extended Kalman filter, which effectively fuses kinematic predictions with visual observations. Furthermore, a dedicated Convolutional Neural Network (CNN) is leveraged to perform real-time grasp quality prediction directly from visual input, enabling the system to select optimal pre-grasp poses. Comprehensive experimental validations demonstrate the efficacy of our approach, showing a 39% improvement in localization accuracy compared to vision-only systems and a significant increase in successful ball collection rates. The proposed architecture provides a comprehensive and reliable solution for autonomous robotic systems that require dynamic interaction with fast-moving objects in complex settings.