GelFusion: Enhancing Robotic Manipulation under Visual Constraints via Visuotactile Fusion

Visuotactile sensing offers rich contact information that can help mitigate performance bottlenecks in imitation learning, particularly under vision-limited conditions, such as ambiguous visual cues or occlusions. Effectively fusing visual and visuotactile modalities, however, presents ongoing challenges. We introduce GelFusion, a framework designed to enhance policies by integrating visuotactile feedback, specifically from high-resolution GelSight sensors. GelFusion using a vision-dominated cross-attention fusion mechanism incorporates visuotactile information into policy learning. To better provide rich contact information, the framework's core component is our dual-channel visuotactile feature representation, simultaneously leveraging both texture-geometric and dynamic interaction features. We evaluated GelFusion on three contact-rich tasks: surface wiping, peg insertion, and fragile object pick-and-place. Outperforming baselines, GelFusion shows the value of its structure in improving the success rate of policy learning.

@article{jiang2025_2505.07455,
  title={ GelFusion: Enhancing Robotic Manipulation under Visual Constraints via Visuotactile Fusion },
  author={ Shulong Jiang and Shiqi Zhao and Yuxuan Fan and Peng Yin },
  journal={arXiv preprint arXiv:2505.07455},
  year={ 2025 }
}