Masked Sensory-Temporal Attention for Sensor Generalization in Quadruped Locomotion

With the rising focus on quadrupeds, a generalized policy capable of handling different robot models and sensor inputs becomes highly beneficial. Although several methods have been proposed to address different morphologies, it remains a challenge for learning-based policies to manage various combinations of proprioceptive information. This paper presents Masked Sensory-Temporal Attention (MSTA), a novel transformer-based mechanism with masking for quadruped locomotion. It employs direct sensor-level attention to enhance the sensory-temporal understanding and handle different combinations of sensor data, serving as a foundation for incorporating unseen information. MSTA can effectively understand its states even with a large portion of missing information, and is flexible enough to be deployed on physical systems despite the long input sequence.

@article{liu2025_2409.03332,
  title={ Masked Sensory-Temporal Attention for Sensor Generalization in Quadruped Locomotion },
  author={ Dikai Liu and Tianwei Zhang and Jianxiong Yin and Simon See },
  journal={arXiv preprint arXiv:2409.03332},
  year={ 2025 }
}