Eye on the Target: Eye Tracking Meets Rodent Tracking

Analyzing animal behavior from video recordings is crucial for scientific research, yet manual annotation remains labor-intensive and prone to subjectivity. Efficient segmentation methods are needed to automate this process while maintaining high accuracy. In this work, we propose a novel pipeline that utilizes eye-tracking data from Aria glasses to generate prompt points, which are then used to produce segmentation masks via a fast zero-shot segmentation model. Additionally, we apply post-processing to refine the prompts, leading to improved segmentation quality. Through our approach, we demonstrate that combining eye-tracking-based annotation with smart prompt refinement can enhance segmentation accuracy, achieving an improvement of 70.6% from 38.8 to 66.2 in the Jaccard Index for segmentation results in the rats dataset.

@article{mededovic2025_2503.10305,
  title={ Eye on the Target: Eye Tracking Meets Rodent Tracking },
  author={ Emil Mededovic and Yuli Wu and Henning Konermann and Marcin Kopaczka and Mareike Schulz and Rene Tolba and Johannes Stegmaier },
  journal={arXiv preprint arXiv:2503.10305},
  year={ 2025 }
}