TrackOcc: Camera-based 4D Panoptic Occupancy Tracking

Comprehensive and consistent dynamic scene understanding from camera input is essential for advanced autonomous systems. Traditional camera-based perception tasks like 3D object tracking and semantic occupancy prediction lack either spatial comprehensiveness or temporal consistency. In this work, we introduce a brand-new task, Camera-based 4D Panoptic Occupancy Tracking, which simultaneously addresses panoptic occupancy segmentation and object tracking from camera-only input. Furthermore, we propose TrackOcc, a cutting-edge approach that processes image inputs in a streaming, end-to-end manner with 4D panoptic queries to address the proposed task. Leveraging the localization-aware loss, TrackOcc enhances the accuracy of 4D panoptic occupancy tracking without bells and whistles. Experimental results demonstrate that our method achieves state-of-the-art performance on the Waymo dataset. The source code will be released atthis https URL.

@article{chen2025_2503.08471,
  title={ TrackOcc: Camera-based 4D Panoptic Occupancy Tracking },
  author={ Zhuoguang Chen and Kenan Li and Xiuyu Yang and Tao Jiang and Yiming Li and Hang Zhao },
  journal={arXiv preprint arXiv:2503.08471},
  year={ 2025 }
}