Methods for navigation based on large-scale learning typically treat each episode as a new problem, where the agent is spawned with a clean memory in an unknown environment. While these generalization capabilities to an unknown environment are extremely important, we claim that, in a realistic setting, an agent should have the capacity of exploiting information collected during earlier robot operations. We address this by introducing a new retrieval-augmented agent, trained with RL, capable of querying a database collected from previous episodes in the same environment and learning how to integrate this additional context information. We introduce a unique agent architecture for the general navigation task, evaluated on ObjectNav, ImageNav and Instance-ImageNav. Our retrieval and context encoding methods are data-driven and heavily employ vision foundation models (FM) for both semantic and geometric understanding. We propose new benchmarks for these settings and we show that retrieval allows zero-shot transfer across tasks and environments while significantly improving performance.
View on arXiv@article{monaci2025_2504.03524,
title={ RANa: Retrieval-Augmented Navigation },
author={ Gianluca Monaci and Rafael S. Rezende and Romain Deffayet and Gabriela Csurka and Guillaume Bono and Hervé Déjean and Stéphane Clinchant and Christian Wolf },
journal={arXiv preprint arXiv:2504.03524},
year={ 2025 }
}