Work smart, not hard: Recalling relevant experiences for vast-scale but time-constrained localisation

This paper is about life-long vast-scale localisation in spite of changes in weather, lighting and scene structure. Building upon our previous work in Experience-based Navigation [1], we continually grow and curate a visual map of the world that explicitly supports multiple representations of the same place. We refer to these representations as experiences, where a single experience captures the appearance of an environment under certain conditions. Pedagogically, an experience can be thought of as a visual memory. By accumulating experiences we are able to handle cyclic appearance change (diurnal lighting, seasonal changes, and extreme weather conditions) and also adapt to slow structural change. This strategy, although elegant and effective, poses a new challenge: In a region with many stored representations - which one(s) should we try to localise against given finite computational resources? By learning from our previous use of the experience-map, we can make predictions about which memories we should consider next, conditioned on how the robot is currently localised in the experience-map. During localisation, we prioritise the loading of past experiences in order to minimise the expected computation required. We do this in a probabilistic way and show that this memory policy significantly improves localisation efficiency, enabling long-term autonomy on robots with limited computational resources. We demonstrate and evaluate our system over three challenging datasets, totalling 206km of outdoor travel. We demonstrate the system in a diverse range of lighting and weather conditions, scene clutter, camera occlusions, and permanent structural change in the environment.