Galatea: an optimal transport framework for merging global and local descriptors in visual place recognition on extraterrestrial surfaces
Raunak Sarbajna and Christoph F. Eick · Department of Computer Science, University of Houston · manuscript in preparation
In short
A rover on the Moon has no GPS. It can still locate itself by recognising places: it compares its camera view against a set of reference images with known positions, much as you might recognise a street corner. This is called visual place recognition. It is hard on the Moon for three reasons: the Sun casts black, sharp shadows that move over the lunar day; crater fields look statistically alike everywhere; and lunar soil reflects light in unusual ways.
A benchmark
LunarSim-VPR renders the same rover positions under seven Sun directions, so changes in accuracy are caused by lighting alone.
A framework
Galatea treats both stages of place recognition as optimal transport: pooling image features into one descriptor, and checking local feature matches.
A controlled study
Three seeds per model and paired confidence intervals show which design choices matter for lunar navigation, and which do not.
The LunarSim-VPR benchmark
No public lunar dataset combines exact camera positions with systematic changes of lighting, so we built a lightweight renderer. Each scene combines a 16.4 km regional elevation model with a detailed 0.3 m local one. Craters follow the size–frequency statistics of real lunar terrain, and distant massifs form the horizon. Light follows the lunar-Lambert reflectance model with an opposition surge, and shadows are cast by both local and regional relief.
The test set has three unseen scenes: 450 map images and 3,150 queries. Every query position is rendered under seven Sun conditions, from the map’s own Sun (C0) to the Sun behind the camera and only 8° above the horizon (C6). Training uses five other scenes lit from many Sun directions.
See one location under all seven Sun positions · Move the Sun yourself
The method
Galatea is a two-stage system. Both stages are optimal transport problems: they decide how much “mass” to move between two sets of things.
A family of pooling rules
The first stage solves a semi-relaxed unbalanced transport problem with one parameter, λ. At λ = 0 it is exactly the assignment of NetVLAD, at λ = 1 exactly that of SALAD, and Galatea can also learn λ. The dust-bin lets the model throw away features that say nothing about the place, such as black sky.
We prove that the iterations converge at rate λ, that the dust-bin acts as a logistic gate, and that after normalisation the descriptor depends on the transport plan only through a weight per feature. That last result predicts that, with a frozen backbone, the choice of λ should barely matter.
Fusing local and global evidence
The second stage matches keypoints between the query and each candidate, rewarding matches that keep their relative layout. It needs no training and is unaffected by rotating, shifting or scaling either image.
Its score is fused with the global similarity through a saturating rule. We prove this rule can only overturn the global ranking when the global margin is small, which protects confident global decisions from local matches on moving shadow edges.
Results
Recall@1 is the share of query images whose top-ranked map image shows the correct place (within 10 m and 60° heading). “Hard” averages the three hardest Sun conditions (C2, C5, C6). Lunar-trained rows: mean ± standard deviation over three seeds.
| Model | Recall@1 | Recall@5 | Hard |
|---|---|---|---|
| Earth-trained, used as is | |||
| CosPlace-R50 | 26.8 | 50.3 | 14.1 |
| EigenPlaces-R50 | 28.1 | 53.0 | 16.6 |
| DINOv2-SALAD | 35.8 | 65.4 | 23.4 |
| Lunar-trained head, frozen backbone | |||
| GeM pooling | 31.5 ± 0.4 | 64.7 | 21.7 |
| NetVLAD | 58.2 ± 0.7 | 86.6 | 46.7 |
| OT, λ = 0 | 59.9 ± 0.8 | 87.6 | 48.4 |
| OT, λ = 1 (SALAD) | 59.4 ± 0.7 | 87.3 | 48.6 |
| OT, learned λ (Galatea) | 59.8 ± 0.8 | 87.3 | 47.6 |
| Lunar-trained, backbone fine-tuned | |||
| GeM pooling | 50.7 ± 3.2 | 81.4 | 42.0 |
| NetVLAD | 76.4 ± 0.7 | 94.3 | 64.3 |
| OT, λ = 0 | 73.1 ± 1.1 | 93.3 | 61.9 |
| OT, λ = 1 (SALAD) | 73.5 ± 0.5 | 94.0 | 62.2 |
| OT, learned λ (Galatea) | 71.3 ± 1.2 | 93.0 | 60.1 |
| With global–local fusion (partial FGW) | |||
| Galatea head, frozen | 66.1 ± 0.1 | – | – |
| Galatea head, fine-tuned | 74.5 ± 0.8 | – | 63.0 |
| NetVLAD, fine-tuned (full system) | 78.8 ± 0.5 | 95.3 | 66.9 |
Varied lighting in training has the largest effect
Training on the same scenes but with a single Sun direction costs 13.6 points with a frozen backbone and 21.5 points with a fine-tuned one. A more flexible model fits the training light more closely, so fine-tuning only helps together with varied lighting.
The choice of λ has little effect
With a frozen backbone, every λ from 0 to 1 gives the same accuracy within 1.5 points, as the theory predicts. With fine-tuning the spread stays small, and a learned λ simply stays at its starting value. NetVLAD’s residual pooling, a different design choice, pulls ahead by 4.5 points.
How evidence is fused matters more than the matcher
Ranking by local matches alone rescues 15% of queries but breaks 16% that were right. Fused with the global score, it keeps half the rescues and breaks only 2.4%. With fusion, a training-free matcher, simple nearest neighbours and the learned LightGlue perform alike.
What this means for lunar navigation
- Train under many Sun directions. An illumination sweep in the training data, from a renderer or from elevation-model pipelines, matters more than any architecture choice.
- Adapt the backbone, then choose the pooling. Fine-tuning adds 11–19 points for every model. Fix λ rather than learning it; λ = 0 is the cheapest.
- Use local evidence to adjust the global ranking, not to replace it. Local evidence on the Moon is plentiful but unreliable. Let it adjust the global ranking only where that ranking is uncertain; then the cheapest matcher is good enough.
All results come from simulation. The comparisons are paired and made on unseen scenes, so we expect them to carry over better than the absolute numbers, but they still need to be confirmed on real lunar imagery.
Cite
@misc{sarbajna2026galatea,
title = {Galatea: An Optimal Transport Framework for Merging Global and Local Descriptors
in Visual Place Recognition on Extraterrestrial Surfaces},
author = {Sarbajna, Raunak and Eick, Christoph F.},
year = {2026},
note = {Manuscript in preparation. Code: https://github.com/RaunakDune/galatea-lunar-vpr}
}
Authors. Raunak Sarbajna (ORCID · Google Scholar) and Christoph F. Eick, Department of Computer Science, University of Houston.
Code, renderer and rendered benchmark are released under the MIT license. Apollo photographs: NASA, public domain.