Add semi-dense matching support (LoFTR)
- LoFTR module (
hloc/matchers/loftr.py) - semi-dense matching on image pairs (
hloc/match_dense.py) - quantization to store features and matches in hloc format
- supports quantization to pre-extracted features (e.g. superpoint)
I've had some good results using this. I was wondering though, if there is a reason the matching is hard-coded to a batch size of 1? It occurs to me that more fully utilising the GPU & maybe allowing data-parallel across multiple GPUs might make working with this a lot faster / more practical.
I'm also slightly confused about the use of conf.top_k. It appears to be referenced, but not assigned?
Is this still in the pipeline to be merged?
Yes this is still in the pipeline to be merged, and should be ready now. Tested on Aachen v1.1 with setup loftr_aachen (no superpoint keypoints), which should run on machines with 32GB RAM:
Reconstruction:
num_reg_images = 6697
num_cameras = 4331
num_points3D = 6106347
num_observations = 29844267
mean_track_length = 4.88742
mean_observations_per_image = 4456.36
mean_reprojection_error = 1.31786
Localization results:
Day: 88.7 / 96.4 / 99.0
Night: 75.9 / 90.6 / 99.0
Re batch size: I tested it and for me it only works with a batch size of 1 on Aachen (RTX 3080, 6 pairs / second). Nevertheless, I put the match extraction using LoFTR in a separate for-loop in case someone is willing to make this multi-GPU compatible.
cc @JiamingSuen