[WIP] 3-view reprojection-error based ViewGraphEstimator

[johnwlambert]

Register 3 views using averaging or PNP, then perform BA, and check the support for the triplet via reprojection error.

[travisdriver]

Considering >2 views for outlier rejection purposes is a great idea. Would it be possible to make this work for N views? Why stop at 3?

[johnwlambert]

Considering >2 views for outlier rejection purposes is a great idea. Would it be possible to make this work for N views? Why stop at 3?

Thanks for taking a look Travis. I think with very low probability all the triangulated keypoints and relative poses for erroneous geometry + matches would align for 3 views with low reprojection error. But we can certainly validate loops of length 4, or 5, or 6, etc. afterwards.

I added a note in one of the files about the "Growing Consensus" algorithm. Moving from consistent 2 to consistent 3, and onwards, is consistent with that algorithm: https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Son_Solving_Small-Piece_Jigsaw_CVPR_2016_paper.pdf

[akshay-krishnan]

@johnwlambert do we see an improvement in metrics? it'd be nice to add relevant metrics to PR description.

[johnwlambert]

@johnwlambert do we see an improvement in metrics? it'd be nice to add relevant metrics to PR description.

Hi @akshay-krishnan, no improvement yet. We should discuss this in more detail on Wednesday. We get unusually low reprojection error after BA even for some egregiously bad matches, and need to investigate it more.

[johnwlambert]

[johnwlambert]

vge-no-ba-visual_comparison_dashboard.html.zip

[akshay-krishnan]

Thanks John, its a nice improvement, some of them have regressions though. We should look into those.

[johnwlambert]

vge-no-ba-v2-visual_comparison_dashboard.html.zip

Thanks for taking a look @akshay-krishnan. 2nd run yielded similar results. @travisdriver it's interesting that more precision here, but tiny bit less recall, seems to hurt AstroNet's translation_distance error metric so much.

[johnwlambert]

Before:

After:

I guess the units for AstroNet don't mean anything though, so maybe its converged? Rotation and translation angular error only change in the decimal point.

[travisdriver]

@johnwlambert Thank you for taking a closer look at AstroNet. I think there is clearly some degradation in the results by the looks of the plots (would probably be more apparent from a top-down view). Increasing the reprojection threshold will likely do the trick.

Looking at the dashboard, it seems like there are a lot of regressions. Only 2/11 of the benchmarks seem to benefit across the board (deep skydio-8 and SIFT rc3). Maybe some of the other datasets would benefit from a higher reprojection threshold; the threshold previously used for pairs may be too restrictive when considering triplets.