UniMVSNet icon indicating copy to clipboard operation
UniMVSNet copied to clipboard

The evaluation results on DTU evaluation set are different between paper and released checkpoints

Open YANG-SOBER opened this issue 3 years ago • 2 comments

Dear Rui Peng:

Thank you very much for your contribution and nice work.

I evaluate your released checkpoints "unimvsnet_dtu.ckpt" on the DTU evaluation set. (I do not change any parameters)

The results are: 0.4173 for mean accuracy and 0.2966 for mean completeness.

However, in the paper, the two metrics are 0.352 and 0.278, respectively.

May I know whether this released checkpoint is the one you used for the paper?

Thanks for your help.

Looking forward to your response.

YANG-SOBER avatar Dec 29 '22 07:12 YANG-SOBER

After set the align_corners=True in the F.grid_sample(), the result is 0.3685 (acc) and 0.2785 (comp), the other difference will be attributed to the gipuma (fusible). The compilation of which is dependent on the compute capability of the particular cuda and gpu version, please refer to https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/.

YANG-SOBER avatar Dec 31 '22 13:12 YANG-SOBER

The test results are indeed related to the environment, maybe you can try pytorch1.2.

prstrive avatar Dec 31 '22 14:12 prstrive