EventNeRF icon indicating copy to clipboard operation
EventNeRF copied to clipboard

PSNR is not the same as in the paper

Open 520jz opened this issue 1 year ago • 7 comments

Hello, I reproduced your experiment, currently testing on the lego dataset, all the parameters have not been touched, but I found that even if I iterate to 1 million times, I can't achieve the effect of PSNR25 in the paper, in fact it is only about 17, why is that? The parameters are kept the same as in your github repository. image

520jz avatar Mar 06 '24 10:03 520jz

Another problem is that the formulas in your paper do not correspond to the codes

520jz avatar Mar 06 '24 11:03 520jz

Hello. The logged PSNR should not be used for evaluation as it is derived from the training event-based loss of the current mini-batch. The numbers reported in the paper are computed after training for 500,000 iterations using the procedure described in metric/README.md.

r00tman avatar Mar 06 '24 16:03 r00tman

Hello. The logged PSNR should not be used for evaluation as it is derived from the training event-based loss of the current mini-batch. The numbers reported in the paper are computed after training for 500,000 iterations using the procedure described in metric/README.md.

Thank you for your reply. You mean to use metric/README.md. to manually calculate the PSNR of the results, right? I'll give it a try

520jz avatar Apr 13 '24 09:04 520jz

Hello. The logged PSNR should not be used for evaluation as it is derived from the training event-based loss of the current mini-batch. The numbers reported in the paper are computed after training for 500,000 iterations using the procedure described in metric/README.md.

But I found that there seems to be no comp folder in the source code you provided? image The command requires entering the comp folder image

520jz avatar Apr 13 '24 09:04 520jz

Hello. The logged PSNR should not be used for evaluation as it is derived from the training event-based loss of the current mini-batch. The numbers reported in the paper are computed after training for 500,000 iterations using the procedure described in metric/README.md.

Sorry, I misread it. I downloaded the comp folder manually from here, but there doesn’t seem to be an sh file in it. image image

520jz avatar Apr 13 '24 09:04 520jz

Hello. The logged PSNR should not be used for evaluation as it is derived from the training event-based loss of the current mini-batch. The numbers reported in the paper are computed after training for 500,000 iterations using the procedure described in metric/README.md.

Sorry, I misread it. I downloaded the comp folder manually from here, but there doesn’t seem to be an sh file in it. image image Where is this main.py file? It seems that the main.py file is not used in the source code you provided. image

520jz avatar Apr 13 '24 11:04 520jz

Hello. The logged PSNR should not be used for evaluation as it is derived from the training event-based loss of the current mini-batch. The numbers reported in the paper are computed after training for 500,000 iterations using the procedure described in metric/README.md.

Where is this main.py file? It seems that the main.py file is not used in the source code you provided. image

520jz avatar Apr 13 '24 11:04 520jz