HAT icon indicating copy to clipboard operation
HAT copied to clipboard

why train's log is different from the pretrained models(paper)?

Open Zency-Sun opened this issue 2 years ago • 4 comments

Hello authors, thank you very much for your great work, I have come across this problem while studying your work:

  • In your log log file when training HAT_SRx4, the metrics on different datasets are Set5 (33.0399/0.9053), Set14 (29.1625/0.7963), Urban100 (27.9554/0.8365) and optimal at different iterations
  • The validation metrics reported in Table 6 of your paper (I have tested and your pre-trained models all reproduce the results) are Set5 (33.04/0.9056), Set14 (29.23/0.7973), Urban100 (27.97/0.8368)

How do you view the inconsistency of the data? How did you select the final model from the training model? I look forward to your reply!

Zency-Sun avatar May 04 '23 02:05 Zency-Sun

@Zency-Sun There are some slight differences for data production between the primary experiment and the final inference for the paper results, due to a small mistake. Thus the values in the training logs are not totally the same as reported in the paper. The final model is simply selected according to the average performance in the training log.

chxy95 avatar May 04 '23 03:05 chxy95

Thanks for the quick reply! I have one more question: Comparing the results of HAT_SRx4's log with your paper table 6 for the three validation sets Set5,Set14,Urban100, why is the best result in the log smaller than the one in the paper (especially in Set14 and Urban100)? Isn't the final model picked from the training results? Can you elaborate on how you determined the final model? I'm confused by this question and look forward to your response!

Zency-Sun avatar May 04 '23 06:05 Zency-Sun

@Zency-Sun SRx2 pretraining is used for SRx4, which has influence on the performance of Set5/Set14. The final model is simply choosen based on the approximate average performance on the validation sets from the latest several models.

chxy95 avatar May 04 '23 07:05 chxy95

  • That is, the x4SR results in your paper are the result of x2SR pre-training, not the scratch log, and the results of using the x2SR pre-trained model are better, right?
  • Also, I would like to know if you are selecting the model based on the average PSNR value or SSIM?
  • And,Is it just averaged with three validation sets (Set5+Set14+Urban100) or five validation sets (Set5+Set14+Urban100+BSD100+Mange109)?
  • If it is convenient, I hope you can publish the x2SR, x3SR, x4SR (pretrain from x2SR) log files

Looking forward to and thanking you for your quick reply!

Zency-Sun avatar May 04 '23 07:05 Zency-Sun