ShoufaChen
What may cause the AP result difference
I run the evaluate command, but get the different AP results,
note that I only change the command arg --num-gpus 8 to --num-gpus 1 since I only have one gpu device
But the results are slight different compared to the paper experiments.
I evaluate it with your pretrained weight, any ideas on why the difference occurred?
the scores in "()" are from paper, and the scores out of "()" are my evaluate results.
| model | AP | AP50 | AP75 | APs | APm | APl |
|---|---|---|---|---|---|---|
| coco.res50 | 45.776(45.5) | 65.417(65.1) | 49.313(48.7) | 27.809(27.5) | 48.298(48.1) | 61.726(61.2) |
| coco.res101 | 46.528(46.6) | 66.290(66.3) | 49.969(50.0) | 29.977(30.0) | 49.468(49.3) | 62.079(62.8) |
| coco.swinbase | 52.301(52.3) | 72.812(72.7) | 56.473(56.3) | 35.481(34.8) | 56.004(56.0) | 68.613(68.5) |
Hi,
Can you get the strictly same results when evaluate twice on your machine?
Hi,
Can you get the strictly same results when evaluate twice on your machine?
Yes, I run the command twice with same setting, and get same results.
May I know your system and detectron2 version (git commit tag)?
It seems that you got higher results than ours in most cases. I am sorry that I can't reproduce your results on my machine.
May I know your system and detectron2 version (git commit tag)?
It seems that you got higher results than ours in most cases. I am sorry that I can't reproduce your results on my machine.
detectron2 version should be 0.6, I build it from source.
and GPU is v100 32G, only one gpu device.
During inference time, DiffusionDet will randomly generate noisy boxes, which may affect the AP results.
