RobustDet icon indicating copy to clipboard operation
RobustDet copied to clipboard

The training results did not meet the results mentioned in the paper

Open CrazyBrick opened this issue 2 years ago • 0 comments

Reproduced on the VOC dataset using the scripts and parameters used in readme, with only two parameters modified (batch size reduced and lr reduced). The local trained model and the provided model were evaluated, and the results showed a significant difference. Why ?Did I make a mistake?

[INFO] <11-17 13:45:17> VOC07 metric? Yes
[INFO] <11-17 13:45:17> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:17> AP for aeroplane = 0.2715
[INFO] <11-17 13:45:17> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:17> AP for bicycle = 0.3174
[INFO] <11-17 13:45:17> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:18> AP for bird = 0.0304
[INFO] <11-17 13:45:18> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:18> AP for boat = 0.0411
[INFO] <11-17 13:45:18> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:19> AP for bottle = 0.0775
[INFO] <11-17 13:45:19> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:19> AP for bus = 0.3412
[INFO] <11-17 13:45:19> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:21> AP for car = 0.4596
[INFO] <11-17 13:45:21> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:21> AP for cat = 0.1192
[INFO] <11-17 13:45:21> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:21> AP for chair = 0.0608
[INFO] <11-17 13:45:21> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:22> AP for cow = 0.0397
[INFO] <11-17 13:45:22> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:22> AP for diningtable = 0.3488
[INFO] <11-17 13:45:22> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:22> AP for dog = 0.0670
[INFO] <11-17 13:45:22> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:22> AP for horse = 0.3076
[INFO] <11-17 13:45:22> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:23> AP for motorbike = 0.2049
[INFO] <11-17 13:45:23> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:26> AP for person = 0.3152
[INFO] <11-17 13:45:26> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:27> AP for pottedplant = 0.0223
[INFO] <11-17 13:45:27> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:27> AP for sheep = 0.0428
[INFO] <11-17 13:45:27> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:27> AP for sofa = 0.1608
[INFO] <11-17 13:45:27> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:28> AP for train = 0.2479
[INFO] <11-17 13:45:28> datasets/VOCdevkit/VOC2007/ImageSets/Main/test.txt
[INFO] <11-17 13:45:28> AP for tvmonitor = 0.2768
[INFO] <11-17 13:45:30> Mean AP = 0.1876
[INFO] <11-17 13:45:30> ~~~~~~~~
[INFO] <11-17 13:45:30> Results:
[INFO] <11-17 13:45:30> 0.272
[INFO] <11-17 13:45:30> 0.317
[INFO] <11-17 13:45:30> 0.030
[INFO] <11-17 13:45:30> 0.041
[INFO] <11-17 13:45:30> 0.078
[INFO] <11-17 13:45:30> 0.341
[INFO] <11-17 13:45:30> 0.460
[INFO] <11-17 13:45:30> 0.119
[INFO] <11-17 13:45:30> 0.061
[INFO] <11-17 13:45:30> 0.040
[INFO] <11-17 13:45:30> 0.349
[INFO] <11-17 13:45:30> 0.067
[INFO] <11-17 13:45:30> 0.308
[INFO] <11-17 13:45:30> 0.205
[INFO] <11-17 13:45:30> 0.315
[INFO] <11-17 13:45:30> 0.022
[INFO] <11-17 13:45:30> 0.043
[INFO] <11-17 13:45:30> 0.161
[INFO] <11-17 13:45:30> 0.248
[INFO] <11-17 13:45:30> 0.277
[INFO] <11-17 13:45:30> 0.188
[INFO] <11-17 13:45:30> ~~~~~~~~
[INFO] <11-17 13:45:30> 
[INFO] <11-17 13:45:30> --------------------------------------------------------------
[INFO] <11-17 13:45:30> Results computed with the **unofficial** Python eval code.
[INFO] <11-17 13:45:30> Results should be very close to the official MATLAB eval code.
[INFO] <11-17 13:45:30> --------------------------------------------------------------

Mean AP = 0.1876 VS 0.415

CrazyBrick avatar Nov 17 '23 07:11 CrazyBrick