NVlabs
IoU for each category in PASCAL VOC
Thanks a lot for your contribution!
Do you have IoU for each category for PASCAL VOC dataset? This would help understand how your model performs for each category and sometimes helps to interpret some results. Also, are background class masks obtained with query "background"?
+------------+-------+-------+
| Class | IoU | Acc |
+------------+-------+-------+
| background | 80.63 | 88.04 |
| aeroplane | 38.08 | 91.94 |
| bicycle | 31.41 | 82.44 |
| bird | 51.6 | 92.92 |
| boat | 32.7 | 80.97 |
| bottle | 63.46 | 84.18 |
| bus | 78.83 | 89.82 |
| car | 65.07 | 77.16 |
| cat | 79.22 | 92.35 |
| chair | 18.75 | 23.81 |
| cow | 73.4 | 85.33 |
| table | 31.63 | 42.33 |
| dog | 76.42 | 84.66 |
| horse | 59.41 | 81.66 |
| motorbike | 55.31 | 90.56 |
| person | 43.96 | 46.05 |
| plant | 40.92 | 61.29 |
| sheep | 66.57 | 82.72 |
| sofa | 31.52 | 46.53 |
| train | 49.47 | 92.3 |
| monitor | 29.74 | 49.96 |
+------------+-------+-------+
For the background, we use some threshold to predict. See details here https://github.com/NVlabs/GroupViT/blob/13b786155a1dfffe4703f40d028c92be58e1178d/segmentation/evaluation/group_vit_seg.py#L242
Hi @xvjiarui thanks for sharing the class wise accuracy and IoU scores. Can you please tell me which checkpoint are you using for computing the validation scores on VOC? I saw there are there three checkpoints available on your GitHub:
- https://github.com/xvjiarui/GroupViT/releases/download/v1.0.0/group_vit_gcc_yfcc_30e-879422e0.pth
- https://github.com/xvjiarui/GroupViT/releases/download/v1.0.0/group_vit_gcc_yfcc_30e-74d335e6.pth
- https://github.com/xvjiarui/GroupViT/releases/download/v1.0.0/group_vit_gcc_redcap_30e-3dd09a76.pth
Thanks a lot.
@xvjiarui thank you for sharing your results. Can you please also share per-class IoU on Pascal Context? I would like to know how well this work on classes like 'floor' and 'ceiling'. Thanks again.
Hi @GodzSom
Please find pascal context 59 classes mIoU here
+------------+-------+-------+ [24/1134]
| Class | IoU | Acc |
+------------+-------+-------+
| airplane | 41.15 | 97.13 |
| bag | 23.86 | 33.32 |
| bed | 48.31 | 61.93 |
| bedclothes | 33.78 | 36.26 |
| bench | 30.23 | 54.03 |
| bicycle | 45.37 | 91.13 |
| bird | 30.87 | 98.19 |
| boat | 29.57 | 94.21 |
| book | 9.88 | 10.44 |
| bottle | 63.28 | 91.6 |
| building | 26.59 | 31.28 |
| bus | 65.42 | 94.53 |
| cabinet | 26.6 | 71.75 |
| car | 59.36 | 90.57 |
| cat | 63.45 | 98.2 |
| ceiling | 24.86 | 46.25 |
| chair | 39.42 | 55.76 |
| cloth | 24.02 | 26.71 |
| computer | 17.45 | 68.98 |
| cow | 49.53 | 96.43 |
| cup | 11.75 | 15.09 |
| curtain | 26.05 | 32.87 |
| dog | 69.12 | 97.01 |
| door | 13.43 | 21.63 |
| fence | 23.83 | 41.87 |
| floor | 36.3 | 52.24 |
| flower | 41.92 | 50.09 |
| food | 26.34 | 63.53 |
| grass | 17.52 | 18.47 |
| ground | 23.24 | 36.37 |
| horse | 43.85 | 97.22 |
| keyboard | 30.82 | 37.36 |
| light | 6.87 | 17.52 |
| motorbike | 56.73 | 95.09 |
| mountain | 29.15 | 60.39 |
| mouse | 8.97 | 9.67 |
| person | 52.53 | 88.76 |
| plate | 18.68 | 57.6 |
| platform | 24.31 | 56.76 |
| plant | 57.32 | 76.65 |
| road | 20.4 | 34.15 |
| rock | 37.81 | 56.41 |
| sheep | 43.88 | 97.16 |
| shelves | 21.37 | 38.05 |
| sidewalk | 15.36 | 55.68 |
| sign | 19.15 | 40.64 |
| sky | 38.43 | 40.64 |
| snow | 41.59 | 75.13 |
| sofa | 48.69 | 63.47 |
| table | 35.37 | 68.45 |
| track | 16.31 | 68.59 |
| train | 62.35 | 86.74 |
| tree | 33.21 | 41.65 |
| truck | 32.58 | 46.52 |
| monitor | 56.04 | 70.05 |
| wall | 10.26 | 10.64 |
| water | 37.92 | 45.97 |
| window | 27.14 | 52.19 |
| wood | 23.89 | 26.06 |