PointTransformer Test result with pre-trained model on S3DIS dataset

[yudopan]

Checklist

• [X] I have searched for similar issues.
• [X] I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for master branch).

My Question

Thanks for the work! I have a question about the test result of PointTransformer(pytorch) on S3DIS dataset with the pretrained model you provided. I tried to do the test on Stanford3dDataset_v1.2_Aligned_Version and Stanford3dDataset_v1.2. I get mIoU 67.35 for aligned version and 67.79 for the other one, never get 69.2 that report in the table. Do you have any idea why? Best, Pan

[aoligei178]

I have the same problem! I also get mIOU 67.3% for aligned version, but I didn't try another version. No matter how I train Point Transformer, mIOU on test is always lower than validation result by 2%. Have you solved this problem?

[yudopan]

unfortunately no :(

[aoligei178]

Hello! I have found something wrong with their test script. When running training script and validation script, they use 50000 points as input, but when running testing script, they use all the points as input, they are different!! I found other codes such as RandLA-Net keep them the same. I haven't try yet, so I can not tell you whether this operation makes difference, I will tell you once upon I got conclusion. You can also have a try!

[yudopan]

thank you for your reply:-) But I thought it should not be the reason. In testing, it downsamples the point cloud, too.