LSMOL icon indicating copy to clipboard operation
LSMOL copied to clipboard
verified publisher icon Robertwyq

Evaluation 3D instance segmentation

Open chreisinger opened this issue 2 years ago • 2 comments

Hello,

thank you for your excellent work! I have two questions about your 3D instance segmentation:

  • Why are TP instances below the threshold not added as false positives (FP)? Another else for if in L122 https://github.com/Robertwyq/LSMOL/blob/d35cb4a58764ef57b6f47082101c643bf3bd5013/evaluation/eval_3d_instance.py#L115-L129
  • The indices in here do not match up when you do not add scores here.

Please give me your thoughts on both points.

Thank you in advance!

chreisinger avatar Dec 04 '23 15:12 chreisinger

Thanks for your attention. We will carefully examine the issues you have raised and provide you with a response.

Robertwyq avatar Dec 05 '23 03:12 Robertwyq

Hello, are there any updates regarding this issue?

I have another question regarding the case of true positives: https://github.com/Robertwyq/LSMOL/blob/d35cb4a58764ef57b6f47082101c643bf3bd5013/evaluation/eval_3d_instance.py#L121-L129

There is no handling for the case of two clusters assigned to the same ground truth instance (with valid IoU, e.g., 0.4). In that case, both clusters would count as TPs, and the FN count would be negative. https://github.com/Robertwyq/LSMOL/blob/9a39b26804285d9a03e56664a34abab187c38ff4/evaluation/eval_3d_instance.py#L130

Do I miss here something?

Thank you for your time.

Best regards, Christian

chreisinger avatar Mar 20 '24 08:03 chreisinger