Question about Baxter dataset evaluation criteria

[xiaotian-zhang]

Hello,
Thank you for your great work!

I am currently trying to reproduce the results from your paper and have some questions regarding the Baxter dataset.

From the dataset I obtained, I can see only the 3D coordinates of the end-effector (ee) and its corresponding 2D coordinates in the image. However, the dataset does not provide ground truth (GT) 2D PCK and 3D PCK for each joint.

I would like to confirm:

• Did you evaluate the dataset's 2D PCK and 3D PCK solely based on the end-effector (ee)?
• Additionally, I would like to ask about the selection criteria for 2D PCK and 3D PCK in the mask or rendered images. How were these values computed or selected?

For reference, one of the poses in the dataset is as follows:
{'ee_2d': [1642, 667],
'ee_3d': [0.49483262696492686, -0.08692254723444083, 0.787991035104306],
'joints': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.8]}

Looking forward to your response!

[ootts]

1. Yes, we only evaluate ee's 2D and 3D instead of all joints to ensure a fair comparison to previous works.
2. I didn't fully understand this question, could you clarify it?

[xiaotian-zhang]

Thank you for your quick reply! Since you only used the end-effector (ee) coordinates for evaluation, question 2 is no longer an issue. Thanks!

Can I add you on WeChat to further discuss the details?

[ootts]

Sure. My wechat id is clhfsky.