Scripts not compatible with H3 datasets

[MedericFourmy]

Changes introduced in the new H3 dataset hot3d breaks their compatibility with some existing utility scripts (unrelated to evaluation). The two main causes are:

• file structure: hot3d scene annotations are done per stream (rgb, gray1, etc.) and the scene camera format has changed (classic datasets -> "cam_K", hot3d -> "cam_model")
• camera model: New dataset Hot3d includes fisheye camera images, which can be handled with the hand tracking toolkit camera model and rasterizer.

Candidates scripts for update:

• [x] ~~annotation_tools: file structure, relies on depth data (not present in HOT3d and HANDAL), assumes pinhole model~~ (cf https://github.com/thodan/bop_toolkit/issues/137#issuecomment-2348489294)
• [x] calc_gt_distribution: file structure, add stream choice
• [x] calc_gt_masks: file structure, relies on depth data
• [x] calc_gt_info: file structure, relies on depth data
• [ ] crop_image_and_gt: file structure, assumes pinhole model
• [x] enumerate_test_targets: file structure
• [ ] render_train_imgs: file structure, assume pinhole camera
• [ ] subsample_dataset: file structure
• [x] vis_gt_poses: file structure, assumes pinhole type rasterizer (we need to switch to bop_toolkit_lib/renderer_htt.py for hot3d).
• [x] vis_est_poses: same as vis_gt_poses
• [x] vis_object_symmetries: file structure, pinhole camera stored in dataset folder (could replace by a user defined cam with reasonable defaults).

[AnasIbrahim]

The annotation tool cannot be extended to the new H3 BOP24 format. There is no easy way to do this as the tool uses depth and point cloud visualization. So I would exclude it from this list.

[MartinSmeyer]

Hey @MedericFourmy, Since the new BOP robotics datasets will be in H3 format, can we continue to update the scripts?

[MedericFourmy]

Yes indeed, which ones would you prioritize?

[MartinSmeyer]

Thanks Mederic! I would start with these:

calc_gt_distribution: file structure, add stream choice calc_gt_masks: file structure, relies on depth data calc_gt_info: file structure, relies on depth data vis_object_symmetries: file structure, pinhole camera stored in dataset folder (could replace by a user defined cam with reasonable defaults).