ARISE-Initiative
Standardizing End-Effector Frame Conventions for Robot-Gripper Compatibility
Issue: Current code shows misaligned gripper mounting orientations, requiring custom gripper_mount_pose adjustments for different robot-gripper combinations.
Proposed frame convention:
Robot end-effector frame:
- Z: gripper mounting direction
- Y: lateral direction, pointing to the left in front view
- X: Y × Z
Gripper frame:
- Z: mounting direction (same as robot end-effector frame)
- X: Palm normal/gripper closing direction
- Y: Z × X
It makes all robot end-effector and gripper base frames consistent, enabling plug-and-play gripper changes without pose adjustments. Also eliminates the need for custom gripper_mount_quat_offset configurations for PandaDexLH and PandaDexRH.
More examples:
PS: We'd better add some unit tests to verify frame consistency in new robot and gripper integrations.
This is really nice @Dingry , thanks!
Going to CC @xieleo5 @Steve-Tod: does this change require updating anything for our datasets? Also, @Dingry does this change anything for the 1) franka + panda gripper / 2) GR1 + inspire/fourier hand datasets?
Yes I believe if you change the site orientation, very likely we need to re-generate the data on latest robosuite because the IK target will change. But I also think this unification is important if in the future we want to add more grippers.
Well, I did not realized the impact to the generated dataset before. I tried to playback the dataset of GR1+Fourier hand collected before, and found it works well. If other datasets are the same case, maybe we can simply replay them and record the necessary robot states as the new dataset without re-generation.
Agreed with unification @xieleo5; @Dingry gotcha, does action playback also work?