walk_imitation data format

[KevinBian107]

Hi team,

I am trying to do imitation with rodent data and I was thinking to convert our STAC data to a similar format with you guys' data. However, I am not too sure about what is joint_quat, root2site, root_qpos, and root_qvel. Would it be possible to also give some guidance on your data preprocessing steps? Thanks for all the help.

This is what I got from reading the fly walking h5 file: Dataset: trajectories/00000/joint_quat shape: (90, 102, 4) dtype: float32 Dataset: trajectories/00000/root2site shape: (90, 6, 3) dtype: float32 Dataset: trajectories/00000/root_qpos shape: (90, 7) dtype: float32 Dataset: trajectories/00000/root_qvel shape: (90, 6) dtype: float32

I saw this processing with the walker data: (https://github.com/TuragaLab/flybody/blob/2268560c5ea06cb6ed062ca5a8491f0442d67f09/flybody/tasks/rewards.py#L37 ) is this a similar process done on the reference trajectory?

[vaxenburg]

Hi Kevin, yes, the final step in the reference data preparation was exactly the same as in the get_walker_features function you referenced. And the starting point for that was the inverse kinematics fitting described in the last two paragraphs of the "Reference walking data preparation" section of our preprint. Let me know if it's not very clear?

[KevinBian107]

Hi, thanks for the response, I read through the section and the code again and I think there are a few points that still confuses me a bit. I have all the reference qpos, qvel, body_positions, and CoM. I noticed that the joint_quat for you guys is a (102, 4) array when neglecting clip dimension. I understand it as a quat value for each of the joints. However, I am a bit confused on how wec an actually get that value.

In addition, is the root_qpos here refering to the CoM or is it something else?

Thanks for all the help!

[vaxenburg]

Technically, root_qpos refers to the root joint, but it's pretty close to the actual CoM in the fly model.

joint_quat is a rotation quaternion, one for each of the 102 joints, that expresses two things at once: the direction in space of the joint axis, and also the rotation joint angle qpos around that axis for that joint. There is a function to calculate that here. Also, joint_quat is expressed in the fly's local reference frame, this function makes sure of that.

[KevinBian107]

I see, thanks for the help again, I see that we need to get the xaxis for pointing directions of the joints to combine with qpos. I see how we can extract such value from the xml model for the Mujoco agent, but I am not sure how we would do so for trajectory data (250 frame per clips).

Thanks for all the help!

[vaxenburg]

Probably one way of getting the joint axis direction is first to do inverse kinematics fitting of the body model to the reference data, and then get the joint axis direction from that fitted body model -- that's how we do it anyway. But there could be other ways I guess?