Exporting .PKL SMPL Meshes and defining whether to use male or female models

[nshreyasvi]

Hello, Thank you for the amazing work. I would like to know how can I export the output .npy meshes into .obj, .npz or .pkl format? I would also like to know if the output .pkl file would follow the standard SMPL keys or would the keys be changed? Moreover, is there a way to use Male/Female model and if it can be defined anywhere?

[fabienbaradel]

Hi @nshreyasvi Thanks for your interest in our work.

You can save into a .obj file using this code snippet for the first human in the list:

vertices = humans[0]['v3d'].cpu().numpy()
faces = model.smpl_layer['neutral_10'].bm_x.faces
mesh = trimesh.Trimesh(vertices=vertices, faces=faces)
filename = "human_0.obj"
mesh.export(filename)

The mesh follow sthe SMPL-X topology (not SMPL). And we use the neutral model so we cannot choose between male or female at inference time.

[nshreyasvi]

Hello, Thank you so much. I also wanted to know if the generated meshes are fitted according to the body shape of the person and if it is able to generate meshes accurately using the neural mesh for male and female bodies without using the male and female SMPL-X templates/prompt?

[fabienbaradel]

Hi, the mesh that we are outpouting is gender neutral. So I do not have a precise answer for your question sorry.

[nshreyasvi]

Hello, Thank you so much for the response. I was able to generate the .obj files. However, I currently face the following 2 issues:

• The output .obj files that are generated have the mesh upside down. Is that normal?
• I wanted to export the output as SMPL-X pkl files so that I can use the converter provided here to convert them into SMPL files. However, when I checked the output .npy files, I can't see the keys that are needed (betas, body_pose, transl). Do you happen to know how can the output SMPL-X mesh can be saved in the standard format so that it is possible to change it?

[fabienbaradel]

Hi @nshreyasvi

• I think that it is just a matter of convention, we use OpenCV convention for 3D axis.
• if you use the converter for moving to SMPLX you will get the mesh only and not the associated parameters. If you want to save the SMPL-X parameters you can add them in the .npy file. Best,

[nshreyasvi]

Hello, Thank you for the answer.

• I would like to know if there is a way to convert the OpenCV convention so that it is aligned with respect to the vertical axis?
• I also tried to extract the output SMPL-X to carry out analysis, However, I was a bit confused by the notations. I would like to know which key is used to get the body_pose of the person? I tried to check for v3d, j3d and rotvec and they tend to have (127, 3), (53, 3) and (10475, 3) shape. I believe that it should be in the form of (1, 72) or (1, 23, 3)?

[vinmila]

Dear Author @fabienbaradel, Thanks for your work and detailed answers, I have tried using the model which works exceptionally well. In addition to the above question, I also would like to point out the difference between the pose parameters as per SMPL-X is 54X3 and multi hmr claims 53X3, is there any parameter reduction considered in Multi HMR model. It would be great if you could through some light how to access the SMPL-X parameter keys which can be used with the SMPL-X blender plugin.

Thanks in advance..

[vinmila]

Hi again, adding the shape of the extracted model parameters below

Shape of transl is (3,) - the primary head point position Shape of rotvec is (53, 3) - I assume this is the pose parameter vector Shape of expression is (10,) - Expression paramters Shape of joints_3d is (127, 3) - Joint positions in 3d Shape of joints_2d is (127, 2) - Joint position in pixel coordinates Shape of shape is (10,) - Shape paramters Shape of transl_pelvis is (1, 3) - Reference pelvis key point location

as per the previous query pose parameters do not match the SMPL pose parameters. For your reference I also plug in the joint hierarchy of SMPL-X

root pelvis left_hip left_knee left_ankle left_foot right_hip right_knee right_ankle right_foot spine1 spine2 spine3 neck head jaw left_eye_smplhf right_eye_smplhf left_collar left_shoulder left_elbow left_wrist left_index1 left_index2 left_index3 left_middle1 left_middle2 left_middle3 left_pinky1 left_pinky2 left_pinky3 left_ring1 left_ring2 left_ring3 left_thumb1 left_thumb2 left_thumb3 right_collar right_shoulder right_elbow right_wrist right_index1 right_index2 right_index3 right_middle1 right_middle2 right_middle3 right_pinky1 right_pinky2 right_pinky3 right_ring1 right_ring2 right_ring3 right_thumb1 right_thumb2 right_thumb3

Regards VinO..

[MPForte]

I would like to know if there is a way to convert the OpenCV convention so that it is aligned with respect to the vertical axis?

I am struggling with a similar problem. Everything works for me only with: person_center = 'head'.

However, if I select the pelvis the vertical alignment is lost. From what I understood up to now, this has to do with the camera parameters that cannot be anymore the identity matrix for the rotation and 0,0,0 for the camera translation. @nshreyasvi have you made any progress on this?

[fabienbaradel]

Hi, You can find explanations about the joints ordering here. So in short:

global_orient = pose[:,0]
body_pose = pose[:,1:22]
left_hand_pose = pose[:,22:37]
right_hand_pose = pose[:,37:52]
jaw_pose = pose[:,52:53]

Our models have been trained using person_center='head' so you cannot use them at inference time with a different keypoint name (for example the pelvis). I hope that it helps,

[vinmila]

Hi Fabien, Greetings. Thanks for your input; it clarifies my query.