Setting camera pose in pyrender

[oneThousand1000]

Hi there!

I spent several days trying to properly set the camera pose in pyrender, and now I want to share how I convert camera matrices to pyrender camera poses.

I find the problem of the incorrect camera pose is caused by the flipped y-axis and z-axis in pyrender.

So if you want to set a camera's pose as R @ T (R denotes rotation matrix and T denotes translation matrix), you should:

# R : rotation matrix, [4,4]
# T : translation matrix, [4,4]

# flicp the y and z coordinate
T_in_pyrender = T
T_in_pyrender [[1,2],3] *= -1

# get angles 
angles= cv2.Rodrigues(R[:3,:3])[0]

# flip y and z angles
angles[[1,2],:] *= -1
# get rotation matrix
new_R = cv2.Rodrigues(angles)[0]
R_in_pyrender  = np.eye(4)
R_in_pyrender [:3,:3] = new_R

# compute camera pose

camera_pose = R_in_pyrender  @  T_in_pyrender 

# Then you can do something like:

scene.add(camera, pose=camera_pose)

In sum, you should flip the y-axis and z-axis (in translation and rotation).

[Jianghanxiao]

Some lookAt() functions which can potential be easy for people to use https://github.com/Jianghanxiao/Helper3D/blob/master/trimesh_render/src/camera.py#L4

[oneThousand1000]

Some lookAt() functions which can potential be easy for people to use https://github.com/Jianghanxiao/Helper3D/blob/master/trimesh_render/src/camera.py#L4

Thanks!