Relative Camera Transforms

[mohammed-amr]

Hello!

I'm having trouble getting the camera to world matrices for perspective views to align. When visualizing image frustums and backprojecting depths into a point cloud, the images don't quite match up and the point cloud is delaminated. This is true for all sets of perspective images.

I'm using the provided matrix R. I'm converting it to opencv convention for my backprojection. I've also tried building the the transform myself given the provided function - the original function as is and an opencv version.

As you can see below, the backprojected depth maps don't quite line up (although they almost do). This is true for every set of perspective images.

Would you please resolve this? Is this perhaps an artifact of the pano tilt/pitch/yaw?

I then tried rendering the rgb and depth panoramas myself, but unfortunately the depth and rgb panoramas are not aligned. See below.

Any help is appreciated please. Thanks!