How do I transfer both velodyne and pose to the camera coordinate system?

[Lee-JaeWon]

I hope that both the pos and velodyne are represented in the camera coordinate system "Perspective cameras (image_00, image_01): x = right, y = down, z = forward"

But I don't know what to do. Velodyne data was performed with inv(cam_to_velo), The poses.txt data also moved the coordinate system through inv(cam_to_pose)

It doesn't work very well.

Thank you for your answer.

[yiyiliao]

Hi, here you can find how to convert the Velodyne data to the camera view: https://github.com/autonomousvision/kitti360Scripts/blob/master/kitti360scripts/viewer/kitti360Viewer3DRaw.py#L232

Let me know if this does not address your question.

[Lee-JaeWon]

@yiyiliao

Thank you for your response.

I kept referring to that code as well.

Thanks to this, the process of projecting velodyne points to the image was successful.

However, i am trying now accumulate these velodyne points depending on the corresponding position (cam0_to_world.txt).

So I tried it as below,

I felt that the image and the coordinate system didn't fit when I multiplied it by the pos.

Am I doing it right?

'velo' is pure velodyne points from .bin, Tr_velo_to_camis np.inv(cam_to_velo). 'trans_cam_to_world' is the poses taken directly from cam0_to_world.txt

            TF_points = (R0_rect.dot(Tr_velo_to_cam.dot(velo)).T)
                    
            trans_cam_to_world = self.get_poses()

            campose = np.array(trans_cam_to_world[idx]).reshape(3, 4)
            campose = np.vstack((campose,np.array([0,0,0,1])))

            points = np.matmul(TF_points, campose.T)

[yiyiliao]

Hi, you can refer to the code we provided in C++ for accumulation: https://github.com/autonomousvision/kitti360Scripts/blob/9defe218f4d05a1c920924f38ebfa8efc27318da/kitti360scripts/devkits/accumuLaser/src/accumulation.cpp#L460-L472

Note that the raw velodyne points should be corrected for ego-motion (CurlVelodyneData).