关于petr的img2lidar

[Shaquille-Wu]

petr这个模型需要用到图像坐标系到lidar坐标系的转换，apollo源码中也提供了相关的参数，具体如下：

std::vector k_data_{ -1.40307297e-03, 9.07780395e-06, 4.84838307e-01, -5.43047376e-02,

  -1.40780103e-04, 1.25770375e-05,  1.04126692e+00,  7.67668605e-01,

  -1.02884378e-05, -1.41007011e-03, 1.02823459e-01,  -3.07415128e-01,

  0.00000000e+00,  0.00000000e+00,  0.00000000e+00,  1.00000000e+00,

  -9.39000631e-04, -7.65239349e-07, 1.14073277e+00,  4.46270645e-01,

  1.04998052e-03,  1.91798881e-05,  2.06218868e-01,  7.42717385e-01,

  1.48074005e-05,  -1.40855671e-03, 7.45946690e-02,  -3.16081315e-01,

  0.00000000e+00,  0.00000000e+00,  0.00000000e+00,  1.00000000e+00,

  -7.0699735e-04,  4.2389297e-07,   -5.5183989e-01,  -5.3276348e-01,

  -1.2281288e-03,  2.5626015e-05,   1.0212017e+00,   6.1102939e-01,

  -2.2421273e-05,  -1.4170362e-03,  9.3639769e-02,   -3.0863306e-01,

  0.0000000e+00,   0.0000000e+00,   0.0000000e+00,   1.0000000e+00,

  2.2227580e-03,   2.5312484e-06,   -9.7261822e-01,  9.0684637e-02,

  1.9360810e-04,   2.1347081e-05,   -1.0779887e+00,  -7.9227984e-01,

  4.3742721e-06,   -2.2310747e-03,  1.0842450e-01,   -2.9406491e-01,

  0.0000000e+00,   0.0000000e+00,   0.0000000e+00,   1.0000000e+00,

  5.97175560e-04,  -5.88774265e-06, -1.15893924e+00, -4.49921310e-01,

  -1.28312141e-03, 3.58297058e-07,  1.48300052e-01,  1.14334166e-01,

  -2.80917516e-06, -1.41527120e-03, 8.37693438e-02,  -2.36765608e-01,

  0.00000000e+00,  0.00000000e+00,  0.00000000e+00,  1.00000000e+00,

  3.6048229e-04,   3.8333174e-06,   7.9871160e-01,   4.3321830e-01,

  1.3671946e-03,   6.7484652e-06,   -8.4722507e-01,  1.9411178e-01,

  7.5027779e-06,   -1.4139183e-03,  8.2083985e-02,   -2.4505949e-01,

  0.0000000e+00,   0.0000000e+00,   0.0000000e+00,   1.0000000e+00};

很明显，这是一个6x4x4的矩阵，每个4x4是旋转+位移吗？ 旋转矩阵部分，不是单位的，这堆参数是不是有错？ 比如，1.041，这个数已经超过1了 1.140732，这个数也超过1了 其他几个旋转矩阵也都有这个问题， 所以，我想请教一下，这个6x4x4的矩阵是想表达什么？如果是旋转矩阵+位移，那么旋转矩阵部分为什么不是单位的？

[LordonCN]

Same to bevformer, it was calculate by this: https://github.com/fundamentalvision/BEVFormer/blob/20923e66aa26a906ba8d21477c238567fa6285e9/projects/mmdet3d_plugin/datasets/nuscenes_dataset_v2.py#L144

  if self.modality['use_camera']:
      image_paths = []
      lidar2img_rts = []
      lidar2cam_rts = []
      cam_intrinsics = []
      for cam_type, cam_info in info['cams'].items():
          image_paths.append(cam_info['data_path'])
          # obtain lidar to image transformation matrix
          lidar2cam_r = np.linalg.inv(cam_info['sensor2lidar_rotation'])
          lidar2cam_t = cam_info[
              'sensor2lidar_translation'] @ lidar2cam_r.T
          lidar2cam_rt = np.eye(4)
          lidar2cam_rt[:3, :3] = lidar2cam_r.T
          lidar2cam_rt[3, :3] = -lidar2cam_t
          intrinsic = cam_info['cam_intrinsic']
          viewpad = np.eye(4)
          viewpad[:intrinsic.shape[0], :intrinsic.shape[1]] = intrinsic
          lidar2img_rt = (viewpad @ lidar2cam_rt.T)
          lidar2img_rts.append(lidar2img_rt)

          cam_intrinsics.append(viewpad)
          lidar2cam_rts.append(lidar2cam_rt.T)

[LordonCN]

Now we are testing for bevformer in new coming apollo version, lidar2img plays an important role in it, code will be release in the future.