Add option to skip jacobian calculation for cv2.projectPoints() to speed up calculation

[weidinger-c]

cv2.projectPoints takes very long because it always calculates and returns a jacobian matrix, which is huge for millions of points. It would be nice to just skip the calculation to only get the projected points

[asmorkalov]

The statement is not correct. Jacobians output is optional and has NoArray() value by default. In Python, it means that you get function overload without the output at all. Internally the function checks, if output is needed and calculates it on demand only. See https://github.com/opencv/opencv/blob/c71d4952733a0e1dd1f88ac87066c802f1119d97/modules/calib3d/src/calibration.cpp#L879 and https://github.com/opencv/opencv/blob/c71d4952733a0e1dd1f88ac87066c802f1119d97/modules/calib3d/src/calibration.cpp#L918.

[weidinger-c]

@asmorkalov Thanks for the reply. Could you please tell me how to call the python function correctly then? I have not been able to get a result, that does not contain the jacobian matrix

[weidinger-c]

https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#ga1019495a2c8d1743ed5cc23fa0daff8c The python call always returns a jacobian based on the documenation? Python: cv.projectPoints( objectPoints, rvec, tvec, cameraMatrix, distCoeffs[, imagePoints[, jacobian[, aspectRatio]]] ) -> imagePoints, jacobian

[asmorkalov]

You can try None as the function parameter as empty OutputArray.

[weidinger-c]

No that does not work ("Expression cannot be assignment target")

[asmorkalov]

image_points = cv2.projectPoints(..., None)

[weidinger-c]

image_points = cv2.projectPoints(..., None)

no unfortunately that does not work. Then I still get an object containing the points (1st array) and the jacobian matrix (2nd array).

[weidinger-c]

@asmorkalov I guess there should be an option to calculate the jacobian or pass the None argument correctly to the C++ implementation?

[IanRiera]

image_points = cv2.projectPoints(..., None)

no unfortunately that does not work. Then I still get an object containing the points (1st array) and the jacobian matrix (2nd array).

So I encountered the same issue, and as you mention it does not skip the Jacobian computation in any way. So my fix was to get rid of the opencv function, and get to a DIY solution that I will leave here just in case is of any help. I reduced the runtime by an order of magnitude (from 0.003 to 0.0003).

def project_3D_points(
        self,
        points_3D,
        rotation_matrix,
        translation_vector,
        camera_intrinsics):
    # Step 1: Apply rotation and translation (extrinsic transformation)
    points_camera = (rotation_matrix @ points_3D) + translation_vector[:, np.newaxis]

    # Step 2: Apply intrinsic matrix
    points_pixel = camera_intrinsics @ points_camera

    # Step 3: Normalize by the z-coordinate (homogeneous)
    points_pixel = points_pixel[:2] / points_pixel[2]

    return np.rint(points_pixel).astype(int)

Example of usage:

projected_2D_points = self.project_3D_points(
    points_3D,
    camera_extrinsics[:3, :3],
    camera_extrinsics[:3, 3],
    camera_intrinsics)