Rotated bboxes transforms

[AntoineSimoulin]

Add Transforms support for Rotated Boxes

This PR implements the last transforms for rotated boxes and follows what has been implemented in #9095 and #9084. This PR implements in particular the following modifications :

• Add support for perspective for rotated boxes;
• Fix missing tests for affine transformation and rotated boxes;
• Fix the _affine_bounding_boxes_with_expand function for rotated boxes when expand=True;
• Fix clamp_bounding_boxes function with behavior detailed below;
• Add support for elastic for rotated boxes;
• Add support for crop for rotated boxes.
• Add missing tests for TestConvertBoundingBoxFormat;
• Remove the SUPPORTED_BOX_FORMATS and NEW_BOX_FORMATS variable in the tests as tests for transform now full cover rotated boxes
• Add support for sanitize for rotated boxes

Details on the clamping function

For the clamping, we re-order the point of the box such that the point with the lowest value on the x-axis is the point 1 (c.f. _order_bounding_boxes_points). Given the position of the 4 vertices with respect to the y-axis (c.f. cases above), we are going to adjust the points (x1, y1), (x2, y2), and (x4, y4) to make sure the point (x1, y1) is on the right side of the y-axis. We loop through the four vertices of the rotated box and apply the same operation. In the end we are guaranteed that the bounding box will be within the canvas size and will be completely included within the area of the original box.

We propose some illustration examples below (original boxes in grey and corresponding clamped boxes in blue.

Please note that depending on the order in which we loop through the vertices, we are not guaranteed the output boxes is the box with the largest area that meet the condition above (we might be too aggressive with the clamping. This can occur if the box is largely out of bounds along multiple axis).

Test plan

Please run the following tests:

pytest test/test_transforms_v2.py -vvv -k "TestPerspective and test_kernel_bounding_boxes"
pytest test/test_transforms_v2.py -vvv -k "TestPerspective and test_correctness_perspective_bounding_boxes"

pytest test/test_transforms_v2.py -vvv -k "TestAffine and test_transform_bounding_boxes_correctness"

pytest test/test_transforms_v2.py -vvv -k "TestRotate and test_kernel_bounding_boxes"
pytest test/test_transforms_v2.py -vvv -k "TestRotate and test_functional_bounding_boxes_correctness"
pytest test/test_transforms_v2.py -vvv -k "TestRotate and test_transform_bounding_boxes_correctness"

pytest test/test_transforms_v2.py -vvv -k "TestClampBoundingBoxes and test_kernel"
pytest test/test_transforms_v2.py -vvv -k "TestClampBoundingBoxes and test_functional"

pytest test/test_transforms_v2.py -vvv -k "TestElastic and test_kernel_bounding_boxes"

pytest test/test_transforms_v2.py -vvv -k "TestConvertBoundingBoxFormat and test_kernel"
pytest test/test_transforms_v2.py -vvv -k "TestConvertBoundingBoxFormat and test_kernel_noop"

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/9104

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:white_check_mark: You can merge normally! (1 Unrelated Failure)

As of commit 809b71c4b02e377fd5b31640ab9d354e90146530 with merge base 6473b779bdb8ba02bab0fc9e0f4ef4661ebb632a ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• CMake / windows (windows.g5.4xlarge.nvidia.gpu, cuda, 11.8) / windows-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[AntoineSimoulin]

Hey @NicolasHug I publish a fix which should fix the test and address your comments. Here is the list of the modifications:

• Modify the make_bounding_boxes function to add clamping and padding, this ensuring that rotated boxes are build within the range of the canvas size;
• Re-placing the reference_perspective_bounding_boxes function within the TestPerspective class to reduce the number of lines modified in this PR and since this function is only used within the class;
• Decreasing the tightness for the test in TestAffine to atol=1e-5, rtol=2e-5 as the rotation angle had slightly higher variation when computed with the test function. Also let some tolerance for TestConvertBoundingBoxFormat;
• Not applying the function _parallelogram_to_bounding_boxes to int rotated box as the truncation of the point from float to int does not preserve the rectangular shape of the box;
• Apply clamping after resizing rotated bounding boxes;
• Improve docstring for the _clamp_rotated_bounding_boxes function.

Please run the tests with:

pytest test/test_transforms_v2.py -k box -v
...
2372 passed, 1432 skipped, 5025 deselected in 67.68s (0:01:07)

[github-actions[bot]]

Hey @AntoineSimoulin!

You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py