[Feature Request] Support same_on_batch option for transforms

[xvjiarui]

🚀 Feature

In the latest torchvison, transforms support tensor input (very nice feature, cheers!).

Motivation

As titled, same_on_batch option is not available for torchvision transforms. For example, RandomResizedCrop will crop at the same location for all images in the batch (which is ideal for video, but not for batched images).

Pitch

Kornia has this feature off the shelf. Maybe torchvision could follow the same API.

cc @vfdev-5

[vfdev-5]

@xvjiarui thanks for the feature request ! Yes, we are also thinking about applying random transformations on each image in the batch.

[fmassa]

Hi,

Thanks for opening this issue, this is indeed something we should plan to do at some point in the future.

But I think it might take some time before we get into it, because I think for a full-fleshed experience this will require torch.vmap and NestedTensor, both of which are currently prototypes in PyTorch.

I think we should be in close touch with @cpuhrsch (who is working on NestedTensor) and @zou3519 (who is working on vmap) so that we can beta-test those features in the context of this feature request. But I wouldn't expect this feature to be included in torchvision in the next 6 months.

[mizoru]

This never materialized, right?

[AlessandroMondin00]

Any updates here? I think this would be super important

[juliendenize]

Hi,

I developed a library based on Torchvision v2 called Torchaug that aims to have batched transforms with random parameters for each sample in the batch.

For example RandomResizedCrop can be batched to handle different transforms.

I believe that as soon as vmap and nested tensors will be truly released by Pytorch, Torchvision will implement a more efficient way of doing it but this should work in the meantime :).