Support Mixup and Cutmix transformation

[JiYuanFeng]

HI, the more strong transformation "RandCoarseDropout" has been released in the latest version. It's possible to support Mixup or Cutmix in MonAI?

[Nic-Ma]

Hi @JiYuanFeng ,

Thanks for your feedback. Seems like a new transform feature request, I marked for feature discussion. @ericspod @wyli Do you guys think it's a common approach in medical imaging augmentation?

Thanks in advance.

Thanks.

[wyli]

for the mixup, I can find this paper as a reference https://openreview.net/pdf?id=rkBBChjiG. not sure if we have an example/renference for cutmix.

[sandylaker]

@wyli For CutMix, you can refer to this paper. These two augmentations mix one image with another randomly selected one, which can be selected from the same batch or the whole dataset. Thus, the API of these two augmentations are slightly different from the other ones.

[wyli]

Thanks, I think these are good augmentations to have. @Nic-Ma these could be in v0.8?

[Nic-Ma]

Sounds good to me. I can try to implement in v0.8.

Thanks.

[Nic-Ma]

Hi @wyli ,

Currently, all the MONAI transforms work with a single image, so I think maybe this feature is not a transform, It should be a layer before the network to mix up a batch data?

Thanks.

[ericspod]

My thoughts on mixup that I wanted to play with at some point was as a special sort of DataLoader that would combine images from a batch to make a smaller batch with some controls for how the mixing would be done. Alternatively if we wanted to do it as a transform we would have to first create a dataset of image pairs that are to be mixed (this wouldn't involve copying the original images though so should be lightweight), a normal transform sequence, and a final transform that does the mixing of the two images to produce one. This can get us around the issue of transform sequences being one-to-one for the most part, they operate on individual items in a batch rather than being able to operate on a whole batch.

[masadcv]

@Nic-Ma Just to add to discussion here, there is an existing implementation of this for medical images, perhaps for paper @wyli linked (https://openreview.net/pdf?id=rkBBChjiG), in NiftNet at: https://github.com/NifTK/NiftyNet/blob/935bf4334cd00fa9f9d50f6a95ddcbfdde4031e0/niftynet/application/segmentation_application.py#L325

[Nic-Ma]

Hi @masadcv ,

Thanks for your sharing. Let me investigate it and get back ASAP.

Thanks.

[JohnMasoner]

Hi @Nic-Ma ,

It seems that Mixup and Cutmix are still unsupported.

Is there any consideration to include this data transformation method? Because I've been working on related content lately, if consider adding these transformations. I will submit pr.

Thanks.

[Nic-Ma]

Hi @JohnMasoner ,

Thanks for your interest here. If @ericspod and @wyli don't have any other comments, welcome to contribute a PR.

[JohnMasoner]

Thanks for your reply,

I readed @ericspod comments, i will contribute a PR about CutOut( Maybe like 'RandCoarseDropout', also similar to Mixup and Cutmix)

[ericspod]

Hi @JohnMasoner, thanks for looking into this. I'd definitely like to see some contributions along the lines of mixup/cutmix, something new you're working on would be great as well as reference implementations of what's been published already. I had suggested implementing it as a dataloader of some sort but it's possible to be instead a special collate function, or some other implementation entirely, so feel free to propose your own solution.