MONAI Resolve the confusion about dimension order and orientation through their respective keyword arguments in `LoadImage`

Is your feature request related to a problem? Please describe.

As a PyTorch user, I was surprised that MONAI does not follow PyTorch's dimension order convention (MONAI: WHD, PyTorch: DHW).

PyTorch 3D Conv docs:

I also thought that it is odd that the images are not represented in the way that a radiologist would look at them, with the patient facing up or down, instead of left or right. However, I understand that this is the convention that MONAI settled on. I will only be proposing a more transparent and flexible approach further below.

Finally, the reverse_indexing option in LoadImage is misleading - I tried it with NrrdReader, NibabelReader, and ITKReader, and it only worked with ITKReader. At least in theory for my case (not to clutter the post, I will upload the picture in the response).

Here are a few issues/discussions describing the confusion/differences/conventions:

https://github.com/Project-MONAI/MONAI/discussions/2189#discussioncomment-732827
https://github.com/Project-MONAI/MONAI/discussions/2888
https://github.com/Project-MONAI/MONAI/discussions/4872
https://github.com/Project-MONAI/MONAI/issues/362#issuecomment-625675026
https://github.com/Project-MONAI/MONAI/issues/4862

Describe the solution you'd like Similarly to how LoadImage has ensure_channel_first, it would be really transparent, flexible, and a non-breaking change to have:

order/dim_order, set by default to WHD as per MONAI convention, but a user could change it to DWH or any other combination.
orientation, can be set to RAS, LPS, etc. I am not sure right now if there is a consistent convention for different formats or if it varies. It could be implemented in a "non-breaking" way, similar to how LoadImage's reader is implemented, but it might be useful to have it consistent for all formats.
[Optional] If 1. is implemented, reverse_indexing is probably unnecessary.

Additional context Personally, even though I have been using MONAI for quite a while now, I always transpose the images so that: 1) they follow the PyTorch convention, 2) we see the axial slice (first dim after batch and channel dims) in the same way that a radiologist would look at it. I also orient the images LPS, but that's irrelevant rn.

The proposed approach may be particularly important for sharing models/MONAI Bundle - the dimension order or orientation used can be different between models and easily overlooked, rendering the weights almost useless if the data is not in the correct format. The proposed feature would make it more transparent as well as more easily documented by developers/sharers.

Jan 04 '24 16:01 ibro45

The ITKReader with and without reverse_indexing. When True, the values ended up being really small.

reverse_indexing=False

reverse_indexing=True, low values

Jan 04 '24 16:01 ibro45

Hi @ibro45, thanks for the reporting, the dimension order, as here said may not be a problem. And for the reverse_indexing result, I don't think it will change the intensity of the image, why does the pic you pasted show a large difference between two max and min intensities?

Jan 05 '24 07:01 KumoLiu

as here https://github.com/Project-MONAI/MONAI/issues/362#issuecomment-626422782 may not be a problem

Indeed, networks don't care about the order, but users do, and, judging by the list of issues/discussions above, they get confused at times (including me). That definitely has to do with all sorts of conventions out there, and that issue cannot be solved, but I believe that it can be transparent, more easily changed, and consistent across readers.

A LoadImage(order="WHD", orientation="RAS") would make it much simpler to know what's going on by default and to manipulate it as you wish. Currently, you would have to use Transpose and Orientation to make it look identical to when read with SimpleITK and turned into numpy. Don't get me wrong, these transforms definitely should exist, but a more transparent, flexible, and consistent LoadImage would be beneficial.

reverse_indexing feels awkward - it's simply not used by some readers, without any warning. With others, it works. Moreover, there are other inconsistencies between readers and orientation, @surajpaib will open a different issue.

For ease of use, I believe all MONAI readers should be consistent with regard to the order and the orientation of the images they load.

Jan 05 '24 17:01 ibro45

Alright, so there's something we realized just now. We used to get it into the PyTorch convention by doing:

Orientation(axcodes="LPS")
Transpose(indices=[0, 3, 2, 1]) # cxyz -> czyx

And we used to have issues with Transpose not updating the metadata accordingly. After finding https://github.com/Project-MONAI/MONAI/issues/5975, we figured out that we can just do Orientation with "SPL" instead of "LPS" and no Transpose to achieve the PyTorch convention:

Orientation(axcodes="SPL")

I think that, therefore, having orientation argument in LoadImage set to "RAS" by default with an explanation in the docstrings on how to manipulate it to achieve a certain convention would be really nice and understandable.

Jan 13 '24 23:01 ibro45