mediapy `display_images()` downsamples images seemingly arbitrarily....

[erictabellion]

Issue tested with version: https://github.com/google/mediapy/pull/41

a) If I call media.show_images() once passing it a dict of 32 images, each with shape (512, 512, 3), it stubbornly displays a row matrix of images that are (256, 256, 3) each.

b) If I call media.show_images() 32 times passing it a dict with a single image each time, each with shape (512, 512, 3), it displays the images at the correct resolution (however the images are displayed one below each other which is not what I want).

c) In case a), if I pass the argument height=512 then the images are displayed pixelated 2x: it looks like mediapy makes a 256x256 internal image buffer and displays it pixelated at 512x512.

d) Passing downsample=False doesn't fix the issue either (too easy-no points).

e) I also have a repro case where it dowsamples from 480 to 240 so this seems like a 2x downsample bug.

f) Aha!!! It looks like displaying a smaller number of images (6 vs. 32) works as intended.

Looks like it is some kind of cap on total memory it can consume?

[hhoppe]

Hi Eric!

I can't seem to be able to reproduce it. In Colab, I tried:

import mediapy as media
images = {f'image{index}': media.color_ramp((512, 512)) for index in range(32)}
media.show_images(images)

and it shows 32 images of size (512, 512, 3). When I right-click Save_as_image on one of them, it does save as a 512x512 image.

In Jupyter Lab, it shows 32 shrunk images such that all of them appear in the window width. But again when I shift-right-click Save_as_image it is natively 512x512. (The scaling there might be adjustable using CSS styles.)

Please give us more details. Perhaps try Chrome Developer Tools (Control-Shift-i) to examine the native image resolution? Is there any OS display scaling (e.g., Windows "Make everything bigger") or Chrome Zoom scaling?

[erictabellion]

Hi Hugues!

All the input image resolutions I reported were obtained printing the np.ndarray.shape. All the display resolutions I reported were obtained after right-clicking on the image displayed by mediapy and selecting "Open in a new tab" in Google Chrome. I then hover the mouse over the new tab and a tooltip appears showing the native resolution of the image being displayed among other things. right-click+save the image produces an image file of that resolution.

In my use-case I get an image_batch from my ML pipeline data-loader, with shape (1, 32, 512, 512, 3) and dtype=np.float32. I display it with:

def display_image_row(image_row):
  image_dict = {}
  for col in range(image_row.shape[0]):
    image_dict[f'{col}'] = image_row[col]
  media.show_images(image_dict, height=512, downsample=False, ylabel='row0')

display_image_row(image_batch[0])

Whether or not I convert the input image_batch first to uint8, I get the same issue.

image_batch = image_batch * 255
image_batch = image_batch.astype(np.uint8)

[hhoppe]

Are you using Colab? Please see my shared Colab notebook in https://colab.research.google.com/drive/1pkTsS0v40kbf66d6a_rfMS1V6VrS247d?usp=sharing which seems to work fine with both np.uint8 and np.float32 and with/without the height and downsample flags.

[futscdav]

Hi Hugues @hhoppe ,

I ran into this somewhat annoying limitation too with no apparent way to tell the function to disable the downsampling.

I've edited your snippet to show the issue

import mediapy as media
import numpy as np 
ramp_images = {f'image{index}': media.color_ramp((512, 512)) for index in range(32)}
rand_images = {f'image{index}': np.random.uniform(0, 255, size=[512, 512, 3]).astype(np.uint8) for index in range(32)}
media.show_images(rand_images)
media.show_images(ramp_images)

Looks like maybe there's a limitation based on how much data the images have? Ramp is fine, random images are not.

[futscdav]

Okay after some code reading I found the issue, there's a conditional (and a very crude) downsampling step in https://github.com/google/mediapy/blob/main/mediapy/init.py#L1113-L1116

However, running mediapy._IPYTHON_HTML_SIZE_LIMIT = mediapy._IPYTHON_HTML_SIZE_LIMIT * 10

seems to be fine and produced desired result. Perhaps this could be an argument to the function rather than hidden global?

[hhoppe]

@futscdav Fantastic detective work!

(This explains why I couldn't reproduce @erictabellion 's issue --- my synthetic ramp images compressed too well.)

Now I vaguely remember introducing this threshold because large HTML output would cause some issue in the browser notebook (Colab and/or Jupyter Lab, I don't remember) -- maybe a front-end crash or hang. This was >5 years ago, so the size threshold may be too small today. Also, mediapy should issue a warning that it is downsampling the image content!