Memory leak when encoding images

[stbnps]

Version

0.2.0

Describe the bug.

Calling the encoder, eg: image_encoder.write(paths, images, params=image_encoding_params) leaks memory, this becomes noticeable when you call it for a long time with a lot of images.

Minimum reproducible example

# Just call this a lot of times with many images
image_encoder.write(paths, images, params=image_encoding_params)

Environment details

No response

Relevant log output

No response

Other/Misc.

No response

Check for duplicates

• [X] I have searched the open bugs/issues and have found no duplicates for this bug report

[letmejoin]

meet same issue with nvimgcodec.Decoder(), when decode a lots of image with arange shapes

[jantonguirao]

@stbnps Thank you for reporting this. To better understand this issue. I'd like to ask:

• Do you change the number of paths between your calls?
• What format are you encoding?
• Are you referring to CPU or GPU memory?
• Does it leak if you use the same arguments (images and paths) over and over again in a loop?

[stbnps]

@jantonguirao

Do you change the number of paths between your calls?

Yes, different images and paths for every call

What format are you encoding?

.jpg

Are you referring to CPU or GPU memory?

CPU memory

Does it leak if you use the same arguments (images and paths) over and over again in a loop?

It happens regardless:

import os
from uuid import uuid4

import numpy as np
from nvidia import nvimgcodec
import psutil


process = psutil.Process(os.getpid())
image_encoder = nvimgcodec.Encoder()

fixed_paths = []
fixed_arrays = []
for j in range(100):
    fixed_paths.append(f'ims/{uuid4()}.jpg')
    fixed_arrays.append((np.random.rand(500,500,3) * 255).astype(np.uint8))


print(f'Fixed images / paths test:')
for i in range(100):
    image_encoder.write(fixed_paths, fixed_arrays)
    mem_info = process.memory_info()
    print(f'RSS: {mem_info.rss/1000000:.2f} MB')

print(f'\nVariable images / paths test:')
for i in range(100):    
    arrays = []
    paths = []
    for j in range(100):
        paths.append(f'ims/{uuid4()}.jpg')
        arrays.append((np.random.rand(500,500,3) * 255).astype(np.uint8))
    image_encoder.write(paths, arrays)

    mem_info = process.memory_info()
    print(f'RSS: {mem_info.rss/1000000:.2f} MB')

[jantonguirao]

@stbnps I can confirm that there's a memory leak in the encoder API. Thank you for spotting that. It will be fixed in the following release (0.4.0 that is, since we are just rolling out 0.3.0 as we speak).

@letmejoin I could not reproduce it with the decoder. Could you please provide a simple reproduction script like the one above?

[letmejoin]

@stbnps I can confirm that there's a memory leak in the encoder API. Thank you for spotting that. It will be fixed in the following release (0.4.0 that is, since we are just rolling out 0.3.0 as we speak).

@letmejoin I could not reproduce it with the decoder. Could you please provide a simple reproduction script like the one above? After detailed debugging, it was found that the memory leak problem was caused by the cupyx.scipy.ndimage.rotate() module

[jantonguirao]

After detailed debugging, it was found that the memory leak problem was caused by the cupyx.scipy.ndimage.rotate() module

Thank you for letting us know!