Memory leak using func_timeout .

[Utsav13]

def get_video_capture2(timeout=1):
        try:
            ReturnValue = func_timeout(timeout , cap .read)
            return ReturnValue
        except FunctionTimedOut:
            print("cap .read could not complete within {} seconds and was terminated".format(timeout))
        except Exception as e:
            print("Exception in cap.read() = {}".format(e))

Here is my python function where I use func_timeout. without func_timeout memory leak is not found in this code. I also try to free an object using del obj, but no changes were detected.

using del obj, python code

def get_video_capture2(timeout=1):
        global frame
        try:
            ReturnValue = func_timeout(timeout , cap .read)
            frame = ReturnValue
            del  ReturnValue
            return True
        except FunctionTimedOut:
            print("cap .read could not complete within {} seconds and was terminated".format(timeout))
        except Exception as e:
            print("Exception in cap.read() = {}".format(e))

so how can I free my memory ??

[kata198]

First, is this in python2 or python3?

Second, everything should be referenced in the garbage collector. You can access this via the "gc" module, and "gc.collect" (takes optional "generation" argument) will garbage collect.

The garbage collector SHOULD be independent, but it is possible that there is a bug in python2 that is keeping a reference on the dead thread (related to issue #10, see my latest update).

You can play around with the gc module to track your objects, and see if they still hold reference (they should not). "gc.get_referrers(ReturnValue)" for example, will show you WHAT is specifically still holding reference to that variable.

I can look into this more for you in a bit, but my time is a bit limited right now due to personal life things. Please let me know what you discover, and/or if you need further assistance! I will keep watch and support when I am able.

Thanks!

[kata198]

btw, I should add that the "del" statement does NOT free memory, it just removes a reference. So the garbage collector still needs to run for it to clean up.

You can probably just run gc.collect() after the exception is raised and everything will be honkey-dorey [this will also happen automatically after a configurable number of objects in each generation, and upon certain triggers], but in the case that it does not, see the additional methods available in the previous comment!

[fnavarrogonzalez]

Im able to reproduce it using python2:

timeout
gc: collectable <type 0x562e63c9a620>
gc: collectable <dict 0x7fd7a0cccb40>
gc: collectable <tuple 0x7fd7a0cb73c0>
gc: collectable <tuple 0x7fd7a1c06650>
gc: collectable <function 0x7fd7a0cc6b18>
gc: collectable <tuple 0x7fd7a0cb7310>
gc: collectable <type 0x562e63c9a260>
gc: collectable <cell 0x7fd7a1c01130>
gc: collectable <cell 0x7fd7a1c01168>
gc: collectable <cell 0x7fd7a1c01440>
gc: collectable <cell 0x7fd7a1c01b78>
gc: collectable <dict 0x7fd7a0ccd050>
gc: collectable <tuple 0x7fd7a3bd5fc8>
timeout
gc: collectable <type 0x562e63c9a9e0>
gc: collectable <cell 0x7fd7a1c0b590>
gc: collectable <cell 0x7fd7a0cbb280>
gc: collectable <cell 0x7fd7a0cbb2f0>
gc: collectable <dict 0x7fd7a0ccd4b0>
gc: collectable <tuple 0x7fd7a0cb74c8>
gc: collectable <type 0x562e63c9ada0>
gc: collectable <cell 0x7fd7a1c0b088>
gc: collectable <tuple 0x7fd7a1c00dd0>
gc: collectable <function 0x7fd7a0cc6c80>
gc: collectable <dict 0x7fd7a0ccd280>
gc: collectable <tuple 0x7fd7a0cb75d0>
gc: collectable <tuple 0x7fd7a0cb7520>
timeout
gc: collectable <type 0x562e63c9b160>
gc: collectable <cell 0x7fd7a1c010f8>
gc: collectable <cell 0x7fd7a0cbb398>
gc: collectable <cell 0x7fd7a0cbb328>
gc: collectable <dict 0x7fd7a0cccc58>
gc: collectable <tuple 0x7fd7a0cb7680>
gc: collectable <type 0x562e63c9b520>
gc: collectable <cell 0x7fd7a1c01478>
gc: collectable <tuple 0x7fd7a1c06610>
gc: collectable <function 0x7fd7a0cc6cf8>
gc: collectable <dict 0x7fd7a0ccc910>
gc: collectable <tuple 0x7fd7a0cb7418>
gc: collectable <tuple 0x7fd7a0cb76d8>

from time import sleep
from func_timeout import func_timeout, FunctionTimedOut
import gc
import os
import psutil
from pydexdump import dexdump
gc.set_threshold(1, 1, 1)
gc.set_debug(gc.DEBUG_LEAK)


def function(argument):
    a = {"key": argument}
    sleep(3)
    return a

def execute():
    returnValue = func_timeout(3, function, args=("hii",))
    return returnValue


while True:
    try:
        gc.collect()
        a = execute()
        del a
        gc.collect()
    except FunctionTimedOut:
        gc.collect()
        print("timeout")

Also with the help of mprof you can see that the memory is increasing each time, better with a function that is expensive memory wise.