shelved_cache icon indicating copy to clipboard operation
shelved_cache copied to clipboard

Functions with the same arguments conflict with each other

Open uninstall-your-browser opened this issue 1 year ago • 2 comments

When 2 functions have different names but the same parameter signature, they conflict with each other when using this utility

import cachetools
from shelved_cache import PersistentCache
from cachetools import LRUCache

pc = PersistentCache(LRUCache, filename="/tmp/test_cache", maxsize=1000)

@cachetools.cached(pc)
def square(x):
    return x * x

@cachetools.cached(pc)
def cube(x):
    return x * x * x

if __name__ == '__main__':
    print(f"square(2) = {square(2)}") # = 4
    print(f"cube(2) = {cube(2)}") # = 4 !?

Expected that cube(2) = 8. ~~The key function in the decorator should at least consider something like method.__qualname__ to prevent this in most cases~~ This seems a bit more complex than that, cachetools relies on each function/method having a unique cache/dict, so sharing the cache with cachetools.cached will have this problem unless you have another wrapper like with asynccached or cachedasyncmethod and add the function name (or something else that identifies the function) to the key args

uninstall-your-browser avatar Jul 13 '24 06:07 uninstall-your-browser

wow, thanks for posting this issue! I was on the fence about using this library... not anymore

tadamcz avatar Jul 16 '24 09:07 tadamcz

Thanks for posting this, and already diving into it. @uninstall-your-browser, would you like to create a PR for this? Life is a bit busy at the moment for me, so I don't have time to work on this package in the near future.

mariushelf avatar Jul 17 '24 11:07 mariushelf

I have published my own fork: https://pypi.org/project/shelved_cache_fixed/

uninstall-your-browser avatar Oct 19 '24 03:10 uninstall-your-browser

Looking into this again. Actually, the behavior that you describe is expected.

You are using the same cache on both functions, and the cache looks up the cached value only based on the arguments. The same happens if you use, e.g., the LRUCache from cachetools:

    import cachetools
    from cachetools import LRUCache

    pc = LRUCache(maxsize=1000)

    @cachetools.cached(pc)
    def square(x):
        return x * x

    @cachetools.cached(pc)
    def cube(x):
        return x * x * x

    assert square(2) == 4
    assert cube(2) == 8, "if this is 4, cube() uses the same cache as square()"

    > AssertionError: if this is 4, cube() uses the same cache as square()

You can see e.g., in the last example for cachetools.cached that you need to use a new instance of the cache for each method.

Sorry for getting back to you only now (life is still very busy), but I am going to close this issue.

mariushelf avatar Oct 19 '24 18:10 mariushelf

Thanks for clarifying, @mariushelf. This is indeed a (somewhat surprising) behaviour of cachetools that has nothing to do with shelved_cache.

tadamcz avatar Oct 20 '24 11:10 tadamcz