loopy icon indicating copy to clipboard operation
loopy copied to clipboard

`memoize_on_disk` can produce filenames that are too long

Open wence- opened this issue 2 years ago • 1 comments

Since https://github.com/inducer/loopy/pull/710 (perhaps), memoize_on_disk produces quite long filenames, for example:

.../.cache/pytools/pdict-v4-loopy-memoize-cache-buffer_array-LoopyKeyBuilder.LoopyKeyBuilder-v0-2022.1-islpy2023.1.2-cgen2020.1-8158afdb18f100fb21b6cd86c6178e3c5cd19058-v1-py3.10.12.final.0

Unfortunately, some filesystems have quite short name restrictions. e.g. we had a bug report where this failed because the cache was on an encrypted filesystem where the max filename length is 144 characters (rather than the usual ext4 limit of 255).

Would it make sense to hash the (long) prefix as well as the contents?

wence- avatar Sep 05 '23 11:09 wence-

I'd be open to moving a bunch of that stuff from the directory name to the cache key. Perhaps it would make sense to retain pdict-v4-loopy-buffer_array-3.10? Want to cook up a PR?

inducer avatar Sep 05 '23 13:09 inducer