prefix-dev
Support cache strategy
Problem description
I hope to cache the ~/.cache/rattler/cache directory in GitLab CI, but the cached conda and PyPI packages in this directory are all decompressed, which causes my cache to be quite large.
Additionally, I will meet the following warning:
Therefore, I hope to provide some kind of caching strategy to control whether decompression is necessary.
I think this would make sense. For the implementation details I would like to ask @baszalmstra to give some input.
One side note from me, I liked that mamba saved the packages as real conda packages, this made it extremely easy to reshare them and use the cache as a local channel.
Curious what you think.
@zen-xu Just a note for integration with GitLab. If you are using self-hosted runners it might be nice to mount the cache on the runner. This automatically shares it between the runs on that runner. With enough time all runners have a decent cache. If your disk ever blows up, deleting the full cache is fine as it will automatically refill it.
If you are using self-hosted runners it might be nice to mount the cache on the runner.
Good idea！
In addition, is it possible to support system-level caching, such as PIXI_CACHE=/etc/cache/pixi, so that different users can incrementally download cache files to this directory, and save disk usage.
Another interesting feature would be to have an option in pixi.toml to either disable or specify the cache directory. This would allow to people using several pixi projects in various file systems to specify, for each project, a cache directory that is compatible with hard links.
Disabling cache or having the possibility to set a value relative to PIXI_PROJECT_ROOT would make it possible to build a fully self contained project directory.
