Daniel Holth

Finally an interesting graph after configuring the benchmark correctly. This test downloads and extracts 30 `.conda` packages. On the `main` branch, it takes almost 12 seconds with latency=0.25 (30 *...

https://asciinema.org/a/7hf1BkGd8ZdUsXW8qx85Wl4Zt

As for the link/unlink transaction, there are a few more parameters like this one, or `default_threads`; we should experiment again, it would not be surprising if 2 or 3 shows...

The other day I tried orjson for something repodata related and stdlib json worked better. For one you have to feed it a whole string. msgspec looks promising https://github.com/jcrist/msgspec which...

``` In [10]: with timeme(): ...: json.load(open('repodata.json')) Took 2.2460153102874756s In [11]: with timeme(): ...: ujson.load(open('repodata.json')) Took 3.087876081466675s ``` Python 3.10

@jaimergp I am interested in all kinds of conda performance, but I haven't figured out the role SubdirData plays when new solvers are in use.

Can we do all network access in conda and tell libmamba "here is the cached json"?

https://gist.github.com/dholth/aab4f8cff8ef89806eec701da37c8727 is based on what we do for https://metayaml-conda.fly.dev/ ``` % python jamie_speed.py Have repodata.json libmamba: 1618.15 ms conda+json: 4572.29 ms conda+orjson: 4457.42 ms conda+msgspec: 4348.76 ms sqlite: 1554.47 ms...

The other point would be to make it more difficult to unintentionally use more toolz. Did you say we already have a bug for removing toolz altogether? IMO we should...

This is a good idea but we're preparing to replace the cache with a per-subdir sqlite3 database. In that branch we try to extract as few files from info/ as...