Yurij Mikhalevich
Yurij Mikhalevich
I agree with @hhsecond that we should auto-convert `pathlib.Path` to `storage.Path` if it starts with `lit://` 👍
Stats from a dataset of 269 photos on a much slower NAS (using the v1.2.3 and v2.0.0-pre AppImage distributions): - ingestion: 3m56.626s vs 125m0.507s (+3070%) - search: 0m18.064s vs 3m19.742s...
I'll probably be using the new model anyway for the sake of quality improvements, but it probably will make sense to keep an option to use the smaller model.
Ingestion of 3275 photos, larger model: 1682m40.463s (~28 hours)
To speed up the "querying" part I can use faiss, the effect of this switch will be more noticeable on larger datasets. But, given that model loading attributes to most...
Or consider storing the vectors in LMDB to speed up the search and reduce the RAM consumption during querying. Needs testing.
@Seon82, I like the suggestion to split the model. Thank you! About zero-copy, if I understand correctly, this means having to keep the Ray always running, which isn't much different...
Hi @ramayer! Great job on the rclip-server! :-) The name is cool, I don't mind it. I'm worried that a daemon that has a CLIP model loaded will still consume...
> relevant columns from the rclip database into memory. When dealing with millions of images, I would instead load feature vectors into memory in batches of, let's say, 100_000. This...
Hi, @Fyrd 👋 Is it possible to add `Intl` and `Intl.DisplayNames` to this database? I am using `browserslist` with `eslint` to check whether I am using unsupported API, and having...