Olicorne
Olicorne
A few more changes for the crawler: - applied black to cleanup the code - switched to loguru for better logging - add an env var for configurable log level...
Ive been running the crawler for a couple of hour now and it seems to be running very smoothly apart from 504 errors from mwmbl.org that seems to struggle, I...
Btw [here](https://lmdb.readthedocs.io/en/release/#lmdb.Environment) is the most important python doc for lmdb. They state: > On 64-bit there is no penalty for making this huge (say 1TB). Must be Please try and...
> Another thought - if you're rewriting the crawler using LMDB as the index, this might be a good time to rewrite it in Rust, if you feel confident doing...
Related quote from a lwn pagr about YACY: > Finally, for common terms, the number of URL references for a given word can become unwieldy, so YaCy partitions the DHT...
Am I correct if I think that this code is only relevant if we use batch generation?
> it only works on model >4B Just in case: have you checked wether the error message `Error at layer 27: 'Qwen3DecoderLayer' object has no attribute 'set_control'` repeated over and...
Up :) anything I can do to help?
Up :) we are in july and I'm still interested
Same issue here. Why was this closed?