Rodrigo Nonose
Rodrigo Nonose
@fredwu it's supervised by `WorkerSupervisor` which is a `ConsumerSupervisor`, `one_for_one` strategy. The restart is `temporary` so it'll not be restarted, but that's on `OPQ`'s side.
@robvolk it seems like it, but I'll take a closer look
@robvolk I had to increase the number of workers and depth a little bit to get similar numbers of results as before (`workers: 30`, `max_depth: 5`), but other than that...
@robvolk send me an e-mail with how you're running the crawler, maybe I can help.
@fredwu nothing Merging with master, it still breaks 3 tests. I'll take a look to fix it.
@mazz you can try using the latest version of `opq` which I addresses some similar issues, the PR was merged but I'm not sure if there's a patch in `hex`
There's a bunch of changes since my fork, it might be something new. Also, `opq` is not the latest.
I don't use the crawler in a while unfortunately.
@adamelmore we ran our crawler without issues a couple of times If you want to check the exact dependencies: https://github.com/emcasa/hammock I had to use forks and, since we're not using...
I'm also interested in solving this issue. I'll have time this following week to look into it and come up with something. @robvolk do you mind giving some direction? I'm...