Rob Volk

We're seeing the same issue and we're going to have to pull Redmon out because of this. Redmon is consuming 90% of the memory on my Redis server and we...

Here's the code I used and I'll try to open a PR at some point: ```ex uri = "#{Elastix.config(:uri)}/index_name/type/#{id}/_update" body = %{ script: %{ inline: "ctx._source.tags.add(params.tag)", lang: "painless", params: %{...

thanks so much @rhnonose! I ran my crawler with your fork and I don't see it actually crawling pages. Is it working for you locally?

When I run the crawler with your change, it starts crawling, but then stops pretty quickly. I can't figure out why. I feel like it's now actually calling the parser,...

@rhnonose 's changes work! Thanks so much for fixing the crawler!! Now I'm able to crawl a site deep and my server doesn't run out of memory. Had to use...

In addition to the latest crawler at master, I’m using latest OPQ On Wed, Aug 15, 2018 at 7:33 PM Michael wrote: > @rhnonose ok I'll just try > >...

Here's my `mix.exs` @mazz. I'm not sure if I need to reference @rhnonose's `OPQ` directly anymore, but I did need it back when he created his branch for `crawler`. ```elixir...

I did some more digging and if I use this branch, then it does indeed fix the memory issue. It introduces a new issue though - URLs will be crawled...

Thanks for the reply. If @rhnonose 's changes were merged in, why am I seeing different behavior when I reference his fork and yours? When I reference your main branch,...

With your new changes have you noticed pages are scraped multiple times? On Mon, Oct 1, 2018 at 12:29 PM Rodrigo Nonose wrote: > There's a bunch of changes since...