Zubair Mohsin

hey did you got your answer yet from anyone??

Yupp, I also ended up using the same.

@hsleewis can you share your approach of sleep function? I love queues but I can't have shell access to client's server, so sleep functionality is the only go, can you...

@hsleewis can you share your code ? gist? it would be very helpful. I am confused by following : I have helper classes like ShopifyWebhook , ShopifyProduct, ShopifyPage to make...

Is it possible that multiple instances of same Spider are using same requests??

This is how I am dispatching jobs from a console command. ```php public function handle(): int { for ($offset = 1; $offset $this->offset]); } ``` These logs are from different...

```php protected function initialRequests(): array { return ShopifyStore::query() ->offset($this->context['offset']) ->limit(50) ->get() ->map(function (ShopifyStore $shopifyStore) { $request = new Request( 'GET', "https://" . $shopifyStore->url, [$this, 'parse'] ); return $request->withMeta('store_id', $shopifyStore->id); })->toArray();...

After your comment I went ahead and checked for duplicates in the table. There were indeed some duplicates. Removed them. But problem still happening. Below is my Spider's full source...

- I am using `redis` + Laravel Horizon for queues - I can verify that in a short time ( but it can be assumed that this processor is just...

Hey @ksassnowski , you are right about the second part. In my `SocialLinksDatabaseProcessor` I am not getting duplicate items for the duplicate URLs. So your thinking about the extensions like...