Wenxing Zheng comments

Results 9 comments of


                                            Wenxing Zheng

Middleware settings for scrapy-splash with scrapy-cluster, SplashRequest not work

so the scrapy-splash can't work with scrapy-cluster now?

Middleware settings for scrapy-splash with scrapy-cluster, SplashRequest not work

Thanks to @kmike Do you happen to know where the problem is?

Middleware settings for scrapy-splash with scrapy-cluster, SplashRequest not work

Thanks to @kmike After some investigation, found that python is not quite easy to support the serialization and deserialization of class instance. Therefore, I turn to another way: 1. add...

Middleware settings for scrapy-splash with scrapy-cluster, SplashRequest not work

@hustshawn the basic idea is to not use the scrapy-splash stuffs, but to make use of the functionalities of the scrapy-cluster + scrapy. The followings are mainly for PoC without...

Middleware settings for scrapy-splash with scrapy-cluster, SplashRequest not work

we need to ask @kmike whether the 'basic' solution is acceptable or not? If yes, we can start the PR work.

Middleware settings for scrapy-splash with scrapy-cluster, SplashRequest not work

I need to do both as I need to bypass the proxy to Splash too

Middleware settings for scrapy-splash with scrapy-cluster, SplashRequest not work

@FinnFrotscher check the code snippets above, hope it can help.

use origin input key by default

which version will include this change?

use origin input key by default

Thanks to @shaoyue On Tue, Nov 28, 2017 at 7:47 PM, shaoyue wrote: > almost forgot about this patch... I'll commit a update for tutorial in > this patch later...