Nicolas Cadou
Nicolas Cadou
Seeing the same thing with 0.7.6, also macOS Monterey. Clearing the cache doesn't help.
Looks like an issue in PhantomJS, not scrapy-webdriver. Is there a bug report for this on Selenium's bug tracker?
Take a look at scrapy_webdriver.tests.test_request_queue. One way to do it would be to base the test on TestRequestQueue.test_priorization, have the mock raise an exception midway, and make sure subsequent requests...
If you could submit a pull request with a failing test case, that'd be awesome.
Not that I know of. To make some sense of twisted, I looked at its documentation and at scrapy source code, and googled for specific problems I encountered. This blog...
If I remember correctly, dont_filter=True comes from an earlier experiment where requests were not queued up in the spider middleware. They would be rescheduled in the scrapy queue and then...
Requests are running concurrently in scrapy in the sense that they won't block the main twisted event loop. Stock scrapy requests will therefore go through concurrently even if an unfinished...
You got that exactly right, support for multiple webdriver instances would be a new feature for scrapy-webdriver. And no worries about being stupid, you have no idea how much head-banging...
I would certainly not mind contributions. As for the low usage, this project is still very young, so I'm not surprised.
There are ways with webdriver to create tabs and windows, and switch between them, so it should be possible to implement that support in scrapy-webdriver.