GerapyPlaywright
GerapyPlaywright copied to clipboard
Gerapy
Downloader Middleware to support Playwright in Scrapy & Gerapy
这种报错会是什么原因呢... 2022-03-23 06:21:06 [scrapy.core.scraper] ERROR: Error downloading Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/twisted/internet/defer.py", line 1656, in _inlineCallbacks result = current_context.run( File "/usr/local/lib/python3.8/dist-packages/twisted/python/failure.py", line 489, in throwExceptionIntoGenerator return g.throw(self.type, self.value,...
使用Sscrapy的时候，利用 yield PlaywrightRequest(article_url, callback=self.parse_result, wait_until='domcontentloaded', headers=self.headers) 出现错误：gzip.BadGzipFile: Not a gzipped file (b'
 想要在actions那里给page对象传参数，不知道应该怎么写才好
**程序时原封不动的运行 我的scrapy版本时2.5** ` 2021-12-29 14:10:14 [scrapy.utils.log] INFO: Scrapy 2.5.0 started (bot: example) 2021-12-29 14:10:14 [scrapy.utils.log] INFO: Versions: lxml 4.6.3.0, libxml2 2.9.5, cssselect 1.1.0, parsel 1.6.0, w3lib 1.22.0, Twisted 21.2.0, Python...
Hello, I am running into an issue with 'yield' for gerapy-playwright when I need to access a login page. I try to run: yield PlaywrightRequest(login_page_url, self.parse_login, actions = self.login_action) in...
如何爬去的一个网站返回的response里面的headers包含了 content-encoding: "gzip"的话，那么就会报上述错误，虽然作者在 downloadermiddlewares.py 的代码段中去掉了这个属性： # Necessary to bypass the compression middleware # 这个地方只能去掉 headers 中的content-encoding，但是response.headers中的依然存在，所以下面应该直接改为 headers=headers, headers = response.headers headers.pop('content-encoding', None) headers.pop('Content-Encoding', None) response = HtmlResponse( page.url, status=response.status, headers=response.headers,...
 环境说明： scrapy 2.5.1 gerapy_playwright 0.2.2 Python 3.6
