requests-html
requests-html copied to clipboard
response.html.render can not connect to storage.googleapis.com
So I have the following code:
session = requests_html.HTMLSession()
response: requests_html.HTMLResponse = session.get(some_url)
response.html.render()
^ | this throws an error:
Download may take a few minutes.
Traceback (most recent call last):
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\contrib\pyopenssl.py", line 488, in wrap_socket
cnx.do_handshake()
File "C:\Users\User\source\WORK\venv\lib\site-packages\OpenSSL\SSL.py", line 1934, in do_handshake
self._raise_ssl_error(self._ssl, result)
File "C:\Users\User\source\WORK\venv\lib\site-packages\OpenSSL\SSL.py", line 1671, in _raise_ssl_error
_raise_current_error()
File "C:\Users\User\source\WORK\venv\lib\site-packages\OpenSSL\_util.py", line 54, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 677, in urlopen
chunked=chunked,
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
self._validate_conn(conn)
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 976, in _validate_conn
conn.connect()
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connection.py", line 370, in connect
ssl_context=context,
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\util\ssl_.py", line 377, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\contrib\pyopenssl.py", line 494, in wrap_socket
raise ssl.SSLError("bad handshake: %r" % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "C:\Users\User\source\WORK\venv\lib\site-packages\requests_html.py", line 586, in render
self.browser = self.session.browser # Automatically create a event loop and browser
File "C:\Users\User\source\WORK\venv\lib\site-packages\requests_html.py", line 730, in browser
self._browser = self.loop.run_until_complete(super().browser)
File "C:\Program Files\Python375\lib\asyncio\base_events.py", line 579, in run_until_complete
return future.result()
File "C:\Users\User\source\WORK\venv\lib\site-packages\requests_html.py", line 714, in browser
self._browser = await pyppeteer.launch(ignoreHTTPSErrors=not(self.verify), headless=True, args=self.__browser_args)
File "C:\Users\User\source\WORK\venv\lib\site-packages\pyppeteer\launcher.py", line 305, in launch
return await Launcher(options, **kwargs).launch()
File "C:\Users\User\source\WORK\venv\lib\site-packages\pyppeteer\launcher.py", line 119, in __init__
download_chromium()
File "C:\Users\User\source\WORK\venv\lib\site-packages\pyppeteer\chromium_downloader.py", line 146, in download_chromium
extract_zip(download_zip(get_url()), DOWNLOADS_FOLDER / REVISION)
File "C:\Users\User\source\WORK\venv\lib\site-packages\pyppeteer\chromium_downloader.py", line 85, in download_zip
data = http.request('GET', url, preload_content=False)
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\request.py", line 76, in request
method, url, fields=fields, headers=headers, **urlopen_kw
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\request.py", line 97, in request_encode_url
return self.urlopen(method, url, **extra_kw)
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\poolmanager.py", line 336, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 765, in urlopen
**response_kw
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 765, in urlopen
**response_kw
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 765, in urlopen
**response_kw
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 725, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\util\retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='storage.googleapis.com', port=443): Max retries exceeded with url: /chromium-browser-snapshots/Win_x64/588429/chrome-win32.zip (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
Win 10, Py 3.7.5
I had the same problem a couple of days ago. To solve it you simply have to run the command.
pyppeteer-install
And it will install Chromium for you. Source.
I had the same problem a couple of days ago. To solve it you simply have to run the command.
pyppeteer-installAnd it will install Chromium for you. Source.
Thank you!
pyppeteer-install
is not working, same SSL issue
What worked for me is to install python certificates with sudo, for example:
sudo /Applications/Python\ 3.6/Install\ Certificates.command