requests-html icon indicating copy to clipboard operation
requests-html copied to clipboard

response.html.render can not connect to storage.googleapis.com

Open DobroAlex opened this issue 5 years ago • 4 comments

So I have the following code:

session = requests_html.HTMLSession()

response: requests_html.HTMLResponse = session.get(some_url)

response.html.render()

^ | this throws an error:

Download may take a few minutes.
Traceback (most recent call last):
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\contrib\pyopenssl.py", line 488, in wrap_socket
    cnx.do_handshake()
  File "C:\Users\User\source\WORK\venv\lib\site-packages\OpenSSL\SSL.py", line 1934, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "C:\Users\User\source\WORK\venv\lib\site-packages\OpenSSL\SSL.py", line 1671, in _raise_ssl_error
    _raise_current_error()
  File "C:\Users\User\source\WORK\venv\lib\site-packages\OpenSSL\_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connection.py", line 370, in connect
    ssl_context=context,
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\util\ssl_.py", line 377, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\contrib\pyopenssl.py", line 494, in wrap_socket
    raise ssl.SSLError("bad handshake: %r" % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\User\source\WORK\venv\lib\site-packages\requests_html.py", line 586, in render
    self.browser = self.session.browser  # Automatically create a event loop and browser
  File "C:\Users\User\source\WORK\venv\lib\site-packages\requests_html.py", line 730, in browser
    self._browser = self.loop.run_until_complete(super().browser)
  File "C:\Program Files\Python375\lib\asyncio\base_events.py", line 579, in run_until_complete
    return future.result()
  File "C:\Users\User\source\WORK\venv\lib\site-packages\requests_html.py", line 714, in browser
    self._browser = await pyppeteer.launch(ignoreHTTPSErrors=not(self.verify), headless=True, args=self.__browser_args)
  File "C:\Users\User\source\WORK\venv\lib\site-packages\pyppeteer\launcher.py", line 305, in launch
    return await Launcher(options, **kwargs).launch()
  File "C:\Users\User\source\WORK\venv\lib\site-packages\pyppeteer\launcher.py", line 119, in __init__
    download_chromium()
  File "C:\Users\User\source\WORK\venv\lib\site-packages\pyppeteer\chromium_downloader.py", line 146, in download_chromium
    extract_zip(download_zip(get_url()), DOWNLOADS_FOLDER / REVISION)
  File "C:\Users\User\source\WORK\venv\lib\site-packages\pyppeteer\chromium_downloader.py", line 85, in download_zip
    data = http.request('GET', url, preload_content=False)
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\request.py", line 76, in request
    method, url, fields=fields, headers=headers, **urlopen_kw
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\request.py", line 97, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\poolmanager.py", line 336, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 765, in urlopen
    **response_kw
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 765, in urlopen
    **response_kw
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 765, in urlopen
    **response_kw
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "C:\Users\User\source\WORK\venv\lib\site-packages\urllib3\util\retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='storage.googleapis.com', port=443): Max retries exceeded with url: /chromium-browser-snapshots/Win_x64/588429/chrome-win32.zip (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))

Win 10, Py 3.7.5

DobroAlex avatar Jun 16 '20 15:06 DobroAlex

I had the same problem a couple of days ago. To solve it you simply have to run the command.

pyppeteer-install

And it will install Chromium for you. Source.

gerdm avatar Jun 19 '20 14:06 gerdm

I had the same problem a couple of days ago. To solve it you simply have to run the command.

pyppeteer-install

And it will install Chromium for you. Source.

Thank you!

coding-ax avatar Aug 31 '20 04:08 coding-ax

pyppeteer-install

is not working, same SSL issue

orihomie avatar Nov 16 '20 13:11 orihomie

What worked for me is to install python certificates with sudo, for example:

sudo /Applications/Python\ 3.6/Install\ Certificates.command

orihomie avatar Nov 17 '20 18:11 orihomie