examples icon indicating copy to clipboard operation
examples copied to clipboard
verified publisher icon pytorch

SSL Error When downloading dataset

Open junpuf opened this issue 2 years ago • 3 comments

Your issue may already be reported! Please search on the issue tracker before creating one.

Context

  • Pytorch version: 2.2.0
  • Operating System and version: Ubuntu 20.04

Your Environment

  • Installed using source? [yes/no]: No
  • Are you planning to deploy it using docker container? [yes/no]: No
  • Is it a CPU or GPU environment?: GPU
  • Which example are you using: gcn, gat
  • Link to code or data to repro [if any]: https://github.com/pytorch/examples/tree/main/gcn, https://github.com/pytorch/examples/tree/main/gat

Expected Behavior

Passing

Current Behavior

Failing for SSL error when downloading dataset

Possible Solution

Steps to Reproduce

  1. download this repo
  2. bash run_python_examples.sh 'gcn,gat' ... or just run
import requests
requests.get('https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz', stream=True)

Failure Logs [if any]

bash run_python_examples.sh 'gcn,gat'
using cuda
Starting gcn
Running example: gcn
Using cuda device
Downloading dataset...
Traceback (most recent call last):
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn
    conn.connect()
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 1104, in _create
    self.do_handshake()
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 1375, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='linqs-data.soe.ucsc.edu', port=443): Max retries exceeded with url: /public/lbc/cora.tgz (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/examples/gcn/main.py", line 244, in <module>
    with requests.get(cora_url, stream=True) as tgz_file:
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='linqs-data.soe.ucsc.edu', port=443): Max retries exceeded with url: /public/lbc/cora.tgz (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))
graph convolutional network failed
Finished gcn, status 0
Starting gat
Running example: gat
Using cuda device
Downloading dataset...
Traceback (most recent call last):
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn
    conn.connect()
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connection.py", line 419, in connect
    self.sock = ssl_wrap_socket(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 1104, in _create
    self.do_handshake()
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/ssl.py", line 1375, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='linqs-data.soe.ucsc.edu', port=443): Max retries exceeded with url: /public/lbc/cora.tgz (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/examples/gat/main.py", line 341, in <module>
    with requests.get(cora_url, stream=True) as tgz_file:
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/fsx/autobench/runtime/weekly_benchmarks_opt_p5_aws_py3.10_pt_2.2.0_cu12.1_2024-02-13-00-17-03/env/lib/python3.10/site-packages/requests/adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='linqs-data.soe.ucsc.edu', port=443): Max retries exceeded with url: /public/lbc/cora.tgz (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))
graph attention network failed
Finished gat, status 0
Some examples failed:

graph convolutional network failed
graph attention network failed

@ebrahimpichka for Graph Attention Network example @JoseLuisC99 for Graph Convolutional Network example

junpuf avatar Feb 15 '24 02:02 junpuf

Hi, I'm checking them.

ebrahimpichka avatar Feb 15 '24 22:02 ebrahimpichka

I can just bypass the SSL verification by passing verify=False into the get request, for both gat and gcn. But I'm not sure if it causes any security issues or not. @msaroufim

    # Load the dataset
    cora_url = 'https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz'
    path = './cora'

    if os.path.isfile(os.path.join(path, 'cora.content')) and os.path.isfile(os.path.join(path, 'cora.cites')):
        print('Dataset already downloaded...')
    else:
        print('Downloading dataset...')
        with requests.get(cora_url, stream=True, verify=False) as tgz_file:
            with tarfile.open(fileobj=tgz_file.raw, mode='r:gz') as tgz_object:
                tgz_object.extractall()

ebrahimpichka avatar Feb 15 '24 23:02 ebrahimpichka

@ebrahimpichka so because this works in CI with no issues what we can do is add an option to argparser to pass in an no-ssl flag which would disable verification. if you wanna send me a PR for that i can merge easily

msaroufim avatar Feb 19 '24 19:02 msaroufim