streamlink
exceptions: PluginError gets shared between HTTPSession requests and validation schemas
Checklist
- [X] This is a plugin issue and not a different kind of issue
- [X] I have read the contribution guidelines
- [X] I have checked the list of open and recently closed plugin issues
- [X] I have checked the commit log of the master branch
Streamlink version
Latest stable release
Description
While trying to use the SSLKEYLOGFILE feature of python's SSL module, I set the value of this env variable to an invalid path string (I added quotes around the path which are not expected by the Windows API) which gets rejected by openssl/kernel layer.
This raises a specific kind of error (obtained by running the process in Pdb):
Traceback (most recent call last):
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 701, in urlopen
httplib_response = self._make_request(
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 384, in _make_request
self._validate_conn(conn)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 1013, in _validate_conn
conn.connect()
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connection.py", line 402, in connect
self.ssl_context = create_urllib3_context(
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\ssl_.py", line 350, in create_urllib3_context
context.keylog_filename = sslkeylogfile
OSError: [Errno 22] Invalid argument: '"C:\\Users\\Admin\\SSLKEYLOGFILE.txt"'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py", line 439, in send
resp = conn.urlopen(
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 757, in urlopen
retries = retries.increment(
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\retry.py", line 532, in increment
raise six.reraise(type(error), error, _stacktrace)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\packages\six.py", line 769, in reraise
raise value.with_traceback(tb)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 701, in urlopen
httplib_response = self._make_request(
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 384, in _make_request
self._validate_conn(conn)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 1013, in _validate_conn
conn.connect()
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connection.py", line 402, in connect
self.ssl_context = create_urllib3_context(
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\ssl_.py", line 350, in create_urllib3_context
context.keylog_filename = sslkeylogfile
urllib3.exceptions.ProtocolError: ('Connection aborted.', OSError(22, 'Invalid argument'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlink\plugin\api\http_session.py", line 153, in request
res = super().request(
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', OSError(22, 'Invalid argument'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlink\plugins\twitch.py", line 619, in _access_token
sig, token = self.api.access_token(is_live, channel_or_vod)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlink\plugins\twitch.py", line 424, in access_token
return self.call(query, schema=validate.Schema(
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlink\plugins\twitch.py", line 264, in call
res = self.session.http.post(
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 590, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlink\plugin\api\http_session.py", line 172, in request
raise err
streamlink.exceptions.PluginError: Unable to open URL: https://gql.twitch.tv/gql (('Connection aborted.', OSError(22, 'Invalid argument')))
However this relatively clearly communicated error is swallowed by ~the twitch plugin code~a generic streamlink exception, and is converted into a vague NoStreamsError here:
def _access_token(self, is_live, channel_or_vod):
try:
sig, token = self.api.access_token(is_live, channel_or_vod)
except (PluginError, TypeError):
raise NoStreamsError(self.url)
...which eventually results in the end of the process with only a single and misleading message like this:
error: No playable streams found on this URL: twitch.tv/insert_live_channel
While the root cause in this case is user error, I think it's worth noting that the exception catch clause is way too broad; unexpected exceptions bubbling up from the ssl/os/socket layers shouldn't be treated as "no stream available to play", they should bubble up to the top and be properly logged to the end user in some form.
My guess is that the "true" no-stream-to-play case raises a few or just one specific kind of (inner)exception; the best practice would be for only those to be caught, while everything else should be raised further up.
Debug log
Loaded config from deprecated path, see CLI docs for how to migrate: C:\Users\Admin\AppData\Roaming\streamlink\streamlinkrc
[cli][warning] Loaded config from deprecated path, see CLI docs for how to migrate: C:\Users\Admin\AppData\Roaming\streamlink\streamlinkrc
[cli][debug] OS: Windows 10
[cli][debug] Python: 3.10.8
[cli][debug] Streamlink: 5.1.2
[cli][debug] Dependencies:
[cli][debug] certifi: 2021.10.8
[cli][debug] isodate: 0.6.0
[cli][debug] lxml: 4.6.4
[cli][debug] pycountry: 20.7.3
[cli][debug] pycryptodome: 3.11.0
[cli][debug] PySocks: 1.7.1
[cli][debug] requests: 2.26.0
[cli][debug] urllib3: 1.26.7
[cli][debug] websocket-client: 1.2.1
[cli][debug] Arguments:
[cli][debug] url=twitch.tv/insert_live_channel
[cli][debug] stream=['best']
[cli][debug] --loglevel=debug
[cli][debug] --player="C:\Program Files\MPC-HC\mpc-hc64.exe"
[cli][debug] --twitch-low-latency=True
[cli][info] Found matching plugin twitch for URL twitch.tv/insert_live_channel
[plugins.twitch][debug] Getting live HLS streams for insert_live_channel
error: No playable streams found on this URL: twitch.tv/insert_live_channel
Not a plugin issue...
The reason for that behavior is the PluginError exception, which gets shared by the HTTPSession as exception class when an HTTP request fails (for whatever reason) and by the validate module, when the validation schema fails.
The NoStreamsError which gets raised instead of the PluginError is meant for cases where the validation schema doesn't match because of invalid authentication, etc. when acquiring an access token.
The shared PluginError exception class is a relict from the Livestreamer fork. Fixing this is unfortunately a bit challenging due to the HTTPSession and how it is used throughout the entire set of plugins and stream implementations. There isn't even proper HTTP error handling with specific exception classes in Streamlink. All these errors raise PluginError by default, which is bad.
https://github.com/streamlink/streamlink/blob/716e24d1098343910935e05b6744caa0ebddffd1/src/streamlink/plugin/api/http_session.py#L132-L182
Should requests ever get replaced by httpx (#4915), then the entire error handling logic will need to be rewritten.