Running into `RemoteDisconnected` error when running explainaboard with multiple metrics/some metrics.

[iglee]

Hello, I ran into RemoteDisconnected error when I was running multiple metrics at once to generate reports. So, I tried running them individually, and found that some metrics worked and ran into the same issues for some other metrics.

Example traceback of the errors:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/Users/iglee/opt/anaconda3/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Users/iglee/opt/anaconda3/lib/python3.9/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Users/iglee/opt/anaconda3/lib/python3.9/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/Users/iglee/opt/anaconda3/lib/python3.9/http/client.py", line 1377, in getresponse
    response.begin()
  File "/Users/iglee/opt/anaconda3/lib/python3.9/http/client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "/Users/iglee/opt/anaconda3/lib/python3.9/http/client.py", line 289, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/iglee/opt/anaconda3/lib/python3.9/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/Users/iglee/opt/anaconda3/lib/python3.9/site-packages/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/Users/iglee/opt/anaconda3/lib/python3.9/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/iglee/opt/anaconda3/lib/python3.9/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/Users/iglee/opt/anaconda3/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Users/iglee/opt/anaconda3/lib/python3.9/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Users/iglee/opt/anaconda3/lib/python3.9/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/Users/iglee/opt/anaconda3/lib/python3.9/http/client.py", line 1377, in getresponse
    response.begin()
  File "/Users/iglee/opt/anaconda3/lib/python3.9/http/client.py", line 320, in begin
    version, status, reason = self._read_status()
  File "/Users/iglee/opt/anaconda3/lib/python3.9/http/client.py", line 289, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Failed command I tried: (multiple metrics, bart_score_mt, comet) explainaboard --task machine-translation --custom_dataset_paths data/wmt21/formatted/data/challengeset.de-en.system1.data --system_outputs data/wmt21/formatted/sysout/challengeset.de-en.system1.sysout --metrics comet explainaboard --task machine-translation --system_outputs data/wmt21/formatted/challengeset.de-en.system2.tsv --metrics chrf bleu comet

Worked: (chrf, mover_score, rouge2) explainaboard --task machine-translation --custom_dataset_paths data/wmt21/formatted/data/florestest2021.bn-hi.GTCOM.data --system_outputs data/wmt21/formatted/sysout/florestest2021.bn-hi.GTCOM.sysout --metrics chrf

[iglee]

oh, actually, bart_score_mt was a different error. just checked the log again, and it was ValueError: Invalid metric specification.

I tried it because I saw it here: https://github.com/neulab/ExplainaBoard/blob/5055013af62dd9c50c46e389ed3c678031a3a319/explainaboard/tasks.py but there might be a mismatch in names or maybe I used it wrong?