coveralls publish fails on CI

[jku]

This happens on every windows builds but does not show up as overall failure since publish has been flaky.

Submitting coverage to coveralls.io...
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coveralls\reporter.py", line 122, in report
    from coverage.report import Reporter  # pylint: disable=import-outside-toplevel
ImportError: cannot import name 'Reporter' from 'coverage.report' (C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coverage\report.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\runpy.py", line [19](https://github.com/theupdateframework/python-tuf/actions/runs/3821917724/jobs/6501541997#step:6:20)3, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts\coveralls.exe\__main__.py", line 7, in <module>
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coveralls\cli.py", line 95, in main
    result = coverallz.wear()
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coveralls\api.py", line 254, in wear
    json_string = self.create_report()
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coveralls\api.py", line 332, in create_report
    data = self.create_data()
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coveralls\api.py", line 386, in create_data
    self._data = {'source_files': self.get_coverage()}
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coveralls\api.py", line 407, in get_coverage
    src_dir).coverage
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coveralls\reporter.py", line [23](https://github.com/theupdateframework/python-tuf/actions/runs/3821917724/jobs/6501541997#step:6:24), in __init__
    self.report(cov, conf)
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coveralls\reporter.py", line 1[25](https://github.com/theupdateframework/python-tuf/actions/runs/3821917724/jobs/6501541997#step:6:26), in report
    return self.report5(cov)
  File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coveralls\reporter.py", line 64, in report5
    from coverage.files import FnmatchMatcher, prep_patterns  # pylint: disable=import-outside-toplevel
ImportError: cannot import name 'FnmatchMatcher' from 'coverage.files' (C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\site-packages\coverage\files.py)
Error: Process completed with exit code 1.

[jku]

Current error is something completely different

Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.10.9\x64\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\hostedtoolcache\windows\Python\3.10.9\x64\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\hostedtoolcache\windows\Python\3.10.9\x64\Scripts\coveralls.exe\__main__.py", line 7, in <module>
  File "C:\hostedtoolcache\windows\Python\3.10.9\x64\lib\site-packages\coveralls\cli.py", line 95, in main
    result = coverallz.wear()
  File "C:\hostedtoolcache\windows\Python\3.10.9\x64\lib\site-packages\coveralls\api.py", line 254, in wear
    json_string = self.create_report()
  File "C:\hostedtoolcache\windows\Python\3.10.9\x64\lib\site-packages\coveralls\api.py", line 332, in create_report
    data = self.create_data()
  File "C:\hostedtoolcache\windows\Python\3.10.9\x64\lib\site-packages\coveralls\api.py", line 386, in create_data
    self._data = {'source_files': self.get_coverage()}
  File "C:\hostedtoolcache\windows\Python\3.10.9\x64\lib\site-packages\coveralls\api.py", line 406, in get_coverage
    return CoverallReporter(workman, workman.config, base_dir,
  File "C:\hostedtoolcache\windows\Python\3.10.9\x64\lib\site-packages\coveralls\reporter.py", line 23, in __init__
    self.report(cov, conf)
  File "C:\hostedtoolcache\windows\Python\3.10.9\x64\lib\site-packages\coveralls\reporter.py", line 123, in report
    self.reporter = Reporter(cov, conf)
TypeError: Reporter() takes no arguments

It's as if windows was using a newer incompatible coverage version (that has a Reporter() class)... but according to the install logs, the versions are the same?

[shabeebk]

@jku Looking back to the code from coveralls and its dependency coverage, it looks like broken compatibility.

1. The first issue you reported popped up when the FnmatchMatcher was replaced with GlobMatcher with this commit at coverage.
2. In the case of the second one, it is a bit different. The coveralls was expecting to fail a try block and fall back to hacky implementation with this block. However, when the coverage introduced the Reporter class with this commit, this also started failing because the import was a success and the Reporter class is available, which also expects no arguments.

I have tried a couple of trials to see how is it passing in env other than windows, but I couldn't prove it. However, I have confirmed that both windows and other os pass the coveralls publish if we fall back to coverage version 6.5.0 in our requirements file.

[jku]

That looks like a good explanation...

I'm unsure what to suggest as a fix though:

• coveralls-python has issues filed that look like the same thing already, but it's not a project that sees a lot of development
• I'm not keen on pinning coverage to an old version (currently just half a year old but considering point 1 above... that could turn to years quite quickly)

A possible fix might be to sidestep coveralls-python altogether and use https://github.com/coverallsapp/github-action for reporting instead...

[shabeebk]

Yes, I agree with @jku considering that the action is hosted and maintained by coveralls team itself.

[lukpueh]

A possible fix might be to sidestep coveralls-python altogether and use https://github.com/coverallsapp/github-action for reporting instead...

FYI: We opted for the cli (~2y ago) due to security concerns (https://github.com/theupdateframework/python-tuf/issues/1246), and because the action did not work well for Python sources (see https://github.com/coverallsapp/github-action/issues/4).

I'm pretty sure the security concerns are no longer relevant, and the Action works better. Might still be worth reading up on the two linked issues.