Testsuite fails under pytest==8.2.0 with `'AsyncHTTPTestCase' has no attribute 'runTest'`

[sirosen]

We currently have some test failures in basically all python versions, starting ~2 days ago. At first glance, it looks like an issue with tornado's AsyncHTTPTestCase, but tornado doesn't have a recent release.

Looking at what projects updated recently, I flagged pytest as a good candidate for investigation, and testing with pytest=8.1.2 works fine. So something related to unittest TestCases changed in 8.2.0 in a way that breaks tornado tests.

For reference, here's one of the error traces:

____________________ ERROR collecting tests/test_tornadoparser.py ____________________
.tox/py311/lib/python3.11/site-packages/tornado/testing.py:180: in __init__
    setattr(self, methodName, _TestMethodWrapper(getattr(self, methodName)))
E   AttributeError: 'AsyncHTTPTestCase' object has no attribute 'runTest'
____________________ ERROR collecting tests/test_tornadoparser.py ____________________
.tox/py311/lib/python3.11/site-packages/tornado/testing.py:180: in __init__
    setattr(self, methodName, _TestMethodWrapper(getattr(self, methodName)))
E   AttributeError: 'TestApp' object has no attribute 'runTest'
____________________ ERROR collecting tests/test_tornadoparser.py ____________________
.tox/py311/lib/python3.11/site-packages/tornado/testing.py:180: in __init__
    setattr(self, methodName, _TestMethodWrapper(getattr(self, methodName)))
E   AttributeError: 'TestValidateApp' object has no attribute 'runTest'

runTest looks like a dynamically created method in unittest, whose name gets passed on init to TestCase. I started looking at changes to pytest's unittest module ( https://github.com/pytest-dev/pytest/blame/main/src/_pytest/unittest.py ), but I'm out of time for digging into this right now.

[sirosen]

I reported this to one of the relevant upstreams (tornado) and then started looking at reporting to pytest. I see that there's a thread over there about this: https://github.com/pytest-dev/pytest/issues/12263

And that the pytest devs want to submit a patch to tornado. I haven't really followed the details on this one that well, so no opinion yet. But let's wait on this to see if there's a fix.

If tornado gets patched, it may have strange implications for our minimal version testing. We might have to pin down pytest for that as well.

[lafrech]

Thanks for investigating. I saw the failures but didn't have the time to look into it yet.

Good to know it is being taken care of upstream.