test_tls_client_auth: previous item was not torn down properly

[jaraco]

When running the tests on macOS, I get an error about the previous test not being torn down properly:

――――――――――――――――――――――――――――― ERROR at setup of test_tls_client_auth[VerifyMode.CERT_NONE-True-localhost-builtin] ――――――――――――――――――――――――――――――
[gw1] darwin -- Python 3.10.4 /Users/jaraco/code/public/cherrypy/cheroot/.tox/python/bin/python

cls = <class '_pytest.runner.CallInfo'>, func = <function call_runtest_hook.<locals>.<lambda> at 0x107c87a30>, when = 'setup'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: "Callable[[], TResult]",
        when: "Literal['collect', 'setup', 'call', 'teardown']",
        reraise: Optional[
            Union[Type[BaseException], Tuple[Type[BaseException], ...]]
        ] = None,
    ) -> "CallInfo[TResult]":
        """Call func, wrapping the result in a CallInfo.
    
        :param func:
            The function to call. Called without arguments.
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: Optional[TResult] = func()

cls        = <class '_pytest.runner.CallInfo'>
duration   = 0.00021595798898488283
excinfo    = <ExceptionInfo AssertionError('previous item was not torn down properly') tblen=6>
func       = <function call_runtest_hook.<locals>.<lambda> at 0x107c87a30>
precise_start = 75296.118347958
precise_stop = 75296.118563916
reraise    = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)
result     = None
start      = 1655863389.6226811
stop       = 1655863389.6228979
when       = 'setup'

.tox/python/lib/python3.10/site-packages/_pytest/runner.py:338: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/python/lib/python3.10/site-packages/_pytest/runner.py:259: in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
        ihook      = <_HookCaller 'pytest_runtest_setup'>
        item       = <Function test_tls_client_auth[VerifyMode.CERT_NONE-True-localhost-builtin]>
        kwds       = {}
.tox/python/lib/python3.10/site-packages/pluggy/_hooks.py:265: in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
        argname    = 'item'
        args       = ()
        firstresult = False
        kwargs     = {'item': <Function test_tls_client_auth[VerifyMode.CERT_NONE-True-localhost-builtin]>}
        self       = <_HookCaller 'pytest_runtest_setup'>
.tox/python/lib/python3.10/site-packages/pluggy/_manager.py:80: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
        firstresult = False
        hook_name  = 'pytest_runtest_setup'
        kwargs     = {'item': <Function test_tls_client_auth[VerifyMode.CERT_NONE-True-localhost-builtin]>}
        methods    = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from '/Users/jaraco/code/public/cherrypy/cheroot/.tox/pyt...pper name='/dev/null' mode='r' encoding='UTF-8'>> _state='suspended' _in_suspended=False> _capture_fixture=None>>, ...]
        self       = <_pytest.config.PytestPluginManager object at 0x10313cac0>
.tox/python/lib/python3.10/site-packages/_pytest/runner.py:154: in pytest_runtest_setup
    item.session._setupstate.setup(item)
        item       = <Function test_tls_client_auth[VerifyMode.CERT_NONE-True-localhost-builtin]>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_pytest.runner.SetupState object at 0x105e2fe80>, item = <Function test_tls_client_auth[VerifyMode.CERT_NONE-True-localhost-builtin]>

    def setup(self, item: Item) -> None:
        """Setup objects along the collector chain to the item."""
        needed_collectors = item.listchain()
    
        # If a collector fails its setup, fail its entire subtree of items.
        # The setup is not retried for each item - the same exception is used.
        for col, (finalizers, exc) in self.stack.items():
>           assert col in needed_collectors, "previous item was not torn down properly"
E           AssertionError: previous item was not torn down properly

col        = <Module test_server.py>
exc        = None
finalizers = [<bound method Node.teardown of <Module test_server.py>>]
item       = <Function test_tls_client_auth[VerifyMode.CERT_NONE-True-localhost-builtin]>
needed_collectors = [<Session cheroot exitstatus=<ExitCode.OK: 0> testsfailed=0 testscollected=157>, <Package test>, <Module test_ssl.py>, <Function test_tls_client_auth[VerifyMode.CERT_NONE-True-localhost-builtin]>]
self       = <_pytest.runner.SetupState object at 0x105e2fe80>

.tox/python/lib/python3.10/site-packages/_pytest/runner.py:482: AssertionError
                                        

[jaraco]

It appears the test that's running before and leaving itself not torn down properly is test_high_number_of_file_descriptors. Tests pass if I run:

tox -- -k "not test_high_number_of_file_descriptors"

[jaraco]

Removing the forked marker also works around the issue. Skipping the test does not, however.

[jaraco]

I can replicate the issue by using this test as the sole "forked" test:

@pytest.mark.forked
def test_null():
    pass

So the bug is with forked.

[webknjaz]

So the bug is with forked.

Yes: https://github.com/pytest-dev/pytest/issues/9621 / https://github.com/pytest-dev/pytest-forked/issues/67.

[jaraco]

It doesn't looks like there's been progress on those issues, so maybe cheroot should abandon forked or just disable those tests until forked gets fixed.

[jaraco]

Unpininng pytest results in new errors.

[jaraco]

Pinning to pytest < 7.2 allows the tests to run, but only on Python 3.11 or earlier. Attempting to run pytest 7 on Python 3.12 fails with a DeprecationWarning.