Flaky test detected.

[jzysak25]

I have detected some flaky tests using pytest plugin flake finder, the details of this plugin can be found at: https://github.com/dropbox/pytest-flakefinder Flaky test: test_transports.py::test_httptransport_timeout location:

def test_httptransport_timeout():
    start_time = time.time()
    # non-routable IP address, will result in a timeout
    http_transport = HTTPTransport('http://10.255.255.1', 'token')
    trace = trace_factory.get_or_create_trace()

    # This will make sure we get TimeoutError and not MaxRetryError
    with pytest.raises(urllib3.exceptions.TimeoutError):
        http_transport.send(trace)

    duration = time.time() - start_time

    # Making sure that an unreachable url will result in duration almost equal to the
    # timeout duration set
    assert http_transport.timeout < duration < http_transport.timeout + 0.3

Where duration exceeds the timeout. Snapshots of the error report: FAILED tests/test_transports.py::test_httptransport_timeout[0] - assert 1.6475169658660889 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[1] - assert 1.4870219230651855 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[2] - assert 1.4910881519317627 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[3] - assert 1.4848833084106445 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[4] - assert 1.482806921005249 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[5] - assert 1.4943721294403076 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[6] - assert 1.500823974609375 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[7] - assert 1.4800028800964355 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[8] - assert 1.4843499660491943 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[9] - assert 1.4824299812316895 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[10] - assert 1.4858860969543457 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[11] - assert 1.488236665725708 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[12] - assert 1.505213975906372 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[13] - assert 1.5001637935638428 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[14] - assert 1.4944648742675781 < (1.0 + 0.3) FAILED tests/test_transports.py::test_httptransport_timeout[15] - assert 1.4844911098480225 < (1.0 + 0.3) How to re-appear the problem: in the root directory, install the plugin: pip install pytest-flakefinder then run: pytest --flake-finder Conclusion: I have encountered this flaky test, I have tried to read through the code but do not understand exactly why this happened, I am posting this as an issue to see if there is a solution to improve or this is designed on purpose.