python
Windows debug main test_os, test_venv failures
python -m test -uall test_os
test test_os failed -- Traceback (most recent call last):
File "f:\dev\3x\Lib\test\test_os.py", line 2229, in test_fdopen
self.check_bool(os.fdopen, encoding="utf-8")
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\dev\3x\Lib\test\test_os.py", line 2224, in check_bool
with self.assertRaises(RuntimeWarning):
f(fd, *args, **kwargs)
AssertionError: RuntimeWarning not raised
python -m test.test_os crash box EDIT: this is with direct unittest run
minkernal\crts\ucrt\src\appcrt\lowio\isatty.cpp
Assertion failed (fh>=0 && (unsigned)fh < (unsigned)_nhandle
python -m test test_venv
Traceback (most recent call last):
File "f:\dev\3x\Lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
~~~~~~~~^^
File "f:\dev\3x\Lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\dev\3x\Lib\test\_test_venv_multiprocessing.py", line 10, in drain_queue
if code != queue.get():
~~~~~~~~~^^
File "f:\dev\3x\Lib\multiprocessing\queues.py", line 102, in get
with self._rlock:
res = self._recv_bytes()
File "f:\dev\3x\Lib\multiprocessing\synchronize.py", line 98, in __exit__
return self._semlock.__exit__(*args)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
OSError: [WinError 6] The handle is invalid
test test_venv failed -- Traceback (most recent call last):
File "f:\dev\3x\Lib\test\test_venv.py", line 277, in test_prefixes
self.assertEqual(out.strip(), expected.encode(), prefix)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: b'f:\\dev\\3x' != b'f:\\dev\\3x\\' : base_prefix
python -m test -uall -v test_venv hangs with test_with_pip (test.test_venv.EnsurePipTest.test_with_pip)
python -m test.test_venv -vv shows traceback and then hangs
Linked PRs
- gh-116925
- gh-117065
- gh-117076
- gh-117116
test_os failure bisected to 652fbf88c4c422ff17fefd4dcb5e06b5c0e26e74 cc @serhiy-storchaka
For me both of failures reproducible on default debug build (I mean this is not free-threaded build). Can you confirm this @terryjreedy ?
test_venv failure bisected to fddc829236d7b29a522a2160e57b2d7ca23b9b95 cc @zooba
I wonder how this failures were not caught by our CI/CD?
Nice to know it is not my disk failing. With plain build (no debug, gil):
python -m test -uall test_os Same 'RuntimeWarning not raised' as above.
python -m test.test_os (fixed above) Warning box (application unable to start correctly)
followed by 2 failures: 'RuntimeWarning not raised' as with regrtest, plus 30 second timeout
File "f:\dev\3x\Lib\test\test_os.py", line 2610, in _kill_with_event
proc.wait(timeout=support.SHORT_TIMEOUT)
python -m test test_venv hangs as
test_multiprocessing_recursion (test.test_venv.BasicTest.test_multiprocessing_recursion)
python -m test.test_venv does not hang but gives a traceback similar to initial report above
Traceback (most recent call last):
File "f:\dev\3x\Lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
~~~~~~~~^^
File "f:\dev\3x\Lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\dev\3x\Lib\test\_test_venv_multiprocessing.py", line 10, in drain_queue
if code != queue.get():
~~~~~~~~~^^
File "f:\dev\3x\Lib\multiprocessing\queues.py", line 102, in get
with self._rlock:
res = self._recv_bytes()
File "f:\dev\3x\Lib\multiprocessing\synchronize.py", line 95, in __enter__
return self._semlock.__enter__()
~~~~~~~~~~~~~~~~~~~~~~~^^
File "f:\dev\3x\Lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
~~~~~~~~^^
PermissionError: [WinError 5] Access is denied
File "f:\dev\3x\Lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\dev\3x\Lib\test\_test_venv_multiprocessing.py", line 6, in fill_queue
queue.put(code)
~~~~~~~~~^^^^^^
File "f:\dev\3x\Lib\multiprocessing\queues.py", line 89, in put
if not self._sem.acquire(block, timeout):
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
PermissionError: [WinError 5] Access is denied
followed by the prefixes error in original report.
python -m test.test_os(fixed above) Warning box (application unable to start correctly)
I think this is expected. We rely on libregrtest to suppress assert dialogs from passing invalid arguments into CRT functions, and if you launch the test this way then we don't get that suppression.
OSError: [WinError 6] The handle is invalid
Perhaps something has changed with how multiprocessing creates its child processes? This would suggest that the semaphore handle is not being properly shared to the child process.
AssertionError: b'f:\dev\3x' != b'f:\dev\3x\' : base_prefix
This one just needs a more flexible comparison, perhaps via pathlib rather than strings. The exact value is likely to be influenced by PYTHONHOME or other overrides, but the test is useful for ensuring that we actually created the venv we intended to.
I dug into the test_os failure and found that the underlying file object created for os.fdopen depends on the fd's type:
https://github.com/python/cpython/blob/b8d808ddd77f84de9f93adcc2aede2879eb5241e/Modules/_io/_iomodule.c#L323-L329
On the GHA, stdin and stdout (fileno 0 and 1) are directed to files, so _io.FileIO is used. When running the test via python -m test test_os, stdin and stdout are normal, so _io.WindowsConsoleIO is used. And currently _io.WindowsConsoleIO does not raise RuntimeWarning as the test case expected if fd is a bool object.
I'm working on a PR to fix it.
The test_venv.test_prefixes test failed because python.bat set the PYTHONHOME variable like this:
@set PYTHONHOME=C:\Users\xxxxx\Source\cpython\
The Python runtime will use this as sys.base_prefix. However, it has a trailing slash, which differs from the behavior when executing Python via python.exe normally. The test_venv test will create a new python.exe and launch it normally, resulting in the test's failure.
The GitHub Actions (GHA) run the test case using python[_d].exe, so it wouldn't trigger this issue.
Since anyone can set PYTHONHOME to whatever they want, we either need to normalise it when we read it, or handle that it may not be perfectly as expected when we do the test. I'd prefer to enhance the test to deal with this situation.
Traceback (most recent call last):
File "f:\dev\3x\Lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
~~~~~~~~^^
File "f:\dev\3x\Lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:\dev\3x\Lib\test\_test_venv_multiprocessing.py", line 10, in drain_queue
if code != queue.get():
~~~~~~~~~^^
File "f:\dev\3x\Lib\multiprocessing\queues.py", line 102, in get
with self._rlock:
res = self._recv_bytes()
File "f:\dev\3x\Lib\multiprocessing\synchronize.py", line 98, in __exit__
return self._semlock.__exit__(*args)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
OSError: [WinError 6] The handle is invalid
This error is raised by test_venv.test_multiprocessing_recursion.
It's caused by python.bat setting the PYTHONHOME environment variable, which affects the sys._base_executable whenever Python is launched directly or from a venv. This leads to breaking the venv detection code here: https://github.com/python/cpython/blob/6547330f4e896c6748da23704b617e060e6cc68e/Lib/multiprocessing/popen_spawn_win32.py#L26
So, the simple way to fix this is to remove the PYTHONHOME environment variable before running this test and restore it later. If some user sets the PYTHONHOME environment variable and uses a venv, they will have the same problem. But I think this is not unexpected, as PYTHONHOME can break the venv's isolation and we have a note about this.
The three failed test cases mentioned in the original issue in test_os and test_venv have already been fixed.
Additionally, there are some warning logs like:
minkernal\crts\ucrt\src\appcrt\lowio\isatty.cpp
Assertion failed (fh>=0 && (unsigned)fh < (unsigned)_nhandle
As mentioned in this GitHub issue, this behavior is expected. Therefore, I believe we can close this issue now.
All PRs are merged or closed. Thanks all. I am getting other errors now, which if persistent, should be other issue(s).