pytest-dev
pdb prompt is nonresponsive in Python 3.13.0 from python-build-standalone
With Python 3.13.0 installed with uv (i.e. from python-build-standalone), the pdb prompt is completely nonresponsive when triggered either by the --pdb flag or by a breakpoint.
This can be reproduced by the following test in test_foo.py:
def test_bar():
raise ValueError
and running:
pytest test_foo.py --pdb
I tested the same code with the official source release from python.org for 3.13.0 on the same computer and the same code works just fine, so it seems it's somehow related to the build by python-build-standalone. Note that putting a breakpoint in a regular Python script and running it in the same interpreter does work, so this seems to be an issue with the combination with pytest.
OS: Arch Linux.
Pytest version
❯ pytest --version
pytest 8.3.3
`pip list` output
❯ pip list
Package Version
--------- -------
iniconfig 2.0.0
packaging 24.1
pip 24.2
pluggy 1.5.0
pytest 8.3.3
I've never used uv before, but this was a good opportunity to give it a try:
-
uv python install -
uv venv -
uv pip install -e .
All that took about a second, which is indeed pretty damn crazy :exploding_head:
And indeed I can reproduce with that. A breakpoint() in a normal Python file (outside of pytest) seems to work fine. And disabling pytest's output capturing with -s seems to fix things too!
Can this be related to the readline quirk documented @ https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html#backspace-key-doesn-t-work-in-python-repl ?
Hmm, I don't think so. I don't see such an error message, and for what it's worth, setting TERMINFO_DIRS=/etc/terminfo:/lib/terminfo:/usr/share/terminfo doesn't change anything either.
Oh, this one seems like a likely culprit, however:
Use of libedit on Linux
Python 3.10+ Linux distributions link against libedit (as opposed to readline) by default, as libedit is supported on 3.10+ outside of macOS.
Most Python builds on Linux will link against readline because readline is the dominant library on Linux.
Some functionality may behave subtly differently as a result of our choice to link libedit by default. (We choose libedit by default to avoid GPL licensing requirements of readline.)
Perhaps something similar to #1281 could be going on with libedit (or their usage of it) too?
Yep, it's that indeed. Built a vanilla CPython 3.13.0 with ./configure --with-readline=editline and it behaves in the exact same way. It would also explain why @jaraco saw this on macOS over in https://github.com/pytest-dev/pytest/issues/12497#issuecomment-2345094725 because from what I understand, CPython uses it on macOS by default.
Tried CPython v3.12.7 built the same way, that works. Will now bisect between the two...
Well, this was a bit annoying as two build fixes for readline were in between:
cpython$ git log --oneline v3.12.7..v3.13.0 -- Modules/readline.c
417dd3aca7b gh-116322: Rename PyModule_ExperimentalSetGIL to PyUnstable_Module_SetGIL (GH-118645)
c2627d6eea9 gh-116322: Add Py_mod_gil module slot (#116882)
!! 8515fd79fef gh-117845: Detect libedit hook function signature in configure (#117870)
e7e1116a781 gh-105323: Remove `WITH_APPLE_EDITLINE` to use the same declaration for all editline (gh-112513)
c2982380f82 gh-112510: Add `readline.backend` for the backend readline uses (GH-112511)
2df26d83486 gh-112105: Make completer delims work on libedit (gh-112106)
154f099e611 gh-112292 : Catch import error conditions with readline hooks (gh-112313)
!! f4cb0d27cc0 gh-109191: Fix build with newer editline (gh-110239)
501939c9c14 gh-105323: Update readline module to detect apple editline variant (gh-108665)
e7de0c5901b gh-108765: Python.h no longer includes <sys/time.h> (#108775)
b32d4cad15f gh-108444: Replace _PyLong_AsInt() with PyLong_AsInt() (#108459)
d228825e088 gh-106320: Remove _PyOS_ReadlineTState API (#107034)
c9ce983ae1a gh-106320: Remove private pylifecycle.h functions (#106400)
a9305b5e80e gh-105240: add missing function prototypes (#105241)
9ab587b7146 gh-89886: Rely on HAVE_SYS_TIME_H (#105058)
but with a bit of git cherry-pick to get them to whatever commit I was testing, I was able to figure out the culprit:
commit 01e7405da400e8997f8964d06cc414045e144681 (HEAD)
Author: Tian Gao <[email protected]>
Date: Mon Mar 25 08:18:09 2024 -0700
gh-112948: Make pdb completion similar to repl completion (#112950)
Lib/pdb.py | 43 ++++++++++++++++++++++++++++---------------
Lib/test/test_pdb.py | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
Misc/NEWS.d/next/Library/2023-12-11-00-51-51.gh-issue-112948.k-OKp5.rst | 1 +
3 files changed, 80 insertions(+), 15 deletions(-)
create mode 100644 Misc/NEWS.d/next/Library/2023-12-11-00-51-51.gh-issue-112948.k-OKp5.rst
that is:
- https://github.com/python/cpython/issues/112948
- https://github.com/python/cpython/pull/112950
which notably seems to use rlcompleter now in pdb.
Not sure where to go from here. If we want to open a CPython bug about this, we probably need a more minimal reproducer? But I have no clue whether the culprit is CPython or pytest here.
@The-Compiler wow that's some impressive investigative skills right there! I certainly wouldn't have been able to figure this all out so fast...
All that took about a second, which is indeed pretty damn crazy :exploding_head:
Yeah it's pretty sweet ^^
... disabling pytest's output capturing with
-sseems to fix things too!
@The-Compiler my hero. My sanity owes you a debt of gratitude!
Chiming in to confirm that I've been experiencing this bug as well, specifically in a uv-generated venv (but haven't checked using any other package managers):
- kernel 6.8
- python 3.13
- uv 0.4.25
@The-Compiler similar bug happen in gdb: (because of https://github.com/python/cpython/issues/112948 )
$ gdb
(gdb) st<tab>
start starti step stepi stepping stop strace
(gdb) pi
>>> [o for o in sys.modules.keys() if 'readline' in o]
[]
>>> import pdb
>>> [o for o in sys.modules.keys() if 'readline' in o]
['readline']
>>> quit
(gdb) st<tab><tab>
After import pdb in py3.13, autocomplete is broken
Someone (not me) will need to report this to gdb, and ideally cook up a reproducer and report it to CPython and/or libedit. I won't be spending any more time on this, as it doesn't affect me and we have a workaround for pytest.
so there's a bit of an annoying side-effect of this patch. at least on ubuntu 24.04 importing readline has a silent side-effect of adding the COLUMNS and LINES environment variables
for me this unfortunately breaks curses and the testsuite for my text editor
I say "silently" because it's invisible to python due to the readline C extension modifying the environment variables:
>>> import readline
>>> import os
>>> os.environ['COLUMNS']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<frozen os>", line 685, in __getitem__
KeyError: 'COLUMNS'
however it is visible in subprocesses:
$ diff -u <(env -i python3 -c 'import subprocess; subprocess.check_call(("bash", "-c", "env | sort"))') <(env -i python3 -c 'import readline, subprocess; subprocess.check_call(("bash", "-c", "env | sort"))')
--- /dev/fd/63 2025-03-30 17:32:29.681398545 -0400
+++ /dev/fd/62 2025-03-30 17:32:29.681398545 -0400
@@ -1,4 +1,6 @@
+COLUMNS=80
LC_CTYPE=C.UTF-8
+LINES=40
PWD=/tmp/z
SHLVL=1
_=/usr/bin/env
I can hack around it with an additional session fixture to undo the environment variable side-effect:
@pytest.fixture(autouse=True, scope='session')
def _pytest_readline_workaround():
# see https://github.com/pytest-dev/pytest/issues/12888
# is the workaround even needed?
assert 'readline' in sys.modules
os.environ['COLUMNS'] = os.environ['LINES'] = ''
del os.environ['COLUMNS'], os.environ['LINES']
