python
test_ctx_mgr_rollback_if_commit_failed assumes outdated SQLite locking behavior
Bug report
Bug description:
It's possible this is user error or similar, but I'm observing the following when compiling Python3.14.
The test test_ctx_mgr_rollback_if_commit_failed in Lib/test/test_sqlite3/test_dbapi.py seems to fail with modern SQLite versions because it assumes that a read-only transaction plus a user-defined function will block concurrent writers. The current test encodes a stronger assumption than SQLite provides, making it brittle across SQLite versions.
With recent SQLite releases (e.g. 3.47+), this assumption seems to be no longer valid, causing the parent process to successfully write to the database and the child process to fail with: sqlite3.OperationalError: user-defined function raised exception
This results in a false-negative test failure. The test contains the following assertion in the child process:
assert "database is locked" in input()
However, modern SQLite allows the parent to acquire a write lock in this scenario, so the parent correctly sends "no error" instead.
This causes the assertion to fail even though SQLite and the context manager are behaving correctly.
The proposed fix is relax the assertion to accept both valid outcomes:
line = input()
assert "database is locked" in line or "no error" in line
This preserves the original intent of verifying correct rollback behavior while remaining compatible with modern SQLite locking semantics.
Environment • SQLite: 3.47.0 • Python: 3.14-dev / 3.12+ • Platform: Amz Linux 2023 (aarch64, but not architecture-specific) • Compiler: Clang20
CPython versions tested on:
3.14
Operating systems tested on:
Linux
Linked PRs
- gh-143373
Can you show the exact traceback please?
Sorry thought I had included:
======================================================================
FAIL: test_ctx_mgr_rollback_if_commit_failed (test.test_sqlite3.test_dbapi.MultiprocessTests.test_ctx_mgr_rollback_if_commit_failed)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/eugo/cpython/Lib/test/test_sqlite3/test_dbapi.py", line 1972, in test_ctx_mgr_rollback_if_commit_failed
self.assertEqual(proc.returncode, 0)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0
----------------------------------------------------------------------
Ran 493 tests in 0.643s
FAILED (failures=1, skipped=6)
test test_sqlite3 failed
0:01:42 load avg: 10.43 [36/43] test_sqlite3 failed (1 failure)
For more context, this is the only test that fails in the suite.
I will try to have a look but I will need to check which sqlite version we are expecting (if the version is too recent we may have missed some guards)
I will try to have a look but I will need to check which sqlite version we are expecting (if the version is too recent we may have missed some guards)
thanks.
Ok, so after a little investigation:
- We assume that SQLite version is at least 3.15.2.
- The test passes on my machine which uses sqlite3 v3.44.0
- So something changed between 3.44 and 3.47 but unfortunately, I'll need to dig into the sources to find what changed (I'm going to do it). I want to guard the assertion checks, or possibly document it.
I have a more important question: is is a flaky test? or is it consistently failing?
I have a more important question: is is a flaky test? or is it consistently failing?
Consistently failing in the ~12 builds I've run.
LMK how else I can help.
Ok, looking at the commit history between 3.44 and 3.47, there may be some changes related to locks. I won't ask you to compile sqlite and then check for the regression commit if any because it's as tiring for you or me.
The original issue was reported in https://github.com/python/cpython/issues/71521 and this was a regression test for this one.
The way we need to fix this is:
- Make sure that we rollback when the commit fails.
- So maybe we should think of a better test.
I think the alternative is to use one read-only operation and one write-only. Can you try the following patch:
diff --git a/Lib/test/test_sqlite3/test_dbapi.py b/Lib/test/test_sqlite3/test_dbapi.py
index 20e39f61e4d..42cdb96dbac 100644
--- a/Lib/test/test_sqlite3/test_dbapi.py
+++ b/Lib/test/test_sqlite3/test_dbapi.py
@@ -1916,7 +1916,7 @@ def wait():
cx.executescript('''
-- start a transaction and wait for parent
begin transaction;
- select * from t;
+ insert into t values("ok");
select wait();
rollback;