PEP 703 -- Making the Global Interpreter Lock Optional in CPython

[colesbury]

Feature or enhancement

The steering council has accepted PEP 703. This is intended as a top-level issue to keep track of integration status.

The "up for grabs" list contains issues that no one is currently working on and are ready to be implemented. If you are interested in working on one of them, please comment on the specific issue and CC me (@colesbury) on related PRs.

### Up For Grabs (comment on specific issue to take)

### Linked issues
- [ ] https://github.com/python/cpython/issues/111506
- [ ] https://github.com/python/cpython/issues/111870
- [ ] https://github.com/python/cpython/issues/111924
- [ ] https://github.com/python/cpython/issues/111926
- [ ] https://github.com/python/cpython/issues/112075
- [ ] https://github.com/python/cpython/issues/112069
- [ ] https://github.com/python/cpython/issues/112529
- [ ] https://github.com/python/cpython/issues/112606
- [ ] https://github.com/python/cpython/issues/114203
- [ ] https://github.com/python/cpython/issues/114214
- [ ] https://github.com/python/cpython/issues/114271
- [ ] https://github.com/python/cpython/issues/115999
- [ ] https://github.com/python/cpython/issues/116024
- [ ] https://github.com/python/cpython/issues/116322
- [ ] https://github.com/python/cpython/issues/116329
- [ ] https://github.com/python/cpython/issues/116522
- [ ] https://github.com/python/cpython/issues/116664
- [ ] https://github.com/python/cpython/issues/116738
- [ ] https://github.com/python/cpython/issues/116818

Completed Issues

### Completed Issues
- [ ] https://github.com/python/cpython/issues/108223
- [ ] https://github.com/python/cpython/issues/108374
- [ ] https://github.com/python/cpython/issues/108337
- [ ] https://github.com/python/cpython/issues/108724
- [ ] https://github.com/python/cpython/issues/109549
- [ ] https://github.com/python/cpython/issues/109740
- [ ] https://github.com/python/cpython/issues/110119
- [ ] https://github.com/pypa/packaging/issues/727
- [ ] https://github.com/python/cpython/issues/111062
- [ ] https://github.com/python/cpython/issues/111569
- [ ] https://github.com/python/cpython/issues/112062
- [ ] https://github.com/python/cpython/issues/111903
- [ ] https://github.com/python/cpython/issues/111916
- [ ] https://github.com/python/cpython/issues/111928
- [ ] https://github.com/python/cpython/issues/112070
- [ ] https://github.com/python/cpython/issues/111956
- [ ] https://github.com/python/cpython/issues/111965
- [ ] https://github.com/python/cpython/issues/112213
- [ ] https://github.com/python/cpython/issues/112071
- [ ] https://github.com/python/cpython/issues/111972
- [ ] https://github.com/python/cpython/issues/112538
- [ ] https://github.com/python/cpython/issues/111962
- [ ] https://github.com/python/cpython/issues/112535
- [ ] https://github.com/python/cpython/issues/112205
- [ ] https://github.com/python/cpython/issues/111971
- [ ] https://github.com/python/cpython/issues/111650
- [ ] https://github.com/python/cpython/issues/113750
- [ ] https://github.com/python/cpython/issues/111964
- [ ] https://github.com/python/cpython/issues/113884
- [ ] https://github.com/python/cpython/issues/114569
- [ ] https://github.com/python/cpython/issues/114329
- [ ] https://github.com/python/cpython/issues/112066
- [ ] https://github.com/python/cpython/issues/110481
- [ ] https://github.com/python/cpython/issues/112532
- [ ] https://github.com/python/cpython/issues/112050
- [ ] https://github.com/python/cpython/issues/111968
- [ ] https://github.com/python/cpython/issues/115491
- [ ] https://github.com/python/cpython/issues/112175
- [ ] https://github.com/python/cpython/issues/113743
- [ ] https://github.com/python/cpython/issues/115103
- [ ] https://github.com/python/cpython/issues/112087
- [ ] https://github.com/python/cpython/issues/116616
- [ ] https://github.com/python/cpython/issues/116167
- [ ] https://github.com/python/cpython/issues/112536

### Deferred tasks
- [ ] Revisit and update [`--disable-gil`](https://docs.python.org/dev/using/configure.html#cmdoption-disable-gil) configure documentation
- [x] Reenable test_cppext on `--disable-gil` builds
- [x] Audit usage of PyObject_Malloc for non-PyObject allocations
- [ ] Consider avoiding refcounting `tp_mro` during PyType_IsSubtype

Upstream functionality from nogil-3.12

This is a list of commits from the nogil-3.12 PR plan. The crossed-out entries are commits that do not need to be upstreamed, usually because the functionality is already in the main branch.

• [ ] cefe5dfee9 configure: disallow "--with-trace-refs" for "--disable-gil" builds
• [x] dcddbe2ddb configure: add support for --with-thread-sanitizer (#112536)
• [x] f546dbf16a Enable/disable the GIL at runtime (gh-116167)
• [x] f30d8d8f50 Add pyatomic.h
• [x] ~~385eb1d99c pyport: add new macros~~
• [x] ~~de2be447b3 Make PyThreadState_GET thread-local~~
• [x] a24dc2ecc3 pystate: keep track of attached vs. detached state
• [x] 4584be5950 parking_lot: add mutexes and one-time notifications
• [x] 6845b133cc critical_section: helpers for fine-grained locking (gh-111569)
• [x] 8ed62cab6a pystate: use _PyRawMutex for internal mutexes (https://github.com/python/cpython/issues/111924)
• [x] e15443b1f2 ceval: move eval_breaker to per-thread state (#112175)
• [x] b6b12a9a94 Implement biased reference counting
• [x] b6b12a9a94 Implement BRC inter-thread queue (#110481)
• [x] ~~7b6b6f1a01 unicode: immortalize interned strings~~
• [x] ~~fc173e3711 unicode: always immortalize interned strings~~
• [x] dd9b78460c Add safe memory reclamation scheme based on FreeBSD's GUS
• [x] 901e134921 Add mimalloc v2.0.9 (DinoV)
• [x] b6980856df Additional mimalloc changes (https://github.com/python/cpython/issues/112532)
• [ ] d13c63dee9 pymem: remove uses of _PyMem_SetDefaultAllocator during finalization (https://github.com/python/cpython/issues/111924)
• [x] 654be8ffd6 gc: make the garbage collector non-generational (#112529)
• [x] 967fe31473 gc: Traverese mimalloc heaps to find all objects. (#112529)
• [x] 2864b6b36e Implement stop-the-world pauses (#111964)
• [x] 2864b6b36e gc: implement stop-the-world GC (#112529)
• [ ] c1befd7689 Stop the world before fork() and Python shutdown (#116522)
• [ ] 82800d8ec8 ceval: stop the world when enabling profiling/tracing for all threads (#116818)
• [ ] 7423dff344 pystate: use stop-the-world in a few places
• [ ] 86efa7dfe3 pystate: implement _PyRuntime.multithreaded
• [ ] d896dfc8db dict: make dict thread-safe (#112075)
• [x] df4c51f82b list: make list thread-safe (#112087)
• [x] 9c1f7ba1b4 mro: thread-safe MRO cache (#113743)
• [x] 7a7aca096b getargs.c: make parser_init thread-safe (#111956)
• [ ] 0dddcb6f9d weakref: make weakrefs thread-safe without the GIL (#111926)
• [x] 410ba1036a dtoa: make dtoa thread-safe (#111962)
• [x] 6540bf3e6a unicode: make unicodeobject.c thread-safe (#111971)
• [x] 5d006db9fa codecs.c: fix race condition (#111972)
• [x] d1b5ed128e _threadmodule: make _thread.lock thread-safe (#114271)
• [ ] cfc11bcb1a typeobject: thread safety
• [ ] cfecf6f4eb threading: remove _tstate_lock from threading.Thread (#114271)
• [x] ~~74df7785f5 pyqueue: add internal queue data structure~~
• [x] 4450445f51 pymem: add _PyMem_FreeQsbr (#115103)
• [x] 7e60a01aee queue: make SimpleQueue thread-safe (#113884)
• [x] 4ca2924f0d set: make set thread-safe (#112069)
• [x] ~~3cfbc49229 moduleobject: fix data races~~
• [ ] 5722416ef5 _threadmodule: thread-safety fixes (#114271)
• [ ] 31ec6f0290 pystate: refcount threads to handle race between interpreter shutdown and thread exit
• [x] 45bdd27ee5 threading: make _thread.lock thread-safe
• [x] 07f5f8c318 slice: move slice_cache to per-thread state (#111968)
• [ ] ea1160c6d7 asyncio: fix race conditions in enter_task and leave_task
• [x] 212fef480e object.c: fix race when accessing attributes and methods (#111789)
• [x] 70856f126d asdl: use _PyOnceFlag in Python-ast.c (#111956)
• [x] 360a79cb88 socketmodule.c: use relaxed atomics for global 'defaulttimeout' (#116616)
• [x] 041a08e339 functools: make lru_cache thread-safe (#112070)
• [x] 9bf62ffc4b random: add a mutex to guard random.Random (#112071)
• [x] a8251a8d25 clinic: support '@' syntax for recursive mutexes (#111903)
• [x] ffade9d6f6 bufferedio: add locks to make print() thread-safe (#111965)
• [x] 5b83c16dcd textio: add locks to make textio thread-safe (#111965)
• [x] 6323ca60f9 stringio: make stringio thread-safe (#111965)
• [x] f1e4742eaa deque: make most functions thread-safe (#112050)
• [ ] 78825e0508 importlib: fix data race in imports (PyImport_ImportModuleLevelObject)
• [ ] 2f5c90a284 semaphore.c: decrease count before release sem_lock
• [x] 22eca6e215 sha1: make sha1module thread-safe (https://github.com/python/cpython/issues/111916)
• [x] ada9b73feb _struct: fix race condition in cache_struct_converter (#112062)
• [ ] 0e0b3899d1 signalmodule: fix thread-safety issue on macOS (Unclear if this is still an issue on macOS)
• [x] 964bb33962 json: make JSON scanner thread safe (https://github.com/python/cpython/issues/111928)
• [ ] 86e7772c64 http: fix dependency on finalization order
• [ ] 9ab96964e7 faulthandler: don't dump all threads when running without the GIL
• [ ] cff32694a4 test_gdb: skip test_threads when running without GIL
• [ ] 2ae5ee5ed4 tests: fix and work around some race conditions in tests
• [ ] 9f9b3d085f ceval: fix some thread-safety issues
• [x] 2a4c17e896 pystate: move freelists to per-thread state (#111968)
• [ ] 149ea9dc43 Deferred reference counting
• [ ] 7e7568672d specialize: make specialization thread-safe
• [ ] 90d34f0d18 specialize: optimize for single-threaded programs
• [ ] 42d3e11d8c code: make code object use deferred reference counting
• [ ] c9fc49666c test: add support for checking for TSAN
• [x] 7507a77a98 thread: don't use sem_clockwait with TSAN
• [ ] a62d37674c _posixsubprocess: disable vfork when running with ASAN
• [ ] 398204d57b object: fix reported TSAN races
• [ ] 4526c07cae Disable the GIL by default in --disable-gil builds (gh-116329)