crash when compiled with nuitka

[remdragon]

I got the following crash trying to compile my app with nuitka. I'm guessing this is something they need to fix, but I thought I would share it here in case it helps:

Traceback (most recent call last):
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\trio\_core\_run.py", line 2003, in unrolled_run
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\trio\_core\_run.py", line 1388, in spawn_impl
AttributeError: 'NoneType' object has no attribute 'f_locals'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\ItasPinClientPortal.py", line 3253, in <module>
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\trio\_core\_run.py", line 1887, in run
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\trio\_core\_run.py", line 2202, in unrolled_run
trio.TrioInternalError: internal error in Trio - please file a bug!

Python 3.8.5 Trio 0.16.0 Quart 0.12.0 Quart-Trio 0.5.1 Nuitka 0.6.9.2

[oremanj]

Nuitka apparently produces coroutine objects that don't have an associated frame. This is not in keeping with standard Python behavior, and breaks Trio's KeyboardInterrupt protection mechanism. (In general, producing a fully drop-in Python compiler is extremely challenging; there are many dark corners like this. I wasn't previously familiar with Nuitka but I took a brief look at their documentation and I would expect potentially more points of difference like this.)

If you want your Trio app to run faster, you'll probably have better luck running it under PyPy than under Nuitka.

[njsmith]

To be fair, the way trio messes with f_locals there is a particularly egregious use of some of Python's dark corners. I don't expect Nuitka will ever be able to support that, but we could potentially switch to another mechanism for the control-C handling stuff. My suggestion would be to try temporarily commenting out that code, to see how much you have to change to get trio running under Nuitka and how much speedup it gives, and whether there are any other problems you have to work around.

[remdragon]

Fortunately Ctrl+C handling isn't a huge priority for this app, so I decided to pursue this further.

I changed that line to this:

        if coro.cr_frame: coro.cr_frame.f_locals.setdefault(LOCALS_KEY_KI_PROTECTION_ENABLED, system_task)

and then I had to change _core/_ki.py line 124 to this:

                if coro.cr_frame: coro.cr_frame.f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = enabled

then _core/_ki.py line 141 to this:

                if gen.gi_frame: gen.gi_frame.f_locals[LOCALS_KEY_KI_PROTECTION_ENABLED] = enabled

At this point I'm geting this traceback. I feel I'm in over my head so I'm stopping here, but posting it in case it's useful to someone:

Traceback (most recent call last):
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\ItasPinClientPortal.py", line 3253, in <module>
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\trio\_core\_run.py", line 1897, in run
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\ItasPinClientPortal.py", line 3229, in main
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\ItasPinClientPortal.py", line 3074, in init_db
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\contextlib.py", line 171, in __aenter__
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\ItasPinClientPortal.py", line 490, in portal_adt_connect
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\ItasPinClientPortal.py", line 141, in run_sync2
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\trio\_threads.py", line 191, in to_thread_run_sync
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\trio\_sync.py", line 296, in acquire_on_behalf_of
  File "C:\cvs\itas\IT20FB~1\ITASPI~1.DIS\trio\_core\_traps.py", line 43, in cancel_shielded_checkpoint
AttributeError: 'NoneType' object has no attribute 'unwrap'

[smurfix]

Huh. That's a big "cannot ever happen" as Trio never sends None into a coroutine.

So at first glance this looks like a Nuitka bug.

[JamesOldfield]

I have found the Trio (0.19.0, on Windows) works fine in Nuitka (0.6.18) when I disable keyboard interupt handling in Trio. So at least @remdragon's issue has been fixed.

To disable keyboard interrupt handling I did two things:

• I commented out line 1460 of trio/_core/_run.py: coro.cr_frame.f_locals.setdefault(LOCALS_KEY_KI_PROTECTION_ENABLED, system_task)
• I replaced enable_ki_protection and disable_ki_protection in trio/_core/_ki.py with identity wrappers: def foo(fn): return fn

Personally, I don't need Trio's special KI handling anyway. I call trio.run in a separate thread, so it never gets KeyboardInterrupt anyway. My main thread just sits and waits for the interupt, at which point it sends a request to the Trio thread to shutdown gracefully. Really, this is the best way to handle this in a mature program anyway, and since Nuitka is often used for release-ish type programs, it probably often applies.

Feature request: Could Trio please be given a simple global way to shutdown KI handling? I think it will be straightforward to wrap the above two changes into a Nuitka plugin, which is a bit cleaner than manually patching an installed version of Trio, but it will still be fragile because it would depend on the exact source code in Trio.

[jimkring]

I'm having a very similar issue trying to build an app that uses PySide6 and Trio. I followed the instructions on Qt's site for how to create an async app with trio. The application builds and runs, OK, but when I try to run the main asynchronous entry point, I'm seeing an error that looks like Trio is trying to do something that doesn't work when built with Nuitka. I'm wondering if anyone has seen this and if there's a workaround/fix. Everything works fine running from source. Thanks!

Traceback (most recent call last):
  File "/MyProject/app.dist/trio/_core/_run.py", line 2121, in unrolled_run
  File "/MyProject/app.dist/trio/_core/_run.py", line 1514, in spawn_impl
AttributeError: 'NoneType' object has no attribute 'f_locals'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/MyProject/app.dist/trio/_core/_run.py", line 1291, in guest_tick
  File "/MyProject/app.dist/outcome/_impl.py", line 113, in send
  File "/MyProject/app.dist/trio/_core/_run.py", line 2326, in unrolled_run
trio.TrioInternalError: internal error in Trio - please file a bug!

[JamesOldfield]

@jimkring Did you try the trio plugin for Nuitka that I wrote? It comes with Nuitka so you don't need to install anything, just enable it. I haven't tried it for a while so it's possible it doesn't work with the latest version of Trio. Beware that it disables graceful handing of KeyboardInterrupt.

[jimkring]

@JamesOldfield that fixed the issue! Thank you very much.