API function PySignal_SetWakeupFd is not exported and unusable

[c450e467-56c6-437b-b557-3bcb2eb89127]

BPO	45201
Nosy	@vstinner

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-09-15.06:03:15.175>
labels = ['expert-C-API', 'type-bug', '3.9']
title = 'API function PySignal_SetWakeupFd is not exported and unusable'
updated_at = <Date 2021-09-29.08:18:20.534>
user = 'https://bugs.python.org/cy'

bugs.python.org fields:

activity = <Date 2021-09-29.08:18:20.534>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['C API']
creation = <Date 2021-09-15.06:03:15.175>
creator = 'cy'
dependencies = []
files = []
hgrepos = []
issue_num = 45201
keywords = []
message_count = 2.0
messages = ['401810', '402828']
nosy_count = 2.0
nosy_names = ['vstinner', 'cy']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45201'
versions = ['Python 3.9']

[c450e467-56c6-437b-b557-3bcb2eb89127]

I want to wait on curl_multi_wait which messes up python's signal handling since libcurl doesn't know to exit the polling loop on signal. I created a pipe that curl could read from when python registered a signal, and PySignal_SetWakeupFd would write to the pipe, thus letting curl leave its polling loop, letting my C module return an exception condition.

Except PySignal_SetWakeupFd cannot be used.

In Modules/signalmodule.c, it says:

    int
    PySignal_SetWakeupFd(int fd)

when it should say:

    PyAPI_FUNC(int)
    PySignal_SetWakeupFd(int fd)

This probably isn't a problem for most, since gcc has visiblity=public by default, but I guess Gentoo's process for building python sets -fvisibility=hidden, so I can't access it, and all Microsoft users should have no access to PySignal_SetWakeupFd, since Microsoft does have hidden visibility by default.

[vstinner]

See also bpo-45316: [C API] Functions not exported with PyAPI_FUNC().

[NicolasT]

Also running into this. The function is documented, but not exported by various builds of libpython (likely because of symbol-visibility hidden by default). Applying PyAPI_FUNC seems to fix this.

If the function isn't intended to be public, maybe the docs should be removed? Otherwise, the symbol should be available :smile:

Given this function's prototype has (to my knowledge) not changed since its introduction in Python 2.6, any chance this could, then, become part of the limited API as well?

[vstinner]

Also running into this.

Would you mind to elaborate your use case? Also, since the function is not usable, how do you work around this issue? Import the signal module and call the set_wakeup_fd() function?

[NicolasT]

Would you mind to elaborate your use case?

Sure! We're working on a framework combining Python and OCaml, including integration with a library called Async and its scheduler/event-loop (also integrating PEP-492-style coroutines with it). Being able to promptly react to signals delivered to such process (which are handled by the Python runtime, since that's the "main" entry-point and semantics we want to provide to users), even when the Async main-loop is running (i.e., pure OCaml code is executing) is important. I'd like to register a socketpair (or self-pipe) using PySignal_SetWakeupFd before entering the main-loop, and have this then select on the socket (among other things, of course), such that we can then use PyErr_CheckSignal whenever the socket became readable (i.e., a (C) signal handler wrote a byte on the socket) and return control to the Python runtime if appropriate.

since the function is not usable, how do you work around this issue?

I created a custom build of libpython with PyAPI_FUNC applied to the function's declaration, and this did result in the symbol being exported by the library (unlike the default).

Import the signal module and call the set_wakeup_fd() function?

We could use signal.set_wakeup_fd, though then we'll need to call into Python code etc., whilst a simple C function is a lot easier (and cheaper) to call.

If PySignal_SetWakeupFd isn't meant to be part of the (public, exported) CPython C API, then maybe it should not be documented, and shouldn't be declared through Python.h (from pyerrors.h), and we'll use signal.set_wakeup_fd by importing/calling it using the applicable C APIs. The C function seems useful though for anyone integrating with some "foreign" event loop.

[vstinner]

I created PR gh-121537 to export the function.

[vstinner]

Fixed by change https://github.com/python/cpython/commit/ca0fb3423c13822d909d75eb616ecf1965e619ae.