foundationdb icon indicating copy to clipboard operation
foundationdb copied to clipboard

Provide ability for Python bindings to avoid `multiprocessing`

Open paulreimer opened this issue 3 years ago • 2 comments

I am wanting to use the Python binding from a unikernel-type context -- the concept of processes doesn't apply, so the use of Python multiprocessing doesn't work here: https://github.com/apple/foundationdb/blob/main/bindings/python/fdb/impl.py#L648

threading is available, so ideally that could be used instead? Perhaps selecting multiprocessing vs threading in a user-configurable way?

paulreimer avatar Sep 18 '22 01:09 paulreimer

Looking at the Python documentation for multiprocessing.Semaphore, it seems like the implementations in multiprocessing and threading are the same except for one argument name:

A semaphore object: a close analog of threading.Semaphore.

A solitary difference from its close analog exists: its acquire method’s first argument is named block, as is consistent with Lock.acquire().

If true and if there is no other reason that using threading would cause a problem, we could probably just switch to threading instead. This should be tested that it still solves the original issue where we were unable to capture Ctrl+C in a Python REPL when blocked on a future.

sfc-gh-abeamon avatar Sep 19 '22 17:09 sfc-gh-abeamon

Yup; for my own purposes I'm using a similarly patched version (import threading as multiprocessing), and it seems to be working OK (though not exhaustively tested).

paulreimer avatar Sep 19 '22 18:09 paulreimer