axboe
Asynchronously cancelling operations from another thread.
Hello :wave:,
I wanted to request a feature (if it is not already possible with the current API) for being able to asynchronously cancel SQE's via. io_uring_prep_cancel across rings in separate threads.
The use case is for canceling timeouts that are live on a separate thread away from the main thread of my program.
The workaround I am using right now, which is a bit cumbersome, is 1) requesting a thread from the main thread of my program to submit an SQE to their ring to cancel a live timeout via. message passing, 2) having the thread await for a CQE indicating the result of the cancellation, and 3) having the thread report back to my main thread with the result of the cancellation via. message passing.
With this feature, rather than rely on message passing, I would be able to have the main thread of my program simply perform the cancellation of a timeout that is live on a ring instance in a separate thread via. its own local ring instance.
One-off hack would be terrible, so I'd think of a way to submit requests bypassing userspace sync, even if it's relatively slow. So, there are two parts to it:
- Submit requests from another thread without using the SQ.
- Somehow reap CQEs from that second thread.
Interestingly, those are same problems as for BPF requests. One way would be to use it once it gets to completion: https://lore.kernel.org/io-uring/[email protected]/
Another way is to solve 1. by an interface, e.g. via syscall io_uring_register, which would do submission as BPF does. There can be multi-SQ support, but there should be more justification for that. The completion part would be naturally solved with multi-CQ already needed for BPF.
In any case, even if prototyped, those won't get upstreamed quickly
Maybe there needs to be "One Ring" (to rule them all!!!, giggle)
- One Ring - fd - thread/process-1 (can send, receive, monitor all rings)
- io_uring - fd - thread/process-2 (can send, receive from One Ring only, if enabled)
- io_uring - fd - thread/process-3 ... ...
struct one_ring;
// parent
io_uring_queue_init_lord_of_the_rings(&one_ring); // hehe
// child
io_uring_queue_init(..., parent=secret_key); // secret key used to find/register the parent, vice versa.
One Ring & io_uring would use each others memory as its own, no copying.
