Luminarys
Switch to mio
Looks like mio finally removed its restriction on passing FDs between polls on threads, we should migrate from amy now.
I wanted to take a stab at this but it appears mio 0.7 no longer supports user space queues and the recommended approach is to use a crossbeam queue along with Wake to notify the Poll. The limitation here is one can register at most one Wake with a given Poll, hence the reason for a queue. However, I do not think this approach lends itself to a better design than the one exists in the codebase. One would have to use a single channel between CIO and the components(rpc, tracker, etc) and send a predefined ID in each message as well as queue the ID. What are your thoughts on this? Open to suggestions.
Awesome, I really would like to switch, but the issue with channels (and more minor timers) has been blocking this. I'm happy to give feedback on your plan, but it's also fine to just send a PR when you have a prototype.
I've thought about this a bit, I may attempt it myself if you're not actively working on it. Here's my idea:
- Create a new thread whose sole purpose is to globally manage channels and timers
- New thread will manage creation of all timers and channels
- Wrap all channels with an Arc<Mutex<Waker>> which wakes the new thread's Poll
- Set the new thread to loop in polling with a timeout of say 1ms. After this timeout it updates a timerwheel and notifies wakers of other threads based on channel states.
I need to mull on this some more but it's one way of approaching the issue.
Actually we can simplify the above model by just wrapping all channels with the Waker of the receiver. On sending to that channel we just directly notify the thread. The global thread then can just manage a timerwheel and dispatch timer events based on that, simplifying things greatly. We also could just not even use a global thread and use a timer wheel per thread.
At this point, I have a solution that simulates registering of multiple channels with mio via a Waker. I have a hunch that I will be able incorporate that solution into the synapse code without making major design changes. I also have a solution for a timer. I think the timer in mio-extras is more complicated than it needs to be. I will try to share this mio-specific work with you by pushing it to a repo, and will tag you on the PR to get your review as well. After that, I plan to use that work to switch synapse over to mio.
This is the library I recently built to be able to register channel and timer events with Poll. I intend to use it as a foundation while making changes to transition from amy to mio. Feedback on the library is very much appreciated.
Overall this looks good, my main point of concern is the scheduling code. I think there should be no need to spawn a thread per timer task, we should be able to use a single timerwheel here.
@Luminarys Thank you for the feedback. I created a PR that addresses the issue. Merge is pending your review :)