ipython
pause receiving while submitting tasks
results coming in during submission causes thread contention while submitting tasks, which can thrash performance with context switches
This pauses receiving messages while we are preparing tasks to be submitted, and resumes when they are done submitting. The result is the client focusing on task submission, delaying handling of results until it is done sending.
Need to do some testing, but prior to this PR, I could create a task pipeline that looks like this:
where a 'bubble' was created as the main task-creation thread paused while receiving the first bunch of tasks before submitting the next batch. This is hard to reproduce because it's thread contention with async network events, but I can at least compare overall execution time.
whereas a similar computation with this PR is denser:
I need to do some more tests and visualization to make better comparisons.
Also working on adding a notebook to the docs for visualizing an asyncresult's timestamps for profiling, etc. which is how I created these graphs.
Run again with 480 engines, 20 tasks/engine (top is main, 3093386f , bottom is this pr, a98a4fa). Workload is load-balanced submission of random 0-1s tasks (same seed), 20 tasks/engine for a total of 9600 tasks.
Can see that while the client is working to produce the tasks, there is still contention between serializing in the main thread and actually sending in the io thread until the main thread is done (purple line). This completes 1s faster in this PR (7.6s vs 8.4s). The first result doesn't arrive for 2 more seconds, which is really around when the last real send completes and receives start being processed.
The bubble can be seen around 11s in main, which is where sends and receives are both being processed, and this is gone after this PR.
The failure on Windows appears to be a reliably reproducible one, which makes me think I've introduced some possible inconsistent state when pausing/unpausing handlers (windows CI is typically much slower than others, so it tends to have different behavior when there are race conditions).