membraneframework
Tracking issue: optimization
This issue is a place to discuss the ideas regarding optimizations and should track progress on that matter.
The ideas so far (after meeting from 27.03.2020):
Grouping of the outgoing buffers
Right now, even when element processes a list of buffers, the output buffers are sent immediately and that may cause the following elements to process one buffer at the time and thus sending lots of messages.
Solution
We should add time-based grouping on the output pads. 1st thing to do could be the grouping of :buffer actions coming from handling a list of buffers together with a maximum timeout that would send the grouped buffers if the list processing takes too long. This could be achieved without timers by adding a timestamp and checking it after each processed buffer.
That, however, does not help in case of sources like UDP that would emit buffers one by one, even if they are parts of one video frame sent over RTP and thus arrive almost at the same time. The additional output buffering would be required, most likely with configurable maximum delay allowed for the output buffers
Priority for buffers/events sitting in message queue
Currently, any element handles messages one-by-one. This may cause the buffers to be queued inside the process' mailbox instead of InputBuffer. Huge mailbox may cause slowdown of sending new messages and drop messages if it's full.
Solution
Every time we process messages in an element, we should read all messages that would be put inside InputBuffer effectively giving them a priority over other messages.
Sending 'pause' messages instead of demands
Instead of sending demands, a consuming element could send 'pause' messages upstream to notify the preceding element not to send further packets till a 'continue' message. That would eliminate the need for sending any flow control related messages when the stream flows smoothly, except for rare 'keep-alive' messages for detection of deadlocks or similar abnormalities. Care needs to be taken to preserve the current API.
Automating demands: #285 Linking multiple elements within one process: #293