stream-demux icon indicating copy to clipboard operation
stream-demux copied to clipboard

Add a sharding mechanism for streams in order to improve performance when there are a large number of streams

Open jondubois opened this issue 7 years ago • 0 comments

Currently, all streams listen to a single source stream and pick up data which is relevant to them.

There needs to be some kind of sharding mechanism (based on stream name) to improve performance when there are a large number of streams - That way each stream only consumes from the source shard which is relevant to them.

...Maybe in some kind of tree hierarchy - That would allow stream-demux to achieve O(log n) time complexity with respect to the number of streams.

jondubois avatar Nov 18 '18 20:11 jondubois