StackExchange.Redis icon indicating copy to clipboard operation
StackExchange.Redis copied to clipboard

Streams implementation with Rx

Open aboimpinto opened this issue 5 years ago • 4 comments

Hello, I'm trying to find a way to not use the loop to get the data from the stream and bring it to my app using reactive extensions.

I open a StackOverflow thread where I'm putting all the possible solutions

  • using Pub/Sub to notify when the client that we have something in the stream
  • using a timer that goes check every 200ms
  • using the loop

The examples are very simple but not aligned with reality. Can we find a solution?

https://stackoverflow.com/questions/62012495/how-to-implement-redis-streams-with-c-sharp-rx

Paulo Aboim Pinto

aboimpinto avatar May 27 '20 06:05 aboimpinto

the "fix" here is probably to use blocking waits, but: that is blocked behind some work we need to do to enable blocking waits; with blocking, you aren't depoendent on the pub/sub, and you don't have the latency of poll

mgravell avatar May 27 '20 08:05 mgravell

the "fix" here is probably to use blocking waits, but: that is blocked behind some work we need to do to enable blocking waits; with blocking, you aren't depoendent on the pub/sub, and you don't have the latency of poll

that is true ... but I don't think that block will solve everything because we still need to keep a loop and refresh the block after a timeout or after getting something from the stream. I think with the "block" feature we could have a less footprint in terms of memory and CPU usage, but the problem would be the same.

do you have a solution, how this could be implemented with the current code base?

Paulo Aboim Pinto

aboimpinto avatar May 27 '20 10:05 aboimpinto

Ok, so that we're on the "same page" - exactly what problem would be the same? Imagine we had blocking stream reads: what is the issue you're still trying to solve?

mgravell avatar May 27 '20 17:05 mgravell

according to the current implementation of Block, we can block for X seconds or until receives the a message in the stream ... right? After we need to refresh the block ... and make sense to have it inside a loop.

Am I seeing this wrong?

aboimpinto avatar May 27 '20 21:05 aboimpinto