websocket-client icon indicating copy to clipboard operation
websocket-client copied to clipboard

Listening thausands messages/sec

Open rst630 opened this issue 4 years ago • 4 comments

I listening binance socket and 100 pairs have about 14k/msg/s. When I doing blocking operations when recieving messages (insert to db, calculating indicators etc.) After 10-20min of listening queue of socket messages have huge backlog and server can't going faster. Then socket client crashes w/o any errors.

How to deal with it? How I know how much msgs pending to proceed? I need to listen much more then 14k/s x10 more. I think better hardware does't resolve problem - because CPU is't 100% when client crashes

rst630 avatar Dec 10 '21 14:12 rst630

The best thing you can do is getting rid of blocking calls, e.g by using amphp/mysql or amphp/postgres instead of your current blocking DB client.

kelunik avatar Dec 10 '21 15:12 kelunik

Sorry, I didn't intend to close this, yet.

kelunik avatar Dec 10 '21 15:12 kelunik

does Amp\call will do the same if mysql operation in it's callback?

rst630 avatar Dec 10 '21 17:12 rst630

No, Amp\call won't magically make a blocking I/O call non-blocking.

kelunik avatar Dec 10 '21 20:12 kelunik