graylog-plugin-beats icon indicating copy to clipboard operation
graylog-plugin-beats copied to clipboard

Support back pressure protocol

Open samnf opened this issue 8 years ago • 0 comments

According to elastic, "Filebeat uses a backpressure-sensitive protocol when sending data to Logstash or Elasticsearch to account for higher volumes of data. If Logstash is busy crunching data, it lets Filebeat know to slow down its read." More info here: https://www.elastic.co/guide/en/logstash/current/persistent-queues.html#backpressure-persistent-queue

I searched through the graylog-plugin-beats code and didn't find any reference to it, and my testing indicates its not supported.

This would be useful to prevent Graylog from having its journal maxed out and having it crash if filebeats starts shipping more logs than Graylog can handle.

samnf avatar Oct 04 '17 05:10 samnf