zio-http icon indicating copy to clipboard operation
zio-http copied to clipboard

OOM when receiving large streaming requests

Open davidlar opened this issue 1 year ago • 0 comments

Describe the bug We have a service that proxies large streams to different backends. It's compiled with GraalVM native-image and is running in docker with a quite small heap ~80Mb. It has worked fine for a long time but when we updated to zio-http 3.0.0 from 3.0.0-RC9 it started to get OOM instantly. I did some investigation and the problem is the unbounded queue in AsyncBody.asStream that was introduced. It completely disables back pressure for incoming streams. So if the producer is faster than the consumer, the data will be buffered in the unbounded queue. This could lead to OOM.

To Reproduce Steps to reproduce the behaviour:

  1. Create a service with request streaming enabled and limit the heap. Make processing of incoming data a bit slow
  2. Create a test case that sends a request larger than the heap of the service.
  3. Watch OOM in the service

Expected behaviour It should work like it did before (but without blocking netty, of course)

davidlar avatar Sep 26 '24 09:09 davidlar