busboy icon indicating copy to clipboard operation
busboy copied to clipboard

Accumulating files in memory

Open Pooyahmti opened this issue 1 year ago • 2 comments

Running below code, with evey request, the app memory usage ( RSS ) increases until server runs out of memory. Why the buffers are not garbage collected after response is sent ?

import http from 'http'
import busboy from 'busboy'

http.createServer((req, res) => {

  if (req.method === 'POST') {
    const bb = busboy({ headers: req.headers });
    bb.on('file', (name, file, info) => {
      const chunks = [];
      file.on('data', (data) => {
        chunks.push(data);
      })
      file.on('end', () => {
        console.log(info, Buffer.concat(chunks));
      })
    })
    bb.on('close', () => {
      res.writeHead(200, { Connection: 'close' });
      res.end("Data Received")
    });
    req.pipe(bb);
  }
}).listen(3000, () => {
  console.log("Listening for requests")
})

Pooyahmti avatar Aug 28 '24 20:08 Pooyahmti

Because V8 has a lazy garbage collector.

If you set the --expose-gc CLI flag you can explicitly trigger the V8 GC with global.gc(), but that should only be for troubleshooting and not for production use. For controlling memory you're better off using the other V8 flags that dictate various maximum memory usages.

mscdex avatar Aug 28 '24 21:08 mscdex

Manually calling the garbage collector had no effect. I suspect that something holds a strong reference to buffers, preventing them from being garbage collected.

Pooyahmti avatar Aug 29 '24 05:08 Pooyahmti