simdjson_nodejs Memory Leak

When running the benchmarks locally, the process gradually consumes all memory on my machine until it runs out of memory (~28GB)

Commenting out simdjson.lazyParse in the benchmark removes leak behavior, so I don't think its the test setup. Printing the node heap statistics shows that it is stable ~80 mb, so the leak is likely in the c++ code.

Sep 05 '20 03:09 ChrisSmith

isValid and parse don't seem affected, so its specific to lazyParse

Sep 05 '20 03:09 ChrisSmith

Hi @ChrisSmith, thanks for your input. I will investigate that and get back to you, cheers!

Sep 05 '20 04:09 luizperes

Hi @ChrisSmith, I took some time to investigate that today and in my machine (2018 Macbook i7 with 16GB RAM) it doesn't leak memory as we would expect, but had a peak of around 5GB but soon released it and had an average usage of 2GB. This on 7 threads as I didn't find a way to control the number of threads from within benchmark lib.

We also have just updated the library. Now it's currently on version 0.5.0, so it could've been a bug on the upstream.

Please also note that simdjson.lazyParse and simdjson.isValid should do basically the same work, although we keep a pointer to the parsed object here. But that is always released here once the record (pointer owner) is removed from the stack.

I will need more info from you, if it still happens. Thanks!

Sep 07 '20 07:09 luizperes

I can also confirm the memory leak.

Jul 30 '22 14:07 Uzlopak