influxdb icon indicating copy to clipboard operation
influxdb copied to clipboard

why using off-heap memory for indirect index offsets slice

Open foobar opened this issue 3 years ago • 0 comments

Discussed in https://github.com/influxdata/influxdb/discussions/23449

Originally posted by foobar June 14, 2022 hi there, when running InfluxDB 1.8 with large dataset(10TB, 50 shards), and the series cardinality is high(10million), we found the go runtime HeapSys is small(less than 10GB) but the process RSS memory is above 200GB. This is a big gap. After some investigation we found lots of memory is allocated off-heap, which means it is not managed by Go runtime. This behavior seems by design with commit https://github.com/influxdata/influxdb/commit/d3e832b462546eae2517d8e32d4c93c0847d6354.

But I'm wondering what's the benefit of using off-heap memory and why those memory spaces are out of statistics.

foobar avatar Jun 21 '22 03:06 foobar