influxdb
influxdb copied to clipboard
why using off-heap memory for indirect index offsets slice
Discussed in https://github.com/influxdata/influxdb/discussions/23449
Originally posted by foobar June 14, 2022 hi there, when running InfluxDB 1.8 with large dataset(10TB, 50 shards), and the series cardinality is high(10million), we found the go runtime HeapSys is small(less than 10GB) but the process RSS memory is above 200GB. This is a big gap. After some investigation we found lots of memory is allocated off-heap, which means it is not managed by Go runtime. This behavior seems by design with commit https://github.com/influxdata/influxdb/commit/d3e832b462546eae2517d8e32d4c93c0847d6354.
But I'm wondering what's the benefit of using off-heap memory and why those memory spaces are out of statistics.