RecurrentComplex icon indicating copy to clipboard operation
RecurrentComplex copied to clipboard

Structuredata file is way too big

Open DBotThePony opened this issue 7 years ago • 3 comments

This is an long term issue, but it should be resolved somewhere in future

Eventually, saving structuredata for any reason will become a bottleneck in server's work, because file will grow >50 mb, casuing huge lag (>10 seconds) when Recurrent Complex save it's data (sometimes lag even bigger than jungle maze generation). Disk is not a bottleneck - server (Linux based) runs on SSD.

Edit: Maybe store dat files per region? (32x32 chunks)

DBotThePony avatar Sep 25 '18 06:09 DBotThePony

Would second this, helping run a big server and tracked down a regular crash happening due to the server running out of memory trying to read a 161MB structuredata file. When I tried opening it in NBTExplorer it was well over 4GB uncompressed.

Zekrom64 avatar Feb 21 '19 02:02 Zekrom64

Yep, per-region (or per-structure? hrm) storing is the way to go. I will tackle this some point after the other issues are through.

Ivorforce avatar Mar 11 '19 22:03 Ivorforce

Workaround btw is deleting the structureData file. It contains stuff that's important for structure generation and can mean structures at the edges of your world are cut off, and /#whatisthis will fail on old structures, but otherwise it has no real downsides.

Ivorforce avatar Mar 19 '19 21:03 Ivorforce