ndjson-cli icon indicating copy to clipboard operation
ndjson-cli copied to clipboard

hitting limits with large GeoJSON file

Open pmgh1 opened this issue 7 years ago • 1 comments

I have a large hexagon grid that I thought would be a perfect candidate for topojson.

But I seem to be hitting the limits of ndjson and geo2topo.

There are a few million (3 or 4) hexagons in the GeoJSON file and the file is about 1GB.

Thanks!

Can anything be done ? Running on Windows 7 node: 10.13 npm: 5.6 ndjson-split 0.3.1 geo2topo 3.0.0

D:>ndjson-split 'd.features' < output_hexgrid_oneline.json > hexgrid.ndjson readline.js:421 string = this._line_buffer + string; ^ RangeError: Invalid string length at Interface._normalWrite (readline.js:421:32) at ReadStream.ondata (readline.js:149:10) at ReadStream.emit (events.js:182:13) at addChunk (_stream_readable.js:283:12) at readableAddChunk (_stream_readable.js:264:11) at ReadStream.Readable.push (_stream_readable.js:219:10) at lazyFs.read (internal/fs/streams.js:181:12) at FSReqWrap.wrapper [as oncomplete] (fs.js:460:17)

D:>geo2topo tracts=output_hexgrid_oneline.json > hex_topo.json buffer.js:646 return this.utf8Slice(0, this.length); ^ Error: Cannot create a string longer than 0x3fffffe7 characters at Buffer.toString (buffer.js:646:17) at JSON.parse () at ReadStream. (C:\Users\steve.o'brien\AppData\Roaming\npm\node_modules\topojson\node_modules\topojson-server\bin\geo2topo:107:46) at ReadStream.emit (events.js:187:15) at endReadableNT (_stream_readable.js:1094:12) at process._tickCallback (internal/process/next_tick.js:63:19)

pmgh1 avatar Nov 26 '18 18:11 pmgh1

I am hitting this with https://github.com/jgoodall/us-maps/blob/master/geojson/zcta5.geo.json

dfrankow avatar Apr 01 '19 15:04 dfrankow