Forby
Forby
Hi, reading the map chunked is quite trivial I think but transforming inlcding writing is harder. I started on this branch: https://github.com/Inconcessus/OTBM2JSON/tree/add-stream-reader. See `examples/stream` for a demo.
I made some more changes using a simple synchronous callback instead of using an event emitter. The problem I encountered is that the application uses a depth-first approach. We get...
Look for `examples/stream` for a demonstration of the transformation API. You have to define a transformation callback function that is applied for every `feature` encountered. Remember to `return` the feature...
I tested this on a 25MB OTBM file and the memory usage went from 300MB to 50MB. Let me know what your findings are!
So there's a big improvement but not as much as I would expect. The streaming function handles one `OTBM_TILE_AREA` at a time (and when it is completed, it is converted...
If memory remains a huge problem we can always write the completed features to disk and compile them afterwards using `fs.createReadStream`. As of now they are kept fully in memory...