Startup time is over 15s on some systems
We run a simple service that uses this package on Kubernetes, and startup time is over 15 seconds. Memory usage explodes to over 2G during this time, and then over a minute or so normalizes at ~500MB. The service is running on a high frequency c2-standard-4 (Google Cloud) and is allowed 1.5CPUs and 2.5GB of RAM.
Since the .csv is embedded anyways, would it be feasible to instead embed a pre-parsed binary file that can much more efficiently be loaded into memory? I have previously used a similar strategy in another language and it cut down the loading time from a few seconds to a few hundred milliseconds.
Hi,
Thats indeed the current expected behavior and is considered normal. Using a binary could help, but would require a process to generate it when a new version of the source files are released. We're open to PRs for this request, if you want to contribute.