Support deterministic slice sizes when fetching data from backend
Currently MetricBackends will fetch the data for an entire series and range at once, which requires that an entire range fits into memory up until it reaches the configured or provided quota (default: 20M).
For each time series discovered for a filter a fetch* is performed for the entire range queried for, which has to fit into memory.
*: https://github.com/spotify/heroic/blob/master/heroic-component/src/main/java/com/spotify/heroic/metric/MetricBackend.java#L62
The implementation should perform the fetches piece-by-piece at a fixed size to make memory usage independent of the range. Short term the range could be broken down, but mid term it would be better for performance reasons if the backend itself could determine a suitable size.
⚠ This issue has been automatically marked as stale because it has not been updated for at least two months. If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open. This issue will automatically be closed in ten days if no further activity occurs. Thank you for all your contributions.
For Bigtable this was beginning to be solved by https://github.com/spotify/heroic/pull/231 which would limit the number of cells being read.