aws-analytics
aws-analytics copied to clipboard
Add task to make old indexes read-only and merge shards
Elasticsearch can be much more performant with time based indexes if the shards are merged and the index is set to be readonly after there are no more writes going to it. This should improve query times.
- https://www.elastic.co/guide/en/elasticsearch/reference/7.17/indices-forcemerge.html
Acceptance criteria:
- [ ] Indexes that are not read-only excluding the index for the current day should be force merged on a daily basis
- [ ] Make absolutely certain the current day's index is not processed - times are in UTC