Add support to create Elasticsearch rolling indexes in visibility

[jeacott1]

I would like to be able to keep an Elasticsearch cluster performant, manageable, and affordable by having the ES visibility storage provider roll to a new indexname periodically. ie per month index names might be {namespace}-202301, {namespace}-202302, etc, or per year {namespace}-2023, {namespace}-2024. A configurable user defined pattern that allows custom field values as target indexnames would also be a nice add. There should then be a means to search across a range of indexes (with the default range being configurable, or all?)

This would allow searches to target the indexes by date range (or custom field etc) thereby being much more efficient. This would also allow automatic ES migration of older content to cheaper ES tiers, or archiving them.

Describe alternatives you've considered I am not aware of any real alternative.

[rodrigozhou]

We don't have support for this at this moment. But I wonder if you created index aliases, maybe it might work.

[jeacott1]

@rodrigozhou I'm not sure how aliases would help with creating the index names. ES will create an index based on whatever name you write to - afaik there's no option to make this dynamic with temporal right now. This would be a small change that would at least relieve the single index problem. As for reading across the multiple generated indexes, yes you could use an alias in the short term, but ideally you generate a list of index names covering the timespan you care about and query with those. Otherwise with a simple alias pattern of my-index-* you'll always be querying the ever growing list of all the indexes which will eventually break. It also means you cant move old indexes to cold nodes because they'll constantly be hit.

edit - ES does have a feature where you can write to an index with a construct like index-{now/d{YYYYMMDD} but it's not at all clear to me if it accepts real timestamps instead of now. In order for this feature to be useful and idempotent, the date needs to come from an actual document field, not just now.