Bryan Burkholder

Add disaster recovery option for Zookeeper cluster loss

1

### Description As we already have read and write access from the manager to S3, we should be able to have automatic backups of the non-ephemeral ZK data to S3....

Split admin ports from application ports

1

### Description We should look into splitting the admin ports (ie, metrics scraping) from the application ports (ie query). This allows for easier access control with something like Nebula, and...

### Describe the bug The indexer currently has logic that prevents running when the Kafka offset is rolled back. While this seemed like a good idea at the time, in...

Dataset rate limit of 0 doesn't work as expected

2

### Describe the bug Setting a rate limit of 0 appears to disable a dataset instead of still consuming and discarding everything.

Add redaction to protobuf definitions to prevent PII logging

2

### Description Depends on https://github.com/protocolbuffers/protobuf/issues/13576 This is to help prevent any PII from escaping to the logs with someone logging a protobuf object. As we cannot override the `toString()`, something...

Add shard request caching

1

### Description For requests that come through to a given shard that are identical to previous queries (think long-range queries, dashboarding, etc) these should use a request cache and not...

Cache node should gracefully handle over-provisioned LRU cache configs

2

### Description By default Lucene allocates 32MB of ram for a LRU cache for each instance. When allocating very large amounts of cache slots (ie, 200, 500) this can represent...

Snapshot delete failing to delete snapshots for very large buckets

2

### Describe the bug For what appears to be very large buckets, we have ceased to correctly delete snapshots ``` {"@timestamp":"2024-04-08T20:16:29.384Z","log.level":"INFO","log.message":"Completed snapshot deletion - successfully deleted 35 snapshots, failed to...

### Describe the bug Chunks should rollover once reaching disk size, or at a fixed amount of time since creation. This will prevent data loss when a low volume topic...

Add persistent ephemeral node support to metadata store

1

### Summary This adds an optional **opt-in** mode to run the `KaldbMetadataStore` using a combination of `PersistentNode` and `PersistentWatcher` for ephemeral nodes. This system property flag will be removed once...