Ashish Singh

> > Are existing ITs exercising these code paths or do we need to explicitly add them > > +1, Are we planning to run hash prefix/fixed mode in a...

> ❌ Gradle check result for [173514b](https://github.com/opensearch-project/OpenSearch/commit/173514b9ce3cb2b0947fc8cba5e20f797c335235): [FAILURE](https://build.ci.opensearch.org/job/gradle-check/45596/) > > Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure [a flaky...

> ❌ Gradle check result for [6f526e4](https://github.com/opensearch-project/OpenSearch/commit/6f526e4cd61b60d6e71acb509d382fcaaa5d2480): [FAILURE](https://build.ci.opensearch.org/job/gradle-check/45700/) > > Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure [a flaky...

> ❌ Gradle check result for [798cf4b](https://github.com/opensearch-project/OpenSearch/commit/798cf4bc57c95209e652642689c9c6bb32b17aef): [FAILURE](https://build.ci.opensearch.org/job/gradle-check/46034/) > > Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure [a flaky...

Thanks for reporting the issue @tomchlee. I have couple of questions: 1. Were you able to take a thread dump when the `snapshot_deletion` active thread count showed up as 1...

Thanks @tomchlee for the details. Are there any other log lines that you see around the issue (with the snapshot shard blobs deletion) that is happening when the threads get...

@tomchlee - For cluster which are on 2.18 or earlier versions supporting async deletion, you can disable async deletion by setting `cluster.snapshot.async-deletion.enable` to false.

> [@ashking94](https://github.com/ashking94) that is the only type of log lines i see that were snapshot related. there was nothing about what caused the failed to delete. Are there any other...

Based on the analysis of the code and diagnostic data shared, I can think of the following cases where the `snapshot_deletion` thread can wait indefinitely for future to complete. ###...

Tried validating the 1st theory through a opensearch UT and it did not work. Also tried this below snippet of code and this disproves the 1st theory. ``` CompletableFuture future...