graylog2-server icon indicating copy to clipboard operation
graylog2-server copied to clipboard

Set elasticsearch_index_optimization_jobs default to 10

Open todvora opened this issue 3 years ago • 1 comments

This PR sets the elasticsearch_index_optimization_jobs configuration value to 10 (originally 20), to avoid depleting all the elasticsearch/opensearch connections (elasticsearch_max_total_connections_per_route) available.

Fixes https://github.com/Graylog2/graylog2-server/issues/12025

Motivation and Context

With default 20 threads of elasticsearch_index_optimization_jobs and default 20 threads for search-server clients, it could easily happen that all client threads are blocked by the optimization and the graylog server stops indexing new messages, as observed in https://github.com/Graylog2/graylog2-server/issues/12025

When we decrease the default optimize threads to 10, there will be still 10 client threads available for ingestion and other tasks.

How Has This Been Tested?

Tested by the cloud team, see https://github.com/Graylog2/graylog2-server/issues/12025#issuecomment-1112959172

Types of changes

  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [] Refactoring (non-breaking change)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • [x] My code follows the code style of this project.
  • [ ] My change requires a change to the documentation.
  • [ ] I have updated the documentation accordingly.
  • [x] I have read the CONTRIBUTING document.
  • [ ] I have added tests to cover my changes.

todvora avatar Sep 22 '22 10:09 todvora

Good idea @boosty, done now!

todvora avatar Sep 22 '22 11:09 todvora

@todvora As the change is really small, could you backport this to 4.3?

boosty avatar Sep 23 '22 07:09 boosty