graylog2-server icon indicating copy to clipboard operation
graylog2-server copied to clipboard

Index field type data out of sync

Open patrickmann opened this issue 3 years ago • 1 comments

Under certain conditions, the field type metadata maintained in MongoDB gets out of sync with ES. This results in duplicate key exceptions whenever we try to refresh that data.

You can get into this state just via UI operations; but you can't get out of it again without losing data by completely deleting the index; or manually modifying the MongoDB.

Possible Solution

Always delete data from index_field_types when deleting an index-set; even when the "delete index data" checkbox has been unchecked.

Steps to Reproduce (for bugs)

  1. Create index test1 with prefix test
  2. Delete test1, but uncheck the "delete data" checkbox in the dialog
  3. Create an index test2 with prefix test

Observe a duplicate key exception:

[IndexFieldTypePollerPeriodical] Uncaught exception in periodical
com.mongodb.DuplicateKeyException: Write failed with error code 11000 and error message 'E11000 duplicate key error collection

HS-780520567

patrickmann avatar Feb 23 '22 11:02 patrickmann

So is there a fix or workaround for this problem yet?

ja-wa avatar Jul 07 '22 10:07 ja-wa