TheHive-Project
Fix broken compatibility with Elasticsearch 8.x/Opensearch 2.x
Request Type
Bug
Work Environment
| Question | Answer |
|---|---|
| OS version (server) | Ubuntu |
| OS version (client) | 20.04 |
| Cortex version / git hash | 3.1.6-withdeps |
| Package Type | Docker |
| Browser type & version | N/A |
Problem Description
Hi, I identified problem with creating cortex database in Elasticsearch >=8.x and OpenSearch >= 2.x. Parameter include_type_name is removed from newest ES/OS versions (reference Moving from types to typeless APIs in Elasticsearch 7.0 for ES, and Remove mapping types #150 for OS).
Steps to Reproduce
- Setup Opensearch 2.x or ElasticSearch 8.x.
- Run clean cortex install.
- Wait for cortex to setup, and click migrate database button.
- See error message in logs.
Possible Solutions
Cortex uses elastic4play library, which uses elastic4s under the hood. Bumping elastic4s version from 7.17.2 to 8.x should enable compatibility with new Elasticsearch/Opensearch versions.
Complementary information
[error] o.e.d.DBConfiguration - ElasticSearch request failure: PUT:/cortex_6?include_type_name=false
StringEntity({"settings":{"index":{"number_of_shards":5,"number_of_replicas":1,"mapping.nested_fields.limit":100}},"mappings":{"date_detection":false,"numeric_detection":false,
...
"job":["dummy-job","report"],"sequence":["dummy-sequence"],"report":["artifact"],"audit":["dummy-audit"],"user":["dummy-user"],"dblist":["dummy-dblist"]}}}}},Some(application/json))
=> ElasticError(illegal_argument_exception,request [/cortex_6] contains unrecognized parameter: [include_type_name],None,None,None,List(ElasticError(illegal_argument_exception,request [/cortex_6] contains unrecognized parameter: [include_type_name],None,None,None,null,None,None,None,List())),None,None,None,List())
https://github.com/TheHive-Project/elastic4play/blob/86665bfe13a5cb34104482ebe49039d309f23f43/build.sbt#L46
Would be nice if this was fixed, we are moving to open search for our whole stack
Would be nice if this was fixed, we are moving to open search for our whole stack
Same thing for us, we use Opensearch in its latest versions. To waiting the resolution, we will use an additional Opensearch node with an old version to continue using Cortex.
