nextcloud
illegal_argument_exception while starting a new index
Hi daita,
today I wanted to add a new instance of nextcloud (16.0.1) with Fulltextsearch (1.3.4), Fulltextsearch-Elasticsearch (1.3.3), Fulltextsearch-Files (1.3.2) on my ElasticSearch 6.2.4-Server (with ingest-attachment an readonlyrest 1.18).
Starting with ./occ fulltextsearch:test gives
.Testing your current setup: Creating mocked content provider. ok Testing mocked provider: get indexable documents. (2 items) ok Loading search platform. (Elasticsearch) ok Testing search platform. ok Locking process ok Removing test. ok Pausing 3 seconds 1 2 3 ok Initializing index mapping. fail Error detected, unlocking process ok In Connection.php line 620:
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/cloud_index] contains unrecognized parameter: [include_type_name]"}],"type":"illegal_argument_exception","reason":"request [/cloud_index] contain s unrecognized parameter: [include_type_name]"},"status":400}
fulltextsearch:test [--output [OUTPUT]] [-j|--json] [-d|--platform_delay PLATFORM_DELAY]
Starting with ./occ fulltextsearch:index (after ./occ fulltextsearch:reset) and building an elasticsearch index by hand with
curl --noproxy -XPOST "http://cloud_user:[email protected]:9200/cloud_index/type" -H "Content-Type: application/json" -d "{ "field" : "value"}"
{"_index":"cloud_index","_type":"type","_id":"O-zMrGsBbpR17RLI-dox","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
and
curl -H'Content-Type: application/json' -XPUT 'http://cloud_user:[email protected]:9200/cloud_index/_settings' -d ' { "index.mapping.total_fields.limit": 10000 }'
{"acknowledged":true}
gives the following error for every file:
┌─ Errors ──── │ Error: 112/112 │ Index: files:10780 │ Exception: Elasticsearch\Common\Exceptions\BadRequest400Exception │ Message: Rejecting mapping update to [cloud_index] as the final mapping would have more than 1 type: [standard, type] │ │ └──
│ Exception: Elasticsearch\Common\Exceptions\BadRequest400Exception │ Message: pipeline with id [attachment] does not exist
Do You have an idea, where to start debugging?
I did not want to stop/reset/restart the other indeces for tests, because the ./occ fulltextsearch:live is still running in the screen-sessions of these instances.
Thanks in advance!
here some more logs from elasticsearch server when running ./occ fulltextsearch:test
[2019-07-01T14:57:39,646][INFO ][t.b.r.a.l.AclLoggingDecorator] ALLOWED by { name: 'Accept requests from cloud on cloud_index', policy: ALLOW, rules: [groups,indices] req={ ID:2109615002-1775924130#263, TYP:IndexRequest, CGR:N/A, USR:cloud_user, BRS:false, KDX:null, ACT:indices:data/write/index, OA:1.2.3.5/32, XFF:null, DA:1.2.3.4/32, IDX:<N/A>, MET:PUT, PTH:/cloud_index/standard/test_provider%3Asimple, CNT:<OMITTED, LENGTH=217.0 B> , HDR:Accept=application/json, Authorization=<OMITTED>, Content-Length=217, Content-Type=application/json, Host=1.2.3.4:9200, HIS:[Accept requests from cloud on cloud_index-> RULES:[groups->true, indices->true], RESOLVED:[user=cloud_user;group=cloud;av_groups=cloud]] }
[2019-07-01T14:57:39,658][DEBUG][o.e.a.a.i.m.p.TransportPutMappingAction] [nextcloud] failed to put mappings on indices [[[cloud_index/KXYJLP_uQp-_VZ3a7f0fIw]]], type [standard] java.lang.IllegalArgumentException: Rejecting mapping update to [cloud_index] as the final mapping would have more than 1 type: [standard, type] at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:501) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:353) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:285) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:313) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:643) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:273) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:198) [elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:133) [elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:573) [elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:244) [elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:207) [elasticsearch-6.2.4.jar:6.2.4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
[2019-07-01T14:57:39,659][DEBUG][o.e.a.b.TransportShardBulkAction] [cloud_index][2] failed to execute bulk item (index) BulkShardRequest [[cloud_index][2]] containing [index {[cloud_index][standard][test_provider:simple], source[{"owner":"user1","users":[],"groups":[],"circles":[],"links":[],"metatags":[],"subtags":[],"tags":[],"hash":"d5d7561f875d4ca56e82d24c0fdf1a93","provider":"test_provider","source":"","title":"","parts":[],"content":""}]}] java.lang.IllegalArgumentException: Rejecting mapping update to [cloud_index] as the final mapping would have more than 1 type: [standard, type] at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:501) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:353) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:285) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:313) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:643) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:273) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:198) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:133) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:573) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:244) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:207) ~[elasticsearch-6.2.4.jar:6.2.4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
some more information - going forward:
Starting from new with ./occ fulltextsearch:reset, followed by creating a new index on a different way to aboves version:
curl --noproxy -XPUT "http://cloud_user:[email protected]:9200/cloud_index"
{"acknowledged":true,"shards_acknowledged":true,"index":"cloud_index"}
gives success on ./occ fulltextsearch:test afterwards.
The following ./occ fulltextsearch:index leads to
┌─ Errors ──── │ Error: 32/32 │ Index: files:394 │ Exception: Elasticsearch\Common\Exceptions\BadRequest400Exception │ Message: pipeline with id [attachment] does not exist │ │ └──
elasticsearch-log says:
[2019-07-02T10:05:53,371][DEBUG][o.e.a.b.TransportBulkAction] [nextcloud] failed to execute pipeline [attachment] for document [cloud_index/standard/files:394] java.lang.IllegalArgumentException: pipeline with id [attachment] does not exist at org.elasticsearch.ingest.PipelineExecutionService.getPipeline(PipelineExecutionService.java:194) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.ingest.PipelineExecutionService.access$100(PipelineExecutionService.java:42) ~[elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.ingest.PipelineExecutionService$2.doRun(PipelineExecutionService.java:94) [elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:672) [elasticsearch-6.2.4.jar:6.2.4] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.4.jar:6.2.4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
I tested with all kinds of activated/deactivated parameter in fulltextsearch-settings (sources, types).
perhaps this helps for finding a way, getting fulltextsearch working again.
daita, what version of elasticsearch would You suggest? the latest 6tys (6.8.1)?
Because I am not able to switch to elasticsearch 6.8.1, I was not able to fix the problem.
I can index all files, but, as written in my last comment, without any fulltext-index (a file "today.txt" with the content "weather" gives a hit by searching for "today" but not for "weather").
It looks like a problem with ingest-attachment for elasticsearch 6.2.4 (or 6.5.4) in combination with the current version of fulltextsearch/fulltextsearch-elasticsearch/files-fulltextsearch....
Thankful for any ideas or help!
Me too. 6.5.4 installed and currently there seems to be no newer version available for FreeBSD.
Update: I installed 6.8.0 manually with make install (there is no readonlyrest version for 6.8.1 available yet) and after fulltextsearch:reset it works.
Update: I installed 6.8.0 manually with
make install(there is no readonlyrest version for 6.8.1 available yet) and afterfulltextsearch:resetit works.
I installed it as a docker image, which is the way that messes with the system least.
As indicated here https://www.elastic.co/guide/en/elasticsearch/reference/7.17/removal-of-types.html the parameter "include_type_name" is deprecated since Elasticsearch version 7.x and was removed in version 8.x
So in fact, Nextcloud is incompatible with the current version 8.x of Elasticsearch.
What are the plans regarding this issue? Especially, since version 7.x of Elastic was release in April 2019, so the issue is known already quite a while...
