kibble-1 icon indicating copy to clipboard operation
kibble-1 copied to clipboard
verified publisher icon apache

Problems installing Kibble using Docker compose

Open sharanf opened this issue 5 years ago • 10 comments

Description:

I am having problems installing Kibble using docker-compose -f docker-compose-dev.yaml up setup

Initially reported as part of #50 and agreed to continue looking to resolve afterwards.

As requested I have now run docker-compose -f docker-compose-dev.yaml up elasticsearch and it is exiting with a code. I have copied the complete log in the section below.

Reproduction steps:

  1. Change into to Kibble code directory (cd /opt/.../kibble
  2. Run docker-compose -f docker-compose-dev.yaml up elasticsearch
  3. Exits with message kibble_elasticsearch_1 exited with code 78

Actual result: Command runs but then exists with message kibble_elasticsearch_1 exited with code 78

OS: Ubuntu 18.04.5 LTS

Logs: Log details copied below:

$ docker-compose -f docker-compose-dev.yaml up elasticsearch
Starting kibble_elasticsearch_1 ... 
Starting kibble_elasticsearch_1 ... done
Attaching to kibble_elasticsearch_1
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:02,320Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "version[7.9.2], pid[6], build[default/docker/d34da0ea4a966c4e49417f2da2f244e3e97b4e6e/2020-09-23T00:45:33.626720Z], OS[Linux/5.4.0-48-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/15/15+36]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:02,323Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "JVM home [/usr/share/elasticsearch/jdk]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:02,323Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-15979048660884328998, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Des.cgroups.hierarchy.override=/, -Xms256m, -Xmx256m, -XX:MaxDirectMemorySize=134217728, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,941Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [aggs-matrix-stats]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,941Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [analysis-common]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,941Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [constant-keyword]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,941Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [flattened]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,942Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [frozen-indices]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,942Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [ingest-common]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,942Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [ingest-geoip]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,942Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [ingest-user-agent]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,942Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [kibana]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,943Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [lang-expression]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,943Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [lang-mustache]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,943Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [lang-painless]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,943Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [mapper-extras]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,943Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [parent-join]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,943Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [percolator]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,944Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [rank-eval]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,944Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [reindex]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,944Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [repository-url]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,944Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [search-business-rules]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,944Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [searchable-snapshots]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,945Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [spatial]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,945Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [tasks]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,945Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [transform]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,945Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [transport-netty4]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,945Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [vectors]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,946Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [wildcard]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,946Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-analytics]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,946Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-async]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,946Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-async-search]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,946Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-autoscaling]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,946Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-ccr]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,946Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-core]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,947Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-data-streams]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,947Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-deprecation]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,947Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-enrich]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,947Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-eql]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,947Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-graph]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,947Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-identity-provider]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,947Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-ilm]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,947Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-logstash]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,948Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-ml]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,948Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-monitoring]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,948Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-ql]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,948Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-rollup]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,948Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-security]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,948Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-sql]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,948Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-stack]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,949Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-voting-only-node]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,949Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "loaded module [x-pack-watcher]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,949Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "no plugins loaded" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,975Z", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/nvme0n1p4)]], net usable_space [648gb], net total_space [751.8gb], types [ext4]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:03,975Z", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "heap size [256mb], compressed ordinary object pointers [true]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:04,026Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "node name [es01], node ID [dDdqQsSFTeKH--g5LV5OoQ], cluster name [traefik-tutorial-cluster]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:06,521Z", "level": "INFO", "component": "o.e.x.m.p.l.CppLogMessageHandler", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "[controller/229] [Main.cc@114] controller (64 bit): Version 7.9.2 (Build 6a60f0cf2dd5a5) Copyright (c) 2020 Elasticsearch BV" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:06,965Z", "level": "INFO", "component": "o.e.x.s.a.s.FileRolesStore", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:07,681Z", "level": "INFO", "component": "o.e.t.NettyAllocator", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "creating NettyAllocator with the following configs: [name=unpooled, factors={es.unsafe.use_unpooled_allocator=false, g1gc_enabled=true, g1gc_region_size=1mb, heap_size=256mb}]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:07,729Z", "level": "INFO", "component": "o.e.d.DiscoveryModule", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "using discovery type [zen] and seed hosts providers [settings]" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:08,034Z", "level": "WARN", "component": "o.e.g.DanglingIndicesState", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:08,329Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "initialized" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:08,329Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "starting ..." }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:08,439Z", "level": "INFO", "component": "o.e.t.TransportService", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "publish_address {172.18.0.2:9300}, bound_addresses {0.0.0.0:9300}" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:08,592Z", "level": "INFO", "component": "o.e.b.BootstrapChecks", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "bound or publishing to a non-loopback address, enforcing bootstrap checks" }
elasticsearch_1  | ERROR: [1] bootstrap checks failed
elasticsearch_1  | [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
elasticsearch_1  | ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/traefik-tutorial-cluster.log
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:08,602Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "stopping ..." }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:08,618Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "stopped" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:08,618Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "closing ..." }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:08,625Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "closed" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-10-25T10:59:08,626Z", "level": "INFO", "component": "o.e.x.m.p.NativeController", "cluster.name": "traefik-tutorial-cluster", "node.name": "es01", "message": "Native controller process has stopped - no new native processes can be started" }
kibble_elasticsearch_1 exited with code 78

*Other:

sharanf avatar Oct 25 '20 11:10 sharanf

According to this log:

[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

It seems to be an issue with memory available for docker (?).

As this is rather hard to replicate, can you @sharanf try removing ulimits and bootstrap.memory_lock: "true" from docker-compose definition of elasticsearch: https://github.com/apache/kibble/blob/d7f9031dfd93a2efd676fcbd59443feec01df6ed/docker-compose-dev.yaml#L41-L58

turbaszek avatar Oct 25 '20 13:10 turbaszek

@sharanf let me know if change from #80 helped or not. If not we will look further

turbaszek avatar Oct 27 '20 17:10 turbaszek

@sharanf let me know if change from #80 helped or not. If not we will look further

@turbaszek thanks and will do!

sharanf avatar Oct 27 '20 20:10 sharanf

Hey @sharanf any update on this one?

turbaszek avatar Nov 08 '20 11:11 turbaszek

Hi I had another try but am still getting an error.

Trying to create ES index... File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn setup_1 | conn = connection.create_connection( setup_1 | File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 61, in create_connection setup_1 | for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): setup_1 | File "/usr/local/lib/python3.8/socket.py", line 918, in getaddrinfo setup_1 | for res in _socket.getaddrinfo(host, port, family, type, proto, flags): setup_1 | socket.gaierror: [Errno -2] Name or service not known

I am beginning to think that it is caused by my own pc setup. .I took at look in the usr/local/bin/ptyhon3.8/ directory and the site-packages directory doesn't exist.. plus I have a virualenv for another pyhon version as my default.

sharanf avatar Nov 08 '20 16:11 sharanf

@sharanf please make sure that you have latest changes from main and do docker-compose -f docker-compose-dev.yaml build - this should rebuild the images. Then let's try do the following:

docker-compose -f docker-compose-dev.yaml up elasticsearch

If there's no problem then we can stop container by cmd+c. Then let's try

docker-compose -f docker-compose-dev.yaml up setup

If there's no error, then let's try

docker-compose -f docker-compose-dev.yaml up ui

If there's any problem during any of those steps please let me know

turbaszek avatar Nov 08 '20 16:11 turbaszek

Hey @sharanf any update? It seems that @skekre98 was able to spin up Kibble using docker (https://github.com/apache/kibble/pull/91#issuecomment-726482634)

turbaszek avatar Nov 13 '20 15:11 turbaszek

It docker-compose -f docker-compose-dev.yaml up elasticsearch works on my machine. I rebased my branch yesterday.

michalslowikowski00 avatar Nov 14 '20 20:11 michalslowikowski00

@sharanf please make sure that you have latest changes from main and do docker-compose -f docker-compose-dev.yaml build - this should rebuild the images. Then let's try do the following:

docker-compose -f docker-compose-dev.yaml up elasticsearch

If there's no problem then we can stop container by cmd+c. Then let's try

docker-compose -f docker-compose-dev.yaml up setup

If there's no error, then let's try

docker-compose -f docker-compose-dev.yaml up u

If there's any problem during any of those steps please let me know

Hi @turbaszek sorry for the delay in testing.(way too much work and not enough time to do it in :-). I did a fresh clone of main and ran the rebuild - it didnt highlight any errors. I have copied the last few lines here

 Running setup.py develop for apache-kibble
Successfully installed PyYAML-5.3.1 apache-kibble bcrypt-3.2.0 certifi-2020.6.20 cffi-1.14.3 chardet-3.0.4 click-7.1.2 elasticsearch-7.9.1 future-0.18.2 gunicorn-20.0.4 idna-2.10 oauthlib-3.1.0 psutil-5.7.3 pycparser-2.20 python-dateutil-2.8.1 python-twitter-3.5 requests-2.25.0 requests-oauthlib-1.3.0 six-1.15.0 tenacity-6.2.0 urllib3-1.26.2
Removing intermediate container c2430daf74cd
 ---> 86b556adbb48
Successfully built 86b556adbb48
Successfully tagged apache/kibble:latest
kibble uses an image, skipping
ui uses an image, skipping
kibana uses an image, skipping

Next I ran the docker-compose -f docker-compose-dev.yaml up elasticsearch

and it ended with a out of memory error. Have copied some of the details below

 ERROR: [1] bootstrap checks failed
elasticsearch_1  | [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
elasticsearch_1  | ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/kibble.log
elasticsearch_1  | {"type": "server", "timestamp": "2020-11-20T16:16:18,461Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "kibble", "node.name": "es01", "message": "stopping ..." }
elasticsearch_1  | {"type": "server", "timestamp": "2020-11-20T16:16:18,469Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "kibble", "node.name": "es01", "message": "stopped" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-11-20T16:16:18,469Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "kibble", "node.name": "es01", "message": "closing ..." }
elasticsearch_1  | {"type": "server", "timestamp": "2020-11-20T16:16:18,477Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "kibble", "node.name": "es01", "message": "closed" }
elasticsearch_1  | {"type": "server", "timestamp": "2020-11-20T16:16:18,479Z", "level": "INFO", "component": "o.e.x.m.p.NativeController", "cluster.name": "kibble", "node.name": "es01", "message": "Native controller process has stopped - no new native processes can be started" }
kibble_elasticsearch_1 exited with code 78

So I think this issue isn't a kibble one, it is my configuration and everyone else has better memory. From my side I think it's OK to close this issue as it's very specific to me. In the meantime I will continue to try and get it up and running.

Thanks

sharanf avatar Nov 20 '20 16:11 sharanf

Came in to close this issue but saw thent saw the comment about the ES Memory locking issue so will try again to see if it helps.

sharanf avatar Jan 10 '21 16:01 sharanf