StackStorm
Stackstorm Blueprint box memory usage is high
I have three blueprint boxes in a HA environment. On it, there are only two sensors and a workflow. Still, each of them uses a lot of RAM.
840.066MB 21458 st2 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2garbagecollector --config-file /etc/st2/st2.conf 845.285MB 21468 st2 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2sensorcontainer --config-file /etc/st2/st2.conf 853.816MB 21473 st2 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2timersengine --config-file /etc/st2/st2.conf 863.25MB 21461 st2 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2notifier --config-file /etc/st2/st2.conf 864.566MB 21431 root /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2actionrunner --config-file /etc/st2/st2.conf 864.742MB 21423 root /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2actionrunner --config-file /etc/st2/st2.conf 864.754MB 21415 root /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2actionrunner --config-file /etc/st2/st2.conf 865.145MB 21417 root /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2actionrunner --config-file /etc/st2/st2.conf 865.266MB 21427 root /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2actionrunner --config-file /etc/st2/st2.conf 865.418MB 21434 root /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2actionrunner --config-file /etc/st2/st2.conf 865.578MB 21419 root /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2actionrunner --config-file /etc/st2/st2.conf 866.035MB 21476 st2 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2workflowengine --config-file /etc/st2/st2.conf 866.531MB 21421 root /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2actionrunner --config-file /etc/st2/st2.conf 867.25MB 21429 root /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2actionrunner --config-file /etc/st2/st2.conf 867.492MB 21425 root /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2actionrunner --config-file /etc/st2/st2.conf 888.445MB 21465 st2 /opt/stackstorm/st2/bin/python /opt/stackstorm/st2/bin/st2rulesengine --config-file /etc/st2/st2.conf
ps -eo rss,pid,euser,args:100 --sort %mem | grep -v mistral | grep -i '/opt/stackstorm' | awk '{sum+=$1} END {print sum / 1024}' 15380.1
Stackstorm is using almost 15GB RAM in all the boxes. How can it be resolved?
version of stackstorm? what os? versions of mongodb, rabbitmq, redis?
Also any interesting logs for st2 services?
Mistral is no longer supported, also.
All the st2 services are utilizing more memory Now I can see each services utilizing nearly 1.8GB. Can anyone suggest an idea to reduce the memory usage.
If you answer @guzzijones questions, that should allow us to start investigating.
@nzlosh and @guzzijones apologies for delay. Below are most of the answers. Not sure yet on logs, will check with team. Thank you!
version of stackstorm? $ st2 --version st2 3.7.0, on Python 3.6.9
what os? $ cat /etc/os-release NAME="Ubuntu" VERSION="18.04.6 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.6 LTS" VERSION_ID="18.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=bionic UBUNTU_CODENAME=bionic
versions of mongodb, rabbitmq, redis? $ mongod --version db version v6.0.3 Build Info: { "version": "6.0.3", "gitVersion": "f803681c3ae19817d31958965850193de067c516", "openSSLVersion": "OpenSSL 1.1.1 11 Sep 2018", "modules": [], "allocator": "tcmalloc", "environment": { "distmod": "ubuntu1804", "distarch": "x86_64", "target_arch": "x86_64" } } $ ps -ef | grep rabbitmq among the output I found: /usr/lib/rabbitmq/lib/rabbitmq_server-3.6.10
$ redis-server --version Redis server v=4.0.9 sha=00000000:0 malloc=jemalloc-3.6.0 bits=64 build=9435c3c2879311f3
