elastalert icon indicating copy to clipboard operation
elastalert copied to clipboard

Unable to run Elastalert with Elasticsearch 7.1.1

Open bananth2008 opened this issue 6 years ago • 6 comments

Hi There, On running elastalert as a docker image with elasticsearch 7.1.1, the following error message is thrown. (refer below) Any help is highly appreciated as we have recently moved from elasticsearch 6.x to elasticsearch 7.1.1. Thanks in advance for your help. Regards

07:15:13.046Z INFO elastalert-server: Server: Server listening on port 3030 07:15:13.048Z INFO elastalert-server: Server: Websocket listening on port 3333 07:15:13.048Z INFO elastalert-server: Server: Server started 07:15:13.709Z ERROR elastalert-server: ProcessController: INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent. To send them but remain verbose, use --verbose instead. 07:15:13.710Z ERROR elastalert-server: ProcessController: INFO:elastalert:Starting up 07:15:13.713Z ERROR elastalert-server: ProcessController: Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main 07:15:13.713Z ERROR elastalert-server: ProcessController: "main", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/opt/elastalert/elastalert/elastalert.py", line 1929, in 07:15:13.714Z ERROR elastalert-server: ProcessController: sys.exit(main(sys.argv[1:])) File "/opt/elastalert/elastalert/elastalert.py", line 1925, in main 07:15:13.714Z ERROR elastalert-server: ProcessController: client.start() File "/opt/elastalert/elastalert/elastalert.py", line 1106, in start 07:15:13.715Z ERROR elastalert-server: ProcessController: self.run_all_rules() File "/opt/elastalert/elastalert/elastalert.py", line 1158, in run_all_rules 07:15:13.716Z ERROR elastalert-server: ProcessController: self.send_pending_alerts() File "/opt/elastalert/elastalert/elastalert.py", line 1534, in send_pending_alerts 07:15:13.716Z ERROR elastalert-server: ProcessController: pending_alerts = self.find_recent_pending_alerts(self.alert_time_limit) File "/opt/elastalert/elastalert/elastalert.py", line 1526, in find_recent_pending_alerts 07:15:13.717Z ERROR elastalert-server: ProcessController: size=1000) File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.1-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped 07:15:13.717Z ERROR elastalert-server: ProcessController: return func(*args, params=params, **kwargs) TypeError: search() got an unexpected keyword argument 'doc_type' 07:15:13.742Z ERROR elastalert-server: ProcessController: ElastAlert exited with code 1 07:15:13.743Z INFO elastalert-server: Server: Stopping server 07:15:13.743Z INFO elastalert-server: ProcessController: ElastAlert is not running 07:15:13.743Z INFO elastalert-server: Server: Server stopped. Bye!

bananth2008 avatar Jun 10 '19 07:06 bananth2008

Did you make a new installation or just an upgrade? I'am facing the same issue trying to integrate Elastalert with ES 7.1.1 on centOS 7 server. When I use elastalert-0.2.0b2 it runs but don't send email when value matches. Last week it displays message "email send to @" but don't receive it in my inbox.

Djyamss avatar Jun 11 '19 15:06 Djyamss

Please try the beta release: https://github.com/bitsensor/elastalert/releases/tag/3.0.0-beta.0

martijnrondeel avatar Jun 11 '19 15:06 martijnrondeel

I am not experienced with opensource so is this good command to install: pip install elastalert:3.0.0-beta.0

Got this error:

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. ERROR: Invalid requirement: 'elastalert:3.0.0-beta.0'

Djyamss avatar Jun 11 '19 15:06 Djyamss

I get a similar error when I start the container.

22:59:42.454Z ERROR elastalert-server:
    ProcessController:  Traceback (most recent call last):
      File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
22:59:42.457Z ERROR elastalert-server:
    ProcessController:      "__main__", fname, loader, pkg_name)
      File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
        exec code in run_globals
      File "/opt/elastalert/elastalert/elastalert.py", line 1929, in <module>
22:59:42.459Z ERROR elastalert-server:
    ProcessController:      sys.exit(main(sys.argv[1:]))
      File "/opt/elastalert/elastalert/elastalert.py", line 1925, in main
22:59:42.459Z ERROR elastalert-server:
    ProcessController:      client.start()
      File "/opt/elastalert/elastalert/elastalert.py", line 1106, in start
22:59:42.461Z ERROR elastalert-server:
    ProcessController:      self.run_all_rules()
      File "/opt/elastalert/elastalert/elastalert.py", line 1158, in run_all_rules
        self.send_pending_alerts()
      File "/opt/elastalert/elastalert/elastalert.py", line 1534, in send_pending_alerts
22:59:42.461Z ERROR elastalert-server:
    ProcessController:      pending_alerts = self.find_recent_pending_alerts(self.alert_time_limit)
      File "/opt/elastalert/elastalert/elastalert.py", line 1526, in find_recent_pending_alerts
        size=1000)
      File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.1-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped
22:59:42.462Z ERROR elastalert-server:
    ProcessController:      return func(*args, params=params, **kwargs)
    TypeError: search() got an unexpected keyword argument 'doc_type'
22:59:42.510Z ERROR elastalert-server: ProcessController:  ElastAlert exited with code 1
22:59:42.510Z  INFO elastalert-server: Server:  Stopping server
22:59:42.511Z  INFO elastalert-server: ProcessController:  ElastAlert is not running
22:59:42.511Z  INFO elastalert-server: Server:  Server stopped. Bye!

Image: docker.io/bitsensor/elastalert:2.0.1 Elasticsearch image: docker.io/amazon/opendistro-for-elasticsearch:0.9.0

elasticsearch --version
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Version: 6.7.1, Build: oss/tar/2f32220/2019-04-02T15:59:27.961366Z, JVM: 11.0.1

Is anyone running ElastAlert with opendistro-for-elasticsearch 'ODFE'?

jeff-cook avatar Jun 20 '19 23:06 jeff-cook

3.0.0-beta.0 solved the issue for me (elasticsearch 7.2) :+1:

zatteo avatar Jul 13 '19 17:07 zatteo

hi , i am getting the below error when runing the command "python -m elastalert.elastalert --verbose --rule example_frequency.yaml" can anyone please help me in this ?

image

zaheershaffi avatar Sep 14 '20 07:09 zaheershaffi