intelmq-manager icon indicating copy to clipboard operation
intelmq-manager copied to clipboard

Empty parameter value incorrectly interpreted as empty string, leading to bot failures

Open yper3 opened this issue 2 years ago • 10 comments

Hello,

I have problems with the postgres SQL BOT:

2023-03-22 05:27:37,073 - SQL-Output - INFO - Bot will continue in 15 seconds. 2023-03-22 05:27:52,125 - SQL-Output - ERROR - Bot has found a problem. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/intelmq/lib/bot.py", line 319, in start self.process() File "/usr/lib/python3/dist-packages/intelmq/bots/outputs/sql/output.py", line 56, in process if self.execute(query, values, rollback=True): File "/usr/lib/python3/dist-packages/intelmq/lib/mixins/sql.py", line 115, in execute self.cur.execute(query, values) psycopg2.errors.SyntaxError: zero-length delimited identifier at or near """" LINE 1: INSERT INTO events ("") VALUES () ^

Do you know what can be?

Thanks!!!

yper3 avatar Mar 22 '23 04:03 yper3

Is that event probably empty?

sebix avatar Mar 22 '23 16:03 sebix

It happens with any event. when you configured the output to postgres. for which when inserting into the Query there is a blank column.

I have configured postgres following the guide and it has not been able to work due to this error, even when it is connected and the girl database is created

yper3 avatar Mar 22 '23 16:03 yper3

Can you please show the configuration parameters of the SQL-Output bot and show the output of intelmqctl run SQL-Output message get?

sebix avatar Mar 22 '23 17:03 sebix

Can you please show the configuration parameters of the SQL-Output bot and show the output of intelmqctl run SQL-Output message get?

SQL-Output: Bot has found a problem. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/intelmq/lib/bot.py", line 319, in start self.process() File "/usr/lib/python3/dist-packages/intelmq/bots/outputs/sql/output.py", line 56, in process if self.execute(query, values, rollback=True): File "/usr/lib/python3/dist-packages/intelmq/lib/mixins/sql.py", line 115, in execute self.cur.execute(query, values) psycopg2.errors.SyntaxError: zero-length delimited identifier at or near """" LINE 1: INSERT INTO events ("") VALUES () ^ SQL-Output: Dumping message to dump file.

yper3 avatar Mar 23 '23 02:03 yper3

This is the configuration for Bot

SQL-Output: bot_id: SQL-Output description: Send events to a PostgreSQL or SQLite database enabled: true group: Output module: intelmq.bots.outputs.sql.output name: SQL parameters: autocommit: true bottype: Output database: intelmq-events destination_queues: {} engine: postgresql fields: '' host: xx.159.xx.xx jsondict_as_string: true message_jsondict_as_string: true password: **** port: 5432 reconnect_delay: 0 sslmode: require table: events user: intelmq run_mode: continuous

yper3 avatar Mar 23 '23 02:03 yper3

Why is fields an empty string? Does it work if you set it to null/empty/remove it?

sebix avatar Mar 23 '23 09:03 sebix

thank you. I am going to try. the configuration was done from the web interface.

yper3 avatar Mar 23 '23 15:03 yper3

Hello!!!

It has worked. The error is when you make a change from intelmq-manager (web) and save it. Add the field value "", totally remove "field" from cli and it works. But if you modify it again from web it writes it again.

yper3 avatar Mar 23 '23 15:03 yper3

The problem maybe are in intelmq-manager...

yper3 avatar Mar 23 '23 16:03 yper3

Yeah, the manager is not good at getting the types correctly

sebix avatar Mar 23 '23 18:03 sebix