plugin_thold icon indicating copy to clipboard operation
plugin_thold copied to clipboard

thold_daemon latest commit db error

Open morganfw opened this issue 1 year ago • 11 comments

Describe the bug In the latest thold plugin commit after enabling the thold_daemon.service on the latest Cacti 1.2.x commit, the following error is raised:

2024-11-06 10:51:20 - CMDPHP SQL Backtrace: (/plugins/thold/thold_daemon.php[187]:db_check_reconnect(), /lib/database.php[288]:db_fetch_cell(), /lib/database.php[637]:db_fetch_cell_prepared(), /lib/database.php[659]:db_execute_prepared())
--
2024-11-06 10:51:20 - CMDPHP ERROR: A DB Cell Failed!, Error: MySQL server has gone away
2024-11-06 10:51:20 - CMDPHP SQL Backtrace: (/plugins/thold/thold_daemon.php[174]:thold_prime_distribution(), /plugins/thold/thold_daemon.php[401]:db_fetch_cell(), /lib/database.php[637]:db_fetch_cell_prepared(), /lib/database.php[659]:db_execute_prepared())
2024-11-06 10:51:20 - CMDPHP ERROR: A DB Cell Failed!, Error: MySQL server has gone away
2024-11-06 10:51:20 - CMDPHP SQL Backtrace: (/plugins/thold/thold_daemon.php[174]:thold_prime_distribution(), /plugins/thold/thold_daemon.php[395]:db_fetch_cell(), /lib/database.php[637]:db_fetch_cell_prepared(), /lib/database.php[659]:db_execute_prepared())
2024-11-06 10:51:20 - CMDPHP ERROR: A DB Cell Failed!, Error: MySQL server has gone away
2024-11-06 10:51:20 - CMDPHP SQL Backtrace: (/plugins/thold/thold_daemon.php[172]:thold_truncate_daemon_data(), /plugins/thold/thold_daemon.php[257]:db_execute(), /lib/database.php[420]:db_execute_prepared())
2024-11-06 10:51:20 - CMDPHP ERROR: A DB Exec Failed!, Error: MySQL server has gone away

and we need to revert to previous method by disabling thold_daemon.service issuing:

systemctl stop thold_daemon
systemctl disable thold_daemon

As we remember, the thold_daemon was working in thold v1.7.0 or before 1.8 version.

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Plugin (please complete the following information):

  • Version: [e.g. 1.2]
  • Source: [e.g. cacti.net, package, github]
  • Identifier: [e.g. apt/yum package name or github commit ref]

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context Add any other context about the problem here.

morganfw avatar Nov 06 '24 16:11 morganfw

What commit?

TheWitness avatar Nov 06 '24 22:11 TheWitness

What commit?

Sorry, I meant the latest git pulls

morganfw avatar Nov 07 '24 06:11 morganfw

From where? Which repo? What was the last commit?

TheWitness avatar Nov 07 '24 12:11 TheWitness

Is it happening any more? You backed off right?

TheWitness avatar Nov 07 '24 12:11 TheWitness

From where? Which repo? What was the last commit? Is it happening any more? You backed off right?

Previously we used the thold v1.7 plugin for quite some time on Cacti v1.2.x and the thold_daemon.service was up and running without any issues. A few weeks ago we decided to update the thold plugin to version 1.8.x, as soon as we updated the thold_daemon stopped working, returning the errors above. So the solution was to stop using thold_daemon with the commands above and go back to the default thold.

morganfw avatar Nov 07 '24 13:11 morganfw

I think that there is a thold 1.8.2 that needs to be released. Asking @xmacan or @bmfmancini if they have any clues. I'm pretty much locked into 1.3 right now. Only fixing some compatibility issues with 1.3 at the moment.

TheWitness avatar Nov 07 '24 14:11 TheWitness

It may have been that you simple needed to restart the daemon. The error is more like you were running out of connections to your database, or that the database was restarted, on finally, sometimes an insert will exceed the max_allowed_packet_size which should be around several megabytes in today's Cacti recommendations.

image

TheWitness avatar Nov 07 '24 14:11 TheWitness

We would need to get some details from the database, assuming it has not been restarted since then:

SHOW GLOBAL VARIABLES;
SHOW GLOBAL STATUS;

TheWitness avatar Nov 07 '24 14:11 TheWitness

It may have been that you simple needed to restart the daemon. The error is more like you were running out of connections to your database, or that the database was restarted, on finally, sometimes an insert will exceed the max_allowed_packet_size which should be around several megabytes in today's Cacti recommendations.

image

We restarted the daemon several times, but everytime same error is raised.

The DB was not recently restarted, last restart was 2 weeks ago:

root@cacti:~# systemctl status mariadb.service 
● mariadb.service - MariaDB 10.6.18 database server
     Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2024-10-24 07:11:43 CEST; 2 weeks 0 days ago

image

We would need to get some details from the database, assuming it has not been restarted since then:

SHOW GLOBAL VARIABLES;
SHOW GLOBAL STATUS;

Requested SQL Queries attached below

Cacti_MariaDB_SHOW_GLOBAL_STATUS.txt Cacti_MariaDB_SHOW_GLOBAL_VARIABLES.txt

morganfw avatar Nov 07 '24 15:11 morganfw

Ive seen this happen when you restart mariadb thold_daemon needs to be restarted sometimes According to the files you gave I dont see a max out in connections You have 400 max set and I see max used was 152

I have not been able to reproduce this

bmfmancini avatar Mar 20 '25 02:03 bmfmancini

Was there anything in the log that gave a clue?

TheWitness avatar Jun 27 '25 12:06 TheWitness