nodejs-pool icon indicating copy to clipboard operation
nodejs-pool copied to clipboard

Worker module crashed

Open LegolasGChief opened this issue 7 years ago • 15 comments

The worker failed to return last block header. Please verify if the daemon is running properly.

Anyone have any ideas why? how often should I restart modules have been running for about a week and recently had pretty big jump in hashrate, CPU usage htop is minimal yet modules crashing? :(

I check daemon it is still at right block height

LegolasGChief avatar Jun 09 '18 13:06 LegolasGChief

@LegolasGChief Were you able to upgrade to 12.2? I noticed I had the same issue.

snoski3 avatar Jun 09 '18 15:06 snoski3

I'm also having this issue, but my CPU is being used a lot. (Aeon pool)

Worker module is crashing, like it's losing connection with the daemon, but the daemon is running fine, without errors.

shigutso avatar Jun 09 '18 15:06 shigutso

I ran a git pull, which works. Then I ran npm install, which doesn't work. $ npm install audited 1067 packages in 2.497s found 23 vulnerabilities (9 low, 13 moderate, 1 high) run npm audit fix to fix them, or npm audit for details

I'm not sure if I need to start from scratch or update the modules manually.

snoski3 avatar Jun 09 '18 15:06 snoski3

I think if we can replace the parserjson module, we should be able to upgrade. High │ Regular Expression Denial of Service │ ├───────────────┼──────────────────────────────────────────────────────────────┤ │ Package │ parsejson │ ├───────────────┼──────────────────────────────────────────────────────────────┤ │ Patched in │ No patch available │ ├───────────────┼──────────────────────────────────────────────────────────────┤ │ Dependency of │ socketio

Also, I thought I upgraded to 12.0 but I checked the version in the Monero folder in the main folder and it is still version 11.

snoski3 avatar Jun 09 '18 15:06 snoski3

@LegolasGChief I upgraded NPM to version 6.1. If I didn't, I'm sure I wouldn't have these issues.

@Snipa22 Will I have to start from scratch because I upgraded NPM to version 6.1 or are there plans to patch the vulns in the dependencies?

Thanks,

snoski3 avatar Jun 09 '18 16:06 snoski3

This was for Aeon pool. Daemon is also using a lot if CPU sometimes not sure why maybe it is related to that

LegolasGChief avatar Jun 10 '18 04:06 LegolasGChief

Worker crash !?! Undefined of property blockID or something like that !?! It is not pool’s problem ( all those NPM stuffs are unrelated and OK ). Lost connectons with daemon happens some times ( yeah it can crash worker ) but it is OK. Daemon oftens have CPU usage surges. Nothing to worry.

bobbieltd avatar Jun 10 '18 07:06 bobbieltd

By the way, if someone here are using MoneroOcean’s repo, I can pull request the fix to avoid worker crashes 😌

bobbieltd avatar Jun 10 '18 07:06 bobbieltd

@bobbieltd Has anyone successfully updated to 12.1? I went through the process back in April. I just found out the the daemon version is still 11.1

snoski3 avatar Jun 17 '18 12:06 snoski3

moneromooo gave a workaround for this (it works for me):

go to the running daemon (aeond or monerod) and run the command print_pool_stats, if there are a lot of tx(es), it's probably what is causing the high usage of CPU and crashing the pool modules. To workaround this, run this command on the daemon: flush_txpool. As soon as I do this, the CPU usage of my server goes down to 10% and the modules stop crashing.

shigutso avatar Jun 17 '18 12:06 shigutso

Sometimes, transactions got stuck at txpool but it is rare. flush_txpool (you need to manually resend the stuck transactions to miners ) or try to restart coin daemon. Anyway, it can not cause worker died ( except Out of memory )

bobbieltd avatar Jun 18 '18 15:06 bobbieltd

The worker module crashes because of lack of CPU. I realized that if your server has more than 2 cores the modules don't crash. So to stop the CPU spike, flush_txpool works for me, because my server has only 2 cores.

shigutso avatar Jun 18 '18 15:06 shigutso

Seems like this is something to do with daemon rather than the pool lets hope devs are able to fix it

DurinsMine avatar Jul 08 '18 21:07 DurinsMine

If any module crashes, there is pm2 log error and you can know why.

bobbieltd avatar Jul 08 '18 22:07 bobbieltd

@bobbieltd the daemon is checking tx inputs and becoming unresponsive using too much cpu. The worker module fails to getBlockHeader which then causes the blockmanager module to crash. It says that there is no active block template or header if restart modules while its going crazy. Need to flush_txpool and then pm2 restart all.

There is a related but slightly different problem with pool software. The two slew fee options generating the larger input txs because of fractional fees being taken for the block payments to miners which then causes the daemon/pool to stall/crash so its kind of the root cause of above problem, a knock on effect I guess.

This is just info I have gotten from camping in IRC

DurinsMine avatar Jul 08 '18 23:07 DurinsMine