comet-cache icon indicating copy to clipboard operation
comet-cache copied to clipboard

cron_zencache_cleanup never executing!

Open xberg opened this issue 10 years ago • 58 comments

Hi Using Pro version 160103. The _cron_zencache_cleanup never executes. I use Crontrol to monitor the wp_cron and there's very strange behavior: The "next run" time keeps changing!!! I have it set to run every 3 hours, but the next run time is always in just 1 minute time! When I refresh the crontrol page I see your CRON is still schedule to run in a minute. and it NEVER NEVER runs. So this just fills my cache and I have several times a day to manually delete the cache! Same bahavior with _cron_zencache_auto_cache but I don't use this.

I tried deleting and reinstalling: same problem.

And you CRONs are the ONLY ones I cannot delete or edit with crontrol. It's like your system is continusouly deleting and recreating new ones!

Please help me fix this ASAP.

xberg avatar Jan 09 '16 23:01 xberg

I do NOT have the Disable Cache Expiration If Server Load Average is High enabled, but it seems to behave like it is and that my server is high.

xberg avatar Jan 09 '16 23:01 xberg

@xberg I'm not able to reproduce this behavior. The cronjobs are behaving as expected for me:

40 seconds before running

2016-01-10_07-34-24

While running

2016-01-10_07-35-17

Just after running

2016-01-10_07-35-44


I suggest disabling deletion safeguards (ZenCache → Plugin Options → Plugin Deletion Safeguards) and then deactivating and deleting the ZenCache plugin. Then install it fresh and see if you still have issues.

(Note that if you have a custom plugin configuration that you want to backup, you can use ZenCache → Plugin Options → Import/Export Options.)

Also, it's normal for ZenCache to recreate the cron events if you delete them. ZenCache needs those events to be present for the plugin to work properly, so if ZenCache is active and it detects the events are missing, it will recreate them. If you don't want ZenCache to recreate the events, simply deactivate the plugin.

raamdev avatar Jan 10 '16 12:01 raamdev

@xberg Also, if you want to change the schedule at which the cache cleanup occurs, you can create a custom schedule with WP Crontrol (Dashboard → Settings → Cron Schedules) and then select that custom schedule in ZenCache → Plugin Options → Cache Expiration Time → Cache Cleanup Schedule:

2016-01-10_07-41-00

raamdev avatar Jan 10 '16 12:01 raamdev

Raam, Just because you cannot reproduce the issue does not mean that the issue is not there. I am experienced with CRON jobs and I can tell you that there is a major problem on my setup with just your plugin, and this only started a few weeks ago (at least 2 releases ago). Yes, I can change the schedule to whatever I want, but your cleanup NEVER runs. It keeps reseting itself to run in a minute (but NEVER runs).

Let's try to find a temporary workaround: is there any code I could run thru a PHP to replicate the behavior of the cleanup? ie, I would be creating my own CRON?

Because I am using a 2 GB RAMDRIVE it fills in less than a day and then starts producing a massive number of errors. As such your plugin in unusable now.

xberg avatar Jan 11 '16 09:01 xberg

http://prntscr.com/9oo1oz http://prntscr.com/9oo1t5 http://prntscr.com/9oo1wq http://prntscr.com/9oo1zu http://prntscr.com/9oo22i http://prntscr.com/9oo25s http://prntscr.com/9oo2b0

Here I am refreshing CRONTROL every 20 seconds. You can see that your CRONs are alway reset to run in 1 minute. In the meanwhile you can see that another cron, the NXS decrements correctly.

You have a problem and you need to acknowledge it.

xberg avatar Jan 11 '16 09:01 xberg

@xberg Have you reproduced this issue in a clean WordPress installation? See Testing ZenCache in a Clean WordPress Installation

If you have, then please provide me with a list of steps to reproduce this issue on my site.

raamdev avatar Jan 11 '16 16:01 raamdev

Raamdev,

Do you have anything in your program that resets the counter to 1 minute? It would not surprise me at all that your "Disable Cache Expiration If Server Load Average is High" is creating the problem.

Otherwise is there any instruction I could run to replicate the behavior of your CRON?

Without this, what is your reimbursement policy as I can no longer use your caching with this bug.

xberg avatar Jan 12 '16 08:01 xberg

Ah!! I found your code that was creating the bug. I cannot believe you did not immediately find it yourself after my description!

It's code you added 20151220 which resets the CRON to run in a minute: exactly the description I made initially!!!

It's in CronUtils.php and here it is:

/*

  • Checks Cron setup, validates schedules, and reschedules events if necessary. *

  • @attaches-to init hook. *

  • @since 151220 Improving WP Cron setup and validation of schedules / $self->checkCronSetup = function () use ($self) { if ($self->options['crons_setup'] < 1439005906 || $self->options['crons_setup_on_namespace'] !== NAMESPACE || $self->options['crons_setup_with_cache_cleanup_schedule'] !== $self->options['cache_cleanup_schedule'] || $self->options['crons_setup_on_wp_with_schedules'] !== sha1(serialize(wp_get_schedules())) || !wp_next_scheduled('cron'.GLOBAL_NS.'cleanup') /[pro strip-from="lite"]/ // Auto-cache engine. || !wp_next_scheduled('cron'.GLOBAL_NS.'auto_cache') /[/pro]*/ ) {

    wp_clear_scheduled_hook('_cron_'.GLOBAL_NS.'_cleanup');
    wp_schedule_event(time() + 60, $self->options['cache_cleanup_schedule'], '_cron_'.GLOBAL_NS.'_cleanup');
    
    /*[pro strip-from="lite"]*/ // Auto-cache engine.
    wp_clear_scheduled_hook('_cron_'.GLOBAL_NS.'_auto_cache');
    wp_schedule_event(time() + 60, 'every15m', '_cron_'.GLOBAL_NS.'_auto_cache');
    /*[/pro]*/
    
    $self->updateOptions(
      array(
        'crons_setup'                             => time(),
        'crons_setup_on_namespace'                => __NAMESPACE__,
        'crons_setup_with_cache_cleanup_schedule' => $self->options['cache_cleanup_schedule'],
        'crons_setup_on_wp_with_schedules'        => sha1(serialize(wp_get_schedules()))
      )
    );
    

    } };

/*

  • Resets crons_setup and clears WP-Cron schedules. *
  • @since 151220 Fixing bug with Auto-Cache Engine cron disappearing in some scenarios *
  • @note This MUST happen upon uninstall and deactivation due to buggy WP_Cron behavior. Events with a custom schedule will disappear when plugin is not active (see http://bit.ly/1lGdr78). / $self->resetCronSetup = function ( ) use ($self) { if (is_multisite()) { // Main site CRON jobs. switch_to_blog(get_current_site()->blog_id); /[pro strip-from="lite"]/ // Auto-cache engine. wp_clear_scheduled_hook('cron'.GLOBAL_NS.'auto_cache'); /[/pro]/ wp_clear_scheduled_hook('cron'.GLOBAL_NS.'cleanup'); restore_current_blog(); // Restore current blog. } else { // Standard WP installation. /[pro strip-from="lite"]_/ // Auto-cache engine. wp_clear_scheduled_hook('cron'.GLOBAL_NS.'auto_cache'); /[/pro]*/ wp_clear_scheduled_hook('cron'.GLOBAL_NS.'_cleanup'); } $self->updateOptions( array( // Reset so that crons are rescheduled upon next activation 'crons_setup' => $self->default_options['crons_setup'], 'crons_setup_on_namespace' => $self->default_options['crons_setup_on_namespace'], 'crons_setup_with_cache_cleanup_schedule' => $self->default_options['crons_setup_with_cache_cleanup_schedule'], 'crons_setup_on_wp_with_schedules' => $self->default_options['crons_setup_on_wp_with_schedules'] ) ); };

xberg avatar Jan 12 '16 09:01 xberg

Remving this line fixed your bug:

|| $self->options['crons_setup_on_wp_with_schedules'] !== sha1(serialize(wp_get_schedules()))

xberg avatar Jan 12 '16 09:01 xberg

@xberg Watch it with the tone! Show some respect here, and we will do the same.

I cannot believe you did not immediately find it yourself after my description!

Please cleanup the code that you posted and I will review as well. So far it looks like you solved the problem, but I'm not clear on why you're calling this a :bug: yet. Please explain in greater details and we can work together at solving it.

jaswrks avatar Jan 12 '16 14:01 jaswrks

Youi're right: I'm sorry about the tone.

xberg avatar Jan 12 '16 14:01 xberg

@xberg writes...

$self->options['crons_setup_on_wp_with_schedules'] !== sha1(serialize(wp_get_schedules()))`

The only reason that line would cause problems is if wp_get_schedules() changes on every page load (and that should never happen, which again, is why I asked you to reproduce this in a clean installation of WordPress to rule out issues with other plugins or custom code).

That line basically says, "if there has been a change to the WordPress Cron Schedules (i.e., you added/removed/changed a cron schedule), then set up the ZenCache cron events again". This is necessary because if you changed or delete a cron schedule that ZenCache was using, ZenCache must reconfigure its own cron events to make sure that it keeps working as expected.

I'm not seeing any bug in that code. I have tested it and it is working as expected.

raamdev avatar Jan 12 '16 14:01 raamdev

Raamdev, I am not a developer so I cannot help you. I just know that removing this line fixed my problem. With the line there was a continuous loop which kept creating a new cron job that was to execute in a minute, but immediately got deleted and recreated.

I understand it's hard to debug without being able to replicate, but I am convinced I am not the only one who will have this problem so at least now you know what has helped me fix things. The only reason I noticed this problem is that I am using a 2 GB Ramdrive for zencache and this filled up fast. If I were using harddrive it could have been weeks before I noticed a problem. And I also have crontrol installed which allowed me to see this problem: most users will not have crontrol.

xberg avatar Jan 14 '16 11:01 xberg

@xberg Again, have you reproduced this in a clean WordPress environment? I'm guessing you did not, because if you did I have a strong feeling you'd see that there is no bug there. :-)

raamdev avatar Jan 14 '16 12:01 raamdev

@raamdev I'm seeing the same thing on my multisite installation subsites. This cron schedule gets bumped to 1min away on every page load. There doesn't seem to be anything that would be messing with cron schedules that would cause the hash to change. I'll investigate some more.

EDIT !wp_next_scheduled('_cron_'.GLOBAL_NS.'_auto_cache') always evaluates to true for me, causing the ever-reset symptom.

EDIT I'm just going to flat out return from $self->checkCronSetup() right now pending further instructions. This eliminates slow update queries on every page load.

lkraav avatar Jan 22 '16 23:01 lkraav

Thanks ikraav for sharing your fix. Concretly, what file did you edit?

xberg avatar Jan 23 '16 09:01 xberg

Thanks ikraav for sharing your fix. Concretly, what file did you edit?

./src/includes/closures/Plugin/CronUtils.php the same thing you already found. It's not a fix, just a stopgap until people are able to figure out what's happening.

lkraav avatar Jan 23 '16 10:01 lkraav

@lkraav The wp_next_scheduled() function (see docs) just checks if a given action hook, in this case _cron_zencache_auto_cache, has been scheduled.

If the conditional where that check exists evaluates to true (i.e., the event has not been scheduled), then ZenCache schedules the event with this line:

wp_schedule_event(time() + 60, 'every15m', '_cron_'.GLOBAL_NS.'_auto_cache');

So once the event has been scheduled, !wp_next_scheduled('_cron_'.GLOBAL_NS.'_auto_cache') should return false, since the event has now been scheduled.

https://github.com/websharks/zencache-pro/blob/160103/src/includes/closures/Plugin/CronUtils.php#L23-L58

I can't explain why that's not working properly for you, but in all of our tests it works as expected.

raamdev avatar Jan 27 '16 21:01 raamdev

I will re-evaluate each of the conditions again. It may not be about just wp_next_scheduled() here.

lkraav avatar Jan 27 '16 21:01 lkraav

Today I upgraded to V160222 Pro and the issue was back. I returned to /wp-content/plugins/zencache-pro/src/includes/closures/Plugin/CronUtils.php and commented out as I did before: || $self->options['crons_setup_on_wp_with_schedules'] !== sha1(serialize(wp_get_schedules()))

but this time it was not enough, and the CRON kept resetting itself to 1 minute.

But also commenting out: || $self->options['crons_setup_with_cache_cleanup_schedule'] !== $self->options['cache_cleanup_schedule'] worked and now the CRON is no longer reset.

Just FYI, as I have no time to dig further into this matter and that my fix solves the problem for me. Good luck.

xberg avatar Feb 23 '16 10:02 xberg

@xberg @lkraav I'm reopening this issue.

@lkraav reported in https://github.com/websharks/comet-cache/issues/667#issuecomment-187297229 that this issue occurred again with the Comet Cache update, which tells me this bug is most likely related to a certain combination of events that are not being considered in the checkCronSetup() routine, which is resulting in that routine updating the database repeatedly.

I have yet to figure out what combination of events is causing this, but since I have seen this happen during testing in the past (it happened to me once or twice, but then the problem disappeared and I was not able to reproduce it again at the time).

I'm going to reopen this GitHub issue so that we can continue tracking reports and so that it can be researched.

raamdev avatar Feb 25 '16 03:02 raamdev

Hi: quick update. Today I upgraded from Zencache Pro to Comet cache Pro V 160227 and the problem re-appeared. So this time I had to comment out 3 conditions, as commenting out only 2 still continusouly resets the CRON to 60 seconds. The additional condition I had to comment out was: !wp_next_scheduled('_cron_'.GLOBAL_NS.'_cleanup')

in addition to:

  //|| $self->options['crons_setup_with_cache_cleanup_schedule'] !== $self->options['cache_cleanup_schedule']
        //|| $self->options['crons_setup_on_wp_with_schedules'] !== sha1(serialize(wp_get_schedules()))

Good luck, now that I know the "trick" it's not a big inconvenience to comment this out with each upgrade.

xberg avatar Mar 17 '16 10:03 xberg

@xberg Thank you for the update! Would you mind providing a list of active plugins on your site?

raamdev avatar Mar 17 '16 13:03 raamdev

Sure @raamdev I have emailed this to you. Let me email this to you privately as a public list of plugins might be a security threat in case one gets exposed.

xberg avatar Mar 17 '16 13:03 xberg

@xberg Thank you. I received the email. That's quite a few plugins--I was hoping to review the code for those plugins to see if one of them might be causing problems, but that's too long of a list.

@lkraav Would you be able to provide a list of active plugins as well? That way I can see if there are any plugins you're both using and hopefully narrow this down.

raamdev avatar Mar 17 '16 14:03 raamdev

@raamdev haha, I completely understand. And it might be another 100 things that are causing this such as server settings. Perhpas a better use of your time would be to simply have a geek option somewhere that allows users that see this behavior to simply tick that option which in turns comments out the whole "cron reset" function. Or simply like me tell them to edit the file: no big deal at all.

xberg avatar Mar 17 '16 15:03 xberg

@raamdev Eventually I will set up a test platform and turn the plugins off one by one...

xberg avatar Mar 17 '16 15:03 xberg

@raamdev I created a near clone from my production site and I was not able to reproduce the behavior. On the test platform Comet Cache worked normally. The biggest difference is no traffic on the test site :)

xberg avatar Mar 17 '16 15:03 xberg

@xberg Thanks for the follow-up. I can't imagine a difference in traffic affecting this issue... It must be something else...

raamdev avatar Mar 17 '16 19:03 raamdev

Hi, Another follow up. Today I upgraded Comet Cache Pro to V160417 and the same problem occured. In this version you changed the directory structure, so for those facing the same problem as I have the file to modify is in: /wp-content/plugins/comet-cache-pro/src/includes/traits/Plugin

And this time I had to comment out these 4 lines for the cron to stop resetting: //|| $this->options['crons_setup_on_namespace'] !== __NAMESPACE__ //|| $this->options['crons_setup_with_cache_cleanup_schedule'] !== $this->options['cache_cleanup_schedule'] //|| $this->options['crons_setup_on_wp_with_schedules'] !== sha1(serialize(wp_get_schedules())) //|| !wp_next_scheduled('_cron_'.GLOBAL_NS.'_cleanup')

xberg avatar Apr 18 '16 07:04 xberg