celery-singleton icon indicating copy to clipboard operation
celery-singleton copied to clipboard

Recheck that dup task is actually running

Open Suor opened this issue 5 years ago • 6 comments

This might be done with AsyncResult(existing_task_id).ready().

Suor avatar Apr 17 '20 03:04 Suor

This is probably a bug if it doesn't work Should be no difference between AsyncResult here and with normal celery.

What's the result of this now and is it different from a standard celery task?

steinitzu avatar Apr 17 '20 14:04 steinitzu

Now, there is no recheck. I thought that might help if worker crashed.

Suor avatar Apr 17 '20 15:04 Suor

Ah I see what you mean, to check for zombie locks?

This still returns "pending" for unknown task IDs, right? So we still don't know whether it's queued or not.

steinitzu avatar Apr 17 '20 18:04 steinitzu

Hmm, maybe I'm not familiar enough with this part of celery. So the is no way to fix this only expiry thing?

Suor avatar Apr 18 '20 01:04 Suor

There's not really a reliable way to do this that I know.

You can use celery signals to set a custom state instead of "pending" when the task is actually sent to the broker, and say if task is "pending" it doesn't exist.

But that doesn't guarantee much as you would still get "pending if the check happens sometime in between sending the first task and updating the state.

steinitzu avatar Apr 18 '20 13:04 steinitzu

If you can't really do that then this might be closed.

Suor avatar Apr 18 '20 16:04 Suor