celery-singleton icon indicating copy to clipboard operation
celery-singleton copied to clipboard

Add `release_on_start` task configuration

Open malthejorgensen opened this issue 3 years ago • 1 comments

Adds release_on_start to the task configuration.

Passing release_on_start=True to a singleton task makes the task release the lock when the job starts running, rather than when it has finished.

Note: This is only available in celery versions 5.2 and above since it uses the before_start-method on the celery Task object which was introduced in version 5.2.

Example

@celery_app.task(base=Singleton, release_on_start=True)
def calculate_monthly_total(user_id):
    "Throttled update of `monthly_total` that ensures last user input is always accounted for"
    user = User.objects.get(id=user_id)
    monthly_total = [p.total for p in user.purchases.filter(month=datetime.utcnow().month, user=user)]
    user.update(last_modified=datetime.utcnow(), monthly_total=monthly_total)
    time.sleep(3)

# User triggers `calculate_monthly_total()` through various actions
task1 = calculate_monthly_total.delay(user_id=123)
task2 = calculate_monthly_total.delay(user_id=123) # throttled via singleton
time.sleep(1)
task3 = calculate_monthly_total.delay(user_id=123) # not throttled since the previous task has started
assert task1 == task2
assert task1 != task3

malthejorgensen avatar Jun 10 '22 11:06 malthejorgensen

Thank you for looking at #44!

@steinitzu Could you also look at this?

tony avatar Oct 02 '22 16:10 tony