Feature: Delete Tasks Older than a specified time - Delete All Tasks

[0mars]

Is your feature request related to a problem? Please describe. Due to GDPR, we have a requirement to delete tasks as they contain user info, so a specified time can be specified, for example we want to delete tasks older than 48hours

As shown the example below can be used to achieve such task

import celery
import os
import timedelta
import datetime

broker = os.environ.get('CELERY_BROKER_URL')
app = celery.Celery('tasks', broker=broker)
flower = Flower(capp=app, options=flower_options)

time_delta = timedelta(hours=48)
now = datetime.datetime.now()
delete_before_time = now-time_delta

flower.events.delete_tasks_by_time(delete_before_time.timestamp())

or

flower.events.delete_all_tasks()

Then we can a celery beat scheduler, that runs each hour, and delete the tasks

@app.task(queue='cleanup_tasks')
def clean_up_tasks():
    from flower.app import Flower

    time_delta = timedelta(hours=48)
    now = datetime.datetime.now()
    delete_before_time = now-time_delta
    flower_options = object()
    flower_options.db = 'flower'
    flower_options.persistent = True
    flower_options.purge_offline_workers = 1
    # todo: use env vars
    flower = Flower(capp=app, options=flower_options)
    flower.events.delete_tasks_by_time(delete_before_time.timestamp())

@app.on_after_configure.connect
def add_periodic(**kwargs):
    app.add_periodic_task(crontab(hour="*", minute=0), clean_up_tasks.s(), name='cleanup-tasks')

Describe the solution you'd like The solution is based on some core function deleting tasks from the sync, while the other delete by time, checks the timestamp of each task, and delete the corrosponding task logs

[0mars]

#1188 please review the submitted PR as it holds the solution in the core of flower

as specified in the description:

Events:
  None delete_tasks_by_time(int to_timestamp)
  None delete_all_tasks()

[neosavvy]

Is there a merged solution to this problem - it sure would be nice to have.

[denisolyanyuk]

I created a PR with same feature but different approach. I believe it's better because the only thing you need is to add one config object and that's all. Take a look https://github.com/mher/flower/pull/1310