pyronear
[storage] Set up automatic deletion of old media
For now, the media that are uploaded to the bucket are stored indefinitely. But in the long-term, we need to erase those to avoid filling up the disk.
If that's alright with everyone, I suggest setting a MEDIA_PERSISTENCY value in the config (default 30-60days).
What do you think? cc @jeanpasquier75 @florianriche @fe51
Hi there,
This is indeed a good idea ! Moreover, with regard to the CNIL, it is important to define a storage period for our images.
Concretely, how do you want to implement it ?Is the concept to set up a CRON on a backend script deleting old media ?
About the storage period, is 30 days enough time to potentially do training sessions on vision side ?
Good question, I'm not sure how we could do this :man_shrugging:
We'll let the vision team know, but let's work with the maximum legal duration allowed to give them the best flexibility :+1:
FYI, moving this to a later release, since the problem won't be critical before our first live alerts
FYI found a service that could help us do that https://posthook.io/ (found here https://free-for.dev/, sharing because it's nice)
Wrapping my way around this, apart from our current process of doing it manually, the only viable way we could do this is with a cron job :)
Not a priority since we have limited traffic, I'll move this to a later release
I'd like a second opinion on this @MateoLostanlen but I think we can close this? (Those images are expected to be kept for at least a year for fire fighters, but it's older, the binary image may be removed from the bucket but we keep the "detection" and "sequence" record right?)