Investigate the development of pruner component/service within PANIC

[Cherrett]

Rationale

We must investigate the development and use of a pruner component within PANIC. This component entails the following:

• Delete the keys of expired alerts from Redis
• Delete old data from MongoDB

The Redis key deletion is self-explanatory while the MongoDB data deletion can be set-up in various ways. One option would be to include an ENV variable with a given value such as 90, which would entail that the pruner should remove data older than 90 days from MongoDB. Moreover, we must investigate whether to include this within an existing component or whether this will be a new separate docker service within PANIC.

For ticket closure

Scenario: The new component/service is started Given: The new component/service started When: Checking the log files for said component/service Then: You can see correct startup

Scenario: The new component/service has no error logs Given: The new component/service is running When: Checking the log files for said component/service Then: You can see no error logs

Scenario: The new component/service is removing alerts from Redis correctly Given: Redis is up and contains expired alerts When: Data to be removed exists within Redis Then: You can no longer see the expired alerts in Redis

Scenario: The new component/service is removing old data from MongoDB correctly Given: MongoDB is up and contains old data When: Data to be removed exists within MongoDB Then: You can no longer see the old data in MongoDB

[dillu24]

Another functionality that we may include here is logs pruning. We can explore different avenues such as:

• Deleting files which were created/modified X days ago
• Deleting logs from files which were created/modified X days ago. This would require us to also delete empty log files.

[dillu24]

The outcome of this investigation should be to have a ticket which tackles the development of a pruner.