Filtering old data out of samples
What does this PR do?
On especially large clusters, we sometimes see a significant number of old/unused resources such as:
- Jobs that have finished, but are retained
- Pods that are finished (often as a result of the above), but are retained
- Replicasets that have been cycled out, and are not in use
This data is not useful to us when no active pods are running, and there is no reason to add it to the export or process it on the Apptio side. This PR removes the sending of data in these cases.
Unfortunately this cannot be easily filtered out of the informer stored data (would likely require building our own watchers plus other behind the scenes components), but that is a potential future optimization for agent utilization on larger clusters.
Where should the reviewer start?
How should this be manually tested?
Ran on a test cluster, validated sample data did not contain:
- Pods that were terminated
- Replicasets with 0 replicas
- jobs that had completed a while ago
Which were all present in previous samples.
Any background context you want to provide?
Large clusters tend to orphan more resources, currently we package, upload and store a bunch of stuff that's useless to us.
What picture best describes this PR (optional but encouraged)?
What are the relevant Github Issues?
Developer Done List
- [ ] Tests Added/Updated
- [ ] Updated README.md
- [ ] Verified backward compatible
- [ ] Verified database migrations will not be catastrophic
- [ ] Considered Security, Availability and Confidentiality
For the Reviewer:
By approving this PR, the reviewer acknowledges that they have checked all items in this done list.
Reviewer/Approval Done List
- [ ] Tests Pass Locally
- [ ] CI Build Passes
- [ ] Verified README.md is updated
- [ ] Verified changes are backward compatible
- [ ] Reviewed impact to Security, Availability and Confidentiality (if issue found, add comments and request changes)