make `ore openstack gc` clean up SSH keypairs
We've been bumping up into quota limits for SSH keypairs in VexxHost (our openstack provider for kola testing) and I didn't quite understand why because the web interface doesn't show me any keypairs so I didn't think any existed. However checking via the CLI I see a lot. So somehow sometimes keypairs aren't getting cleaned up. We could/should make our GC that we run in the pipeline just detect when stale keypairs exist and delete them too. I think we can just filter by keypairs with a name starting with kola- and created before a certain time:
$ openstack keypair show kola-09efd27b-7512-4a51-b795-24bb82eafdf2
+-------------+-------------------------------------------------+
| Field | Value |
+-------------+-------------------------------------------------+
| created_at | 2023-08-01T03:21:24.000000 |
| fingerprint | 1b:de:08:1c:d4:c1:0f:91:9d:e9:8b:d6:f5:e8:1d:26 |
| id | kola-09efd27b-7512-4a51-b795-24bb82eafdf2 |
| is_deleted | False |
| name | kola-09efd27b-7512-4a51-b795-24bb82eafdf2 |
| private_key | None |
| type | ssh |
| user_id | c6dffb4547d04cd2bd7ed672c02e2f6d |
+-------------+-------------------------------------------------+
For now I unblocked things by running:
$ export OS_REGION_NAME="ca-ymq-1"
$ openstack keypair list -f json | jq -r .[].Name | grep kola | xargs openstack keypair delete
I've been looking at how to implement this and I'm wondering if the kola-openstack job runs garbage collection at all right now? Is it run by default or do we have to specify? I see that in kola-gcp and kola-azure we are running the garbage collection stages manually, but not in kola-openstack. I can open a PR to run GC in openstack if it needs to be specified manually (i'm assuming it does)
In my mind I had cross linked this ticket back here when I opened it, but in practice that didn't happen. See
- https://github.com/coreos/fedora-coreos-pipeline/issues/900
Ah, perfect. I'll work on that.