metal-cli icon indicating copy to clipboard operation
metal-cli copied to clipboard

E2E tests (run in CI) are not cleaning up server and project artifacts

Open displague opened this issue 2 years ago • 4 comments

What happened?

Servers created through E2E testing are not being deleted after the tests complete.

The project name is the same in each test so it is difficult to say which test triggered the project (and to identify if the devices were orphaned as a result of a failed run).

It is possible that an attempt was made to delete the servers and this failed (on the API side). This would block the deletion of the project.

How can we reproduce it?

Ensure that deletion is always carried out on the servers in the project and the project.

Create test servers with a deletion value so that they are automatically reaped.

CLI version (metal --version):

displague avatar Sep 06 '23 16:09 displague

Here is how that is done in TF:

  • example: https://github.com/equinix/terraform-provider-equinix/blob/main/equinix/resource_metal_device_acc_test.go#L671
  • field use: https://github.com/equinix/terraform-provider-equinix/blob/4825cd259b01bff64074cd1bdf78b6ad04f7a8cb/equinix/resource_metal_device.go#L518-L524

displague avatar Sep 08 '23 13:09 displague

Additionally:

  • Each project, device or any resource should have a random generated uuid append ex: metal-cli-project-xxxxx
  • Need to add t.Helper() in each test helper in helper.go
  • Use t.Cleanup() for declaring the cleanups of the resources created. Suggested here https://github.com/equinix/metal-cli/pull/376#discussion_r1395847583
  • Each resource created, should have a cleanup method called in the test

aayushrangwala avatar Nov 24 '23 18:11 aayushrangwala

/assign @aayushrangwala

aayushrangwala avatar Nov 24 '23 18:11 aayushrangwala

E2E test runs are still leaving resources behind, even after #405 was merged (and it should be noted that they're also taking far longer to complete).

ctreatma avatar Dec 18 '23 19:12 ctreatma