e2e testing enhancements for upgrades and installs
We need to create or validate e2e testing for installs and upgrades for the following cases:
Positive Test Cases (Should Pass) (most of these are already done using provided image field, need to make sure we have similar tests using the API field):
new cluster New Cluster install with valid customer provided image field New cluster install with valid customer provided cockroachVersion using provided env variable New cluster install with valid customer provided cockroachVersion using valid provided customer overriden env variable
existing cluster Patch upgrade customer provided image field Patch upgrade with customer provided cockroachVersion using provided env variable Patch upgrade with customer provided cockroachVersion using provided customer overriden env variable Feature upgrade customer provided image field Feature upgrade with customer provided cockroachVersion using provided env variable Feature upgrade with customer provided cockroachVersion using provided customer overriden env variable
Negative cases we need to test (expected fails):
new cluster New cluster install with bad customer provided image field (can't pull) New cluster install with bad customer provided cockroachVersion (can't pull) New cluster install with valid cockroachVersion using bad customer overriden env variable (can't pull) New cluster install with valid cockroachVersion using invalid customer overriden env variable (can pull - nginx example) New cluster install with invalid customer provided image field (can pull - nginx example) New cluster install with invalid customer provided cockroachVersion
existing cluster Upgrade with bad customer provided image field (can't pull) Upgrade with invalid customer provided image field (can pull - nginx example) Upgrade with valid cockroachVersion using bad customer overriden env variable (can't pull) Upgrade with valid cockroachVersion using invalid customer overriden env variable (can pull - nginx example) Upgrade with invalid customer provided cockroachVersion Upgrade with everything valid except customer is trying to skip a feature release (19.2 directly to 20.2) Upgrade with everything valid except customer hasn't finalized previous upgrade