[Agent][Fleet] Add support for environmental, architecture, packaging permutations (different params / variables) to cover Agent / Endpoint support matrix certification

[EricDavisX]

This is in support of integrations-dev #204 and its original issue: integrations-dev #22 a new test addition is also the Red Hat UBI image: https://github.com/elastic/e2e-testing/issues/239

we can add these individually if needed, but I would prefer and will attempt to design some system that removes as much maintenance needs as possible.

Procuring all of the Docker images for the different environments may be a separate part of the chore.

Thoughts on the feature file + implementation code:

• At a minimum we could start a new feature file that has all of the environmental support. If needed, it may be easier to start a new folder for the feature file (if it makes it easier to 'skip' them when not desired, like during every pr or every night).

• In the new file, we can link to the main Ingest feature file section that has Agent and Endpoint deploy (which is nearly merged!) for good reference.

• We could abstract the Go implementation side code into just a few feature file steps, with a parameterization akin to: Given an Agent is deployed to "" and Endpoint "is / isnt" included in Agent policy Then the "Elastic Agent" process is "running" on the host Then the "Metricbeat" process is "running" on the host Then the "Filebeat" process is "running" on the host And the "Elastic Endpoint" process is "running" on the host And Agent is unenrolled from Fleet And the "Elastic Agent" process is "running" on the host And the "Metricbeat" process is "stopped" on the host And the "Filebeat" process is "stopped" on the host And the "Elastic Endpoint" process is "stopped" on the host

We could go a little deeper with validating the data shows up, but that may need to be worked on at the feature side first. Starting with this would be helpful.

update April 2021: we have a code base in place that should support adding more 'example' blocks to all of our tests, one OS or one artifact at a time. I'd like to see something added that would add to our list of covered artifacts (we don't have tests for all artifacts yet). I'd particularly like to see x86 deb / rpm / .tar.gz tests with some OS (even if it is just centos and debian again).

what we have today, for example, for Agent & Endpoint integration test:

If we could expand on this to add: @x86_centos_tar Examples: x86_centos_tar | os | | x86_centos_tar |

@x86_centos_rpm Examples: x86_centos_rpm | os | | x86_centos_rpm |

@x86_debian_deb Examples: x86_debian_deb | os | | x86_debian_deb |

Then the work is decided which tests (if not all) will get this citation. We can plan on nightly runs having all variations, but PR CI can focus only on just the current Centos and/or Debian tests like they do.

[EricDavisX]

the above PR has an example of how we could add in more support for different environment / OSes to run a regimen of tests against them.

[EricDavisX]

In another ticket the open question as to how much of retesting do we need to do with various variables in the system came up. Like for methods of install 1) enroll vs install, 2) container usage vs standard host vm, 3) deb vs tar package, amd vs arm, and combinations of architecture and packaging and (thankfully we don't have to worry about x86 anymore as support was discontinued before we released).

From one side we need to know the technical / architectural reasons how the risks manifest with the different variables, so we know how to test, and then we need to figure out a way to run the same basic test in X number of ways and configurations with possible different results based on configuration at hand. That complex test running desire isn't new, and the above ideas here may yet be a start to a potential solution.

to acknowledge, for the last year+, the plan was just to repeat the tests with some minimal human readable feature files as needed, copy and paste saves us (for now). it is manageable, but sub-optimal. I'm open to discussion on this here, but it hasn't been calm enough to prioritize that (from my side at least).

Note, that we are close to having support for standard linux (and windows!) vms as the install base for the Agent during a given scenario. Yay.