elastic
Add test support in for explicit release versions tests comparing Agent + Stack versions
some discussion is in this ticket, work is not finished yet for the desired behavior but it is coming we expect for 7.10. I suggest a new feature file that cites the specific versions and can look like the comment here:
https://github.com/elastic/kibana/issues/75176#issuecomment-680921289
point in time copy/paste: section 1) very close versions, only patch release differences - all combos should work Agent a.b.1 < Kibana a.b.2 Agent a.b.2 > Kibana a.b.1
section 2) somewhat close versions, Agent minor release is 1 or 2 versions back and is older than the stack: Agent a.9.x < Kibana a.10.x Agent a.9.x < Kibana a.11.x
section 3) versions where Agent is a minor version or major version greater (newer) than the stack should not work: Agent 7.10.x > Kibana 7.9.x Agent 8.0.x > Kibana 7.9.x
@ruflin @ph does that look fully co
a few thoughts:
-
I'm not sure we can safely have the test introspect what 'base version' to use for the installs of Agent and stack, but if we can come up with an idea I'm open to it.
-
based on a set version (if not dynamically introspected), for now, like '7.9.0' it would be highly desired to key other test versions from it. Would we propose keeping this in the docker compose configuration file(s) we can create for this? Then we need to decide which tests to run (if not all of them).
-
if we can handle the case where the implied test version doesn't exist yet and we cancel / ignore the test failures it would be great! Otherwise, we can figure out what to do and how to code it up.
Doing this based off a given reference point version will be really helpful to avoid constant file version numbers maintenance, but we'll likely need to update the hardcoded version a few times a year, seems an acceptable place to start to me.
Doing an issue review, removing the P3 label - its up there with other fairly urgent concerns. If the 7.x Agent doesn't work with the 7.x+1 release it is a critical bug.