Add "supported fleetd version" to release notes to indicate that a certain version of Fleet works with a certain version of fleetd

[Patagonia121]

In release notes, make it clear which versions of Fleet are expected to work with which versions of fleetd..

Abstract: A customer would like to take advantage of a particular feature in Fleet e.g. scripting, but it's not clear by looking in the release notes which version of fleetd needs to be implemented to actually use said feature

[noahtalerman]

Kathy: Each Fleet release, clarify which version of fleetd was released with it?

Luke: Process change for how we update the release notes. Assign this to me.

[noahtalerman]

TODO @noahtalerman: Schedule call w/ Lucas, Reed, and Sharon to brainstorm cost of testing.

Quick win, document what versions of fleetd we're testing. N-1?

[noahtalerman]

@lucasmrod, @sharon-fdm, and @xpkoala and I met and landed on the following:

Decisions:

• Document this: The latest version of the Fleet server supports the latest version of fleetd. While older versions of fleetd may still function with newer versions of the Fleet server (and vice versa), Fleet does not actively test these scenarios and does not pursue bugs for them.
  • TODO Noah: Document this
• Include this in the release article and/or CHANGELOG: the new fleetd version that corresponds to the new version of Fleet.
  • TODO Noah: Get feedback/thoughts form @lukeheath and @spokanemac

cc @ksatter and @pacamaster

Background:

There's several types of customers:

• Self-managed w/ fleetd that has auto-updates on
  • Best practice:
    • Update the Fleet server when a new version is released.
    • If you're slower to upgrade Fleet don't worry. We follow SEMVER and don’t make breaking changes to fleetd unless we release a major version.
    • So any new fleetd release will continue to sends scheduled query results and reports host vitals. New features in Fleet may not work until you upgrade.
• Self-managed w/ fleetd that has auto-updates off
  • Best practice:
    • When you’re getting ready to update Fleet, you should also update fleetd to the corresponding version.
• Cloud-managed w/ fleetd that auto-updates
  • Best practice:
    • Sit back and relax. Fleet handles upgrading these customers to latest Fleet server and the latest fleetd.

[noahtalerman]

• Include this in the release article and/or CHANGELOG: the new fleetd version that corresponds to the new version of Fleet.
• TODO Noah: Get feedback/thoughts form @lukeheath and @spokanemac

Luke and JD, what do you think?

[noahtalerman]

@spokanemac please see above ^^ (I forgot to tag your correct GitHub username)

[spokanemac]

@noahtalerman Adding this to the changelog and release notes shouldn't be a problem. I am guessing we would also want to link folks to the agent update instructions. Are those instructions current?

An additional FR would possibly include the installed fleetd version on the host summary page and possibly a warning that it's out of date.

[noahtalerman]

I am guessing we would also want to link folks to the agent update instructions.

@spokanemac I think we can hold off on linking to that page. It's an overview for managing fleetd updates (setting up TUF server etc.) and not necessarily the most helpful for someone reading the release notes.

I think we can assume that customers who manage their own fleetd updates will know how to update.

An additional FR would possibly include the installed fleetd version on the host summary page and possibly a warning that it's out of date.

Really good idea here. Please feel free to file a feature request for this and bring it to feature fest.

[lukeheath]

@noahtalerman Thanks for putting this together! Overall, it looks great. My only thought is regarding this:

While older versions of fleetd may still function with newer versions of the Fleet server (and vice versa), Fleet does not actively test these scenarios and does not pursue bugs for them.

We need to keep in mind that because fleetd auto-updates, it's important that newer versions of fleetd work with older versions of Fleet. Right now, fleet-v4.x.x corresponds with orbit-v1.x.x, so I think as long as those major versions match, newer versions of fleetd should work with older versions of Fleet. As you pointed out, some newer features won't be available, but we won't break existing functionality.

At some point, it would probably make sense for us to jump Orbit to orbit-v4.x.x. It won't match exactly because the release cadences aren't the same, but at least the major versions would match.

As for referencing the version, let's list it on the Fleet release page. I'll put in a PR to update the template we use for that. How should we list it? We have the following latest versions:

1. orbit-v1.22.0
2. Fleet Desktop always matches Orbit, so 1.22.0. It does not have its own GitHub tag.
3. fleetd-chrome-v1.2.0

[noahtalerman]

As for referencing the version, let's list it on the Fleet release page. I'll put in a PR to update the template we use for that. How should we list it?

@lukeheath makes sense to put it on the releases page.

I think we can add a new "Fleet's agent" H3 (below "Changes") with the callout about newer versions of fleetd:

### Fleet's agent

The following version of Fleet's agent (fleetd) support the latest changes to Fleet:

1. [orbit-v1.22.0](https://github.com/fleetdm/fleet/releases/tag/orbit-v1.22.0)
2. Fleet Desktop 1.22.0 (does not have its own GitHub tag). 
3. [fleetd-chrome-v1.2.0](https://github.com/fleetdm/fleet/releases/tag/fleetd-chrome-v1.2.0)

> While newer versions of fleetd still function with older versions of the Fleet server (and vice versa), Fleet does not actively test these scenarios and some newer features won't be available.

What do you think?

We need to keep in mind that because fleetd auto-updates, it's important that newer versions of fleetd work with older versions of Fleet.

@lukeheath Ah, that's right.

Sounds like it's important to emphasize newer fleetd + older Fleet works and the bit about new features may not work.

I tweaked the language. See above.

it would probably make sense for us to jump Orbit to orbit-v4.x.x. It won't match exactly because the release cadences aren't the same, but at least the major versions would match.

💯 Agreed. Maybe we can take it a step further and match them exactly even if there are no changes to Orbit.

This would mean we would have one version for Fleet and the agent and make it easier for customers to know which version agent corresponds to Fleet.

[lukeheath]

@noahtalerman Re: release language sounds good to me. I'll update the template.

Maybe we can take it a step further and match them exactly even if there are no changes to Orbit.

This would mean we would have one version for Fleet and the agent and make it easier for customers to know which version agent corresponds to Fleet.

Yeah, I can see the value there. The downside is it would create additional overhead to create releases with no changes. This would include generating the tags, conducting release QA, pushing to edge, then promoting to stable. So there's some cost in engineering time, but it would be worth it if the benefit to customers is high enough.

My question would be if using our TUF server with auto-updates enabled is our recommended best practice, do we want to invest time in maintaining a process that is not a best practice? A simpler rule might be "The best version of fleetd is always the latest."

Happy to have the conversation if you'd like! Feel free to put some time on my calendar if you want to dig deeper into that topic.

[noahtalerman]

The downside is it would create additional overhead to create releases with no changes.

Makes sense. I think it's more of a "for the future" thing.

if using our TUF server with auto-updates enabled is our recommended best practice, do we want to invest time in maintaining a process that is not a best practice?

We'd like to support both workflows: using our TUF server and bringing your own TUF server. Both have their own best practice.

Sorry, my original message didn't make this clear.

The best version of fleetd is always the latest.

I like the simplicity here! I think we can use this when chatting w/ customers/users.

I still think we want to specify a fleetd version in the release notes.

This way, it's clear to customers that manage their own TUF server which version of fleetd they need to use the latest features.

cc @ksatter @lucasmrod

[noahtalerman]

@Patagonia121 heads up, we're now specifying the fleetd version associated w/ each Fleet release (minor releases).

This way, customers who manage fleetd updates on their own (auto-updates off) know which version of fleetd to upgrade to if they want to use the latest features in Fleet.

For example an, here's what this looks like in the 4.47 release notes:

[fleet-release]

Notes guide as stars, Fleet versions in harmony, Scripting path made clear.