Enforce latest OS when macOS, iOS, and iPadOS hosts automatically enroll

[nonpunctual]

Goal

User story
As an IT admin,
I want to require end-users to have a minimum OS version installed on the host before they enroll to MDM (with ADE)
so that I can make my new hosts compliant before they are enrolled into production.

Context

• Requestor(s): @nonpunctual
• Product designer: @marko-lisica

Research

During MDM enrollment, the device will respond with MachineInfo to the server when fetching the enrollment profile. The server needs to check if the device is on specified minimum OS version, if not server sends 403 error

More info here (Create a New Device Management Connection > step 3)

Changes

Product

• [ ] UI changes: Figma wireframes here.
• [ ] Other changes:
  • Enforce the latest macOS, iOS, or iPadOS during Setup assistant, if the host is below the minimum version set in Fleet. Let the end user through setup assistant w/o an upgrade if their host is above the minimum version set in Fleet.
  • Fleet will get the latest macOS from Apple here (Public AssetSets). If the request to Apple fails, Fleet will try again up to 3 times. If all attempts fail, Fleet will let the end user through Setup Assistant w/o an OS update.
  • This is supported only for hosts running macOS 14+ or iOS/iPadOS 17+ that automatically enroll (DEP). Apple only supports this feature in these scenarios.
• [ ] Outdated documentation changes: Add info to OS updates page: this applies only to ADE enrollment and hosts that run macOS 14 / iOS/iPadOS 17 and above. Add one sentence that describes this limitation and experience in the macOS section of the page.

Engineering

• [ ] Database schema migrations: TODO
• [ ] Load testing: TODO

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

• Requires load testing: TODO
• Risk level: Low / High TODO
• Risk description: TODO

Manual testing steps

1. Step 1
2. Step 2
3. Step 3

Testing notes

Confirmation

1. [ ] Engineer (@____): Added comment to user story confirming successful completion of QA.
2. [ ] QA (@____): Added comment to user story confirming successful completion of QA.

[noahtalerman]

Thanks @nonpunctual. Really helpful link to Filewave docs.

We'll weight this at the next feature fest on 2024-06-20.

[nonpunctual]

Apple guide for "Managing software updates for IT" updated June 2024 per WWDC managing-software-updates-v1-0.pdf

[lashomb]

This would be really nice to replace some problematic lagacy policies we have to do this now.

[nonpunctual]

@lashomb added customer label to this issue. Thanks!

[marko-lisica]

@nonpunctual I converted the original issue description to user story format. Moving original here.

Apple has added a feature for denying enrollment if a device is not on a specified, minimum OS version.

Property from the schema documented at https://developer.apple.com/documentation/devicemanagement/machineinfo -

Property: MDM_CAN_REQUEST_SOFTWARE_UPDATE Type: boolean Value: If TRUE indicates that the (MDM) server can trigger the (enrolling) device to do a required software update. Available on iOS 17 and later, and macOS 14 and later. Default value: false

From https://github.com/apple/device-management/blob/release/mdm/errors/softwareupdate.required.yaml -

The schema for a JSON or property list XML document returned in an MDM server's 403 response body. The response headers must include a "Content-Type" header indicating whether JSON or XML is being returned.

This response is returned when a device is enrolling with an MDM server during Setup Assistant, and the MDM server requires the device to perform a software update before enrollment is allowed and setup can proceed.

In order for users to take advantage of this feature Fleet must be enabled to:

• recognize devices in the state of NOT matching the minimum specified OS version at MDM enrollment in the MachineInfo config
• send the correctly formatted error response to the enrolling device

Problem

Fleet is not currently enabled to deny MDM enrollment based on a specified minimum OS version.

Potential solutions

1. Add the ability to recognize the MachineInfo properties to Fleet MDM
2. Enable the response for devices that do not match a specified minimum OS version

Documentation for this feature in Filewave https://kb.filewave.com/books/apple-school-business-manager/page/minimum-os-version-for-enrolling-apple-devices-via-ade

[noahtalerman]

Hey @nonpunctual I pulled the ~feature fest off because this story is in the current design sprint :) You can tell if it has :product

Please check before adding requests to feature fest. Thanks!

[georgekarrv]

@roperzh when you get some time can you review this and make sure we have a good understanding of all of the steps needed to both identify and deliver this specific case? When that's done go ahead and pull this over to specified.

[roperzh]

This should apply only on hosts running macOS 14 and above or iOS/iPadOS 17 and above that are enrolling through ADE

~as I understand this flow is available for manual enrollments too, any reason to limit it to ADE?~

edit: I was wrong, seems like it's ADE only, sorry!

@marko-lisica or @nonpunctual (not sure who added this section)

[georgekarrv]

Hey team! Please add your planning poker estimate with Zenhub @dantecatalfamo @ghernandez345 @gillespi314 @jahzielv @mna @roperzh

[PezHub]

QA Notes: Identified a blocker - Summary: we can't force any version, it has to be one of the versions defined here. If you specify other version, the user can't continue through setup assistant (screenshot in the thread). This is a problem because we currently enforce the version configured by the IT admin for software updates.

Error -

We will work on a fix and delay release.

Good news - setting the minimum version to Apple's latest release (14.6) and hardcoding the BundleVersion (separate PR) does indeed work.

[noahtalerman]

@gillespi314, @marko-lisica, and I met during design review and we decided to pause on this feature and not ship it in the upcoming Fleet release.

Why? Currently, if the IT admin specifies a minimum macOS version that's not included in this public list here (ex. macOS 14.5), the end user won't be able to enroll (see error in the comment here). We don't want to IT admins to shoot themselves in the foot.

What's the plan?

• Send this story back to the drafting board and assign @noahtalerman
• Decide on next steps
  • Verify w/ Apple folks whether this is a bug that's going to be fixed or if this is the intended behavior. There's an older Slack thread in Mac Admins Slack about this (link TODO)
  • If it's a bug, then plan to ship the feature when Apple fix is out
  • If it's the intended behavior, update the designs and bring the issue back through estimation.

Any thoughts/concerns w/ this plan?

cc @lukeheath @georgekarrv @PezHub

[nonpunctual]

@noahtalerman @gillespi314 @marko-lisica @lukeheath @georgekarrv @PezHub @spokanemac @zayhanlon

This is a very important feature.

Why? Because organizations that have not yet built complex Device Health / Conditional Access workflows will get to take advantage of one of the most important checks at the most important time, i.e., they get to validate the minimum OS prior to enrollment. In many orgs, minimum OS is the only attribute that's strictly enforced.

This feature will temporarily take pressure off of design + eng on some of the more complicated FRs related to Conditional Access workflows.

It is part of an admin's job to validate the up-to-date OS versions on that list. There is a Slack RSS feed that delivers them in Mac Admins Slack or to email. They are now in the SOFA feed. Apple publishes them. Other 3rd party sources publish them. This is something most macOS admins check daily.

When we talk about "admins shooting themselves in the foot" I believe our standard should be something like "prevent admins from doing something that will impact the performance overhead of Fleet on the Hosts & on the server." This does not fall in that category of error.

I can also see using this in a postive way: Maybe you want to block all enrollments for some reason today? Well, you could set the minimum version higher than an actual extant version. That would be a lot easier to do than to disable all of your enrollment policies & workflows.

We should work under the assumption that admins using Fleet are being intentional & know what they are doing.

If you give someone a toolbox & they decide to hammer in nails with the monkey wrench, I don't think it's a good solution to that problem to take away the monkey wrench (which they still might need.) Give them all the tools & make it clear how they work.

[roperzh]

@noahtalerman, thanks to @PezHub and @gillespi314, we tried if we really can't provide any version by grabbing a bundle version for macOS 14.5 and trying the whole flow again and 100% confirmed that we can't use any other version except the ones listed under the PublicAssetSets key.

[roperzh]

@nonpunctual if an update is released and the IT admin doesn't catches it on time, then nobody can enroll via ADE until they're back again to fix it , this can be an update released just while the IT admin is in a long meeting, or (for global teams) in a different timezone, or maybe OoO.

I agree with your general premise, but this seems like a scenario where we're setting the IT admin for failure.

[lashomb]

There have and always will be gaps between OOBE provisioning and enforced settings by administrators. When new hardware comes out, it may not be on the version of the OS that you’ve set for the rest of the fleet. But that’s fine, we’ve dealt with that before. It’s not like we’re going to erase things back to an older version like the old days. Newly imaged devices always represent in someway an accumulation of improvements to our process over time, and it’s ok by me if a percentage of the environment is on a later OS than mandated by the org.

[nonpunctual]

Appreciate that feedback @lashomb !! Certainly the intent is not to set up for failure. I think we could build safeguards or even API checks against non-extant OS versions. My emphasis is only on getting the feature out. Thanks!

[noahtalerman]

I hear you @lashomb and @nonpunctual! We're working to ship an improvement ASAP but we don't want to ship something that doesn't work as we intended.

[noahtalerman]

Heads up @lukeheath, @georgekarrv, and @gillespi314, I unassigned Sarah and moved this issue from the release board to the drafting board.

[lukeheath]

@noahtalerman Since we're so close, it would be nice to get this over the finish line next release so we can deliver value to our users. My questions/thoughts:

1. I think this is by design and not an Apple bug. If this feature was introduced in 14.6, it likely doesn't work on any version prior to that. It's worth checking with Apple, but I think it's safe to operate under this assumption in order to get it out next release.

2. I'm not sure where the OS version is being set in this workflow. Presumably at some point it goes through our API. Could our API endpoint enforce the minimum version requirement of 14.6? If you try to set it to 14.5, it returns 400 Bad Request.

3. But what if someone wants 14.5? Can this feature be toggled? In other words, if the feature is enabled, the API enforces a minimum version of 14.6. If it's not enabled, it doesn't enforce any minimum.

[noahtalerman]

it would be nice to get this over the finish line next release

@lukeheath agreed.

The plan is to spend 30 mins on a design review call to decide on an approach. Thanks for the questions/thoughts. Super helpful for starting the conversation.

[noahtalerman]

100% confirmed that we can't use any other version except the ones listed under the PublicAssetSets key

Thanks @roperzh and team!

@PezHub, do we know if/how this affects the OS updates via DDM feature?

That is, if I set 14.5 as my minimum version via Fleet today, will an end user that's below 14.5 see OS update notifications and get updated to 14.5?

Will they get updated to 14.6? (available in PublicAssetSets)

Will they get updated at all?

[noahtalerman]

Brock: Not sure what other MDM solutions are doing b/c they may or not have this feature yet.

[noahtalerman]

Sarah: Here's the link I mentioned where Apple says enforcing a specific software update version no longer fails if it’s not the most recent available update (starting with devices on macOS 14.6)

[gillespi314]

do we know if/how this affects the OS updates via DDM feature?

@noahtalerman, DDM works as expected. The DDM updates aren't limited by the PublicAssetSets issue. So today, if the admin sets the minimum version to 14.5, a Sonoma device using ADE will enroll to Fleet successfully, and the minimum version will be enforced after enrollment and they'll see the OS update notifications and get updated to 14.5 according to the specified deadline.

[noahtalerman]

UPDATE:

New plan

The plan for this features (v1) is for Fleet to always, by default, enforce the latest macOS, iOS, and iPadOS during Setup Assistant.

Fleet will get the latest OS from Apple here (Public AssetSets). If the request to Apple fails, Fleet will try again up to 3 times. If all attempts fail, Fleet will let the end user through Setup Assistant w/o an OS update.

prospect-numa and customer-faltona confirmed that this is the desired behavior.

Later, we can get fancier and allow the IT admin to disable this behavior, enable it for specific platforms (ex. macOS only) and/or present the user with a list of supported versions.

Hey @georgekarrv I updated the issue description to reflect this, moved the story to ready for spec and assigned you.

Can you please work w/ @gillespi314 and the team to get this one re-estimated and pulled into the current sprint? Please let me know if including iOS/iPadOS changed the estimate drastically.

Old plan summary

The old plan was to use the existing "Minimum version" option and allow the IT admin to specify a specific macOS version that's enforced during Setup Assistant.

However, currently, Apple only supports versions included under PublicAssetSets here. This means the IT admin can shoot themselves in the foot and block end user from enrolling if they specify a version that isn't supported.

If folks have concerns/thoughts on the new approach please let us know!

cc @nonpunctual @lukeheath @lashomb @roperzh

[nonpunctual]

@noahtalerman I have already expressed my concern on this but I will again. We are hard-coding what versions of macOS admins can use by having built the Nudge implementation the way we have & now with this. We should allow admins to set a version. Thanks.

[lukeheath]

@nonpunctual Can you help me understand the use case where choosing a less-than-latest version during migration would be desirable?

@noahtalerman Have any users expressed concern about being forced into the latest version?

[noahtalerman]

@nonpunctual Can you help me understand the use case where choosing a less-than-latest version during migration would be desirable?

Yes please!

Have any users expressed concern about being forced into the latest version?

None so far. Thumbs up from prospect-numa and customer-faltona

[nonpunctual]

@noahtalerman it might not be the strongest case, but, Apple allows 90d from the offical OS release day to defer install. During that 90d period orgs might want to hold off on deploying the latest, greatest to test. Once the testing is satisfied, then the up-to-date version could be released.

[dherder]

@noahtalerman I agree with Brock's assessment here. While many enterprise orgs will want the latest and greatest, some will still want the option to defer the upgrade, especially if they are in highly regulated industries or if they have many users with legacy third party software applications that may break with the latest release.