The right stuff: populate fleetctl preview with starter scripts, policies, and queries

[noahtalerman]

Goal

User story
As a new Fleet user trying Fleet for the first time,
I want my fleetctl preview instance to be pre-populated with resources like queries and policies
so that I can more easily understand Fleet's value.

Key result

Q2 OKR: The right stuff

Original requests

• fleetdm/confidential#10153

Context

• Product Designer: @noahtalerman
• Engineer: @lukeheath

Changes

Product

• [ ] UI changes: No changes
• [ ] CLI (fleetctl) usage changes:
  • ~~Make fleetctl preview instances Fleet Premium. To do this, we'll Iaunch Fleet with the --dev-license flag when using fleetctl preview.~~
    • @noahtalerman: fleetdm.com/try-fleet already gives you a license!
  • fleetctl preview instances get these starter queries, policies, scripts, and teams (if there's a Fleet Premium license key).
• [ ] YAML changes: No changes
• [ ] REST API changes: No changes
• [ ] Fleet's agent (fleetd) changes: No changes
• [ ] GitOps changes: No changes
• [ ] Activity changes: No changes
• [ ] Permissions changes: No changes
• [ ] Changes to paid features or tiers: No changes
• [ ] My device and fleetdm.com/better changes: No changes
• [ ] First draft of test plan added
• [ ] Other reference documentation changes: No changes
• [ ] Once shipped, requester has been notified
• [ ] Once shipped, dogfooding issue has been filed

Engineering

• [x] Test plan is finalized
• [x] Contributor API changes: No changes
• [x] Feature guide changes: No changes
• [x] Database schema migrations: No changes
• [x] Load testing: No changes

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

• Requires load testing: No
• Risk level: Low

Test plan

Make sure to go through the list and consider all events that might be related to this story, so we catch edge cases earlier.

• [ ] Spin up a new fleetctl preview instance. Make sure the queries, policies, and scripts are added.
• [ ] Spin up a new fleetctl preview instance with a Fleet Premium license key. Make sure the queries, policies, scripts, and teams are added.
• [ ] After a couple minutes make sure the query reports show some data.

Testing notes

Confirmation

1. [ ] Engineer: Added comment to user story confirming successful completion of test plan.
2. [ ] QA: Added comment to user story confirming successful completion of test plan.

[noahtalerman]

@lukeheath we cut fleetctl preview from the first iteration (user story) so I filed this follow up story to track updating fleetctl preview.

Is this a story you can help build?

[lukeheath]

@noahtalerman Sure I'll take a shot at it.

[noahtalerman]

@lukeheath great! I assigned this one to you and moved it to the "Ready" column.

[lukeheath]

@noahtalerman Currently fleetctl preview does populate with 50 queries and 44 policies. Do you have something else in mind?

[noahtalerman]

Currently fleetctl preview does populate with 50 queries and 44 policies. Do you have something else in mind?

@lukeheath that's correct. fleetctl preview gets the standard query library.

Instead, I think we want fleetctl preview to get our new starter library. So we get scripts, queries, and policies.

[lukeheath]

@noahtalerman fleetctl preview currently launches in free mode, so if we load the starter library it will only receive a single query. If we want to use the starter library, we'd need to also make fleetctl preview premium. If we want to do that, an easy approach would be to launch Fleet with the --dev-license flag when using fleetctl preview. We can package all of that into this issue if you like.

[noahtalerman]

we'd need to also make fleetctl preview premium. If we want to do that, an easy approach would be to launch Fleet with the --dev-license flag when using fleetctl preview. We can package all of that into this issue if you like.

@lukeheath ah, that's right. I think let's do this! I updated the issue description to reflect this.

[lukeheath]

@noahtalerman I'm out next week but will tackle this when I return.

[noahtalerman]

I'm out next week but will tackle this when I return.

Sweet! Thank you.

[noahtalerman]

• ~~Make fleetctl preview instances Fleet Premium. To do this, we'll Iaunch Fleet with the --dev-license flag when using fleetctl preview.~~
  • @noahtalerman: fleetdm.com/try-fleet already gives you a license!

@lukeheath if you visit fleetdm.com/try-fleet, you get a license today! I forgot. Whoops from me.

I think this means that, for this story, we only have to update fleetctl preview to use our new starter library. And verify that the teams, scripts, queries, and policies show up.

[rachaelshaw]

@lukeheath do you think we'll ship this in 4.73?

[lukeheath]

@rachaelshaw Actually I think I can get this into 4.71.0 (merge to main by Monday). One question for you from a design perspective:

Currently, fleetctl preview loads in our standard query library (~40 queries and policies). Do we want to leave that in place for fleetctl preview free users, and only apply the starter library if they're running fleetctl preview with a premium license? The alternative would be to install the starter library for both free and premium, but because the starter library is primarily organized by team, the end result is the fleetctl preview free version would have one query and that's it.

[rachaelshaw]

@lukeheath I'd go with just including the one query; I feel like including >1 page of queries makes Fleet look harder to use. (Maybe we could follow up later with a separate, small library of queries to include for free users?)

[lukeheath]

@rachaelshaw That's a good point. While it does a good job showing all the things Fleet can do with queries and policies, it is also a bit daunting for someone who hasn't used Fleet before. It's also easier to implement just one flow for free and premium, so win win! Thanks for the feedback.

[lukeheath]

@sharon-fdm Please bring this into your release board tomorrow with no estimate. I've already built it, just putting through the Orchestration board for QA: https://github.com/fleetdm/fleet/pull/30519. PR is still in draft as I finalize, but I'll request review later this week.

[AndreyKizimenko]

Hi @noahtalerman, I just finished my test pass, and noticed that Fleet Free doesn't come with any script pre-uploaded. Same goes for the no-team team on Fleet premium.

starter-library.yml has this section on the top, but it doesn't seem to apply scripts globally as expected. Should we make a separate no-team team?

apiVersion: v1
kind: config
spec:
  scripts:
    - it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh
    - it-and-security/lib/windows/scripts/uninstall-fleetd-windows.ps1
    - it-and-security/lib/linux/scripts/uninstall-fleetd-linux.sh
---

This might be because you can't upload scripts for All teams and need to explicitly select No team

[AndreyKizimenko]

Additionally, the only policies that we supply are assigned to Workstations, meaning that Fleet Free won't see any policies either.

[noahtalerman]

I just finished my test pass, and noticed that Fleet Free doesn't come with any script pre-uploaded. Same goes for the no-team team on Fleet premium.

@AndreyKizimenko great catch!

I think that's ok. Let's prioritize Fleet Premium over Fleet Free.

Ideally, they'd show up for Fleet but I'm not sure how to do that. It would take me 30+ mins to dig in. If you know how, go for it!

For now I opened a PR to remove that part of the YAML that doesn't work and moved this issue back to in progress.

the only policies that we supply are assigned to Workstations, meaning that Fleet Free won't see any policies either.

This is intentional. We're prioritizing Fleet Premium over Fleet Free.

"All teams" policies are inherited. And, right now, there are no policies we know of that are applicable to all of Fleet's best practice teams (computers, servers, iphone, etc.)

[noahtalerman]

FYI @lukeheath ^

[AndreyKizimenko]

Thank you!

[noahtalerman]

FYI @AndreyKizimenko I merged the PR that removes the scripts that don't work. Moved this back to "Awaiting QA"

[AndreyKizimenko]

Confirmed that everything still works as before, premium has access to script, policies and queries. Moving to ready for release

[noahtalerman]

@AndreyKizimenko I just tried dogfooding this myself by creating a dummy account on fleetdm.com and grabbing my trial license key on fleetdm.com/try-fleet (part of the fleetdm.com/start flow).

Here's my command:

~/.fleetctl/fleetctl preview --license-key eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJGbGVldCBEZXZpY2UgTWFuYWdlbWVudCBJbmMuIiwiZXhwIjoxNzU4Mzc3MjkxLCJzdWIiOiJGbGVldCIsImRldmljZXMiOjEwLCJub3RlIjoiQ3JlYXRlZCB3aXRoIEZsZWV0IExpY2Vuc2Uga2V5IGRpc3BlbnNlciIsInRpZXIiOiJwcmVtaXVtIiwiaWF0IjoxNzU1Nzg1MjkxfQ.1HqGi-D50AN59cNC4J0Mxy_cnntf4zyC_fXF6a2Eq5DiBMgJTB6q3QyG8CVL1ojtHQPj9hrg06PXWBqe45wMcQ

I see this error when fleetctl preview doesn't start up:

Error: downloading orbit and osqueryd: initialize updates: failed to update metadata: update metadata: tuf: failed to decode timestamp.json: tuf: valid signatures did not meet threshold

I even tried uninstall the fleetd agent from my machine to put myself in the shows of someone trying Fleet for the first time.

Have you seen this error before?

Can you please help me try to repro? Go through my steps of creating a dummy account and grabbing your command from fleetdm.com/try-fleet. If you can, can you please help me file a bug?

Here's the full output:

Pulling Docker dependencies...
Starting Docker containers...
Waiting for server to start up...
Initializing server...
Configured fleetctl in the 'preview' context to avoid overwriting existing config.
Loading standard query library...
[+] applied 50 queries
[+] applied 44 policies
Fleet will now log you into the UI automatically.
You can also open the UI at this URL: http://localhost:1337/previewlogin.
Email: [email protected]
Password: preview1337#
Enrolling local host...
Trying to clear orbit and osquery directories...
Error: downloading orbit and osqueryd: initialize updates: failed to update metadata: update metadata: tuf: failed to decode timestamp.json: tuf: valid signatures did not meet threshold

[noahtalerman]

Can you please help me try to repro? Go through my steps of creating a dummy account and grabbing your command from fleetdm.com/try-fleet. If you can, can you please help me file a bug?

Moved this back to the "Ready for QA" column on the #g-orchestration board.

FYI @sharon-fdm @AndreyKizimenko

[lukeheath]

@noahtalerman That's strange. That error is from TUF and would mean our timestamp is broken, which would be breaking agent updates. I ran fleetctl preview and it launched as expected. When I went to fleetdm.com/try-fleet to try it with a license key, I saw that the license key we're providing has expired. I filed a P1 bug and will look into this further once that is resolved.

[noahtalerman]

@lukeheath @AndreyKizimenko I just tried again and fleetctl preview with my premium license worked ✅

That said, we're throwing a lot of noisy logs in the output. I filed a bug to track this: https://github.com/fleetdm/fleet/issues/32208

Moved this story back to "Confirm and celebrate" on the drafting board.

Downloading dependencies from fleetdm/fleet:main into /Users/noahtalerman/.fleet/preview...
Pulling Docker dependencies...
Starting Docker containers...
Waiting for server to start up...
Initializing server...
Configured fleetctl in the 'preview' context to avoid overwriting existing config.
Loading starter library...
level=debug msg="Applying starter library"
level=debug msg="Created temporary directory for scripts" path=/var/folders/89/_k0w_2rj4v93n2v6lbpl3bnw0000gn/T/fleet-scripts-2523422851
level=debug msg="Found script references in starter library" count=3
level=debug msg="Downloading script" name=it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh url=https://raw.githubusercontent.com/fleetdm/fleet/main//it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh local_path=/var/folders/89/_k0w_2rj4v93n2v6lbpl3bnw0000gn/T/fleet-scripts-2523422851/it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh
level=debug msg="Downloading script" name=it-and-security/lib/windows/scripts/uninstall-fleetd-windows.ps1 url=https://raw.githubusercontent.com/fleetdm/fleet/main//it-and-security/lib/windows/scripts/uninstall-fleetd-windows.ps1 local_path=/var/folders/89/_k0w_2rj4v93n2v6lbpl3bnw0000gn/T/fleet-scripts-2523422851/it-and-security/lib/windows/scripts/uninstall-fleetd-windows.ps1
level=debug msg="Downloading script" name=it-and-security/lib/linux/scripts/uninstall-fleetd-linux.sh url=https://raw.githubusercontent.com/fleetdm/fleet/main//it-and-security/lib/linux/scripts/uninstall-fleetd-linux.sh local_path=/var/folders/89/_k0w_2rj4v93n2v6lbpl3bnw0000gn/T/fleet-scripts-2523422851/it-and-security/lib/linux/scripts/uninstall-fleetd-linux.sh
level=debug msg="Starter library applied successfully"
Fleet will now log you into the UI automatically.
You can also open the UI at this URL: http://localhost:1337/previewlogin.
Email: [email protected]
Password: preview1337#
Enrolling local host...
Trying to clear orbit and osquery directories...
{"level":"info","time":"2025-08-22T10:25:09-04:00","message":"initialize TUF from embedded root keys"}
{"level":"debug","error":"stat /Users/noahtalerman/.fleet/preview/orbit/bin/osqueryd/macos-app/stable/osqueryd.app.tar.gz: no such file or directory","time":"2025-08-22T10:25:10-04:00","message":"stat file"}
{"level":"debug","error":"stat /Users/noahtalerman/.fleet/preview/orbit/bin/orbit/macos/stable/orbit: no such file or directory","time":"2025-08-22T10:25:12-04:00","message":"stat file"}
Waiting for host to enroll...
Starting simulated Linux hosts...
Preview environment complete. Enjoy using Fleet!

[fleet-release]

Preview blooms with scripts, Policies, teams, and queries, Ease for Fleet's new seeds.

[lukeheath]

@noahtalerman Cool I tried and it worked for me, as well: