fleet icon indicating copy to clipboard operation
fleet copied to clipboard

Variable substitution in scripts

Open dherder opened this issue 1 year ago • 27 comments

  • prospect-nishiyama: Gong snippet: https://us-65885.app.gong.io/call?id=620458040443628606&highlights=%5B%7B%22type%22%3A%22SHARE%22%2C%22from%22%3A625%2C%22to%22%3A888%7D%5D
  • customer-hubble: Gong snippet: https://us-65885.app.gong.io/call?id=3446530673982703525
  • prospect-interkosmos: Gong snippet:https://us-65885.app.gong.io/call?id=5145239856872997690&highlights=%5B%7B%22type%22%3A%22SHARE%22%2C%22from%22%3A2613%2C%22to%22%3A2862%7D%5D
  • prospect-pingouin:
    • Slack thread: https://fleetdm.slack.com/archives/C07GLME5P7C/p1728482179107509
    • Gong snippet: Updated Gong snippet (From the call on 1-24-2025)
    • Slack thread: https://fleetdm.slack.com/archives/C07GLME5P7C/p1734542788285989
  • customer-flacourtia: Gong snippet: https://us-65885.app.gong.io/call?id=8250884210103291952&highlights=%5B%7B%22type%22%3A%22SHARE%22%2C%22from%22%3A1980%2C%22to%22%3A2045%7D%5D
  • @noahtalerman: nishiyama requested this because they have 10+ scripts that are exactly the same except for one value. This script is used to bind a workstation to Active Directory. Depending on the location, or some other workstation attribute, these workstations are assigned to different Active Directory groups. The variable in the script is the group that the workstation should be assigned to.
    • @noahtalerman: What is the attribute? @nonpunctual it's likely the workstation is getting some info from LDAP. @harrisonravazzolo maybe they lay down a plist with this info?
    • @noahtalerman: In the interim the user could create a different team for each of the Active Directory groups. They could also have all the workstations in one team and have a different script for all the Active Directory groups.
  • @noahtalerman: nishiyama requested this because they want to put an API token in a script. Instead of putting this API token in the script itself, they can add this token as a parameter in Jamf. We think they're doing this to make updating the secret easier. Instead of having to surgically update the script, they can just update the parameter. Or, they're doing this because it feels more secure.
    • @nonpunctual: The experienced IT admin might be lofting up the script + variables for a less experienced IT admin. The less experienced IT admin can just update the variable instead of having to wrestle with the script.
    • @noahtalerman: In the interim they could use GitOps. Fleet hides these secrets when you view the script in the UI: https://fleetdm.com/guides/secrets-in-scripts-and-configuration-profiles#basic-article
    • @noahtalerman: Eventually Fleet could add a spot in the UI to add/edit secrets that you can use later in scripts.
  • @noahtalerman: User requested this because they want to scope controls (assuming profiles and scripts) based on what department a user is in. I'm not convinced this scoping use case is directly tied to this feature request: variable substitution in scripts.
    • @noahtalerman: Fleet might add the ability to scope scripts based on label. Scoping in Fleet happens with teams (aka baselines) + labels.
  • @noahtalerman: User requested this because they have a complex piece of information that might change in my script (ex. hash). I don't want to have to edit the script. Instead I can just update the variable.
  • @noahtalerman: User requested this because they want to connect their end users to Wi-Fi. In order to do this, the end user workstations needs a SCEP certificate that comes from the organization's third-party SCEP server. On macOS, a SCEP profile is delivered w/ a SCEP challenge (password) to authenticate w/ the SCEP server. Most organizations want this SCEP challenge to be unique per workstation so that they're not using the same password for each workstations and risk exposing the keys to the SCEP server. On Linux, the user thinks the SCEP certificate delivery would be triggered by a script run. In that script, the challenge would be a variable that Fleet would somehow populate using info from the SCEP server.
  • @noahtalerman: User requested this because they want to do security through obscurity. Can use variables to hide pieces of information from less technical users.
  • @noahtalerman: User requested this because they want to place a file (certificate) on the host that has the computer name in it to keep the file unique. Without this variable I'd have to write a specific script for each host.
  • @noahtalerman: User requested this because they want to place something (Welcome to Fleet file in Home folder or certificate) on everyone's host that requires the user (username) in the script to successfully place the thing there. Without this variable I'd have to write a specific script for each host.
    • @noahtalerman: Users might want to put the results of query in a script. Sounds powerful but what are the use cases?
  • @allenhouchins: We have two areas to look at with this. 1) Variable substitution in configuration profiles and 2) variable substitution in scripts. In order to support variable substitution in scripts, we'd have to adopt/build a secrets store concept like GitHub Actions or Parameter Labels. I do agree with using {{ }} for variable substitution coming from a secrets store so an admin isn't confused by the other shell variables they'd be defining within the script using $
  • @nonpunctual Pretty much any customer that has used the Script Parameters features of other deployment tools will want this.

dherder avatar Sep 27 '24 00:09 dherder

I'd request Fleet variables be a different style than shell variables to make it easier for sysadmins to differentiate script variables from fleet variables.

For instance, if doing a shell script as a Jinja2 template, the fleet provided variable would be {{USED_BY}}:

#!/bin/sh

# write the network config
portnoxPath="/var/portnox/"
echo -e "[wifi-security]\nkey-mgmt=wpa-eap\n\n[802-1x]\nca-cert=${portnoxPath}/portnox_ca.crt\nclient-cert=${portnoxPath}/public.crt\neap=tls;\nidentity={{USED_BY}}\nprivate-key=${portnoxPath}/protected.crt\nprivate-key-password=" > /etc/NetworkManager/system-connections/EAPWIFI.nmconnection

# set permissions on network config
chmod 600 /etc/NetworkManager/system-connections/*
chown root:root /etc/NetworkManager/system-connections/*

# restart Network Manager
systemctl restart NetworkManager

This makes it easier to tell that USED_BY is a Fleet injected variable, whereas ${portnoxPath} is a variable defined at runtime on the client.

NOTE - I'm not saying use Jinja2, just please don't make the variable look the same as shell variables. Unless you/others have ideas for why it would be beneficial for the variables to be standard shell variables.

pboushy avatar Oct 03 '24 00:10 pboushy

Moved the original issue description here for safekeeping:

Problem

As an IT admin, I would like to use data from Fleet to populate variables used in script execution. A good example of this would be in certificate generation on Linux hosts when configuring the wifi network connection file. This file requires the user's email address, which we could supply from the "Used by" field in the human to device mapping of the host.

What have you tried?

Performing the variable substitution locally on the host is unreliable because the data is sometimes inaccessible on the host itself.

Potential solutions

As a potential PoC for #21096.

@noahtalerman in order for the approach of script execution with certmonger to work, Fleet needs to be able to handle variables in script execution.

For example, I can issue a bash script to configure certificate generation via certmonger, but when I attempt to leverage the client certificate in a network connection (802.1x), I need to be able to insert a dynamic variable (email address) brokered by Fleet into the network config of the Linux host.

For example, an Ubuntu network config looks like this:

[wifi-security]
key-mgmt=wpa-eap

[802-1x]
ca-cert=/var/portnox/portnox_ca.crt
client-cert=/var/portnox/public.crt
eap=tls;
[email protected]
private-key=/var/portnox/protected.crt
private-key-password=random password

So, I need to be able to supply a bash script that Fleet can interpret and insert the "used-by" field of the host. The bash script added to Fleet (by the admin) would look something like:

#!/bin/sh

# write the network config
echo -e "[wifi-security]\nkey-mgmt=wpa-eap\n\n[802-1x]\nca-cert=/var/portnox/portnox_ca.crt\nclient-cert=/var/portnox/public.crt\neap=tls;\nidentity=$USED_BY\nprivate-key=/var/portnox/protected.crt\nprivate-key-password=" > /etc/NetworkManager/system-connections/EAPWIFI.nmconnection

# set permissions on network config
chmod 600 /etc/NetworkManager/system-connections/*
chown root:root /etc/NetworkManager/system-connections/*

# restart Network Manager
systemctl restart NetworkManager

When the script is delivered to the host, it would look like this:

#!/bin/sh

# write the network config
echo -e "[wifi-security]\nkey-mgmt=wpa-eap\n\n[802-1x]\nca-cert=/var/portnox/portnox_ca.crt\nclient-cert=/var/portnox/public.crt\neap=tls;\[email protected]\nprivate-key=/var/portnox/protected.crt\nprivate-key-password=" > /etc/NetworkManager/system-connections/EAPWIFI.nmconnection

# set permissions on network config
chmod 600 /etc/NetworkManager/system-connections/*
chown root:root /etc/NetworkManager/system-connections/*

# restart Network Manager
systemctl restart NetworkManager

noahtalerman avatar Oct 09 '24 19:10 noahtalerman

@AnthonySnyder8 @phtardif1 can you please add the Gong snippet or Slack thread to the top of the issue description for pingouin and interkosmos? Thanks!

noahtalerman avatar Oct 09 '24 19:10 noahtalerman

@phtardif1 hey just giving you another ping! When you get the chance can you please share the Gong snippet for interkosmos? Thanks :)

noahtalerman avatar Oct 10 '24 19:10 noahtalerman

@noahtalerman I added the gong snippet for interkosmos.

nonpunctual avatar Oct 10 '24 19:10 nonpunctual

@noahtalerman pingouin details have been added

AnthonySnyder8 avatar Oct 11 '24 19:10 AnthonySnyder8

Hey @allenhouchins when you get the chance, can you please follow up w/ pingouin re what workflow they are trying to do w/ variables?

noahtalerman avatar Oct 14 '24 19:10 noahtalerman

  • variable substitution in scripts. In order to support variable substitution in scripts, we'd have to adopt/build a secrets store concept like GitHub Actions or Parameter Labels.

I agree that a secrets/variable store concept like GitHub Actions and Parameter Labels should eventually be implemented to support parameters and/or user-defined variables.

A good first phase of this feature could be to provide variable substitution for items that already exist within fleet:

  • hostname
  • agent version
  • serial number

Personally, I view variable substitution as very different from supporting passing arguments to a script for two core reasons:

Standards

  • Parameters are a standard of shell scripting. They work in a very specific way, and should operate as shell scripts expect (first parameter becomes $1)
  • Variable substitution is not a standard. You can design it to work however you want.

Visibility

  • Parameters show up in history, audit logs, and anything that monitors processes. If it's a "secret" it can become visible to anyone that has the ability to run ps
  • Variable substitution can be done in memory and only visible on the client for a split second. The script and the variables can be sent as a JSON blob to the client, then the client combines the script and variables together. The client only sees the data for the length of time that it's actually running.

pboushy avatar Oct 15 '24 00:10 pboushy

Thanks @pboushy!

secrets/variable store concept like GitHub Actions

This can be accomplished today using Fleet's best practice GitOps (YAML files) and GitHub or GitLab.

Check out an example of how we're using this to manage workstations at Fleet here: https://github.com/fleetdm/fleet/blob/main/it-and-security/lib/configuration-profiles/macos-chrome-enrollment.mobileconfig#L9

cc @dherder

noahtalerman avatar Oct 18 '24 13:10 noahtalerman

Hey @allenhouchins when you get the chance, can you please follow up w/ pingouin re what workflow they are trying to do w/ variables?

Hey @allenhouchins, just checking, any update from pingouin? I saw you opened a Slack thread asking them.

noahtalerman avatar Oct 18 '24 13:10 noahtalerman

@noahtalerman The requestor is out of office this week. If we don't get caught up via Slack, I'll bring it up in the weekly check-in.

allenhouchins avatar Oct 18 '24 14:10 allenhouchins

Hey @pintomi1989 can you please add the Gong snippet for flacourtia?

noahtalerman avatar Oct 28 '24 19:10 noahtalerman

Hey @pintomi1989 just giving you another ping! Can you please add the Gong snippet for flacourtia?

noahtalerman avatar Oct 30 '24 13:10 noahtalerman

@noahtalerman added Gong snippet for customer-flacourtia.

nonpunctual avatar Oct 30 '24 15:10 nonpunctual

  • customer-flacourtia: Gong snippet: https://us-65885.app.gong.io/call?id=8250884210103291952&highlights=%5B%7B%22type%22%3A%22SHARE%22%2C%22from%22%3A1980%2C%22to%22%3A2045%7D%5D
    • @noahtalerman: User requested this because they want to scope controls (assuming profiles and scripts) based on what department a user is in. I'm not convinced this scoping use case is directly tied to this feature request: variable substitution in scripts.

Hey @pintomi1989 we decided not to bring this request to feature fest b/c it doesn't meet the criteria for prioritization.

Context: After the unpacking the why ritual we add all customer requests to feature fest. Up to @zayhanlon to update the Inbox column on the feature fest board before feature fest so that it only includes high priority requests. While there is a customer attached to this issue but I'm not convinced that this is the right request (see reasoning above). Maybe we file a separate request for scoping controls based on end user's department?

noahtalerman avatar Oct 31 '24 18:10 noahtalerman

@noahtalerman I dropped a small soundbyte in from the call we had with customer-pingouin last week, and also included another Slack thread I found as well along the same lines. I haven't had the customer stakeholder join us (yet) on a call, so their team gave feedback in the Gong snippet. I'll continue to ask about this if the right person joins the future call, but hopefully you can at least get a sense of what/why they need this feature. Thanks!

Patagonia121 avatar Jan 16 '25 00:01 Patagonia121

@Patagonia121 thanks for adding the Gong but it doesn't help us understand the specific use case. It's probably because the user in the Gong isn't the user that made the request.

Can you ask them to show us (screenshare) an example workflow in their current macOS MDM solution?

noahtalerman avatar Jan 20 '25 19:01 noahtalerman

@noahtalerman

Here is a script that was used in Jamf with script parameters / variables.

The script parameters in the UI are populated with Bundle ID, app bundle name & version string.

This makes the script generic & the code reusable for removing any application by pasting in the 3 attributes above.

The admin using the script does not need to know anything about the script, they simply need the attributes required to do the removal.

This prevents having to create a unique script with hard-coded variables for each app an admin would want to remove from their fleet.

#!/bin/zsh

# notes:

# $4 Script Parameter in Jamf must be an application bundle id, e.g., com.apple.dt.Xcode
# $5 Script Parameter in Jamf must be an application bundle name, e.g., Xcode.app
# $6 Script Parameter in Jamf must be an application short version string, e.g., 14.2
# $7 Script Parameter in Jamf toggles a test mode. If populated with the string "test" the script will perform a "dry run" without removing files.


# variables (only populate these if Jamf Script Parameters are not used)

varbndl=''
varname=''
varvers=''


###############################
##### DO NOT MODIFY BELOW #####
###############################


# data

appbndl="${4:-$varbndl}"
appname="${5:-$varname}"
appvers="${6:-$varvers}"
tstmode="$7"

IFS=$'\n'
arrdata=($(/usr/bin/mdfind -0 "kMDItemCFBundleIdentifier = $appbndl" | /usr/bin/xargs -0 /usr/bin/mdls -name 'kMDItemVersion' -name 'kMDItemPath' | /usr/bin/sed 's/kMDItemPath    = //g;s/kMDItemVersion = //g'))


# functions

autoload is-at-least

appvr_ck(){ >&2 printf "\nChecking %s versions...\n" "$appname" ; }

appvr_no(){ >&2 printf "\nNo %s found. Exiting...\n" "$appname" ; }

appvr_ok(){ >&2 printf "\nFound %s\nversion = %s\nOk. Leaving %s in place...\n" "${arrdata[i-1]}" "${arrdata[i]}" "$appname" ; }

appvr_rm(){ 
appkill="$(echo "${arrdata[i-1]}" | /usr/bin/sed 's/"//g')"
>&2 printf "\nFound %s\nversion = %s\nInsecure version. Deleting %s...\n" "$appkill" "${arrdata[i]}" "$appkill"; /bin/rm -f -r "$appkill"
>&2 printf "\nValidating deleted path: %s\n" "$appkill"; /bin/ls -ls "$appkill"
exit 0
}

appvr_tm(){
>&2 printf "\nFound %s\nversion = %s\nInsecure version. Deleting %s\n" "${arrdata[i-1]}" "${arrdata[i]}" "${arrdata[i-1]}"
>&2 printf "\n!!! TEST MODE !!! Disabling test mode will remove: %s\n" "${arrdata[i-1]}"
}

not_root(){ >&2 printf "\nThis script must be executed as the root user. Exiting...\n" ; }


# operations

if [ "$EUID" != 0 ]
then
    not_root; exit
fi

if [ -z "${arrdata[*]}" ]
then
    appvr_no; exit
fi

appvr_ck
for ((i=2;i<=${#arrdata[@]};i+=2))
{
    if is-at-least "$appvers" "${arrdata[i]}"
    then
        appvr_ok; continue
    else
        case "$tstmode" in
            'test' ) appvr_tm ;;
                 * ) appvr_rm ;;
        esac
    fi
}

nonpunctual avatar Jan 20 '25 20:01 nonpunctual

@nonpunctual We understand Jamf's functionality and linked to it in the original description issue. We are hoping to capture how exactly the customer is using this feature.

allenhouchins avatar Jan 20 '25 20:01 allenhouchins

@noahtalerman I'll get input this Friday from the customer and provide an updated Gong snippet where they share the workflows they have today in other tools

Patagonia121 avatar Jan 22 '25 05:01 Patagonia121

@noahtalerman For next week, I updated the Gong snippet for customer-pingouin after getting more feedback from them on today's call (1-24-2025). I might check in on this with you during POH as I have a couple other topics to discuss anyway. Thanks!

Patagonia121 avatar Jan 24 '25 23:01 Patagonia121

prospect-nishiyama: https://us-65885.app.gong.io/call?id=620458040443628606&highlights=%5B%7B%22type%22%3A%22SHARE%22%2C%22from%22%3A625%2C%22to%22%3A888%7D%5D

harrisonravazzolo avatar Feb 03 '25 22:02 harrisonravazzolo

  • @noahtalerman: nishiyama requested this because they have 10+ scripts that are exactly the same except for one value. This script is used to bind a workstation to Active Directory. Depending on the location, or some other workstation attribute, these workstations are assigned to different Active Directory groups. The variable in the script is the group that the workstation should be assigned to.
    • @noahtalerman: What is the attribute? @nonpunctual it's likely the workstation is getting some info from LDAP. @harrisonravazzolo maybe they lay down a plist with this info?

@harrisonravazzolo I think we realized that the script's trigger is NOT a smart group for nishiyama. What's the trigger/attribute?

noahtalerman avatar Feb 05 '25 20:02 noahtalerman

  • @noahtalerman: nishiyama requested this because they have 10+ scripts that are exactly the same except for one value. This script is used to bind a workstation to Active Directory. Depending on the location, or some other workstation attribute, these workstations are assigned to different Active Directory groups. The variable in the script is the group that the workstation should be assigned to.
    • @noahtalerman: What is the attribute? @nonpunctual it's likely the workstation is getting some info from LDAP. @harrisonravazzolo maybe they lay down a plist with this info?

@harrisonravazzolo just checking, I think we realized that the script's trigger is NOT a smart group for nishiyama. What's the trigger/attribute?

noahtalerman avatar Feb 07 '25 18:02 noahtalerman

@noahtalerman @harrisonravazzolo Just got off a call with customer-nishiyama

  • There are separate policies for each department / location in their org which is distributed in several geographical locations
  • They are NOT using the Directory Bindings object feature of Jamf I thought they might be using
  • This means they are not using the Directory payload in their Jamf policies
  • There does not need to be separate polices for this
  • I was shown an updated version of their setup script & we talked about ways that the user's identity could be collected
  • If the computer is already bound to Active Directory, the DC & OU match could be done by the setup script itself rather than having the values passed in from the Jamf script parameter
  • I think we should continue to assume that making script parameters ready for this customer is necessary, but
  • A little more investigation into the engineering work needed to change the workflow may make it unnecessary

nonpunctual avatar Feb 07 '25 20:02 nonpunctual

@nonpunctual thanks! Great notes.

Can you please share the Gong snippet from that call?

noahtalerman avatar Feb 07 '25 21:02 noahtalerman

This has come up on various calls with prospect-geeve - here is one: https://us-65885.app.gong.io/call?id=8733277346157782087&highlights=%5B%7B%22type%22%3A%22SHARE%22%2C%22from%22%3A1557%2C%22to%22%3A1632%7D%5D

As it applies to this prospect, they are looking to apply/update a wifi change to a large group of ubuntu devices in a particular location. These being linux devices there is no mobileconfig style payload that might be able to handle the passing of a 'secret' but instead they use all shell scripts to do the modification and thus want to limit the ability to view/pass the wifi credentials in plaintext.

harrisonravazzolo avatar Feb 11 '25 04:02 harrisonravazzolo