masterfiles icon indicating copy to clipboard operation
masterfiles copied to clipboard

CFE-3612: enable yum package inventory in aix

Open flynn1973 opened this issue 4 years ago • 8 comments

see discussion here -> https://groups.google.com/g/help-cfengine/c/jzDzekirYTY

the changes made here enables the package inventory for yum/rpm packages in aix through augments, but there are some unwanted inventory files beside the working ones created in the state directory. this needs to be reviewed by some cfengine expert, as i am unable to figure out whats the reason behind this.

files used for testing: https://github.com/flynn1973/cfengine_aix_yum

flynn1973 avatar Mar 29 '21 12:03 flynn1973

Thanks for submitting a pull request! Maybe @craigcomstock can review this?

cf-bottom avatar Mar 30 '21 10:03 cf-bottom

but there are some unwanted inventory files beside the working ones created in the state directory. this needs to be reviewed by some cfengine expert, as i am unable to figure out whats the reason behind this.

What are the files?

vpodzime avatar Apr 01 '21 07:04 vpodzime

but there are some unwanted inventory files beside the working ones created in the state directory. this needs to be reviewed by some cfengine expert, as i am unable to figure out whats the reason behind this.

What are the files?

packages_updates_$(package_module_knowledge.platform_default).lmdb.lock packages_installed_$(package_module_knowledge.platform_default).lmdb.lock packages_updates_$(package_module_knowledge.platform_default).lmdb packages_updates_$(package_module_knowledge.platform_default).lmdb-lock packages_installed_$(package_module_knowledge.platform_default).lmdb packages_installed_$(package_module_knowledge.platform_default).lmdb-lock

flynn1973 avatar Apr 01 '21 08:04 flynn1973

Those are local cache files for data provided by the package module. But clearly the $(package_module_knowledge.platform_default) variable is not resolved or expanded in the place where it is used. We can fix that once other things here are agreed on.

vpodzime avatar Apr 02 '21 07:04 vpodzime

but there are some unwanted inventory files beside the working ones created in the state directory. this needs to be reviewed by some cfengine expert, as i am unable to figure out whats the reason behind this.

What are the files?

Those are some artifact of variable resolution, annoying but benign as far as I know

nickanderson avatar Apr 02 '21 14:04 nickanderson

@nickanderson @flynn197 it seems like the way forward here is to

  1. conditionally set platform_default to yum where yum is available on aix
  2. set package_inventory to either platform_default (yum in some cases) or { "nimclient", "yum" } if that makes sense? (I am not too familiar with AIX to know if both should be used or just one preferred).

craigcomstock avatar Oct 18 '22 18:10 craigcomstock

@nickanderson @flynn197 it seems like the way forward here is to

1. conditionally set `platform_default` to `yum` where `yum` is available on aix

2. set `package_inventory` to either `platform_default` (yum in some cases) or `{ "nimclient", "yum" }` if that makes sense? (I am not too familiar with AIX to know if both should be used or just one preferred).

@flynn1973 previously said

yes using nimclient and yum together would be way better. https://github.com/cfengine/masterfiles/pull/1952#discussion_r605391601

I am not managing AIX hosts so I would defer to someone who is :D I believe @flynn1973 .

nickanderson avatar Oct 18 '22 19:10 nickanderson

@nickanderson @flynn197 it seems like the way forward here is to

1. conditionally set `platform_default` to `yum` where `yum` is available on aix

2. set `package_inventory` to either `platform_default` (yum in some cases) or `{ "nimclient", "yum" }` if that makes sense? (I am not too familiar with AIX to know if both should be used or just one preferred).

@flynn1973 previously said

yes using nimclient and yum together would be way better. #1952 (comment)

I am not managing AIX hosts so I would defer to someone who is :D I believe @flynn1973 .

as for now, ibm provides yum/dnf as a an add on, means if someone wants to use the so called "IBM AIX Toolbox" one has to rollout yum. nimclient is part of base aix operating system and always available. so the prefered way would be...

  1. set nimclient as default
  2. if yum/dnf is available set "{ "nimclient", "yum" }"

path of binaries to check for existence (setting classes eg...) from the dnf rpm:

root@nimvie: /home/tremch/flask/sudodb # rpm -ql dnf
/opt/freeware/bin/dnf
/opt/freeware/bin/yum
/opt/freeware/share/bash-completion/completions/dnf
/var/cache/dnf

flynn1973 avatar Oct 20 '22 13:10 flynn1973

anything i can do to bring this to an end?

flynn1973 avatar Jan 22 '24 14:01 flynn1973

but there are some unwanted inventory files beside the working ones created in the state directory. this needs to be reviewed by some cfengine expert, as i am unable to figure out whats the reason behind this.

What are the files?

packages_updates_$(package_module_knowledge.platform_default).lmdb.lock packages_installed_$(package_module_knowledge.platform_default).lmdb.lock packages_updates_$(package_module_knowledge.platform_default).lmdb packages_updates_$(package_module_knowledge.platform_default).lmdb-lock packages_installed_$(package_module_knowledge.platform_default).lmdb packages_installed_$(package_module_knowledge.platform_default).lmdb-lock

Do you still see those files generated? I know that I have seen it before, but I don't see it now (3.23.4) and my Jira search doesn't seem to surface anything. Not the point of this PR obviously, but want to be sure if it's still being seen we get a ticket logged.

anything i can do to bring this to an end?

Pestering is best, I am loading this in my brain so I can think it through again.

nickanderson avatar Jan 25 '24 19:01 nickanderson

anything i can do to bring this to an end?

What values do you have for $(sys.flavor) across your infra (if you wanna keep that private you can email me directly, nick dot anderson at northern dot tech.

nickanderson avatar Jan 25 '24 19:01 nickanderson

@flynn1973 after re-reviewing and re-thinking I posted a comment to the related ticket: LMK what you think. https://northerntech.atlassian.net/browse/CFE-3612?focusedCommentId=103760

nickanderson avatar Jan 25 '24 20:01 nickanderson

anything i can do to bring this to an end?

What values do you have for $(sys.flavor) across your infra (if you wanna keep that private you can email me directly, nick dot anderson at northern dot tech.

root@aixtest01: /root # /var/cfengine/bin/cf-promises --show-vars | grep sys.flavour default:sys.flavour aix_7 source=agent

i think sys.os would be a better choice for indentifying aix.

default:sys.os aix source=agent

flynn1973 avatar Jan 29 '24 10:01 flynn1973

but there are some unwanted inventory files beside the working ones created in the state directory. this needs to be reviewed by some cfengine expert, as i am unable to figure out whats the reason behind this.

What are the files?

packages_updates_$(package_module_knowledge.platform_default).lmdb.lock packages_installed_$(package_module_knowledge.platform_default).lmdb.lock packages_updates_$(package_module_knowledge.platform_default).lmdb packages_updates_$(package_module_knowledge.platform_default).lmdb-lock packages_installed_$(package_module_knowledge.platform_default).lmdb packages_installed_$(package_module_knowledge.platform_default).lmdb-lock

Do you still see those files generated? I know that I have seen it before, but I don't see it now (3.23.4) and my Jira search doesn't seem to surface anything. Not the point of this PR obviously, but want to be sure if it's still being seen we get a ticket logged.

anything i can do to bring this to an end?

Pestering is best, I am loading this in my brain so I can think it through again.

those files are not generated any more.

btw.

  • packagesmatching() and packageupdatesmatching() now look for the software inventory databases in the state directory and use them if found. This change enables the usage of these functions in standalone policy files without the demand for specifying the default package inventory attribute in body common control. However, you still need the default package inventory attribute specified in the policy framework for the software inventory databases to exist in the first place and to be maintained. (ENT-9083)

this change in 3.21.3 "seems" to have some effect on package inventory generation on aix. the inventory is updated regulary.

root@aixtest01: /var/cfengine/state # ls -ltr | grep packages -rw------- 1 root system 33729 Jan 01 1970 software_packages.csv -rw------- 1 root system 0 Jan 22 19:31 packages_installed_yum.lmdb.lock -rw------- 1 root system 0 Jan 22 19:31 packages_updates_yum.lmdb.lock -rw------- 1 root system 32768 Jan 28 19:55 packages_updates_yum.lmdb -rw------- 1 root system 192512 Jan 29 11:54 packages_installed_yum.lmdb -rw------- 1 root system 8256 Jan 29 11:54 packages_installed_yum.lmdb-lock -rw------- 1 root system 8256 Jan 29 11:54 packages_updates_yum.lmdb-lock

flynn1973 avatar Jan 29 '24 11:01 flynn1973

@flynn1973 after re-reviewing and re-thinking I posted a comment to the related ticket: LMK what you think. https://northerntech.atlassian.net/browse/CFE-3612?focusedCommentId=103760

looks good, but i would use sys.os instead of sys.flavour

flynn1973 avatar Jan 29 '24 11:01 flynn1973

root@aixtest01: /root # /var/cfengine/bin/cf-promises --show-vars | grep sys.flavour default:sys.flavour aix_7 source=agent

i think sys.os would be a better choice for indentifying aix.

default:sys.os aix source=agent

I suggested sys.flavor because sys.os on linux is simply linux. So I don't think that would work well for an environment that had ubuntu and redhat but wanted to enable a snap module for ubuntu and a flatpak for redhat along with the standard package managers.

Thinking about this again, I considered the potential of the key being a regular expression matching against sys.flavor, that way you could do aix.* to make it take effect for all versions, but it's not clear to me how it would be sane in the event that you have multiple entries that match sys.flavor, e.g.:

{
  "variabless": {
    "default:def.additional_package_inventory": {
        "value": {
            "ubuntu.*": [ "snap" ],
            "ubuntu_22": [ "snap", "flatpak" ],
            "aix": [ "yum" ]
        },
        "comment": "This variable provides the ability to extend the default package managers to inventory for a platform. Keys are based on the value of $(sys.flavor) for the targeted platform."
      }
  }
}

Could do some detection about matching multiple cases, but it seems safer to proceed with using sys.flavor value as the key.

nickanderson avatar Jan 31 '24 18:01 nickanderson

@flynn1973 OK, I pushed to your branch. Can you tell me if those changes work well for you?

nickanderson avatar Jan 31 '24 19:01 nickanderson

Ah there was a conflict, I rebased and lost your original commit, but now it's a clean patch against master.

nickanderson avatar Jan 31 '24 19:01 nickanderson

Squash the fix, please.

vpodzime avatar Feb 01 '24 11:02 vpodzime

@flynn1973 OK, I pushed to your branch. Can you tell me if those changes work well for you?

modified lib/packages.cf and promises.cf

modified def.json

{

        "classes": {
                "services_autorun": ["any"],
                "cfengine_internal_disable_agent_email": ["any"],
                "cfengine_internal_purge_policies": [ "any" ],
                "mpf_augments_control_enabled": [ "any" ],
                "cfe_internal_core_watchdog_enabled": [ "aix.!policy_server::" ],
                "cfe_internal_core_watchdog_disabled": [ "!cfe_internal_core_watchdog_enabled::" ]
        },

        "inputs": ["services/autorun/itsv.cf", "ITSVlib/stdlib.cf"],

        "vars": {

                "acl": ["127.0.0.1", "172.17.8.0/21", "172.18.33.101", "172.18.33.201"],
                "input_name_patterns": [".*\\.cf", ".*\\.dat", ".*\\.txt", ".*\\.conf", ".*\\.mustache",
                        ".*\\.sh", ".*\\.pl", ".*\\.py", ".*\\.rb",
                        "cf_promises_release_id", ".*\\.json", ".*\\.yaml", ".*\\.js", ".*\\.tmpl", ".*\\.csv"
                ],
                "control_server_maxconnections": "800",
                "control_agent_abortbundleclasses": ["ITSV_BUNDLE_ABORT"],
                "control_agent_abortclasses": ["ITSV_ABORT"],
                "mailto": "[email protected]",
                "mailfrom": "[email protected]",
                "smtpserver": "localhost",
                "path[yum]":  "/usr/bin/yum",
                "mpf_update_policy_master_location": "/var/cfengine/masterfiles",
                "control_agent_environment_vars_extra": [ "XPG_SUS_ENV=ON", "TZ=CET-1CEST,M3.5.0/2,M10.5.0/3" ],

                 {
                  "variables": {
                    "default:def.default_package_module": {
                      "value": {
                        "ubuntu_20": "snap",
                         "aix_7": "yum"
                      },
                        "comment": "This variable provides the ability to override the default package manager to use for a platform. Keys are based on the value of $(sys.flavor) for the targeted platform."
                      }
                    }
                  }


                "control_server_itsv_acl_rules": {
                                  "/usr/local/bin": {
                                  "admit": "def.acl",
                                  "comment": "grant access to ulb"
                                  },
                                  "/etc/sudoers.d": {
                                  "admit": "def.acl",
                                  "comment": "grant access to sudoers"
                                  },
                                  "/cferoot": {
                                  "admit": "def.acl",
                                  "comment": "grant access to cferoot"
                                  },
                                  "${sys.workdir}/itsv_reporting": {
                                  "admit": "def.acl",
                                  "comment": "grant access to reporting directory"
                                  }

                               }
                }

}

cf-agent throws an error about def.json

root@aixtest01: /var/cfengine/inputs # /var/cfengine/bin/cf-agent -KI error: Could not parse JSON file /var/cfengine/inputs/def.json: Unable to parse json data as object, object not allowed as l-value info: Installing cfe_internal_non_existing_package... cat: cannot open ../../tests/unit/mock_lslpp_Lc

some lmdb file are created but with nimclient context...

rw------- 1 root system 32768 Feb 01 15:09 packages_installed_nimclient.lmdb -rw------- 1 root system 8256 Feb 01 15:09 packages_updates_nimclient.lmdb-lock -rw------- 1 root system 32768 Feb 01 15:09 packages_updates_nimclient.lmdb

so soemthing is not quite right...maybe variables is meant as vars in def.json?

flynn1973 avatar Feb 01 '24 14:02 flynn1973

Hey @flynn1973,

Thanks for coming back and testing it.

I think I see the confusion ….

Augments has seen some additional functionality since this PR was opened.

Previously, you could only use augments to define variables in the default namespace in the def bundle scope:

So, the following Augments file would define default:def.myvariable with the value My Value.

{
    "vars": {
        "myvariable": "My Value"
        }
}

Since 3.18 new syntax is supported in Augments that lets you do much more. Now you can target any namespace, any bundle and you can additionally set tags ( as if you were to use the meta attribute while defining a variable in policy ), and set comments (just like you would set a comment when defining a variable in policy).

So, using the new syntax to define the same variable as in the above example it would look like this:

{
 "variables": {
   "default:def.myvariable": {
     "value": "My Value"
     }
   }
 }

I wrote the docs for these using the new style and I see that you placed them inside vars. Since these two variables are both in the default:def bundle scope we can use the old style definition like this:

{

        "classes": {
                "services_autorun": ["any"],
                "cfengine_internal_disable_agent_email": ["any"],
                "cfengine_internal_purge_policies": [ "any" ],
                "mpf_augments_control_enabled": [ "any" ],
                "cfe_internal_core_watchdog_enabled": [ "aix.!policy_server::" ],
                "cfe_internal_core_watchdog_disabled": [ "!cfe_internal_core_watchdog_enabled::" ]
        },

        "inputs": ["services/autorun/itsv.cf", "ITSVlib/stdlib.cf"],

        "vars": {

                "acl": ["127.0.0.1", "172.17.8.0/21", "172.18.33.101", "172.18.33.201"],
                "input_name_patterns": [".*\\.cf", ".*\\.dat", ".*\\.txt", ".*\\.conf", ".*\\.mustache",
                        ".*\\.sh", ".*\\.pl", ".*\\.py", ".*\\.rb",
                        "cf_promises_release_id", ".*\\.json", ".*\\.yaml", ".*\\.js", ".*\\.tmpl", ".*\\.csv"
                ],
                "control_server_maxconnections": "800",
                "control_agent_abortbundleclasses": ["ITSV_BUNDLE_ABORT"],
                "control_agent_abortclasses": ["ITSV_ABORT"],
                "mailto": "[email protected]",
                "mailfrom": "[email protected]",
                "smtpserver": "localhost",
                "path[yum]":  "/usr/bin/yum",
                "mpf_update_policy_master_location": "/var/cfengine/masterfiles",
                "control_agent_environment_vars_extra": [ "XPG_SUS_ENV=ON", "TZ=CET-1CEST,M3.5.0/2,M10.5.0/3" ],

                "default_package_module": {
                    "ubuntu_20": "snap",
                    "aix_7": "yum"
                },

                "control_server_itsv_acl_rules": {
                                  "/usr/local/bin": {
                                  "admit": "def.acl",
                                  "comment": "grant access to ulb"
                                  },
                                  "/etc/sudoers.d": {
                                  "admit": "def.acl",
                                  "comment": "grant access to sudoers"
                                  },
                                  "/cferoot": {
                                  "admit": "def.acl",
                                  "comment": "grant access to cferoot"
                                  },
                                  "${sys.workdir}/itsv_reporting": {
                                  "admit": "def.acl",
                                  "comment": "grant access to reporting directory"
                                  }

                }
        }
}

You can use both new and old syntax concurrently if you really want, per the docs: https://docs.cfengine.com/docs/3.21/reference-language-concepts-augments.html#top

vars and variables keys are allowed concurrently in the same file. > If vars and variables keys in the same augments file define the same variable, the definition provided by the variables key wins.

So, this should also work.

{

    "classes": {
        "services_autorun": ["any"],
        "cfengine_internal_disable_agent_email": ["any"],
        "cfengine_internal_purge_policies": [ "any" ],
        "mpf_augments_control_enabled": [ "any" ],
        "cfe_internal_core_watchdog_enabled": [ "aix.!policy_server::" ],
        "cfe_internal_core_watchdog_disabled": [ "!cfe_internal_core_watchdog_enabled::" ]
    },

    "inputs": ["services/autorun/itsv.cf", "ITSVlib/stdlib.cf"],

    "vars": {

        "acl": ["127.0.0.1", "172.17.8.0/21", "172.18.33.101", "172.18.33.201"],
        "input_name_patterns": [".*\\.cf", ".*\\.dat", ".*\\.txt", ".*\\.conf", ".*\\.mustache",
                                ".*\\.sh", ".*\\.pl", ".*\\.py", ".*\\.rb",
                                "cf_promises_release_id", ".*\\.json", ".*\\.yaml", ".*\\.js", ".*\\.tmpl", ".*\\.csv"
                               ],
        "control_server_maxconnections": "800",
        "control_agent_abortbundleclasses": ["ITSV_BUNDLE_ABORT"],
        "control_agent_abortclasses": ["ITSV_ABORT"],
        "mailto": "[email protected]",
        "mailfrom": "[email protected]",
        "smtpserver": "localhost",
        "path[yum]":  "/usr/bin/yum",
        "mpf_update_policy_master_location": "/var/cfengine/masterfiles",
        "control_agent_environment_vars_extra": [ "XPG_SUS_ENV=ON", "TZ=CET-1CEST,M3.5.0/2,M10.5.0/3" ],
        "control_server_itsv_acl_rules": {
            "/usr/local/bin": {
                "admit": "def.acl",
                "comment": "grant access to ulb"
            },
            "/etc/sudoers.d": {
                "admit": "def.acl",
                "comment": "grant access to sudoers"
            },
            "/cferoot": {
                "admit": "def.acl",
                "comment": "grant access to cferoot"
            },
            "${sys.workdir}/itsv_reporting": {
                "admit": "def.acl",
                "comment": "grant access to reporting directory"
            }

        }
    },
    "variables": {
        "default:def.default_package_module": {
            "value": {
                "ubuntu_20": "snap",
                "aix_7": "yum"
            },
            "comment": "This variable provides the ability to override the default package manager to use for a platform. Keys are based on the value of $(sys.flavor) for the targeted platform."
        }
    }
}

If you adjust your def.json accordingly does it work as expect?

nickanderson avatar Feb 01 '24 19:02 nickanderson

the json error is gone now...but still the nimclient in the lmdb file names...

root@aixtest01: /var/cfengine/inputs # ls -ltr /var/cfengine/state/ | grep pack -rw------- 1 root system 33729 Jan 01 1970 software_packages.csv -rw------- 1 root system 0 Feb 02 10:24 packages_installed_nimclient.lmdb.lock -rw------- 1 root system 12288 Feb 02 10:24 packages_installed_nimclient.lmdb -rw------- 1 root system 0 Feb 02 10:24 packages_updates_nimclient.lmdb.lock -rw------- 1 root system 12288 Feb 02 10:24 packages_updates_nimclient.lmdb -rw------- 1 root system 8256 Feb 02 10:24 packages_installed_nimclient.lmdb-lock -rw------- 1 root system 8256 Feb 02 10:24 packages_updates_nimclient.lmdb-lock

flynn1973 avatar Feb 02 '24 09:02 flynn1973

the lmdb files are somewhat "empty" btw...

root@aixtest01: /var/cfengine/inputs # /var/cfengine/bin/cf-check dump /var/cfengine/state/packages_installed_nimclient.lmdb
{
        "<inventory>": "",
}
root@aixtest01: /var/cfengine/inputs # /var/cfengine/bin/cf-check dump /var/cfengine/state/packages_updates_nimclient.lmdb
{
        "<inventory>": "",
}

flynn1973 avatar Feb 02 '24 12:02 flynn1973

Grrr

nickanderson avatar Feb 02 '24 14:02 nickanderson

the lmdb files are somewhat "empty" btw...

root@aixtest01: /var/cfengine/inputs # /var/cfengine/bin/cf-check dump /var/cfengine/state/packages_installed_nimclient.lmdb
{
        "<inventory>": "",
}
root@aixtest01: /var/cfengine/inputs # /var/cfengine/bin/cf-check dump /var/cfengine/state/packages_updates_nimclient.lmdb
{
        "<inventory>": "",
}

Was it not empty previously?

nickanderson avatar Feb 02 '24 14:02 nickanderson

the lmdb files are somewhat "empty" btw...

root@aixtest01: /var/cfengine/inputs # /var/cfengine/bin/cf-check dump /var/cfengine/state/packages_installed_nimclient.lmdb
{
        "<inventory>": "",
}
root@aixtest01: /var/cfengine/inputs # /var/cfengine/bin/cf-check dump /var/cfengine/state/packages_updates_nimclient.lmdb
{
        "<inventory>": "",
}

Was it not empty previously?

did not check this previously, but looking at the file sizes i guess they were always empty. as there are still "nimclient" references in the file names, i guess cfengine still considers nimclient as standard package module even setting the new vars in augments. as nimclient does not provide inventory in lmdb files but in software_packages.csv i suspect this as the reason for the empty inventory files.

flynn1973 avatar Feb 02 '24 15:02 flynn1973

did not check this previously, but looking at the file sizes i guess they were always empty. as there are still "nimclient" references in the file names, i guess cfengine still considers nimclient as standard package module even setting the new vars in augments. as nimclient does not provide inventory in lmdb files but in software_packages.csv i suspect this as the reason for the empty inventory files.

Well, actually bug in my policy. I just pushed a commit, and im writing a note ...

nickanderson avatar Feb 02 '24 15:02 nickanderson

OK, I found a couple of bugs in my implementation where I was getting the data set via Augments.

I am trying snap on Centos 7 since we have package module for it already.

bundle agent snapd_on_centos_7
{
  packages:
      "epel-release" -> { "snapd" }
        policy => "present",
        package_module => yum;

      "snapd"
        policy => "present",
        package_module => yum;

      "vim-enhanced"
        policy => "present",
        package_module => yum;

      "nvim" # actually, I installe this manually, it's "classic" needs the --classic option and I was lazy to see if that is implemtned by the packge module
        policy => "present",
        package_module => snap;

      "etcd" # This one however, is not --classic, so I expect it to get installed via snap since that is what I set as the default package manager
        policy => "present";

  services:
      "snapd.socket"
        service_policy => "enabled";

  files:
      "/snap"
        link_from => ln_s("/var/lib/snapd/snap");

  vars:
      "p" data => packagesmatching( ".*(vim|etcd).*", ".*", ".*", ".*" );

  reports:
      '$(with)' with => storejson( p );
}

I have the following in def.json:

{
  "variables": {
    "default:def.additional_package_inventory_modules": {
        "value": {
            "centos_7": [ "snap"]
        },
        "comment": "This variable provides the ability to extend the default package managers to inventory for a platform. Keys are based on the value of $(sys.flavor) for the targeted platform."
      },

    "default:def.default_package_module": {
        "value": {
          "centos_7": "snap"
        }
        "comment": "This variable provides the ability to override the default package manager to use for a platform. Keys are based on the value of $(sys.flavor) for the targeted platform."
    }
  }
}

When I run the policy with the latest state of the PR that includes the bug fix I get the following output:

[root@hub masterfiles]# cf-agent -KIf update.cf; cf-agent -KI
    info: Copied file '/var/cfengine/masterfiles/lib/packages.cf' to '/var/cfengine/inputs/lib/packages.cf.cfnew' (mode '600')
    info: Backed up '/var/cfengine/inputs/lib/packages.cf' as '/var/cfengine/inputs/lib/packages.cf.cfsaved'
    info: Moved '/var/cfengine/inputs/lib/packages.cf.cfnew' to '/var/cfengine/inputs/lib/packages.cf'
    info: Updated '/var/cfengine/inputs/lib/packages.cf' from source '/var/cfengine/masterfiles/lib/packages.cf' on 'localhost'
    info: Purged '/var/cfengine/inputs/lib/packages.cf.cfsaved' copy dest directory
etcd 3.4.22 from Canonical** installed
    info: Successfully installed package 'etcd'
R: [
  {
    "arch": "none",
    "method": "snap",
    "name": "nvim",
    "version": "v0.9.4"
  },
  {
    "arch": "none",
    "method": "snap",
    "name": "nvim",
    "version": "v0.9.4"
  }
]
R: [
  {
    "arch": "none",
    "method": "snap",
    "name": "etcd",
    "version": "3.4.22"
  },
  {
    "arch": "none",
    "method": "snap",
    "name": "nvim",
    "version": "v0.9.4"
  },
  {
    "arch": "none",
    "method": "snap",
    "name": "etcd",
    "version": "3.4.22"
  },
  {
    "arch": "none",
    "method": "snap",
    "name": "nvim",
    "version": "v0.9.4"
  }
]

Yeah, I have run the policy multiple times as I was testing, but you can see here:

  • etcd is installed from a snap even though the promise does not specify to use the snap module, it picked up the override from Augments.
  • We have inventory from only snap
    • This might seem a bit off since yum is the standard default package manager but:

      • package_inventory is the platform default plus the additional inventory package_inventory => { $(package_module_knowledge.platform_default), @(default:package_module_knowledge.additional_inventory) };
      • default:def.default_package_module is overridden to snap and snap is listed for use in additional inventory as well which results in package_inventory being snap, snap
    • Adjusting the default:def.additional_package_inventory_modules to include yum results in having inventory for both package managers.

      [root@hub masterfiles]# cf-agent -KIf update.cf; cf-agent -KI
          info: Copied file '/var/cfengine/masterfiles/def.json' to '/var/cfengine/inputs/def.json.cfnew' (mode '600')
          info: Backed up '/var/cfengine/inputs/def.json' as '/var/cfengine/inputs/def.json.cfsaved'
          info: Moved '/var/cfengine/inputs/def.json.cfnew' to '/var/cfengine/inputs/def.json'
          info: Updated '/var/cfengine/inputs/def.json' from source '/var/cfengine/masterfiles/def.json' on 'localhost'
          info: Purged '/var/cfengine/inputs/def.json.cfsaved' copy dest directory
      R: [
        {
          "arch": "none",
          "method": "snap",
          "name": "etcd",
          "version": "3.4.22"
        },
        {
          "arch": "none",
          "method": "snap",
          "name": "nvim",
          "version": "v0.9.4"
        },
        {
          "arch": "none",
          "method": "snap",
          "name": "etcd",
          "version": "3.4.22"
        },
        {
          "arch": "none",
          "method": "snap",
          "name": "nvim",
          "version": "v0.9.4"
        },
        {
          "arch": "x86_64",
          "method": "yum",
          "name": "vim-filesystem",
          "version": "7.4.629-8.el7_9"
        },
        {
          "arch": "x86_64",
          "method": "yum",
          "name": "vim-common",
          "version": "7.4.629-8.el7_9"
        },
        {
          "arch": "x86_64",
          "method": "yum",
          "name": "vim-minimal",
          "version": "7.4.629-6.el7"
        },
        {
          "arch": "x86_64",
          "method": "yum",
          "name": "vim-enhanced",
          "version": "7.4.629-8.el7_9"
        }
      ]
      [root@hub masterfiles]# cat def.json
      {
        "variables": {
          "default:def.additional_package_inventory_modules": {
              "value": {
                  "centos_7": [ "snap", "yum" ]
              },
              "comment": "This variable provides the ability to extend the default package managers to inventory for a platform. Keys are based on the value of $(sys.flavor) for the targeted platform."
            },
      
          "default:def.default_package_module": {
              "value": {
                "centos_7": "snap"
              }
              "comment": "This variable provides the ability to override the default package manager to use for a platform. Keys are based on the value of $(sys.flavor) for the targeted platform."
          }
        }
      }
      
      

What do you get with the latest change?

nickanderson avatar Feb 02 '24 16:02 nickanderson

Oh yeah, I also checked with the old style augments format:

{
  "vars": {
      "additional_package_inventory_modules": {
          "centos_7": [ "snap", "yum" ]
      },

      "default_package_module": {
          "centos_7": "snap"
      }
  }
}

nickanderson avatar Feb 02 '24 16:02 nickanderson

WORKS!

root@aixtest01: /root # ls -ltr /var/cfengine/state/ | grep yum
-rw-------    1 root     system            0 Feb 05 10:43 packages_installed_yum.lmdb.lock
-rw-------    1 root     system        65536 Feb 05 10:43 packages_installed_yum.lmdb
-rw-------    1 root     system            0 Feb 05 10:45 packages_updates_yum.lmdb.lock
-rw-------    1 root     system        20480 Feb 05 10:45 packages_updates_yum.lmdb
-rw-------    1 root     system         8256 Feb 05 10:46 packages_updates_yum.lmdb-lock
-rw-------    1 root     system         8256 Feb 05 10:46 packages_installed_yum.lmdb-lock
root@aixtest01: /root # /var/cfengine/bin/cf-check dump /var/cfengine/state/packages_updates_yum.lmdb
{
        "<inventory>": "ansible,7.2.0-1,noarch\napr-util,1.6.3-2,ppc\nbash,5.2.15-1,ppc\ncoreutils,9.3-1,ppc\ncurl,8.5.0-1,ppc\nfile,5.45-1,ppc\nfile-libs,5.45-1,ppc\nfontconfig,2.14.2-1,ppc\ngawk,5.2.2-1,ppc\ngit,2.41.0-1,ppc\ngit-core,2.41.0-1,ppc\ngit-core-doc,2.41.0-1,noarch\nglib2,2.76.3-1,ppc\ngnupg2,2.4.3-1,ppc\nhttpd,2.4.58-1,ppc\nkrb5-libs,1.21.2-1,ppc\nkrb5-workstation,1.21.2-1,ppc\nlibffi,3.4.4-2,ppc\nlibgcrypt,1.10.2-1,ppc\nlibnghttp2,1.55.1-1,ppc\nlibrepo,1.11.0-103,ppc\nlibsmbclient,4.18.9-1,ppc\nlibunistring,1.1-1,ppc\nlzo,2.10-3,ppc\nmake,1:4.4.1-1,ppc\nncurses,6.4-1,ppc\np11-kit,0.24.1-1,ppc\np11-kit-tools,0.24.1-1,ppc\nperl,1:5.38.0-1,ppc\npostgresql-libs,11.21-1,ppc\npython3,3.9.18-1,ppc\npython3-devel,3.9.18-1,ppc\npython3-idle,3.9.18-1,ppc\npython3-jinja2,3.1.2-1,noarch\npython3-librepo,1.11.0-103,ppc\npython3-markupsafe,2.1.2-1,ppc\npython3-test,3.9.18-1,ppc\npython3-tkinter,3.9.18-1,ppc\npython3.9,3.9.18-1,ppc\npython3.9-devel,3.9.18-1,ppc\npython3.9-idle,3.9.18-1,ppc\npython3.9-jinja2,3.1.2-1,noarch\npython3.9-librepo,1.11.0-103,ppc\npython3.9-markupsafe,2.1.2-1,ppc\npython3.9-test,3.9.18-1,ppc\npython3.9-tkinter,3.9.18-1,ppc\nrlwrap,0.46.1-1,ppc\nsamba,4.18.9-1,ppc\nsamba-client,4.18.9-1,ppc\nsamba-common,4.18.9-1,ppc\nsamba-devel,4.18.9-1,ppc\nsamba-libs,4.18.9-1,ppc\nsamba-winbind,4.18.9-1,ppc\nsamba-winbind-clients,4.18.9-1,ppc\nsshpass,1.10-1,ppc\nsudo,1.9.14p3-1,ppc\ntcl,8.6.13-1,ppc\ntk,8.6.13-1,ppc\nvim-common,9.0.1649-1,ppc\nvim-enhanced,9.0.1649-1,ppc\nwget,1.21.4-1,ppc\nxz,5.4.3-1,ppc\nxz-libs,5.4.3-1,ppc\n",
        "N<ansible>": "V<7.2.0-1>A<noarch>\n",
        "N<apr-util>": "V<1.6.3-2>A<ppc>\n",
        "N<bash>": "V<5.2.15-1>A<ppc>\n",
        "N<coreutils>": "V<9.3-1>A<ppc>\n",
        "N<curl>": "V<8.5.0-1>A<ppc>\n",
        "N<file-libs>": "V<5.45-1>A<ppc>\n",
        "N<file>": "V<5.45-1>A<ppc>\n",
        "N<fontconfig>": "V<2.14.2-1>A<ppc>\n",
        "N<gawk>": "V<5.2.2-1>A<ppc>\n",
        "N<git-core-doc>": "V<2.41.0-1>A<noarch>\n",
        "N<git-core>": "V<2.41.0-1>A<ppc>\n",
        "N<git>": "V<2.41.0-1>A<ppc>\n",
        "N<glib2>": "V<2.76.3-1>A<ppc>\n",
        "N<gnupg2>": "V<2.4.3-1>A<ppc>\n",
        "N<httpd>": "V<2.4.58-1>A<ppc>\n",
        "N<krb5-libs>": "V<1.21.2-1>A<ppc>\n",
        "N<krb5-workstation>": "V<1.21.2-1>A<ppc>\n",
        "N<libffi>": "V<3.4.4-2>A<ppc>\n",
        "N<libgcrypt>": "V<1.10.2-1>A<ppc>\n",
        "N<libnghttp2>": "V<1.55.1-1>A<ppc>\n",
        "N<librepo>": "V<1.11.0-103>A<ppc>\n",
        "N<libsmbclient>": "V<4.18.9-1>A<ppc>\n",
        "N<libunistring>": "V<1.1-1>A<ppc>\n",
        "N<lzo>": "V<2.10-3>A<ppc>\n",
        "N<make>": "V<1:4.4.1-1>A<ppc>\n",
        "N<ncurses>": "V<6.4-1>A<ppc>\n",
        "N<p11-kit-tools>": "V<0.24.1-1>A<ppc>\n",
        "N<p11-kit>": "V<0.24.1-1>A<ppc>\n",
        "N<perl>": "V<1:5.38.0-1>A<ppc>\n",
        "N<postgresql-libs>": "V<11.21-1>A<ppc>\n",
        "N<python3-devel>": "V<3.9.18-1>A<ppc>\n",
        "N<python3-idle>": "V<3.9.18-1>A<ppc>\n",
        "N<python3-jinja2>": "V<3.1.2-1>A<noarch>\n",
        "N<python3-librepo>": "V<1.11.0-103>A<ppc>\n",
        "N<python3-markupsafe>": "V<2.1.2-1>A<ppc>\n",
        "N<python3-test>": "V<3.9.18-1>A<ppc>\n",
        "N<python3-tkinter>": "V<3.9.18-1>A<ppc>\n",
        "N<python3.9-devel>": "V<3.9.18-1>A<ppc>\n",
        "N<python3.9-idle>": "V<3.9.18-1>A<ppc>\n",
        "N<python3.9-jinja2>": "V<3.1.2-1>A<noarch>\n",
        "N<python3.9-librepo>": "V<1.11.0-103>A<ppc>\n",
        "N<python3.9-markupsafe>": "V<2.1.2-1>A<ppc>\n",
        "N<python3.9-test>": "V<3.9.18-1>A<ppc>\n",
        "N<python3.9-tkinter>": "V<3.9.18-1>A<ppc>\n",
        "N<python3.9>": "V<3.9.18-1>A<ppc>\n",
        "N<python3>": "V<3.9.18-1>A<ppc>\n",
        "N<rlwrap>": "V<0.46.1-1>A<ppc>\n",
        "N<samba-client>": "V<4.18.9-1>A<ppc>\n",
        "N<samba-common>": "V<4.18.9-1>A<ppc>\n",
        "N<samba-devel>": "V<4.18.9-1>A<ppc>\n",
        "N<samba-libs>": "V<4.18.9-1>A<ppc>\n",
        "N<samba-winbind-clients>": "V<4.18.9-1>A<ppc>\n",
        "N<samba-winbind>": "V<4.18.9-1>A<ppc>\n",
        "N<samba>": "V<4.18.9-1>A<ppc>\n",
        "N<sshpass>": "V<1.10-1>A<ppc>\n",
        "N<sudo>": "V<1.9.14p3-1>A<ppc>\n",
        "N<tcl>": "V<8.6.13-1>A<ppc>\n",
        "N<tk>": "V<8.6.13-1>A<ppc>\n",
        "N<vim-common>": "V<9.0.1649-1>A<ppc>\n",
        "N<vim-enhanced>": "V<9.0.1649-1>A<ppc>\n",
        "N<wget>": "V<1.21.4-1>A<ppc>\n",
        "N<xz-libs>": "V<5.4.3-1>A<ppc>\n",
        "N<xz>": "V<5.4.3-1>A<ppc>\n",
}

kudos nick!!!

flynn1973 avatar Feb 05 '24 09:02 flynn1973