saltstack
[BUG] `prereq`'d state with `onlyif`/`unless` + `fun`: onlyif requisite ignored
Description
A state that is prerequired and itself has an onlyif (or unless) requisite that calls an execution module will run regardless of outcome of the onlyif evaluation.
Setup
- [x] on-prem machine
- [x] VM (Virtualbox, KVM, etc. please specify)
- [ ] VM running on a cloud service, please be explicit and add details
- [ ] container (Kubernetes, Docker, containerd, etc. please specify)
- [ ] or a combination, please be explicit
- [ ] jails if it is FreeBSD
- [ ] classic packaging
- [x] onedir packaging
- [x] used bootstrap to install
Steps to Reproduce the behavior
Copy stuff:
file.copy:
- name: /tmp/aa
- source: /tmp/a
- onlyif:
- fun: file.file_exists
path: /tmp/a
Do sth else:
cmd.run:
- name: true
- prereq:
- Copy stuff
Expected behavior
local:
----------
ID: Do sth else
Function: cmd.run
Result: True
Comment: No changes detected
Started: 06:41:33.485261
Duration: 0.007 ms
Changes:
----------
ID: Copy stuff
Function: file.copy
Name: /tmp/aa
Result: True
Comment: onlyif condition is false
Started: 06:41:33.485360
Duration: 8.103 ms
Changes:
Summary for local
------------
Succeeded: 2
Failed: 0
------------
Total states run: 2
Total run time: 8.110 ms
Screenshots
[WARNING ] no `fun` argument in onlyif: OrderedDict([('path', '/tmp/a')])
[ERROR ] Source file "/tmp/a" is not present
local:
----------
ID: Do sth else
Function: cmd.run
Result: True
Comment: No changes detected
Started: 06:37:15.121723
Duration: 0.008 ms
Changes:
----------
ID: Copy stuff
Function: file.copy
Name: /tmp/aa
Result: False
Comment: Source file "/tmp/a" is not present
Started: 06:37:15.121817
Duration: 80.259 ms
Changes:
Summary for local
------------
Succeeded: 1
Failed: 1
------------
Total states run: 2
Total run time: 80.267 ms
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)Salt Version:
Salt: 3006.3
Python Version:
Python: 3.10.13 (main, Sep 6 2023, 02:11:27) [GCC 11.2.0]
Dependency Versions:
cffi: 1.14.6
cherrypy: unknown
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.2
libgit2: Not Installed
looseversion: 1.0.2
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 22.0
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.9.8
pygit2: Not Installed
python-gnupg: 0.4.8
PyYAML: 6.0.1
PyZMQ: 23.2.0
relenv: 0.13.10
smmap: Not Installed
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: rocky 9.2 Blue Onyx
locale: utf-8
machine: x86_64
release: 5.14.0-284.18.1.el9_2.x86_64
system: Linux
version: Rocky Linux 9.2 Blue Onyx
Additional context
The prereq results in the state to be run twice. During the first run, fun (and args) is popped from the passed-in lowstate chunk and is thus unavailable for the second (non-test) run:
https://github.com/saltstack/salt/blob/fb717a8d4bea3d48c596ae8dda313b7639c23dc6/salt/state.py#L1028-L1034
The lowstate will need to be deep-cloned at an early stage. Hopefully there isn't a feature that relies on being able to change it globally in a state run...
This also occurs when using unless instead of onlyif. Why is this scheduled for Argon?
I think this bugs does not occur any more, the following snippet run correctly in a stateful manner and all cases are correctly covered (watch_in, prereq_in, require_in, unless) :
{%- set master_modules_deps = ['pygit2', 'python-ldap', 'psycopg2-binary'] %}
{%- set master_modules_deps_run_deps = [ "git" ] %}
{%- set master_modules_deps_build_deps = [ "gcc", "libldap2-dev", "libsasl2-dev" ] %} # <- Debian specific
master_modules_deps_run_deps_pkg_installed:
pkg.latest:
- names: {{ master_modules_deps_run_deps }}
- update: True
master_modules_deps_build_deps_pkg_installed:
pkg.latest:
- names: {{ master_modules_deps_build_deps }}
- update: True
master_modules_deps_downgrade_pip_workaround_pip_installed:
pip.installed:
- name: pip==22.3.1
{%- for master_modules_dep in master_modules_deps %}
master_modules_dep_{{ master_modules_dep }}_pip_installed:
pip.installed:
- name: {{ master_modules_dep }}
- upgrade: True
- unless: # dep installed and up to date
- fun: pip.is_installed
pkgname: {{ master_modules_dep }}
- fun: cmd.run
cmd: ! salt-pip list --outdated | grep {{ master_modules_dep }}
- prereq_in:
- pkg: master_modules_deps_run_deps_pkg_installed
- pkg: master_modules_deps_build_deps_pkg_installed
- pip: master_modules_deps_downgrade_pip_workaround_pip_installed
- require_in:
- pkg: master_modules_deps_build_dep_pkg_removed
- watch_in:
- service: salt-master
{%- endfor %}
master_modules_deps_build_dep_pkg_removed:
pkg.removed:
- names: {{ master_modules_deps_build_deps }}
master_pip_pip_update:
pip.installed:
- name: pip
- upgrade: true
- onlyif:
- fun: pip.upgrade_available
pkg: pip
I just get a warning that is irrelevant as the states and logic runs correctly:
[WARNING ] no `fun` argument in unless: OrderedDict([('pkgname', 'pygit2')])
[WARNING ] no `fun` argument in unless: OrderedDict([('pkgname', 'python-ldap')])
[WARNING ] no `fun` argument in unless: OrderedDict([('pkgname', 'psycopg2-binary')])
It is great to finally have this kind of sls run correctly, it wasn't the case few months ago
@sticky-note Are you absolutely certain it's running as expected? The ~~error~~ log message is the same as before and is logged just before the instruction is skipped.
In my case, with this code snippet, all cases are handled correctly, I am running:
salt-call --versions
Salt Version:
Salt: 3006.6
Python Version:
Python: 3.10.13 (main, Nov 15 2023, 04:34:27) [GCC 11.2.0]
Dependency Versions:
cffi: 1.16.0
cherrypy: 18.6.1
dateutil: 2.8.1
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.3
libgit2: 1.7.1
looseversion: 1.0.2
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.2
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 22.0
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: 1.14.0
python-gnupg: 0.4.8
PyYAML: 6.0.1
PyZMQ: 23.2.0
relenv: 0.14.2
smmap: Not Installed
timelib: 0.2.4
Tornado: 4.5.3
ZMQ: 4.3.4
System Versions:
dist: debian 12 bookworm
locale: utf-8
machine: x86_64
release: 6.1.0-17-amd64
system: Linux
version: Debian GNU/Linux 12 bookworm
prereq_in and require_in are correctly run when they should and the (AND) condition on unless also.
The only case when this code snippets does nothing is when all master_modules_deps are installed AND up to date.
Hope this is useful