Don't return directory will be removed during clean if it's part of a required file's path
Description of Issue
When using file.managed with require_in and then a file.directory clean the minion returns back that the path the file is in will be removed due to clean, even though it (fortunately) actually isn't removed.
Spent a few hours trying to figure out how to get Salt to not act like it was going to remove the dir my configs are in (multiple nested dirs I loop through for the state) just to finally figure out it won't actually remove it, thankfully.
This response though is very confusing and makes verifying functionality of any states confusing for anyone that doesn't know it won't actually delete the directory. Can this be adjusted so that if the path detected is part of a required file's path it will not return the "Removed due to clean" message? It should only specify any files/directories it is actually going to remove.
Setup
{% set vhost_conf = "/shared/www/conf/"~ client_conf.path ~ dev_site~"/"~ redirect_conf_path ~ site_config['site_domain']~"-Vhosts.conf" %}
{{ vhost_conf }}:
file.managed:
- source: salt://apache/confs/sites/{{ template_path }}/site-Vhosts.conf.jinja
- template: jinja
- makedirs: True
- require_in:
- file: clean_vhost_confs
- context:
{%- for config_name in site_config %}
{{ config_name }}: {{ site_config[config_name] }}
{%- endfor %}
/shared/www/conf/site-includes.conf:
file.managed:
- source: salt://apache/confs/site-includes.conf.jinja
- template: jinja
- context:
apache_includes: {{ apache_includes }}
{# Clean up all unused Vhost configs #}
clean_vhost_confs:
file.directory:
- name: /shared/www/conf/
- clean: True
- require:
- file: /shared/www/conf/site-includes.conf
Steps to Reproduce Issue
Example here is the config files end up on the minion like this: /shared/www/conf/corp/site.domain.com-Vhosts.conf When you run the state it acts like it is going to remove the /shared/www/conf/corp directory, even though it's needed by the config file. As seen below when actually running the state it still refers to removing due to clean, but thankfully doesn't actually remove the directory. Another directory that wasn't being used for anything did get removed as expected however.
ID: clean_vhost_confs
Function: file.directory
Name: /shared/www/conf/
Result: True
Comment: Files cleaned from directory /shared/www/conf
Started: 14:55:39.891471
Duration: 10.066 ms
Changes:
----------
/shared/www/conf/corp:
----------
removed:
Removed due to clean
/shared/www/conf/corp_dev:
----------
removed:
Removed due to clean
removed:
- /shared/www/conf/corp_dev
When doing a test=true on the state and seeing it talking about removing the directory you do need it is highly disconcerting and nothing I tried could get it to stop. I even tried adding the directory to the state with a file.directory and adding it to the require but it still returned the "Removed due to clean" message.
Versions Report
Master:
Salt Version: Salt: 3005
Dependency Versions: cffi: Not Installed cherrypy: Not Installed dateutil: 2.7.3 docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 2.10 libgit2: Not Installed M2Crypto: Not Installed Mako: Not Installed msgpack: 0.5.6 msgpack-pure: Not Installed mysql-python: Not Installed pycparser: Not Installed pycrypto: Not Installed pycryptodome: 3.6.1 pygit2: Not Installed Python: 3.7.3 (default, Jan 22 2021, 20:04:44) python-gnupg: Not Installed PyYAML: 3.13 PyZMQ: 17.1.2 smmap: Not Installed timelib: Not Installed Tornado: 4.5.3 ZMQ: 4.3.1
System Versions: dist: debian 10 buster locale: UTF-8 machine: x86_64 release: 4.19.0-20-amd64 system: Linux version: Debian GNU/Linux 10 buster
Minion:
Salt Version: Salt: 3005.1
Dependency Versions: cffi: Not Installed cherrypy: Not Installed dateutil: 2.8.1 docker-py: Not Installed gitdb: Not Installed gitpython: Not Installed Jinja2: 2.11.3 libgit2: Not Installed M2Crypto: Not Installed Mako: Not Installed msgpack: 1.0.0 msgpack-pure: Not Installed mysql-python: Not Installed pycparser: Not Installed pycrypto: Not Installed pycryptodome: 3.9.7 pygit2: Not Installed Python: 3.9.2 (default, Feb 28 2021, 17:03:44) python-gnupg: Not Installed PyYAML: 5.3.1 PyZMQ: 20.0.0 smmap: Not Installed timelib: Not Installed Tornado: 4.5.3 ZMQ: 4.3.4
System Versions: dist: debian 11 bullseye locale: utf-8 machine: x86_64 release: 5.10.0-17-amd64 system: Linux version: Debian GNU/Linux 11 bullseye
Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:
- Community Wiki
- Salt’s Contributor Guide
- Join our Community Slack
- IRC on LiberaChat
- Salt Project YouTube channel
- Salt Project Twitch channel
There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at [email protected]. We’re glad you’ve joined our community and look forward to doing awesome things with you!
Thank you, I'm not the only one. I thought I must have an error in my state files and spent quite some time on this topic. For this one state file it happens consistently but if I try to reproduce it with new state file managing "dummy folders" and "dummy files" e.g. in /tmp/ I can't get it to act like that. So far I have not been able to figure out what is triggering it. I've even the same state file on one minion maintaining dir + one file, on another minion maintaining the dir + two files and it only happens on one minion. Drives me nuts, also breaks my monitoring if there are changes waiting or bloats log files where auto apply is active.
Hints about how to track down what is triggering this behavior are most welcome!