Copied config file may diverge from results
Checklist
- [x] I am using the current
masterbranch or the latest release. Please indicate. - [x] I am running on an up-to-date
pypsa-eurenvironment. Update viaconda env update -f envs/environment.yaml.
Describe the Bug
I solved a model, changed the config, resolved the model. Then I look at results/.../config.yaml and find that it's the file from the first run, not from the second. Thus, the stored results have diverged from the stored config. The reason is that snakemake does not execute this rule again:
rule copy_config:
params:
RDIR=RDIR,
output:
RESULTS + "config.yaml",
threads: 1
resources:
mem_mb=1000,
conda:
"../envs/environment.yaml"
script:
"../scripts/copy_config.py"
Together with @p-glaum we tried a bunch of ways to enforce re-executing this rule whenever the config.yaml changes, e.g., by adding config=config to the params section to check for changes in the snakemake.config. However, that did not work. Another idea would be to use one of the prenetworks or postnetworks as dummy input, but nothing was quite satisfactory. Scenario management and the dependence of the different solve_* rules on the copied config.yaml complicate the matter further. Any ideas?