Load experiment configuration context from files even if not defined in experiment template

[mcastellin]

Is your feature request related to a problem? Please describe. When defining controls from file with control-file options like chaosaws.s3.controls.upload it's impossible to set the required configuration properties unless those properties are already part of the experiment template. This can be impractical if you want a control to be transparent to the template.

I may want to use the chaosaws.s3.controls.upload to upload the journal into an S3 bucket but that doesn't mean the experiment is using AWS so I shouldn't be forced to add AWS parameters into every experiment.

Describe the solution you'd like Should be able to set variables from files even if they're not defined in the experiment file.

Describe alternatives you've considered Alternatively we could have an option in the settings and control-file files to specify control-specific configuration.

Additional context Example:

experiment.yaml

title: test
description: test

steady-state-hypothesis:
  title: "All container replicas should be online"
  probes:
    - type: probe
      name: "service-must-respond"
      tolerance: 200
      provider:
        type: http
        url: "http://localhost:8080"
        method: "GET"
        timeout: 5

method: []

upload-journal.yaml

s3-upload:
  provider:
    type: python
    module: chaosaws.s3.controls.upload
    arguments:
      bucket_name: "experiment-journals"
      suffix_with_timestamp: true
      dirpath: 'journals'

local-configuration.yaml

configuration:
  aws_profile_name: default
  aws_region: eu-west-1

Running the following command at the moment doesn't work as suggested:

chaos run --control-file upload-journal.yaml --var-file local-configuration.yaml experiment.yaml

[Lawouach]

Hello @mcastellin I haven't forgotten about you. I will do my best to look at it soon.

[wride95]

(chaostk-env) C:\Users\Administrator\Desktop\chaostoolkit-aws>pip install chaoslib Collecting chaoslib Using cached chaoslib-0.0.1-Alpha.tar.gz (2.1 kB) Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Collecting randomdotorg (from chaoslib) Using cached randomdotorg-0.1.3a3.tar.gz (4.0 kB) Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [23 lines of output] Traceback (most recent call last): File "C:\Users\Administrator\chaostk-env\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 353, in main() File "C:\Users\Administrator\chaostk-env\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Administrator\chaostk-env\Lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Administrator\AppData\Local\Temp\2\pip-build-env-1u1de5f0\overlay\Lib\site-packages\setuptools\build_meta.py", line 355, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=['wheel']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Administrator\AppData\Local\Temp\2\pip-build-env-1u1de5f0\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in _get_build_requires self.run_setup() File "C:\Users\Administrator\AppData\Local\Temp\2\pip-build-env-1u1de5f0\overlay\Lib\site-packages\setuptools\build_meta.py", line 507, in run_setup super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script) File "C:\Users\Administrator\AppData\Local\Temp\2\pip-build-env-1u1de5f0\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup exec(code, locals()) File "", line 22, in File "C:\Users\Administrator\AppData\Local\Temp\2\pip-install-ku68zsv5\randomdotorg_bcb281763b1047a698e8cdf34ab27ef3\randomdotorg.py", line 187 raise ValueError, "sample larger than population" ^ SyntaxError: invalid syntax [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Getting the above error for Python 3.11.4

[cdsre]

I guess the main crux of this is that the configuration loader currently is designed to iterate over items in the configuration and updated them with the value from the extra_vars dict if the key matches. It doesn't provide a mechanism to combine them.

I think some thought is needed around all the use cases and edge cases, should a variable from a var file override an env var etc? I believe it currently does, so it could be case of building all the configuration mapping from env and vault types then merge that with the map from the extra vars with taking precedence. However this really only works for simple types and would create complex cases where the configuration type might not be a simple one.

I think before jumping into this one the community needs some guidance from the BDFL on the road map for variables and configuration in the long term. it would be naive to rush into changes now that might not fit with the longer term goals.