YAML/JSON interface for COSA

[darkmuggle]

One of the pain-points for RHCOS is that we have a rather complicated Openshift pipeline to build RHCOS. Some of the complication comes from business requirements, while others is that the pipeline uses different defaults. While talking data validation @cgwalters raised the idea of using a schema validation tool. I like the idea and wanted to build on it. But rather than build schema validation into he RHCOS pipeline, I would like propose that we add it to COSA and add support for describing builds using a JSON/YAML file.

For example, having the YAML file:

coreos-assembler:
   prepare:
       url: ...
       build_metadata: ...
   build:
       - oscontainer
       - qemu
       - vmarware
...

One would then call coreos-assembler job --yaml build-me-some-bits.yaml.

Core to this proposal is that we would:

• Adopted a schema for describing build, steps and arguments.
• Add a COSA target for executing descriptive steps.
• For the Python code, change argument parsing to support both YAML/JSON and cli args.
• For the bash code, have a CLI arg generator.

[dustymabe]

I think I've proposed something like this somewhere in the past (probably in an old issue somewhere). I think i was proposing something like a file that defines what you want COSA to do and then you just call it (i.e. you don't need the qemu image this time, pull it out of the config).

I'm +1 to something like this.

[ajeddeloh]

I'm +1 as long as we don't mess with the current interface to do it. We ought to have it just be a layer on top on the existing commands that parses the file then runs the other commands with the correct args. I think that's roughly what you're describing, though if that's the case I'm not sure I understand the need for parsing yaml/json in anything but the top level runner.

[darkmuggle]

I'm +1 as long as we don't mess with the current interface to do it.... I think that's roughly what you're describing...

@ajeddeloh you are correct, I don't want to break the current developer or CLI interface, just support an interface that can be used by CI that uses a file. My mental model of this interface is that coreos-assembler job would construct the CLI arguments for non-Python targets, while loading the Python ones.

I also think that breaking the existing interface is a clear deal breaker.

[darkmuggle]

Adding an interface would likely result in more OO code and I'm not sure there is a demand for it.

[cgwalters]

We talked about this again today, I think it makes sense to do something like this!

[travier]

This reminds me of the GitLab CI pipeline configuration and running a local build would be similar to running a pipeline locally.

Edit: It's apparently already possible with gitlab-runner.

[dustymabe]

I'm going to close this out as stale. If we do a re-architecture we'll probably consider something like this.