Step "gate"

[evanchaoli]

[holgerstolzenberg]

What is the current status of this RFC. Has it already landed within Concourse without being documented.

I have a clear use case for it by now:

Assume having a monorepo for Angular apps. Having all the code for all apps in one place means we need tooling to determine what to build. We achieve this by using nx (https://nx.dev/). We managed to use the across step to setup some pretty cool working pipeline, but I could not manage to dynamically push the related ui docker images, as Concourse does not support that by design (https://github.com/concourse/concourse/issues/7560).

Of course we can work around by something like (pseudo):

- in_parallel:
  - try: "determine-app1-image-was-build-task"
    on_succes:
        put: "app1-image"

but this is a bit cumbersome and always means we have to fail "determine-app1-image-was-build-task" task for not doing the image put.

Having the conditional would be great:

- in_parallel:
  - put: "app1-image"
    condition: "master-build/dist/app1/image.tar exists"
  - put: "app2-image"
    condition: "master-build/dist/app2/image.tar exists"

Do not nail me on how the condition could actually look like, but with all the options discussed above, i think it is doable and would ease up a lot of things within Concourse.

Side note: The across step should definitely make it into Concourse.

[suhlig]

In general, I am not convinced that a gate step is needed.

1. Adding more building blocks to Concourse means people need to understand even more things. I would prefer fewer, but more powerful concepts.
2. I would expect it to be very hard to come up with a reasonably universal way to express the condition. For all but the most trivial cases I prefer a programming language where I can write the code in isolation and could even add tests. With this in mind, a gate would not be that much different from a task.

[holgerstolzenberg]

Well okay I would agree that finding a simple yet powerful expression language for evaluating the boolean condition might be hard. Possibly there is already some library out there that might fit.

Second - the step would still be very useful. As I said we have a lot of use cases in our pipes, especially when it comes to monorepo stuff that in fact is a new reality for frontend teams.

A compromise could be: Just have a gate step that evaluates upon a boolean. The boolean has been made available in the pipeline before using e.g. load_var step. How you create the boolean is up to you (param, static constant, created by task). This way, the gate step would be fairly "simple" and Concourse team would not have to worry about inventing an expression language.

[JohannesRudolph]

I've built https://github.com/meshcloud/gate-resource a long while back and it happily serves our production pipelines. Alas, gates can be built as resources. The problem here is that like lots of things concourse - there's great fundamental abstractions and extension point but assembling any non-trivial behavior out of them (like conditional pipeline flows) is quite hard and convoluted. Analogy: Feels like using git without the porcelain around it.

Adding a "gate" as a first class principle to concourse would be a great opportunity to enhance the user experience for a common use case and make it a bit easier.