eddiewebb
[3.1.4] How to block workflows that are triggered with commits and tags
Orb version
3.1.4
What happened
I'm working on a project that defines two main CircleCI workflows:
- One is triggered on every commit on the main branch for staging and run something similar to this:
- [block_e2e] -> [build_deploy_e2e] -> [run_e2e] -> [block_deploy_stg] -> [build_stg] -> [deploy_stg]
- One is triggered on tags on the main branch and run something very much similar but deploys on production instead:
- [block_e2e] -> [build_deploy_e2e] -> [run_e2e] -> [block_deploy_prod] -> [build_prod] -> [deploy_prod]
The issue I have is that the building, deploying and testing of the end to end tests must not ran concurrently, whatever the workflow.
For the block_e2e job of the "commit" workflow I'm using the following definition
- queue/block_workflow:
name: block_e2e
max-wait-time: '180'
this-branch-only: false
job-regex: '^build_deploy_e2e|run_e2e$'
my-pipeline: << pipeline.number >>
confidence: '2'
For the block_e2e job of the "tag" workflow, I'm using the same definition but I'm adding a filters defintion to trigger the job in CircleCI and a tag-pattern:
- queue/block_workflow:
filters:
tags:
only: /^v.*$/
branches:
ignore: /.*/
name: block_e2e
tag-pattern: '^v.*$'
max-wait-time: '180'
this-branch-only: false
job-regex: '^build_deploy_e2e|run_e2e$'
my-pipeline: << pipeline.number >>
confidence: '2'
The issue I'm having is that by adding the tag-pattern variable it ignores the workflow running for the "commit" workflow (at least from what I understand of this line
But if I don't specify the tag-pattern parameter the queue blocker won't run, because of this condition
Expected behavior
I would like to have my workflow blocked according to every workflow (both triggered by tags or commits)
I might not be fully aligned, but if you want the workflow to consider any matching workflow, tagged or commit, then don't specify a tag pattern. You are telling it to ignore commit (and other sources of change) by specifying it.
The job regex you have should cover the screenshot you described(I think)
On Wed, May 29, 2024, 6:17 AM Arthur Knoepflin @.***> wrote:
Orb version
3.1.4 What happened
I'm working on a project that defines two main CircleCI workflows:
- One is triggered on every commit on the main branch for staging and run something similar to this:
- [block_e2e] -> [build_deploy_e2e] -> [run_e2e] -> [block_deploy_stg] -> [build_stg] -> [deploy_stg]
- One is triggered on tags on the main branch and run something very much similar but deploys on production instead:
- [block_e2e] -> [build_deploy_e2e] -> [run_e2e] -> [block_deploy_prod] -> [build_prod] -> [deploy_prod]
The issue I have is that the building, deploying and testing of the end to end tests must not ran concurrently, whatever the workflow.
For the block_e2e job of the "commit" workflow I'm using the following definition
- queue/block_workflow: name: block_e2e max-wait-time: '180' this-branch-only: false job-regex: '^build_deploy_e2e|run_e2e$' my-pipeline: << pipeline.number >> confidence: '2'For the block_e2e job of the "tag" workflow, I'm using the same definition but I'm adding a filters defintion to trigger the job in CircleCI and a tag-pattern:
- queue/block_workflow: filters: tags: only: /^v.*$/ branches: ignore: /.*/ name: block_e2e tag-pattern: '^v.*$' max-wait-time: '180' this-branch-only: false job-regex: '^build_deploy_e2e|run_e2e$' my-pipeline: << pipeline.number >> confidence: '2'The issue I'm having is that by adding the tag-pattern variable it ignores the workflow running for the "commit" workflow (at least from what I understand of this line https://github.com/eddiewebb/circleci-queue/blob/edfb701b59d73a64dcc4c94af6d42ed4a3fe1f9b/scripts/loop.bash#L139
But if I don't specify the tag-pattern parameter the queue blocker won't run, because of this condition https://github.com/eddiewebb/circleci-queue/blob/edfb701b59d73a64dcc4c94af6d42ed4a3fe1f9b/scripts/loop.bash#L54 Expected behavior
I would like to have my workflow blocked according to every workflow (both triggered by tags or commits)
— Reply to this email directly, view it on GitHub https://github.com/eddiewebb/circleci-queue/issues/130, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIEWSXYXOY4AOYCO4D765DZEXBOFAVCNFSM6AAAAABIO2DWA6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGMZDGMJXHA2TANQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>
Should satisfy the scenario you described.
On Thu, May 30, 2024, 7:12 PM Edward A. Webb @.***> wrote:
I might not be fully aligned, but if you want the workflow to consider any matching workflow, tagged or commit, then don't specify a tag pattern. You are telling it to ignore commit (and other sources of change) by specifying it.
The job regex you have should cover the screenshot you described(I think)
On Wed, May 29, 2024, 6:17 AM Arthur Knoepflin @.***> wrote:
Orb version
3.1.4 What happened
I'm working on a project that defines two main CircleCI workflows:
- One is triggered on every commit on the main branch for staging and run something similar to this:
- [block_e2e] -> [build_deploy_e2e] -> [run_e2e] -> [block_deploy_stg] -> [build_stg] -> [deploy_stg]
- One is triggered on tags on the main branch and run something very much similar but deploys on production instead:
- [block_e2e] -> [build_deploy_e2e] -> [run_e2e] -> [block_deploy_prod] -> [build_prod] -> [deploy_prod]
The issue I have is that the building, deploying and testing of the end to end tests must not ran concurrently, whatever the workflow.
For the block_e2e job of the "commit" workflow I'm using the following definition
- queue/block_workflow: name: block_e2e max-wait-time: '180' this-branch-only: false job-regex: '^build_deploy_e2e|run_e2e$' my-pipeline: << pipeline.number >> confidence: '2'For the block_e2e job of the "tag" workflow, I'm using the same definition but I'm adding a filters defintion to trigger the job in CircleCI and a tag-pattern:
- queue/block_workflow: filters: tags: only: /^v.*$/ branches: ignore: /.*/ name: block_e2e tag-pattern: '^v.*$' max-wait-time: '180' this-branch-only: false job-regex: '^build_deploy_e2e|run_e2e$' my-pipeline: << pipeline.number >> confidence: '2'The issue I'm having is that by adding the tag-pattern variable it ignores the workflow running for the "commit" workflow (at least from what I understand of this line https://github.com/eddiewebb/circleci-queue/blob/edfb701b59d73a64dcc4c94af6d42ed4a3fe1f9b/scripts/loop.bash#L139
But if I don't specify the tag-pattern parameter the queue blocker won't run, because of this condition https://github.com/eddiewebb/circleci-queue/blob/edfb701b59d73a64dcc4c94af6d42ed4a3fe1f9b/scripts/loop.bash#L54 Expected behavior
I would like to have my workflow blocked according to every workflow (both triggered by tags or commits)
— Reply to this email directly, view it on GitHub https://github.com/eddiewebb/circleci-queue/issues/130, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIEWSXYXOY4AOYCO4D765DZEXBOFAVCNFSM6AAAAABIO2DWA6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGMZDGMJXHA2TANQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>
Hi, thanks for your answer.
But if I do not specify the tag-pattern the queue just doesn't run on tag trigger (production workflow). I get this message in the logs:
TAG_PATTERN defined, but not on tagged run, skip queueing!
Which seems to come from this condition: https://github.com/eddiewebb/circleci-queue/blob/edfb701b59d73a64dcc4c94af6d42ed4a3fe1f9b/scripts/loop.bash#L54
I see ..
We can drop that check and see if all use cases in bats test still pass.
Desired state is that supplying a tag will only concern itself with tagged builds, but omission will run on all, including tagged.
That sound right?
On Tue, Jun 4, 2024, 3:44 AM Arthur Knoepflin @.***> wrote:
Hi, thanks for your answer.
But if I do not specify the tag-pattern the queue just doesn't run on tag trigger (production workflow). I get this message in the logs:
TAG_PATTERN defined, but not on tagged run, skip queueing!
Which seems to come from this condition: https://github.com/eddiewebb/circleci-queue/blob/edfb701b59d73a64dcc4c94af6d42ed4a3fe1f9b/scripts/loop.bash#L54
— Reply to this email directly, view it on GitHub https://github.com/eddiewebb/circleci-queue/issues/130#issuecomment-2146834309, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIEWSWI7XN5ODYAIJTWDRLZFVV6FAVCNFSM6AAAAABIO2DWA6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBWHAZTIMZQHE . You are receiving this because you commented.Message ID: @.***>
Sorry for the delay I was caught up in work
Yes that would seems to fix my issue