Ignore/exclude files

[gerardo]

It seems like there's no way to ignore build requests if specific files/folders were modified. We increment our project's version and create a git tag after all the build steps were successful, then push those changes back to GHE; that triggers the buildkite webhook and starts another build, thus creating a cycle.

It would be great if Buildkite had a setting to ignore build if the modified files match an "ignore/exclude" regex, ala Bamboo.

Another user case: Don't trigger a build if README.md was updated.

[arjunmukherjee]

+1

[tigris]

I've thought about this a lot. E.g. you could have a .buildkite/hooks/pre-command hook in your repo or something, that poked around at the commit, and "failed" if you didn't wanna run it... Less than ideal since you get a fail instead of a skip, but if running the build is expensive, it may work.

You can also be more diligent when writing commit messages and making sure you use [skip ci] where appropriate. I wonder if you could automate that with a git hook (e.g. something adds the skip ci to the message based on certain rules).

From a CI perspective, it's hard to implement, as you have no idea how many commits this build is for. E.g. if i commit 20 times then push, buildkite is only gonna run a build for the last commit, so when you say "ignore XYZ files", it's only looking at a git diff of the last commit.

[Jolg42]

GitHub Actions have something simple from user perspective that works really well paths-ignore https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#excluding-paths

example

on:
  push:
    paths-ignore:
      - 'docs/**'
      - '*.md'
      - 'something.txt'

Any idea how to this with Buildkite, alternatives?

[HariSekhon]

BuildKite definitely needs to add a paths-ignore equivalent.

Jenkins and GitHub Actions both have this basic functionality.

So it's a matter of CI/CD maturity for BuildKite to get it too.

[HammoTime]

I've just implemented a simple script in my pipeline to provide this functionality. This one specifically is scoped to the containers directory. I then have separate step files that are then loaded in dynamically (you could hard code per directory, etc).

FILES_CHANGED=$(git diff --name-only HEAD~1)
PROJECT_IMAGES=$(ls containers)

if [ "$DEPLOY_ALL" = "true" ]; then
  for image in $PROJECT_IMAGES; do
    echo "'$image' has been queued for build (DEPLOY_ALL)."
    step_file=".buildkite/container_steps/${image}.yaml"
    buildkite-agent pipeline upload "$step_file"
  done
else
  for image in $PROJECT_IMAGES; do
    echo "Checking whether '$image' has been updated."
    image_updated="false"
    for changed_file in $FILES_CHANGED; do
      image_regex="^containers/${image}.*"
      match_count=$(expr "$changed_file" : "$image_regex")

      if [ $match_count -gt 0 ]; then
        image_updated="true"
      fi
    done
    
    if [ "$image_updated" = "true" ]; then
      echo "'$image' has been updated."
      step_file=".buildkite/container_steps/${image}.yaml"
      buildkite-agent pipeline upload "$step_file"
    else
      echo "'$image' has not been updated, skipping."
    fi
  done
fi

The above can then be executed by a simple step:

  - label: ":git: Diff Container Images and Execute Build Steps (if applicable)"
    command: ".buildkite/scripts/diff.sh"
    agents:
      queue: $BUILD_AGENT

[tigris]

FILES_CHANGED=$(git diff --name-only HEAD~1)

I'm curious how you can guarantee this is meaningful? What if someone does a massive commit, then 1 extra commit fixing a simply typo, and pushes all of that at once? The build will only see the last file changed.

I'm also curious how other CI systems handle for that, e.g. the github actions example noted above.

Even if you git diff against the last build (you need a PAT to get that info), what if that build wasn't successful? Should your diff be against the commit sha for the last successful build on this same branch?

[HammoTime]

We currently have builds that can take up to 10 minutes that don't need to be run every time. If someone is debugging, they can easily spend an extra 5 minutes per build on about 10 builds (5 minutes of that build time would happen anyway so I'll discount that). These builds run before our code gets deployed and a code defect is raised. That's 50 minutes of developer time per commit. I've added in an extra "break-glass" option which will run every step and ignore the diff output (New Pipeline -> Add Env Variable "BREAK_GLASS=true"). If someone has a typo, pushes the build, and the build doesn't trigger it will maybe take that person an extra two minutes. I would rather take my 50 minutes to the bank every day, rather than worry about an extra two minutes on a build once or twice a week.

Furthermore, we currently have a pipeline that then triggers two pipelines. One of them is for infrastructure and the other one is related to code deploy. We absolutely do not want to run the infrastructure step every time, because it can take 20-25 minutes to run. However, we cannot run our deploy step until the infrastructure step is run, if it is required. It is very common for developers to not understand the pipeline (it is quite complex), and miss this part which breaks our development environments (main deploys everything, so is not a problem for production). By having the diff step, it improves quality outcomes (i.e. our environments won't break) and reduces time (we don't have to run the infra step every time but can be confident in knowing that if it is required, it will run).

Cumulatively, this small change will save us at least 3-4 days of engineering time per month. You are right, there may be small defects that need to be worked around, but ultimately we're saving so much time those small defects do not matter. I understand wanting to persue perfection, but sometimes you gotta crack a few eggs to make an omellete.