Create pantsbuild `BUILD` files with `./pants tailor ::`

[cognifloyd]

Background

This is another part of introducing pants, as discussed in the TSC Meetings on 12 July 2022, 02 Aug 2022 and 06 Sept 2022. Pants has fine-grained per-file caching of results for lint, fmt (like black), test, etc. It also has lockfiles that work well for monorepos that have multiple python packages. With these lockfiles CI should not break when any of our dependencies or our transitive dependencies release new versions, because CI will continue to use the locked version until we explicitly relock with updates.

To keep PRs as manageable/reviewable as possible, introducing pants will take a series of PRs. I do not know yet how many PRs; I will break this up into logical steps with these goals:

• introduce pants to the st2 repo, and
• teach some (but not all) TSC members about pants step-by-step.

Other pants PRs include:

• https://github.com/StackStorm/st2/pull/5713
• https://github.com/StackStorm/st2/pull/5724
• https://github.com/StackStorm/st2/pull/5725
• https://github.com/StackStorm/st2/pull/5726
• https://github.com/StackStorm/st2/pull/5732
• https://github.com/StackStorm/st2/pull/5733
• https://github.com/StackStorm/st2/pull/5737

Overview of this PR

This PR does 3 things:

• Run ./pants tailor :: (step 5 of the initial config docs)
• Enable the workflow added in #5732
• Fix an edge case with our BUILD metadata to work around our use of git submodules (the test_content_version fixture pack)

I will describe the generated BUILD file contents after I have briefly discussed each of them (the 3 things this PR does).

./pants tailor ::

All but one of the BUILD files in this PR were created by running ./pants tailor :: which is step 5 of the initial config docs.

As described in #5732, BUILD files are an important part of helping pants understand our codebase. As the docs say:

BUILD files provide metadata about your code (the timeout of a test, any dependencies which cannot be inferred, etc). BUILD files are typically located in the same directory as the code they describe. Unlike many other systems, Pants BUILD files are usually very succinct, as most metadata is either inferred from static analysis, assumed from sensible defaults, or generated for you.

In follow-up PRs, we will customize the BUILD metadata further.

GHA workflow

This PR also enables the GHA workflow added in #5732 (it also fixes a typo in the workflow. oops). This workflow will ensure that we have all the BUILD files that pants needs. This is very quick, and serves as an indicator on PRs that people might need to run ./pants tailor ::.

git submodule: test_content_version fixture pack

./pants tailor tried to add BUILD files in the git submodule. But, we cannot guarantee that git submodule is checked out when we run ./pants. So, I added some targets in the parent directory's BUILD file that covers everything in the git submodule. This way, even if the submodule is not checked out, the complete set of metadata is available for pants.

I wrote a plugin for pants that makes the developer UX even better. It checks to make sure the git submodule is checked out, and if not, it provides an error message with instructions on how to fix it. I will submit that in a later PR.

BUILD file metadata

BUILD files contain metadata that guide pants through our source code. That metadata consists of "targets" with "fields".

From the docs: https://www.pantsbuild.org/docs/targets

Targets are an addressable set of metadata describing your code.

Each target type has different fields, or individual metadata values.

One of the nice things about BUILD files, is that they are essentially just python files. In fact, ./pants update-build-files :: actually uses black to reformat them!

So, when looking at BUILD files, defining a target looks like a function call, and the fields are args and/or kwargs in that function call.

There are some restrictions on what python we can use in BUILD files. For example: there are no imports (Pants actually adds all of the required symbols based on which backends are enabled in pants.toml). When we need to add custom targets, we can do that with macros and pants plugins, both of which are also written in python. I have written both macros and pants plugins to add in follow-up PRs.

Targets

These are the targets that ./pants tailor :: added in this PR (linked to the relevant docs page):

• python_requirements (see also: python_requirement)
• python_sources (see also: python_source)
• python_test_utils
• python_tests (see also: python_test)
• shell_sources (see also: shell_source)

I also added some of these:

• resources (see also: shell_source)

Tailor cannot add resources or files targets, so those will be added manually in follow-up PRs.