cli icon indicating copy to clipboard operation
cli copied to clipboard

feat(experimental): Build multiple recipes in parallel

Open gmpinder opened this issue 1 year ago • 5 comments

The build subcommand can now take in any number of recipe files and will build them all in parallel. Along with this new ability, I've added a way to easily distinguish which part of the build log belongs to which recipe. Check out the docker_build action of this PR for an example.

Examples Screen shots

image image

Tasks

  • [x] Make build log follow same pattern as normal logs to keep things consistent
  • [x] Update color ranges based on @xynydev 's feedback
  • [x] Deal with ANSI control characters in log output

gmpinder avatar May 13 '24 01:05 gmpinder

Ok so trying this out, it would seem that although GHAs support using truecolor, Gitlab CI doesn't and makes it look weird. So I'm going to try using standard ANSI colors to keep things as compatible as possible.

gmpinder avatar May 14 '24 17:05 gmpinder

Ok so trying this out, it would seem that although GHAs support using truecolor, Gitlab CI doesn't and makes it look weird. So I'm going to try using standard ANSI colors to keep things as compatible as possible.

This is probably something that shouldn't be used in a CI with build matrixing capabilities, but could be useful for some in local builds or custom CI systems I guess.

The logs being interwoven like that could get cluttered, though the custom colors help, maybe adding an option to write the logs to some directory instead would make this even better?

xynydev avatar May 18 '24 13:05 xynydev

This is probably something that shouldn't be used in a CI with build matrixing capabilities, but could be useful for some in local builds or custom CI systems I guess.

My next goal after this is to make use of buildx bake. There are situations where some stages can be shared amongst builds like compiling stages and bake would help to ensure that they do. Currently this might or might not happen when spawning docker processes during my own testing. I did try it out locally so I guess I could try adding it to this PR now that I know how that works

gmpinder avatar May 18 '24 13:05 gmpinder

The logs being interwoven like that could get cluttered, though the custom colors help, maybe adding an option to write the logs to some directory instead would make this even better?

Yeah that was the reason for the colors. Earthly has a similar pattern for their matrix builds. I think it would be reasonable/fairly easy to add log redirection too. Helps that I already created a system to handle logs

gmpinder avatar May 18 '24 13:05 gmpinder

I'll hold off on the use of buildx bake for this particular PR and add that in later. I will still work on getting logs out

gmpinder avatar May 25 '24 01:05 gmpinder