gtfs-validator icon indicating copy to clipboard operation
gtfs-validator copied to clipboard

ci: build & publish linux/arm64 Docker images

Open derhuerst opened this issue 2 years ago • 10 comments

On an ARM64 system, I would like to use native Docker images in order not to run an emulation layer.

This PR extends the Docker build & publishing CI workflow to also build linux/arm64 images in addition to linux/amd64 images (implicit default).

Please make sure these boxes are checked before submitting your pull request - thanks!

  • [ ] Run the unit tests with gradle test to make sure you didn't break anything
  • [ ] Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • [ ] Linked all relevant issues
  • [ ] Include screenshot(s) showing how this pull request works and fixes the issue(s)

derhuerst avatar Mar 30 '23 18:03 derhuerst

Thanks for opening this pull request! You're awesome. We use semantic commit messages to streamline the release process. Before your pull request can be merged, you should update your pull request title to start with a semantic prefix. Examples of titles with semantic prefixes:

  • fix: Bug with ssl network connections + Java module permissions.
  • `feat: Initial support for multiple @PrimaryKey annotations.
  • docs: update RELEASE.md with new process To get this PR to the finish line, please do the following:
  • Read our Contribution Guidelines
  • Follow Google Java style coding standards
  • Include tests when adding/changing behavior
  • Include screenshots

welcome[bot] avatar Mar 30 '23 18:03 welcome[bot]

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

:white_check_mark: davidgamez
:x: derhuerst
You have signed the CLA already but the status is still pending? Let us recheck it.

CLAassistant avatar Mar 30 '23 18:03 CLAassistant

I think it would make sense to temporarily run the Docker build in this PR too, without publishing of course. What do you think?

derhuerst avatar Mar 30 '23 18:03 derhuerst

@derhuerst do you still want to get this checked in? You'll need to sign the CLA before we can do so.

Regarding running the Docker build against this PR. I'll admit I'm not totally clear on how to do that, since the Docker workflow doesn't run for PRs by default. That said, this change seems mostly reasonable, so we could probably check it in as-is and rollback if it causes some issue with the Docker build? Or did you have a different concern in mind?

bdferris-v2 avatar Apr 18 '23 23:04 bdferris-v2

Am I reading correctly that this failed to run on your repo per https://github.com/derhuerst/gtfs-validator/actions/runs/4741545166/jobs/8418790748 ? Any thoughts?

bdferris-v2 avatar Apr 19 '23 20:04 bdferris-v2

Not sure 🤷

derhuerst avatar Apr 19 '23 22:04 derhuerst

There is a long running bug with Docker (https://github.com/docker/buildx/issues/59) where multi-platform builds don't work with a load, only a push. I think you can see that in the error from your Docker run:

ERROR: docker exporter does not currently support exporting manifest lists

I think the solution here would be to only build a single platform for the test Docker build but then enable multi platforms for the push. Thoughts?

bdferris-v2 avatar Apr 20 '23 17:04 bdferris-v2

I think the solution here would be to only build a single platform for the test Docker build but then enable multi platforms for the push. Thoughts?

Sounds good, as I'm mainly interested in having a native Docker image to pull and use.

derhuerst avatar Apr 20 '23 18:04 derhuerst

@derhuerst I created a draft PR with an alternative solution based on @bdferris-v2 comment. Can I get some feedback on #1528 ? Thanks!

davidgamez avatar Jul 05 '23 13:07 davidgamez

Btw, why not set FROM --platform=$BUILDPLATFORM gradle:7-jdk11-alpine AS build in the Dockerfile so the actual building only has to be done once.

For reference, I ended up just rolling with

FROM eclipse-temurin:17-jdk-jammy as valid-gtfs
COPY --from=ghcr.io/mobilitydata/gtfs-validator:4.1.0 gtfs-validator-cli.jar /
WORKDIR /work
COPY foo-gtfs.zip .
RUN java -jar /gtfs-validator-cli.jar -i /work/foo-gtfs.zip -o /work/out/

for running on ARM.

Side note: The Dockerfile currently uses FROM openjdk, which is deprecated.

jcfj avatar Jul 19 '23 02:07 jcfj