Feature: Use `google/ko` instead of `Dockerfile`

[azeemshaikh38]

Simplify our workflows and instead of Dockerfiles use google/ko.

[naveensrinivasan]

@dlorenc Does ko support tags? The cork job uses couple of tags to “latest” and “stable” to differentiate which between test runs?

[developer-guy]

@dlorenc Does ko support tags? The cork job uses couple of tags to “latest” and “stable” to differentiate which between test runs?

hello @naveensrinivasan, yes google/ko work with tags, let me give you an example:

$ KO_DOCKER_REPO=docker.io/devopps ko publish -B --tags latest,0.3.3,v0 github.com/GoTurkiye/hello-world-cli

Output

2021/10/10 18:09:30 Using base gcr.io/distroless/static:nonroot for github.com/GoTurkiye/hello-world-cli
2021/10/10 18:09:33 Building github.com/GoTurkiye/hello-world-cli for linux/amd64
2021/10/10 18:09:34 Publishing docker.io/devopps/hello-world-cli:latest
2021/10/10 18:09:36 existing blob: sha256:250c06f7c38e52dc77e5c7586c3e40280dc7ff9bb9007c396e06d96736cf8542
2021/10/10 18:09:36 existing blob: sha256:ec52731e927332d44613a9b1d70e396792d20a50bccfa06332a371e1c68d7785
2021/10/10 18:09:36 existing blob: sha256:24a21ecf55f58f84cd43d659032c2ecaad57faa9ec8ba82fdd9e3ab49a932d47
2021/10/10 18:09:37 existing blob: sha256:5a849f7331628c0129d61181f9d9f92dae6c664006cc827785072d1e23bd6442
2021/10/10 18:09:37 docker.io/devopps/hello-world-cli:latest: digest: sha256:ccb2d059265dad0c616f0b0b56bf74a14fa1242ab0cc2a49a7b08be784007d57 size: 952
2021/10/10 18:09:37 Tagging docker.io/devopps/hello-world-cli:0.3.3
2021/10/10 18:09:37 docker.io/devopps/hello-world-cli:0.3.3: digest: sha256:ccb2d059265dad0c616f0b0b56bf74a14fa1242ab0cc2a49a7b08be784007d57 size: 952
2021/10/10 18:09:37 Tagging docker.io/devopps/hello-world-cli:v0
2021/10/10 18:09:38 docker.io/devopps/hello-world-cli:v0: digest: sha256:ccb2d059265dad0c616f0b0b56bf74a14fa1242ab0cc2a49a7b08be784007d57 size: 952
2021/10/10 18:09:38 Published docker.io/devopps/hello-world-cli@sha256:ccb2d059265dad0c616f0b0b56bf74a14fa1242ab0cc2a49a7b08be784007d57
docker.io/devopps/hello-world-cli@sha256:ccb2d059265dad0c616f0b0b56bf74a14fa1242ab0cc2a49a7b08be784007d57

[naveensrinivasan]

Thank you @developer-guy!

[developer-guy]

btw @naveensrinivasan, we can do this if you want us to do, we are looking for an opportunity to contribute 🤩 👀

[naveensrinivasan]

@developer-guy Thank you! I am assigning it to you. Let me know if you have any questions.

Just an FYI there are multiple Dockerfiles in different directories. It would be great if the new ones can support Multi-Platform Images as an option.

KO_DATA_DATE_EPOCH would be useful for reproducibility https://github.com/ossf/scorecard/blob/6c1c789dc5b05cde492334f57b53807c786b038a/scripts/version-ldflags#L24

[azeemshaikh38]

Thanks for your interest in Scorecard @developer-guy and thank you for taking up this issue. Feel free to add this issue to #1121 where we are discussing our next milestone.

[developer-guy]

hello @naveensrinivasan @azeemsgoogle, I did a bunch of things in the PR but I'm not sure about what I did, so, I just want to discuss a bit about the changes that I did in here:

• I edited the dockerbuild target within the Makefile to use google/ko for building container images with cross-platform support enabled of the scorecard project. There are some limitations in google/ko, especially related to ldflags.
• I edited the GitHub Action ( .github/workflows/integration.yml) to enable google/ko support.
• I removed SOURCE_DATE_EPOCH variable from the scorecard/scripts/version-ldflags to use it within the Makefile for environment variable KO_DATA_DATE_EPOCH .

[azeemshaikh38]

Here's a suggestion - how about we start with a simple PR first which does the following:

1. Introduces a .ko.yaml file with the right ldflags and other build settings.
2. Adds a new step to dockerbuild in Makefile, which generates a local scorecard-ko image using ko.
3. Adds a step in the Makefile which diffs the images scorecard-ko and scorecard (image from Dockerfile). Consider using a tool like container-diff.

At this point, we'll have a basic setup to generate and test the scorecard-ko image. If there are important diffs in this image, we can iterate until these diffs are fixed. Once, that is accomplished we can attack the problem of replacing CloudBuild with ko. Repeat for all other Dockerfiles. Wdyt? @naveensrinivasan @developer-guy

[naveensrinivasan]

Here's a suggestion - how about we start with a simple PR first which does the following:

1. Introduces a .ko.yaml file with the right ldflags and other build settings.
2. Adds a new step to dockerbuild in Makefile, which generates a local scorecard-ko image using ko.
3. Adds a step in the Makefile which diffs the images scorecard-ko and scorecard (image from Dockerfile). Consider using a tool like container-diff.

At this point, we'll have a basic setup to generate and test the scorecard-ko image. If there are important diffs in this image, we can iterate until these diffs are fixed. Once, that is accomplished we can attack the problem of replacing CloudBuild with ko. Repeat for all other Dockerfiles. Wdyt? @naveensrinivasan @developer-guy

I agree with the plan @azeemshaikh38! Thanks

[developer-guy]

Hello @naveensrinivasan @azeemshaikh38, thank you so much for helping me.

[x] - Introduces a .ko.yaml file with the right ldflags and other build settings. [x] - Adds a new step to dockerbuild in Makefile, which generates a local scorecard-ko image using ko. [ ] - Adds a step in the Makefile which diffs the images scorecard-ko and scorecard (image from Dockerfile). Consider using a tool like container-diff.

IMHO, this commit resolves the ones that I put [x] in front of it. I'll make the third one ASAP because there are some problems with the image name that ko was built via the --local flag.

Here is why 👇 👀 https://github.com/GoogleContainerTools/container-diff/issues/366

[azeemshaikh38]

A great find by @naveensrinivasan. Adding here since its somewhat relevant to this issue - https://github.com/ImJasonH/ImJasonH/tree/main/articles/moving-and-building-images

[naveensrinivasan]

Thanks @ImjasonH

[naveensrinivasan]

Adds a step in the Makefile which diffs the images scorecard-ko and scorecard (image from Dockerfile). Consider using a tool like container-diff.

Thanks, @developer-guy! Can we skip this for this PR? @azeemshaikh38 Thoughts?

[azeemshaikh38]

Sure we can skip it for this PR.

[azeemshaikh38]

Re-opening this since I assume there is more to be done here. Is that correct @naveensrinivasan ?

[github-actions[bot]]

Stale issue message

[developer-guy]

Kindly ping @naveensrinivasan; what needs to be done? 🙏

[github-actions[bot]]

This issue is stale because it has been open for 60 days with no activity.