Improve `docker` support when using `kpt pkg update`

[julianKatz]

Expected behavior

I'd expect to be able to use the dockerized version of kpt in my Makefile and have it work smoothly. I'm using kpt pkg update. I've laid out the process I went through to get kpt working with docker. It required a good bit of experimentation and docker know-how. Considering that the docs show running in docker as an option, it would be good to show people how to actually run with docker. See my command below:

Actual behavior

A dockerized kpt pkg update reveals a series of usability problems that I'm not able to solve without digging into the kpt Dockefile and codebase.

Information

Example command (my starting point):

docker run gcr.io/kpt-dev/kpt:v1.0.0-beta.13 pkg update ./third_party --strategy force-delete-replace

To start off, this command doesn't work. The directory isn't available within the docker container. This is a pretty typical problem with docker, so I mounted the volume. I also had to figure out that the WORKDIR was not set in the Dockerfile, and I needed that to have a location to mount to (cannot mount to root dir in a container).

docker run --workdir="/work" --volume="$(shell pwd):/work" gcr.io/kpt-dev/kpt:v1.0.0-beta.13 pkg update ./third_party --strategy force-delete-replace

My packages updates! But, wait. Another problem:

❯ ll third_party/src/pod-security-policy/volumes
.rw-r--r-- 1.3k root 21 Mar 13:31 constraint.tmpl
.rw-r--r--  579 root 21 Mar 13:31 src.rego
.rw-r--r-- 4.3k root 21 Mar 13:31 src_test.rego

All of my files have been chown'd b/c the user in the container is root. I fix by adding my user/group:

docker run --user="$(shell id -u):$(shell id -g)" --workdir="/work" --volume="$(shell pwd):/work" gcr.io/kpt-dev/kpt:v1.0.0-beta.13

Another error. kpt is making a temporary directory and the default value doesn't work inside of docker: Error: error creating cache directory for repo: mkdir /.kpt: permission denied

I set the environment variable:

docker run -e "KPT_CACHE_DIR=/work/.kpt" --user="$(shell id -u):$(shell id -g)" --workdir="/work" --volume="$(shell pwd):/work" gcr.io/kpt-dev/kpt:v1.0.0-beta.13

Packages are updating! Hooray. But, note that I had to chown my filesystem back to my user before this worked. Trying to overwrite the root owned files with the docker run command did not work.

Also, kpt doesn't cleanup after itself:

❯ git status --short
?? .kpt/

Considering that dockerization is the easiest way to lock down a dependency across multiple teammembers' machines, this seems like a story worth improving.

[droot]

Thank for for such a detailed report. I totally agree that we can have a tutorial or guide under kpt.dev/guides/how-to-use-kpt-as-container.

@julianKatz will you be up for contributing a basic tutorial, I can help in reviewing it and getting it merged ? (no pressure ofcourse :))

[julianKatz]

I'll have to respectfully decline. Seems like something worthy of a real work item on the kpt team, as it's part of the docs for the product. What I've done here should be a good place to start.

[yuwenma]

@julianKatz are you asking a feature request that using kpt pkg update via kpt image which currently does not (fully) work? If so, I'd like to apply the "enhancement" label. I think we need to iterate the behavior of kpt pkg update to decide whether such feature request is feasible and define the right scope for kpt.

Otherwise, I agree with @droot that this is a good first issue for new contributors

[julianKatz]

Basically I'm saying that using kpt with docker run required me to do a bunch of extra steps to get it to work. Many CLI tools work much better than kpt when used in this way.