runatlantis
"aws: command not found" in local-exec after installing AWS CLI in Atlantis image
Hi, I'm using a Bridgecrew module that, on apply, executes aws sns commands in a local-exec. The aws cli was not installed into the base Atlantis image, so I went ahead and rebuilt my Atlantis image to include the AWS CLI, as well as updated the PATH and symlinked the aws executable with /bin/aws which local-exec uses. I'm still getting the "local-exec: /bin/aws not found" error. Can you help?
I am facing a pretty much similar problem as you but using the Helm and Kubernetes provider.
Error from atlantis plan
Error: Get "https://xxxx.gr7.eu-west-1.eks.amazonaws.com/apis/rbac.authorization.k8s.io/v1/namespaces/grafana/rolebindings/delete-pod-to-platform": getting credentials: exec: executable aws not found
It looks like you are trying to use a client-go credential plugin that is not installed.
To learn more about this feature, consult the documentation available at:
https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins
my provider
kubernetes {
host = module.finder.this.endpoint
cluster_ca_certificate = base64decode(module.finder.this.certificate_authority[0].data)
exec {
api_version = "client.authentication.k8s.io/v1alpha1"
args = ["eks", "get-token", "--cluster-name", module.finder.this.id, "--role-arn", module.accounts.tf_role]
command = "aws"
}
}
}
terraform registry doc:
https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs#exec-plugins
I work fine on my local because I have awscli installed.
The following Dockerfile works and has the aws-cli available on $PATH for any commands/local-exec:
FROM runatlantis/atlantis:v0.17.1@sha256:74a1bfd6c863ef194a3bf15a4a1991a4cdc50375110a294599d7e02200a9dbb0
RUN apk --no-cache add aws-cli
Hello aws-cli is not the latest version. from an alpine with awscli installed from apk cmd:
bash-5.1# aws --version
aws-cli/1.18.177 Python/3.8.10 Linux/5.10.25-linuxkit botocore/1.19.47
from an alpine with a manual installation:
bash-5.1# aws --version
aws-cli/2.2.27 Python/3.8.8 Linux/5.10.25-linuxkit exe/x86_64.alpine.3 prompt/off
bash-5.1#
I suggest you to install awscli using the second way (munual)
FROM runatlantis/atlantis:latest
# https://github.com/aws/aws-cli/issues/4685
ENV GLIBC_VER=2.33-r0
RUN curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk \
&& curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip \
&& apk add --update --no-cache util-linux openssl python3 glibc-${GLIBC_VER}.apk glibc-bin-${GLIBC_VER}.apk \
&& unzip awscliv2.zip && aws/install \
&& rm -rf awscliv2.zip aws glibc-${GLIBC_VER}.apk glibc-bin-${GLIBC_VER}.apk \
/usr/local/aws-cli/v2/*/dist/aws_completer \
/usr/local/aws-cli/v2/*/dist/awscli/data/ac.index \
/usr/local/aws-cli/v2/*/dist/awscli/examples \
/var/cache/apk/*
Is it not currently possible to run Atlantis in a dockerfile, while being able to utilize the aws cli for things like kubernetes provider?
Anything I try from the link above, fail to install with
ERROR: glibc-2.35-r0: trying to overwrite lib/ld-linux-x86-64.so.2 owned by gcompat-1.0.0-r4.
Iām also getting similar output
---> Running in 2eef0326330d
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
(1/2) Installing glibc (2.34-r0)
ERROR: glibc-2.34-r0: trying to overwrite lib/ld-linux-x86-64.so.2 owned by gcompat-1.0.0-r4.
ERROR: glibc-2.34-r0: trying to overwrite lib64/ld-linux-x86-64.so.2 owned by gcompat-1.0.0-r4.
(2/2) Installing glibc-bin (2.34-r0)
Executing glibc-bin-2.34-r0.trigger
1 error; 49 MiB in 48 packages
The command '/bin/sh -c apk add --no-cache glibc-${GLIBC_VER}.apk glibc-bin-${GLIBC_VER}.apk' returned a non-zero code: 1```
@ganeshk1928 see this SO answer.
ARG ALPINE_VERSION=3.15.4
FROM alpine:${ALPINE_VERSION}
ARG GLIBC_VERSION=2.34-r0
ARG AWSCLI_VERSION=2.6.1
# install glibc compatibility for alpine
RUN apk --no-cache add \
binutils \
curl \
&& curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-${GLIBC_VERSION}.apk \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-bin-${GLIBC_VERSION}.apk \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-i18n-${GLIBC_VERSION}.apk \
&& apk add --no-cache \
glibc-${GLIBC_VERSION}.apk \
glibc-bin-${GLIBC_VERSION}.apk \
glibc-i18n-${GLIBC_VERSION}.apk \
&& /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8 \
&& curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64-${AWSCLI_VERSION}.zip -o awscliv2.zip \
&& unzip awscliv2.zip \
&& aws/install \
&& rm -rf \
awscliv2.zip \
aws \
/usr/local/aws-cli/v2/current/dist/aws_completer \
/usr/local/aws-cli/v2/current/dist/awscli/data/ac.index \
/usr/local/aws-cli/v2/current/dist/awscli/examples \
glibc-*.apk \
&& find /usr/local/aws-cli/v2/current/dist/awscli/botocore/data -name examples-1.json -delete \
&& apk --no-cache del \
binutils \
curl \
&& rm -rf /var/cache/apk/*
looks like newer versions of the aws-cli break this glibc shim, but this old version seems enough for configuring the kubectl and helm provider using the new client.authentication.k8s.io/v1beta1 api.
You'll just have to extend FROM the current atlantis image and add the RUN commands from above, or use an init container for Atlantis, etc, and that'll get you a 2.x version of the aws-cli binary.
@cilindrox Can you share the full dockerfile with extended FROM if possible š I'm unable to do that step š
Or
You can edit my dockerfile
# BASE IMAGE
FROM ghcr.io/runatlantis/atlantis:v0.19.6
# COPY ID_RSA KEY TO IMAGE
COPY id_rsa /home/atlantis/.ssh/id_rsa
RUN chmod 400 /home/atlantis/.ssh/id_rsa
RUN ssh-keyscan github.com > /home/atlantis/.ssh/known_hosts
RUN chown -R atlantis:atlantis /home/atlantis/.ssh/
#INSTALLING JQ, CURL AND AWS CLI
ENV GLIBC_VER=2.34-r0
RUN apk --no-cache add binutils curl jq sudo
RUN curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub
RUN curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk
RUN curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk
RUN apk add --no-cache glibc-${GLIBC_VER}.apk glibc-bin-${GLIBC_VER}.apk
RUN curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip && unzip awscliv2.zip && aws/install && rm -rf awscliv2.zip aws /usr/local/aws-cli/v2/*/dist/aws_completer /usr/local/aws-cli/v2/*/dist/awscli/data/ac.index /usr/local/aws-cli/v2/*/dist/awscli/examples
RUN rm glibc-${GLIBC_VER}.apk && rm glibc-bin-${GLIBC_VER}.apk
RUN rm -rf /var/cache/apk/*
#ADD ATLANTIS USER TO SUDO GROUP AND PROVIDE SUDO PREVILIGES
RUN echo '%wheel ALL=(ALL) ALL' > /etc/sudoers.d/wheel
RUN adduser atlantis wheel
RUN echo "atlantis ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
USER atlantis
Sure, here's an example extending from the atlantis base:
FROM ghcr.io/runatlantis/atlantis:v0.19.6@sha256:2d1efea6ae65b075a29df211809f27d8239e77e75a6fa6222778cf04f4119aef
ARG GLIBC_VERSION=2.34-r0
ARG AWSCLI_VERSION=2.6.1
# gcompat (required and installed by atlantis) triggers a conflict when
# installing the glibc shim.
RUN apk --no-cache add \
binutils \
&& curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-${GLIBC_VERSION}.apk \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-bin-${GLIBC_VERSION}.apk \
&& curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VERSION}/glibc-i18n-${GLIBC_VERSION}.apk \
&& apk del gcompat \
&& apk add --no-cache \
glibc-${GLIBC_VERSION}.apk \
glibc-bin-${GLIBC_VERSION}.apk \
glibc-i18n-${GLIBC_VERSION}.apk \
&& /usr/glibc-compat/bin/localedef -i en_US -f UTF-8 en_US.UTF-8 \
&& curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64-${AWSCLI_VERSION}.zip -o awscliv2.zip \
&& unzip awscliv2.zip \
&& aws/install \
&& rm -rf \
awscliv2.zip \
aws \
/usr/local/aws-cli/v2/current/dist/aws_completer \
/usr/local/aws-cli/v2/current/dist/awscli/data/ac.index \
/usr/local/aws-cli/v2/current/dist/awscli/examples \
glibc-*.apk \
&& find /usr/local/aws-cli/v2/current/dist/awscli/botocore/data -name examples-1.json -delete \
&& apk --no-cache del \
binutils \
&& rm -rf /var/cache/apk/*
Keep in mind I'm kinda playing with fire here, but it looks like gcompat was added for convenience on #2175 and it's not mission-critical. However, you might run into issues with some providers or custom workflows that depend on other binaries that aren't compiled against musl.
If you're using arm osx m1, then you need to run the docker build with --platform linux/arm64 or it will build but then aws --version will not work
This works
$ docker build --platform linux/amd64 -t atlantis-test .
$ docker run -it atlantis-test aws --version
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
aws-cli/2.7.19 Python/3.9.11 Linux/5.10.104-linuxkit exe/x86_64.alpine.3 prompt/off
This fails
$ docker build -t atlantis-test-arm .
$ docker run -it atlantis-test-arm aws --version
aws: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory
I wouldn't use a docker image you find on dockerhub unless it's official. It's highly encouraged to create a private Atlantis image for your organization so you can customize the necessary binaries that you workflows need.
It might be as simple as this
FROM ghcr.io/runatlantis/atlantis:v0.23.4
RUN apk add aws-cli