features icon indicating copy to clipboard operation
features copied to clipboard

Terraform devcontainer feature v1.3.4 gpg error

Open f0o opened this issue 2 years ago • 23 comments

0.410 ===========================================================================
0.410 Feature       : Terraform, tflint, and TFGrunt
0.410 Description   : Installs the Terraform CLI and optionally TFLint and Terragrunt. Auto-detects latest version and installs
[2023-09-09T17:26:31.353Z]  needed dependencies.
0.410 Id            : ghcr.io/devcontainers/features/terraform
0.410 Version       : 1.3.4
0.410 Documentation : https://github.com/devcontainers/features/tree/main/src/terraform
0.410 Options       :
0.410     VERSION="latest"
0.410     TFLINT="latest"
0.410     TERRAGRUNT="latest"
0.410     INSTALLSENTINEL="false"
0.410     INSTALLTFSEC="false"
0.410     INSTALLTERRAFORMDOCS="false"
0.410     HTTPPROXY=""
0.410 ===========================================================================
0.863 TERRAFORM_VERSION=1.5.7
1.285 TFLINT_VERSION=0.48.0
1.760 TERRAGRUNT_VERSION=0.50.14
1.762 Downloading terraform...
2.451 (*) Downloading GPG key...
2.549 gpg: keyserver receive failed: Server indicated a failure
2.551 (*) Failed getting key, retring in 10s...
12.55 (*) Downloading GPG key...
12.62 gpg: keyserver receive failed: Server indicated a failure
12.62 (*) Failed getting key, retring in 10s...
22.62 (*) Downloading GPG key...
22.69 gpg: keyserver receive failed: Server indicated a failure
22.69 (*) Failed getting key, retring in 10s...
32.70 (*) Downloading GPG key...
32.78 gpg: keyserver receive failed: Server indicated a failure
32.78 (*) Failed getting key, retring in 10s...
42.78 (*) Downloading GPG key...
42.85 gpg: keyserver receive failed: Server indicated a failure
42.85 (*) Failed getting key, retring in 10s...
52.86 (!) Failed to get gpg key.
52.86 ERROR: Feature "Terraform, tflint, and TFGrunt" (ghcr.io/devcontainers/features/terraform) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/terraform for help troubleshooting this error.
------

how to go from here?

f0o avatar Sep 09 '23 17:09 f0o

Hi 👋

Does your machine use a proxy environment? If so, you would need to update the httpProxy Feature option with the proxy, which will then create a successful connection with the key server.

samruddhikhandale avatar Sep 12 '23 03:09 samruddhikhandale

No Proxies @samruddhikhandale

f0o avatar Sep 12 '23 07:09 f0o

Interesting, terraform Feature with the options specified in the logs works fine for me. @f0o can you provide a sample dev container config for reproducing this issue? Thanks!

samruddhikhandale avatar Sep 12 '23 15:09 samruddhikhandale

I tried rerunning it today and it worked, I guess gpg-servers had a boo-boo?

Anyway the devconfig was:

// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/go
{
	"name": "Go",
	// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
	"image": "mcr.microsoft.com/devcontainers/go:1-1.21-bullseye",
	"features": {
		"ghcr.io/devcontainers/features/terraform:1": {}
	}

	// Features to add to the dev container. More info: https://containers.dev/features.
	// "features": {},

	// Use 'forwardPorts' to make a list of ports inside the container available locally.
	// "forwardPorts": [],

	// Use 'postCreateCommand' to run commands after the container is created.
	// "postCreateCommand": "go version",

	// Configure tool-specific properties.
	// "customizations": {},

	// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
	// "remoteUser": "root"
}

Closing the issue as no longer reproducible; Thanks!

f0o avatar Sep 13 '23 06:09 f0o

Confirmed this is still happening. Windows 11, WSL UBUNTU, No proxies, nothing

0.810 TERRAFORM_VERSION=1.5.7
1.087 TFLINT_VERSION=0.48.0
1.415 TERRAGRUNT_VERSION=0.50.17
1.417 Downloading terraform...
3.301 (*) Downloading GPG key...
33.36 gpg: keyserver receive failed: Server indicated a failure
33.36 (*) Failed getting key, retring in 10s...
43.36 (*) Downloading GPG key...
73.41 gpg: keyserver receive failed: Server indicated a failure
73.41 (*) Failed getting key, retring in 10s...
83.41 (*) Downloading GPG key...
113.5 gpg: keyserver receive failed: Server indicated a failure
113.5 (*) Failed getting key, retring in 10s...
123.5 (*) Downloading GPG key...
153.5 gpg: keyserver receive failed: Server indicated a failure
153.5 (*) Failed getting key, retring in 10s...
163.5 (*) Downloading GPG key...
193.5 gpg: keyserver receive failed: Server indicated a failure
193.6 (*) Failed getting key, retring in 10s...
203.6 (!) Failed to get gpg key.
203.6 ERROR: Feature "Terraform, tflint, and TFGrunt" (ghcr.io/devcontainers/features/terraform) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/terraform for help troubleshooting this error.
------
Dockerfile.extended:24
--------------------
  23 |     
  24 | >>> RUN --mount=type=bind,from=dev_containers_feature_content_source,source=terraform_0,target=/tmp/build-features-src/terraform_0 \
  25 | >>>     cp -ar /tmp/build-features-src/terraform_0 /tmp/dev-container-features \
  26 | >>>  && chmod -R 0755 /tmp/dev-container-features/terraform_0 \
  27 | >>>  && cd /tmp/dev-container-features/terraform_0 \
  28 | >>>  && chmod +x ./devcontainer-features-install.sh \
  29 | >>>  && ./devcontainer-features-install.sh \
  30 | >>>  && rm -rf /tmp/dev-container-features/terraform_0
  31 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c cp -ar /tmp/build-features-src/terraform_0 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/terraform_0  && cd /tmp/dev-container-features/terraform_0  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/terraform_0" did not complete successfully: exit code: 1
[268092 ms] Error: Command failed: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/devcontainercli-jweirich/container-features/0.51.1-1694838573983 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/go:1-1.21-bullseye --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-jenkins-ec2-terraform-74663f5398b6bdab6d4d0b2e62c22d13ac622c82d66011d36915c69c6e88f704-features -f /tmp/devcontainercli-jweirich/container-features/0.51.1-1694838573983/Dockerfile.extended /tmp/devcontainercli-jweirich/empty-folder
[268092 ms]     at gAA (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:462:1066)
[268092 ms]     at FK (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:462:1002)
[268092 ms]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[268093 ms]     at async FAA (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:479:3660)
[268093 ms]     at async GC (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:479:4775)
[268093 ms]     at async VeA (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:611:12240)
[268093 ms]     at async WeA (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:611:11981)
[268099 ms] Exit code 1
[268103 ms] Command failed: /home/jweirich/.vscode-server/bin/8b617bd08fd9e3fc94d14adb8d358b56e3f72314/node /home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js up --container-session-data-folder /tmp/devcontainers-4285b364-d5bb-40d2-a1a1-f81dc3fab5951694838570610 --workspace-folder /home/jweirich/projects/jenkins-ec2-terraform --workspace-mount-consistency cached --id-label devcontainer.local_folder=\\wsl.localhost\Ubuntu\home\jweirich\projects\jenkins-ec2-terraform --id-label devcontainer.config_file=/home/jweirich/projects/jenkins-ec2-terraform/.devcontainer/devcontainer.json --log-level debug --log-format json --config /home/jweirich/projects/jenkins-ec2-terraform/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --mount type=bind,source=/run/user/1000/wayland-0,target=/tmp/vscode-wayland-ba08150a-0827-42fd-abb5-c07c65572656.sock --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true

JDubbTX avatar Sep 16 '23 04:09 JDubbTX

@samruddhikhandale can you please re-open. I'm also getting similar error to @JDubbTX

sethbacon avatar Sep 19 '23 18:09 sethbacon

We were seeing similar issues (see https://github.com/devcontainers/features/issues/323) which were fixed with https://github.com/devcontainers/features/pull/353/files. However, it seemed like for some folks keyserver hkp://keyserver.ubuntu.com works, but for others keyserver hkp://keyserver.ubuntu.com:80 does. Wondering if that's the case with you?

@sethbacon @JDubbTX can you pin the terraform Feature to v1.1.0 and see if it helps?

samruddhikhandale avatar Sep 19 '23 18:09 samruddhikhandale

@samruddhikhandale Got slightly farther... 0.292 =========================================================================== 0.292 Feature : Terraform, tflint, and TFGrunt 0.292 Description : Installs the Terraform CLI and optionally TFLint and Terragrunt. Auto-detects latest version and installs needed dependencies. 0.292 Id : ghcr.io/devcontainers/features/terraform 0.292 Version : 1.1.0 0.292 Documentation : https://github.com/devcontainers/features/tree/main/src/terraform 0.292 Options : 0.292 VERSION="latest" 0.292 TFLINT="latest" 0.292 TERRAGRUNT="none" 0.292 INSTALLTFSEC="false" 0.292 INSTALLTERRAFORMDOCS="true" 0.292 INSTALLSENTINEL="true" 0.292 =========================================================================== 0.815 TERRAFORM_VERSION=1.5.7 1.178 TFLINT_VERSION=0.48.0 1.179 Downloading terraform... 3.135 TERRAFORM_GPG_KEY=72D7468F 3.137 GPG_KEY_SERVERS=keyserver hkps://keyserver.ubuntu.com 3.137 keyserver hkps://keys.openpgp.org 3.137 keyserver hkps://keyserver.pgp.com 3.140 (*) Downloading GPG key... 4.066 gpg: Signature made Thu 07 Sep 2023 06:40:30 PM UTC 4.066 gpg: using RSA key 374EC75B485913604A831CC7C820C6D5CD27AB87 4.067 gpg: Good signature from "HashiCorp Security (hashicorp.com/security) [email protected]" [unknown] 4.068 gpg: WARNING: This key is not certified with a trusted signature! 4.068 gpg: There is no indication that the signature belongs to the owner. 4.068 Primary key fingerprint: C874 011F 0AB4 0511 0D02 1055 3436 5D94 72D7 468F 4.068 Subkey fingerprint: 374E C75B 4859 1360 4A83 1CC7 C820 C6D5 CD27 AB87 4.165 terraform_1.5.7_linux_amd64.zip: OK 4.166 Archive: terraform_1.5.7_linux_amd64.zip 4.166 inflating: terraform
4.621 Downloading tflint... 17.82 TFLINT_GPG_KEY_URI=https://raw.githubusercontent.com/terraform-linters/tflint/master/8CE69160EB3F2FE9.key 18.09 gpg: no valid OpenPGP data found. 18.09 ERROR: Feature "Terraform, tflint, and TFGrunt" (ghcr.io/devcontainers/features/terraform) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/terraform for help troubleshooting this error.

I should mention, no proxy and I've disconnected VPN.

Looks like https://raw.githubusercontent.com/terraform-linters/tflint/master/8CE69160EB3F2FE9.key gives a 404

sethbacon avatar Sep 19 '23 19:09 sethbacon

and now setting it back to latest worked...I hate networking issues. Thanks for your help @samruddhikhandale !

sethbacon avatar Sep 19 '23 19:09 sethbacon

tried specifying terraform feature 1.1.0 and no difference for me. also tried HTTPPROXY set to hkp://keyserver.ubuntu.com. Still hangs on the gpg key download. This is my personal laptop, no vpn involved.

> [dev_containers_target_stage 5/5] RUN --mount=type=bind,from=dev_containers_feature_content_source,source=terraform_0,target=/tmp/build-features-src/terraform_0     cp -ar /tmp/build-features-src/terraform_0 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/terraform_0  && cd /tmp/dev-container-features/terraform_0  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/terraform_0:
0.363 ===========================================================================
0.363 Feature       : Terraform, tflint, and TFGrunt
0.363 Description   : Installs the Terraform CLI and optionally TFLint and Terragrunt. Auto-detects latest version and installs needed dependencies.
0.363 Id            : ghcr.io/devcontainers/features/terraform
0.363 Version       : 1.3.4
[2023-09-19T23:15:28.293Z] 0.363 Documentation : https://github.com/devcontainers/features/tree/main/src/terraform
0.363 Options       :
0.363     VERSION="1.1.0"
0.363     TFLINT="latest"
0.363     TERRAGRUNT="latest"
0.363     INSTALLSENTINEL="false"
0.363     INSTALLTFSEC="false"
0.363     INSTALLTERRAFORMDOCS="false"
0.363     HTTPPROXY=""
0.363 ===========================================================================
0.848 TERRAFORM_VERSION=1.1.0
1.250 TFLINT_VERSION=0.48.0
1.745 TERRAGRUNT_VERSION=0.51.3
1.750 Downloading terraform...
3.424 (*) Downloading GPG key...
33.49 gpg: keyserver receive failed: Server indicated a failure
33.49 (*) Failed getting key, retring in 10s...
43.50 (*) Downloading GPG key...
73.55 gpg: keyserver receive failed: Server indicated a failure
73.55 (*) Failed getting key, retring in 10s...
83.56 (*) Downloading GPG key...
113.6 gpg: keyserver receive failed: Server indicated a failure
113.6 (*) Failed getting key, retring in 10s...
123.6 (*) Downloading GPG key...
153.7 gpg: keyserver receive failed: Server indicated a failure
153.7 (*) Failed getting key, retring in 10s...
163.7 (*) Downloading GPG key...
193.7 gpg: keyserver receive failed: Server indicated a failure
193.7 (*) Failed getting key, retring in 10s...
203.7 (!) Failed to get gpg key.
203.7 ERROR: Feature "Terraform, tflint, and TFGrunt" (ghcr.io/devcontainers/features/terraform) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/terraform for help troubleshooting this error.
------
[2023-09-19T23:15:28.293Z] Dockerfile.extended:24
--------------------
  23 |     
  24 | >>> RUN --mount=type=bind,from=dev_containers_feature_content_source,source=terraform_0,target=/tmp/build-features-src/terraform_0 \
  25 | >>>     cp -ar /tmp/build-features-src/terraform_0 /tmp/dev-container-features \
  26 | >>>  && chmod -R 0755 /tmp/dev-container-features/terraform_0 \
  27 | >>>  && cd /tmp/dev-container-features/terraform_0 \
  28 | >>>  && chmod +x ./devcontainer-features-install.sh \
  29 | >>>  && ./devcontainer-features-install.sh \
  30 | >>>  && rm -rf /tmp/dev-container-features/terraform_0
  31 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c cp -ar /tmp/build-features-src/terraform_0 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/terraform_0  && cd /tmp/dev-container-features/terraform_0  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/terraform_0" did not complete successfully: exit code: 1
[2023-09-19T23:15:28.308Z] Stop (205845 ms): Run: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/devcontainercli-jweirich/container-features/0.51.1-1695165121132 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/base:ubuntu --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-jenkins-ec2-terraform-74663f5398b6bdab6d4d0b2e62c22d13ac622c82d66011d36915c69c6e88f704-features -f /tmp/devcontainercli-jweirich/container-features/0.51.1-1695165121132/Dockerfile.extended /tmp/devcontainercli-jweirich/empty-folder
[2023-09-19T23:15:28.303Z] Error: Command failed: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/devcontainercli-jweirich/container-features/0.51.1-1695165121132 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/base:ubuntu --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-jenkins-ec2-terraform-74663f5398b6bdab6d4d0b2e62c22d13ac622c82d66011d36915c69c6e88f704-features -f /tmp/devcontainercli-jweirich/container-features/0.51.1-1695165121132/Dockerfile.extended /tmp/devcontainercli-jweirich/empty-folder
[2023-09-19T23:15:28.303Z]     at gAA (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:462:1066)
[2023-09-19T23:15:28.304Z]     at FK (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:462:1002)
[2023-09-19T23:15:28.304Z]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[2023-09-19T23:15:28.304Z]     at async FAA (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:479:3660)
[2023-09-19T23:15:28.305Z]     at async GC (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:479:4775)
[2023-09-19T23:15:28.306Z]     at async VeA (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:611:12240)
[2023-09-19T23:15:28.306Z]     at async WeA (/home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js:611:11981)
[2023-09-19T23:15:28.309Z] Stop (207812 ms): Run in Host: /home/jweirich/.vscode-remote-containers/bin/abd2f3db4bdb28f9e95536dfa84d8479f1eb312d/node /home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js up --container-session-data-folder /tmp/devcontainers-c95a211a-c45a-4493-97d7-8b23d42a8f9c1695165118367 --workspace-folder /home/jweirich/projects/jenkins-ec2-terraform --workspace-mount-consistency cached --id-label devcontainer.local_folder=\\wsl.localhost\Ubuntu\home\jweirich\projects\jenkins-ec2-terraform --id-label devcontainer.config_file=/home/jweirich/projects/jenkins-ec2-terraform/.devcontainer/devcontainer.json --log-level debug --log-format json --config /home/jweirich/projects/jenkins-ec2-terraform/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --mount type=bind,source=/run/user/1000/wayland-0,target=/tmp/vscode-wayland-5d0007d5-add3-443b-ab87-573eddae4a8c.sock --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true
[2023-09-19T23:15:28.310Z] Exit code 1
[2023-09-19T23:15:28.323Z] Command failed: /home/jweirich/.vscode-remote-containers/bin/abd2f3db4bdb28f9e95536dfa84d8479f1eb312d/node /home/jweirich/.vscode-remote-containers/dist/dev-containers-cli-0.309.0/dist/spec-node/devContainersSpecCLI.js up --container-session-data-folder /tmp/devcontainers-c95a211a-c45a-4493-97d7-8b23d42a8f9c1695165118367 --workspace-folder /home/jweirich/projects/jenkins-ec2-terraform --workspace-mount-consistency cached --id-label devcontainer.local_folder=\\wsl.localhost\Ubuntu\home\jweirich\projects\jenkins-ec2-terraform --id-label devcontainer.config_file=/home/jweirich/projects/jenkins-ec2-terraform/.devcontainer/devcontainer.json --log-level debug --log-format json --config /home/jweirich/projects/jenkins-ec2-terraform/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --mount type=bind,source=/run/user/1000/wayland-0,target=/tmp/vscode-wayland-5d0007d5-add3-443b-ab87-573eddae4a8c.sock --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true
[2023-09-19T23:15:28.324Z] Exit code 1
[2023-09-19T23:16:21.898Z] Start: Run: wsl -d Ubuntu -e wslpath -u \\wsl.localhost\Ubuntu\home\jweirich\projects\jenkins-ec2-terraform
[2023-09-19T23:16:22.113Z] Stop (215 ms): Run: wsl -d Ubuntu -e wslpath -u \\wsl.localhost\Ubuntu\home\jweirich\projects\jenkins-ec2-terraform

JDubbTX avatar Sep 19 '23 23:09 JDubbTX

I'm getting the same results as @JDubbTX. Is there any way to properly investigate / debug this without having to use a network analyzer like Wireshark or mitmproxy?

Because it would probably help knowing exactly why the "server indicated a failure" or, more specifically, what failure was indicated. I don't know if there was a 404, a 400 or a 500 error, for example.

Manually searching the GPG key on the keyserver yields a successful result anyway...

Ponjimon avatar Sep 27 '23 11:09 Ponjimon

I'm getting the same results as @JDubbTX. Is there any way to properly investigate / debug this without having to use a network analyzer like Wireshark or mitmproxy?

Because it would probably help knowing exactly why the "server indicated a failure" or, more specifically, what failure was indicated. I don't know if there was a 404, a 400 or a 500 error, for example.

Manually searching the GPG key on the keyserver yields a successful result anyway...

Here is the line it is hanging on if that helps. I tried the other night putting it into my bash shell, substituting what I thought were correct values, and got the same hang, but didn't get any info out other than the same "gpg: keyserver receive failed: Server indicated a failure". Unfortunately I'm not too familiar with these commands - perhaps if I get time to try again I can try and get verbose output. https://github.com/devcontainers/features/blob/3a22c419403b5c099c71a74c7079cde813e269d8/src/terraform/install.sh#L73

JDubbTX avatar Sep 27 '23 14:09 JDubbTX

Well, this is interesting. Still trying to figure out why and how, but I tried testing it with the --keyserver flag.

So basically:

( echo "${keys}" | xargs -n 1 gpg -q ${keyring_args} --recv-keys --keyserver keyserver.ubuntu.com) 2>&1 && gpg_ok="true" 

and that fails. However, if I (within the devcontainer!) do nslookup keyserver.ubuntu.comand then use it's IPv4 address, it works.

( echo "${keys}" | xargs -n 1 gpg -q ${keyring_args} --recv-keys --keyserver 185.125.188.26) 2>&1 && gpg_ok="true" 

Is gpg internally using other DNS servers than the system or something like that? Because clearly, the DNS servers the host or the docker container is using, are working, otherwise I wouldn't have received anything from nslookup.

Ponjimon avatar Sep 27 '23 14:09 Ponjimon

Thanks for reporting!

@alexander-smolyakov Can you help prioritize and investigate this issue? Thanks!

samruddhikhandale avatar Sep 27 '23 16:09 samruddhikhandale

Hey all, sharing an intermittent update regarding this issue:

"gpg: keyserver receive failed: Server indicated a failure"

I have checked the source code of gpg and libgpg-error. The Server indicated a failure message is mapped to the GPG_ERR_SERVER_FAILED error code. The gpg returns the GPG_ERR_SERVER_FAILED error code if something goes wrong with DNS-related stuff. It looks like gpg could use several libraries for working with DNS servers based on macros values. Related code:

  • https://github.com/gpg/libgpg-error/blob/master/src/err-codes.h.in#L253
  • https://github.com/gpg/gnupg/blob/master/dirmngr/dns-stuff.c#L1610

This means that gpg could work with DNS servers differently than other tools.

At the moment, I'm investigating how to improve our retry logic to handle issues with the DNS resolver.

alexander-smolyakov avatar Sep 29 '23 14:09 alexander-smolyakov

Update:

We have merged a fix that contains changes to the retry logic. In version 1.3.5, If all previous attempts fail, the feature will try to resolve the IP address of the keyserver and explicitly pass it to the gpg tool.

The same approach was mentioned in the following comment: https://github.com/devcontainers/features/issues/686#issuecomment-1737523182

Please let us know if the issue still appears after these changes.

alexander-smolyakov avatar Oct 05 '23 13:10 alexander-smolyakov

Thank you @alexander-smolyakov !

Closing as fixed with https://github.com/devcontainers/features/pull/713. Feel free to reopen if needed, thanks!

samruddhikhandale avatar Oct 05 '23 16:10 samruddhikhandale

This is still failing for me, can you reopen, or tell me what I might be doing wrong? Here is the relevant part of the log:

64.94 Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 dnsutils all 1:9.18.12-0ubuntu0.22.04.3 [3924 B]
65.87 Fetched 1652 kB in 13s (126 kB/s)
65.91 Selecting previously unselected package libjson-c5:amd64.
(Reading database ... 21144 files and directories currently installed.)
65.93 Preparing to unpack .../0-libjson-c5_0.15-3~ubuntu1.22.04.2_amd64.deb ...
65.94 Unpacking libjson-c5:amd64 (0.15-3~ubuntu1.22.04.2) ...
65.98 Selecting previously unselected package liblmdb0:amd64.
65.99 Preparing to unpack .../1-liblmdb0_0.9.24-1build2_amd64.deb ...
65.99 Unpacking liblmdb0:amd64 (0.9.24-1build2) ...
66.03 Selecting previously unselected package libmaxminddb0:amd64.
66.03 Preparing to unpack .../2-libmaxminddb0_1.5.2-1build2_amd64.deb ...
66.04 Unpacking libmaxminddb0:amd64 (1.5.2-1build2) ...
66.09 Selecting previously unselected package libuv1:amd64.
66.09 Preparing to unpack .../3-libuv1_1.43.0-1_amd64.deb ...
66.10 Unpacking libuv1:amd64 (1.43.0-1) ...
66.14 Selecting previously unselected package bind9-libs:amd64.
66.14 Preparing to unpack .../4-bind9-libs_1%3a9.18.12-0ubuntu0.22.04.3_amd64.deb ...
66.14 Unpacking bind9-libs:amd64 (1:9.18.12-0ubuntu0.22.04.3) ...
66.21 Selecting previously unselected package bind9-host.
66.21 Preparing to unpack .../5-bind9-host_1%3a9.18.12-0ubuntu0.22.04.3_amd64.deb ...
66.21 Unpacking bind9-host (1:9.18.12-0ubuntu0.22.04.3) ...
66.25 Selecting previously unselected package bind9-dnsutils.
66.25 Preparing to unpack .../6-bind9-dnsutils_1%3a9.18.12-0ubuntu0.22.04.3_amd64.deb ...
66.26 Unpacking bind9-dnsutils (1:9.18.12-0ubuntu0.22.04.3) ...
66.29 Selecting previously unselected package dnsutils.
66.29 Preparing to unpack .../7-dnsutils_1%3a9.18.12-0ubuntu0.22.04.3_all.deb ...
66.30 Unpacking dnsutils (1:9.18.12-0ubuntu0.22.04.3) ...
66.34 Setting up liblmdb0:amd64 (0.9.24-1build2) ...
66.36 Setting up libmaxminddb0:amd64 (1.5.2-1build2) ...
66.37 Setting up libuv1:amd64 (1.43.0-1) ...
66.39 Setting up libjson-c5:amd64 (0.15-3~ubuntu1.22.04.2) ...
66.41 Setting up bind9-libs:amd64 (1:9.18.12-0ubuntu0.22.04.3) ...
66.42 Setting up bind9-host (1:9.18.12-0ubuntu0.22.04.3) ...
66.44 Setting up bind9-dnsutils (1:9.18.12-0ubuntu0.22.04.3) ...
66.45 Setting up dnsutils (1:9.18.12-0ubuntu0.22.04.3) ...
66.47 Processing triggers for libc-bin (2.35-0ubuntu3.3) ...
66.51 Processing triggers for man-db (2.10.2-1) ...
67.18 TERRAFORM_VERSION=1.6.0
67.52 TFLINT_VERSION=0.48.0
67.92 TERRAGRUNT_VERSION=0.51.9
67.93 Downloading terraform...
69.38 (*) Downloading GPG key...
99.46 gpg: keyserver receive failed: Server indicated a failure
99.46 (*) Failed getting key, retring in 10s...
109.5 (*) Downloading GPG key...
139.5 gpg: keyserver receive failed: Server indicated a failure
139.5 (*) Failed getting key, retring in 10s...
149.5 (*) Downloading GPG key...
179.6 gpg: keyserver receive failed: Server indicated a failure
179.6 (*) Failed getting key, retring in 10s...
189.6 (*) Downloading GPG key...
219.6 gpg: keyserver receive failed: Server indicated a failure
219.7 (*) Failed getting key, retring in 10s...
229.7 (*) Downloading GPG key...
259.7 gpg: keyserver receive failed: Server indicated a failure
259.7 (*) Failed getting key, retring in 10s...
269.7 (*) Resolving GPG keyserver IP address...
269.9 (*) GPG keyserver IP address 185.125.188.26
269.9 (*) Downloading GPG key...
269.9 gpg: keyserver receive failed: No keyserver available
269.9 (*) Failed getting key, retring in 10s...
279.9 (*) Downloading GPG key...
280.0 gpg: keyserver receive failed: No keyserver available
280.0 (*) Failed getting key, retring in 10s...
290.0 (*) Downloading GPG key...
290.0 gpg: keyserver receive failed: No keyserver available
290.0 (*) Failed getting key, retring in 10s...
300.0 (!) Failed to get gpg key.
300.0 ERROR: Feature "Terraform, tflint, and TFGrunt" (ghcr.io/devcontainers/features/terraform) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/terraform for help troubleshooting this error.

JDubbTX avatar Oct 05 '23 19:10 JDubbTX

Tried debugging this and getting "No keyserver available" on the ip tries. image

JDubbTX avatar Oct 06 '23 05:10 JDubbTX

@JDubbTX got it. I reopened the issue to continue the investigation.

alexander-smolyakov avatar Oct 06 '23 16:10 alexander-smolyakov

@alexander-smolyakov Hoping this sheds some light on the initial error:

root ➜ /workspaces/features (main) $ cat ${GNUPGHOME}/dirmngr.conf
disable-ipv6
keyserver hkps://keyserver.ubuntu.com
keyserver hkps://keys.openpgp.org
keyserver hkps://keyserver.pgp.com

and

root ➜ /workspaces/features (main) $ gpg --recv-keys --debug-level=guru --homedir /tmp/tmp-gn
upg -vvvvv 72D7468F
gpg: using character set 'utf-8'
gpg: enabled debug flags: packet mpi crypto filter iobuf memory cache memstat trust hashing ipc clock lookup extprog
gpg: DBG: [not enabled in the source] start
gpg: DBG: chan_3 <- # Home: /tmp/tmp-gnupg
gpg: DBG: chan_3 <- # Config: /tmp/tmp-gnupg/dirmngr.conf
gpg: DBG: chan_3 <- OK Dirmngr 2.2.27 at your service
gpg: DBG: connection to the dirmngr established
gpg: DBG: chan_3 -> GETINFO version
gpg: DBG: chan_3 <- D 2.2.27
gpg: DBG: chan_3 <- OK
gpg: DBG: chan_3 -> KS_GET -- 0x72D7468F
gpg: DBG: chan_3 <- ERR 219 Server indicated a failure <Unspecified source>
gpg: keyserver receive failed: Server indicated a failure
gpg: DBG: chan_3 -> BYE
gpg: DBG: [not enabled in the source] stop
gpg: keydb: handles=0 locks=0 parse=0 get=0
gpg:        build=0 update=0 insert=0 delete=0
gpg:        reset=0 found=0 not=0 cache=0 not=0
gpg: kid_not_found_cache: count=0 peak=0 flushes=0
gpg: sig_cache: total=0 cached=0 good=0 bad=0
gpg: random usage: poolsize=600 mixed=0 polls=0/0 added=0/0
              outmix=0 getlvl1=0/0 getlvl2=0/0
gpg: rndjent stat: collector=0x0000000000000000 calls=0 bytes=0
gpg: secmem usage: 0/65536 bytes in 0 blocks

JDubbTX avatar Oct 07 '23 00:10 JDubbTX

As I have an ORBI wifi router, this might be my issue. I'll try the suggested fix and report back.

JDubbTX avatar Oct 07 '23 01:10 JDubbTX