pixie icon indicating copy to clipboard operation
pixie copied to clipboard
verified publisher icon pixie-io

Pixie doesn't work on Rancher Desktop

Open ismaeIfm opened this issue 3 years ago • 12 comments

Describe the bug Pixie doesn't work on Rancher Desktop.

To Reproduce Steps to reproduce the behavior:

  1. Install pixie on Rancher Desktop (k8s)
  2. Select dockerd runtime
  3. Install pixie
  4. See error

Expected behavior I expect pixie to work

Screenshots Screenshot 2022-05-31 at 22 27 28

Logs Please attach the logs by running the following command:

./px collect-logs

pixie_logs_20220531221823.zip

App information (please complete the following information):

  • Pixie version: 0.7.9+Distribution.a47d77a.20220510221202.1
  • K8s cluster version Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.16+k3s1", GitCommit:"da16869555775cf17d4d97ffaf8a13b70bc738c2", GitTreeState:"clean", BuildDate:"2021-11-04T00:54:15Z", GoVersion:"go1.15.14", Compiler:"gc", Platform:"linux/arm64"}
  • Node Kernel version
  • Browser version

Additional context Add any other context about the problem here.

ismaeIfm avatar May 31 '22 21:05 ismaeIfm

Turns out that the VM started by rancher-desktop does not have Kernel config file, that’s a list of config entries originally used in building the kernel. Pixie uses it to patch the packaged headers to make them as close to the actual kernel as possible.

https://github.com/lima-vm/alpine-lima/issues/113 I have created this issue on lima-vm, which is the VM tool used by rancher-desktop.

yzhao1012 avatar Jun 02 '22 16:06 yzhao1012

The file is available in the boot image:

$ rdctl shell head /media/sr0/boot/config-virt
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86_64 5.15.40 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=100301
CONFIG_CLANG_VERSION=0
CONFIG_AS_IS_GNU=y
CONFIG_AS_VERSION=23700

jandubois avatar Jun 02 '22 16:06 jandubois

Thanks @jandubois ! I've added /media/sr0/boot/config-virt into Pixie's config search path, and verified that Pixie now works fine on rancher-desktop, on Ubuntu Linux.

I still need to verify on mac. Since initial examination shows that /media/sr0/boot/config-virt does not exist on my m1 macbook air running with Lima VM.

yzhao1012 avatar Jun 02 '22 18:06 yzhao1012

I still need to verify on mac. Since initial examination shows that /media/sr0/boot/config-virt does not exist on my m1 macbook air running with Lima VM.

I don't know why, but on the M1 it seems to be under vda:

lima-rancher-desktop:~# head /media/vda/boot/config-virt
#
# Automatically generated file; DO NOT EDIT.
# Linux/arm64 5.15.40 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=100301
CONFIG_CLANG_VERSION=0
CONFIG_AS_IS_GNU=y
CONFIG_AS_VERSION=23700

I'll create an issue on the alpine-lima repo to see if we can do something to make sure the boot CD is always mounted on /media/cdrom to have a predictable location; I don't know why this isn't the case right now.

jandubois avatar Jun 02 '22 18:06 jandubois

@yzhao1012 Could you include instructions here on how you got Pixie installed on RD, especially once you have it working on macOS? Thank you!

jandubois avatar Jun 02 '22 19:06 jandubois

@yzhao1012 Could you include instructions here on how you got Pixie installed on RD, especially once you have it working on macOS? Thank you!

Re: Once rancher-desktop is installed and launched, follow https://docs.px.dev/installing-pixie/install-guides/community-cloud-for-pixie to install Pixie. That instruction just uses local kubectl config to figure out the target kubernetes cluster and how to reach it. Then deploy Pixie with a predefined packages from public GCR repos.

yzhao1012 avatar Jun 02 '22 21:06 yzhao1012

We have added 2 kernel config paths used by rancher-desktop's CDRom-boot kernel image, under @jandubois' information. See https://github.com/pixie-io/pixie/commit/57e1dfd2f335d59521ec35c2191ca84c4d12e89d

You should be able to test this after this week's Vizier release.

yzhao1012 avatar Jun 08 '22 17:06 yzhao1012

@yzhao1012 I got this exact error (with rancher desktop and dockerd runtime) and I tried to build vizier and deploy it by following the development guide (Pls note I have to manually set the DOCKER_ID and GROUP_ID in latest run_docker.sh script.) But I keep getting the following output.

Output of `skaffold run -f skaffold/skaffold_vizier.yaml`

Generating tags...

  • gcr.io/pixie-oss/pixie-dev/vizier/pem_image -> gcr.io/pixie-oss/pixie-dev/vizier/pem_image:2022-06-12_04-50-10.549_UTC
  • gcr.io/pixie-oss/pixie-dev/vizier/kelvin_image -> gcr.io/pixie-oss/pixie-dev/vizier/kelvin_image:2022-06-12_04-50-10.549_UTC
  • gcr.io/pixie-oss/pixie-dev/vizier/metadata_server_image -> gcr.io/pixie-oss/pixie-dev/vizier/metadata_server_image:2022-06-12_04-50-10.549_UTC
  • gcr.io/pixie-oss/pixie-dev/vizier/query_broker_server_image -> gcr.io/pixie-oss/pixie-dev/vizier/query_broker_server_image:2022-06-12_04-50-10.549_UTC
  • gcr.io/pixie-oss/pixie-dev/vizier/cloud_connector_server_image -> gcr.io/pixie-oss/pixie-dev/vizier/cloud_connector_server_image:2022-06-12_04-50-10.549_UTC
  • gcr.io/pixie-oss/pixie-dev/vizier/cert_provisioner_image -> gcr.io/pixie-oss/pixie-dev/vizier/cert_provisioner_image:2022-06-12_04-50-10.549_UTC Checking cache... WARN[0002] Retrieving Bazel dependencies can take a long time the first time subtask=-1 task=Build
  • gcr.io/pixie-oss/pixie-dev/vizier/pem_image: Error checking cache. getting hash for artifact "gcr.io/pixie-oss/pixie-dev/vizier/pem_image": getting dependencies for "gcr.io/pixie-oss/pixie-dev/vizier/pem_image": getting bazel dependencies: running [bazel query kind('source file', deps('//src/vizier/services/agent/pem:pem_image.tar')) union buildfiles(deps('//src/vizier/services/agent/pem:pem_image.tar')) --noimplicit_deps --order_output=no --output=label]
  • stdout: ""
  • stderr: "Another command holds the client lock: \npid=74\nowner=client\ncwd=/pl/src/px.dev/pixie\n\nWaiting for it to complete...\nAnother command (pid=74) is running. Waiting for it to complete on the server (server_pid=84)...\nAnother command (pid=79) is running. Waiting for it to complete on the server (server_pid=84)...\nAnother command (pid=76) is running. Waiting for it to complete on the server (server_pid=84)...\nLoading: 0 packages loaded\nLoading: 20 packages loaded\n currently loading: @com_llvm_lib//\nINFO: Repository base_image instantiated at:\n /pl/src/px.dev/pixie/WORKSPACE:92:20: in \n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:68:28: in pl_container_images\n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:38:16: in _container_images_setup\n /pl/src/px.dev/pixie/bazel/container_images.bzl:50:18: in base_images\n /pl/src/px.dev/pixie/bazel/container_images.bzl:30:19: in _gcr_io_image\nRepository rule container_pull defined at:\n /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl:294:33: in \nLoading: 34 packages loaded\nERROR: An error occurred during the fetch of repository 'base_image':\n Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl", line 240, column 13, in _impl\n\t\tfail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image "gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee)\nERROR: /pl/src/px.dev/pixie/WORKSPACE:92:20: fetching container_pull rule //external:base_image: Traceback (most recent call last):\n\tFile \"/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl\", line 240, column 13, in _impl\n\t\tfail(\"Pull command failed: %s (%s)\" % (result.stderr, \" \".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image \"gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee)\nLoading: 34 packages loaded\nERROR: /pl/src/px.dev/pixie/BUILD.bazel:137:16: no such package '@base_image//image': Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image "gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee) and referenced by '//:pl_cc_base_image'\nINFO: Repository openjdk-base-musl instantiated at:\n /pl/src/px.dev/pixie/WORKSPACE:92:20: in <toplevel>\n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:68:28: in pl_container_images\n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:38:16: in _container_images_setup\n /pl/src/px.dev/pixie/bazel/container_images.bzl:68:18: in base_images\n /pl/src/px.dev/pixie/bazel/container_images.bzl:30:19: in _gcr_io_image\nRepository rule container_pull defined at:\n /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl:294:33: in <toplevel>\nERROR: An error occurred during the fetch of repository 'openjdk-base-musl':\n Traceback (most recent call last):\n\tFile \"/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl\", line 240, column 13, in _impl\n\t\tfail(\"Pull command failed: %s (%s)\" % (result.stderr, \" \".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f)\nERROR: /pl/src/px.dev/pixie/WORKSPACE:92:20: fetching container_pull rule //external:openjdk-base-musl: Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl", line 240, column 13, in _impl\n\t\tfail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f)\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:73:16: no such package '@openjdk-base-musl//image': Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-musl'\nLoading: 50 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nINFO: Repository com_github_antlr_grammars_v4 instantiated at:\n /pl/src/px.dev/pixie/WORKSPACE:10:8: in \n /pl/src/px.dev/pixie/bazel/repositories.bzl:222:13: in _pl_deps\n /pl/src/px.dev/pixie/bazel/repositories.bzl:158:16: in _cc_deps\n /pl/src/px.dev/pixie/bazel/repositories.bzl:98:28: in _bazel_repo\n /pl/src/px.dev/pixie/bazel/repositories.bzl:39:17: in _http_archive_repo_impl\nRepository rule http_archive defined at:\n /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in \nINFO: Repository 'com_github_antlr_grammars_v4' used the following cache hits instead of downloading the corresponding file.\n * Hash '9858e4a9944cac85830e6cf8edd9d567227af96d8b75f0b31accc525ec842c30' for https://github.com/antlr/grammars-v4/archive/e53d7a1228505bfc80d8637808ef60e7eea92cc2.tar.gz\nIf the definition of 'com_github_antlr_grammars_v4' was updated, verify that the hashes were also updated.\nERROR: An error occurred during the fetch of repository 'com_github_antlr_grammars_v4':\n Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 10, in _http_archive_impl\n\t\tpatch(ctx, auth = auth)\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 167, column 22, in patch\n\t\tctx.patch(patchfile, strip)\nError in patch: Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists.\nERROR: /pl/src/px.dev/pixie/WORKSPACE:10:8: fetching http_archive rule //external:com_github_antlr_grammars_v4: Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 10, in _http_archive_impl\n\t\tpatch(ctx, auth = auth)\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 167, column 22, in patch\n\t\tctx.patch(patchfile, strip)\nError in patch: Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists.\nERROR: /pl/src/px.dev/pixie/src/carnot/funcs/builtins/sql_parsing/BUILD.bazel:21:14: no such package '@com_github_antlr_grammars_v4//': Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists. and referenced by '//src/carnot/funcs/builtins/sql_parsing:cc_library'\nINFO: Repository openjdk-base-glibc instantiated at:\n /pl/src/px.dev/pixie/WORKSPACE:92:20: in \n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:68:28: in pl_container_images\n /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:38:16: in _container_images_setup\n /pl/src/px.dev/pixie/bazel/container_images.bzl:62:18: in base_images\n /pl/src/px.dev/pixie/bazel/container_images.bzl:30:19: in _gcr_io_image\nRepository rule container_pull defined at:\n /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl:294:33: in \nERROR: An error occurred during the fetch of repository 'openjdk-base-glibc':\n Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl", line 240, column 13, in _impl\n\t\tfail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d)\nERROR: /pl/src/px.dev/pixie/WORKSPACE:92:20: fetching container_pull rule //external:openjdk-base-glibc: Traceback (most recent call last):\n\tFile \"/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl\", line 240, column 13, in _impl\n\t\tfail(\"Pull command failed: %s (%s)\" % (result.stderr, \" \".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d)\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:66:16: no such package '@openjdk-base-glibc//image': Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-glibc'\nERROR: /pl/src/px.dev/pixie/BUILD.bazel:137:16: no such package '@base_image//image': Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image \"gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee) and referenced by '//:pl_cc_base_image'\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:66:16: no such package '@openjdk-base-glibc//image': Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-glibc'\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:73:16: no such package '@openjdk-base-musl//image': Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version -os-features -architecture amd64 -variant -features -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-musl'\nERROR: /pl/src/px.dev/pixie/src/carnot/funcs/builtins/sql_parsing/BUILD.bazel:21:14: no such package '@com_github_antlr_grammars_v4//': Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists. and referenced by '//src/carnot/funcs/builtins/sql_parsing:cc_library'\nERROR: Evaluation of subquery "deps(//src/vizier/services/agent/pem:pem_image.tar)" failed (did you want to use --keep_going?): errors were encountered while computing transitive closure\nLoading: 101 packages loaded\nLoading: 101 packages loaded\n"
  • cause: exit status 7

I am just trying to build and deploy vizier with a macbook. Using minikube to build and deploy was also not successful since minikube uses a separate docker runtime and mounting host file system to minikube vm was problematic in mac os.

tharinduwijewardane avatar Jun 12 '22 05:06 tharinduwijewardane

Related issue: lima-vm/lima#450

zasgar avatar Jun 15 '22 23:06 zasgar

You should be able to test this after this week's Vizier release.

@yzhao1012 - I just tested this on my Intel Macbook and it now works for me (I was previously able to reproduce the issue). Given that you've confirmed that it also works on M1 Macbook and Linux, should we add Rancher Desktop as a supported environment to Pixie's Requirements page?

htroisi avatar Jun 16 '22 20:06 htroisi

Related issue: lima-vm/lima#450

with rancher desktop and dockerd runtime

Dockerd runtime wont work. Actually, you have to use the VM driver, like what Hannah and I tested with rancher-desktop. As docker (or any container-based kubernetes desktop environment) is not compatible with Pixie (PEM specifically).

The reason is that PEM needs to access the host kernel's system filesystem to compile BPF code. Container-based kubernetes desktop environment hides the host filesystem from PEM and breaks it.

yzhao1012 avatar Jun 17 '22 20:06 yzhao1012

@yzhao1012 - I just tested this on my Intel Macbook and it now works for me (I was previously able to reproduce the issue). Given that you've confirmed that it also works on M1 Macbook and Linux, should we add Rancher Desktop as a supported environment to Pixie's Requirements page?

@htroisi Is this confirmed to work on Macbook M1/M2? I still get the same error on my Macbook Pro M2 using Pixie 0.8.2+Distribution.401c92c.20230530203620.1.jenkins and the instructions from Install guide for Community Cloud, which installed gcr.io/pixie-oss/pixie-prod/vizier/pem_image:0.14.0 in my RD cluster.

Logs: pixie_logs_20230704095739.zip

rainan16 avatar Jul 04 '23 08:07 rainan16