rules_rust icon indicating copy to clipboard operation
rules_rust copied to clipboard
verified publisher icon bazelbuild

processwrapper failed to find `.so` dependency in bazel

Open allenwang-git opened this issue 10 months ago • 1 comments

rules_rust version: same issue on 0.41.0 and 0.26.0

our project uses bazel and our own cc_toolchain in a hermetic environment, with --subcommands option, we can see our compilation failed at this step:

ERROR: /home/ubuntu/iceoryx2-playground/.bazel/166b5632141acdd19079fcb3e6978d97/external/crate_index__ouroboros-0.18.4/BUILD.bazel:20:13: Compiling Rust rlib ouroboros v0.18.4 (1 files) failed: (Exit 1): process_wrapper failed: error executing command (from target @crate_index__ouroboros-0.18.4//:ouroboros) 
  (cd /home/ubuntu/iceoryx2-playground/.bazel/166b5632141acdd19079fcb3e6978d97/execroot/iceoryx2-playground && \
  exec env - \
    BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 \
    CARGO_BAZEL_REPIN=true \
    CARGO_CFG_TARGET_ARCH=x86_64 \
    CARGO_CFG_TARGET_OS=linux \
    CARGO_CRATE_NAME=ouroboros \
    CARGO_HOME=.cargo_home \
    CARGO_MANIFEST_DIR='${pwd}/external/crate_index__ouroboros-0.18.4' \
    CARGO_PKG_AUTHORS='' \
    CARGO_PKG_DESCRIPTION='' \
    CARGO_PKG_HOMEPAGE='' \
    CARGO_PKG_NAME=ouroboros \
    CARGO_PKG_VERSION=0.18.4 \
    CARGO_PKG_VERSION_MAJOR=0 \
    CARGO_PKG_VERSION_MINOR=18 \
    CARGO_PKG_VERSION_PATCH=4 \
    CARGO_PKG_VERSION_PRE='' \
    SYSROOT=bazel-out/k8-dbg/bin/external/rust_linux_x86_64__x86_64-unknown-linux-gnu__stable_tools/rust_toolchain \
  bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --subst 'pwd=${pwd}' -- bazel-out/k8-dbg/bin/external/rust_linux_x86_64__x86_64-unknown-linux-gnu__stable_tools/rust_toolchain/bin/rustc external/crate_index__ouroboros-0.18.4/src/lib.rs '--crate-name=ouroboros' '--crate-type=rlib' '--error-format=human' '--codegen=metadata=-524816489' '--codegen=extra-filename=-524816489' '--out-dir=bazel-out/k8-dbg/bin/external/crate_index__ouroboros-0.18.4' '--codegen=opt-level=0' '--codegen=debuginfo=2' '--remap-path-prefix=${pwd}=' '--emit=dep-info,link' '--color=always' '--target=x86_64-unknown-linux-gnu' --cfg 'feature="default"' --cfg 'feature="std"' -L bazel-out/k8-dbg/bin/external/rust_linux_x86_64__x86_64-unknown-linux-gnu__stable_tools/rust_toolchain/lib/rustlib/x86_64-unknown-linux-gnu/lib '--cap-lints=allow' '--edition=2018' '--extern=aliasable=bazel-out/k8-dbg/bin/external/crate_index__aliasable-0.1.3/libaliasable-2691302377.rlib' '--extern=static_assertions=bazel-out/k8-dbg/bin/external/crate_index__static_assertions-1.1.0/libstatic_assertions-521553119.rlib' '--extern=ouroboros_macro=bazel-out/k8-opt-exec-2B5CBBC6/bin/external/crate_index__ouroboros_macro-0.18.4/libouroboros_macro-3961516138.so' '-Ldependency=bazel-out/k8-dbg/bin/external/crate_index__aliasable-0.1.3' '-Ldependency=bazel-out/k8-dbg/bin/external/crate_index__static_assertions-1.1.0' '-Ldependency=bazel-out/k8-opt-exec-2B5CBBC6/bin/external/crate_index__ouroboros_macro-0.18.4')
# Configuration: 8b993f2be6475bec43c1e7df280ee22ea63e2c1be0e1a37532fbddecb26aadc0
# Execution platform: @local_config_platform//:host
error[E0463]: can't find crate for `ouroboros_macro`
   --> external/crate_index__ouroboros-0.18.4/src/lib.rs:347:9
    |
347 | pub use ouroboros_macro::self_referencing;
    |         ^^^^^^^^^^^^^^^ can't find crate

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0463`.

the error messages indicate that processwrapper cannot find the dependency enum-iterator-derive under the path bazel-out/k8-opt-exec-2B5CBBC6/bin/external/crate_index__enum-iterator-derive-1.4.0 however we double checked it and found it did exist:

iceoryx2-playground/.bazel/166b5632141acdd19079fcb3e6978d97/execroot/iceoryx2-playground$ ll bazel-out/k8-opt-exec-2B5CBBC6/bin/external/crate_index__ouroboros_macro-0.18.4/libouroboros_macro-3961516138.so
-r-xr-xr-x 1 ubuntu ubuntu 7775704 Mar 29 10:30 bazel-out/k8-opt-exec-2B5CBBC6/bin/external/crate_index__ouroboros_macro-0.18.4/libouroboros_macro-3961516138.so*

And a more interesting fact is, if we copy this shared library file to /tmp and update above failed command to

bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --subst 'pwd=${pwd}' -- bazel-out/k8-dbg/bin/external/rust_linux_x86_64__x86_64-unknown-linux-gnu__stable_tools/rust_toolchain/bin/rustc external/crate_index__ouroboros-0.18.5/src/lib.rs '--crate-name=ouroboros' '--crate-type=rlib' '--error-format=human' '--codegen=metadata=-2161451017' '--codegen=extra-filename=-2161451017' '--out-dir=bazel-out/k8-dbg/bin/external/crate_index__ouroboros-0.18.5' '--codegen=opt-level=0' '--codegen=debuginfo=2' '--remap-path-prefix=${pwd}=' '--emit=dep-info,link' '--color=always' '--target=x86_64-unknown-linux-gnu' --cfg 'feature="default"' --cfg 'feature="std"' -L bazel-out/k8-dbg/bin/external/rust_linux_x86_64__x86_64-unknown-linux-gnu__stable_tools/rust_toolchain/lib/rustlib/x86_64-unknown-linux-gnu/lib '--cap-lints=allow' '--edition=2018' '--extern=aliasable=bazel-out/k8-dbg/bin/external/crate_index__aliasable-0.1.3/libaliasable-2691302377.rlib' '--extern=static_assertions=bazel-out/k8-dbg/bin/external/crate_index__static_assertions-1.1.0/libstatic_assertions-521553119.rlib' '--extern=ouroboros_macro=/tmp/libouroboros_macro-3621082506.so' '-Ldependency=bazel-out/k8-dbg/bin/external/crate_index__aliasable-0.1.3' '-Ldependency=bazel-out/k8-dbg/bin/external/crate_index__static_assertions-1.1.0' '-Ldependency=bazel-out/k8-opt-exec-2B5CBBC6/bin/external/crate_index__ouroboros_macro-0.18.5'

then it can pass.

so our guess is for some reason processwrapper cannot find the dependencies under bazel-out/k8-opt-exec-2B5CBBC6 which contains all shared library format deps but can find deps under bazel-out/k8-dbg which stores all rlib format deps.

we dont know how rules_rust or bazel manage this kind of deps visibility, but the command we use to build is bazel build //srcs:main --config=internal_toolchain_gcc11_libstdc++ --subcommands --sandbox_debug --verbose_failures --spawn_strategy=local

allenwang-git avatar Mar 29 '25 17:03 allenwang-git

I have a similar issue.

But I have a pretty weird observed behaviour:

  • building binary A on macOS aarch64 host cross-compiling to aarch64-pc-windows-msvc
    • ✅ works
  • building binary B on macOS aarch64 host cross-compiling to aarch64-pc-windows-msvc
    • ✅ works
  • building binary A on Windows ARM host compiling to aarch64-pc-windows-msvc
    • ❌ fails
    • fails to locate most of the crates, local and cargo ones alike
  • building binary B on Windows ARM host compiling to aarch64-pc-windows-msvc
    • ✅ works
    • it even uses one of the same local crate that failed to be resolved in the case above, and manages to resolve it properly

I've seen A was using an argfile while B was not, so I changed rules_rust to use rustc_flags.use_param_file("@%s", use_always = True) instead of False, still observing the same failure.

Rules used

  • rules_rust in 0.66.0, with a few minor patches (I can share if necessary)
  • llvm_toolchains in 1.5.0, patched for Windows host to work

ArchangelX360 avatar Dec 09 '25 16:12 ArchangelX360