rules_rust icon indicating copy to clipboard operation
rules_rust copied to clipboard

`crate_vendor` does not work from the root package

Open prestontw opened this issue 3 years ago • 10 comments

I'm trying to set up a project that can be built with both cargo and bazel. The repo is at https://github.com/prestontw/bazel-rust-vendored-experiment/tree/return-to-rust-rules

I'm trying to vendor external dependencies, but when I do, I see:

% bazel run //:crates_vendor
INFO: Analyzed target //:crates_vendor (1 packages loaded, 2 targets configured).
INFO: Found 1 target...
Target //:crates_vendor up-to-date:
  bazel-bin/crates_vendor.sh
INFO: Elapsed time: 0.124s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action

error: failed to sync

Caused by:
  failed to create directory `/crates`

Caused by:
  Read-only file system (os error 30)

Error: Failed to vendor dependencies

Caused by:
    Failed to vendor sources with: exit status: 101

I see the same error if I rely on the default value.

From the error message and the directory path, it seems like the path is absolute rather than relative to the workspace. Indeed, specifying the vendor path as an absolute path (/User/preston/git/bazel-experiment/crates) works.

prestontw avatar May 16 '22 18:05 prestontw

(Quick follow-up, using these successfully vendored crate targets doesn't work. It gives the following error:

% bazel run //:hello_world
ERROR: /Users/preston/git/bazel-experiment/BUILD:4:12: //:hello_world: invalid label '///Users/preston/git/bazel-experiment/crates/axum-0.5.6:axum' in element 0 of attribute 'deps' in 'rust_binary' rule: invalid package name '/Users/preston/git/bazel-experiment/crates/axum-0.5.6': package names may not start with '/'
ERROR: error loading package '': Package '' contains errors
INFO: Elapsed time: 0.075s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded)
FAILED: Build did NOT complete successfully (1 packages loaded)

I've removed the absolute path prefix to get the actual project building again.)

prestontw avatar May 16 '22 19:05 prestontw

Interesting. I wonder what the difference is between your example and https://github.com/bazelbuild/rules_rust/tree/0.4.0/examples/crate_universe/vendor_local_manifests

UebelAndre avatar May 16 '22 19:05 UebelAndre

Interesting. I wonder what the difference is between your example and https://github.com/bazelbuild/rules_rust/tree/0.4.0/examples/crate_universe/vendor_local_manifests

The two big differences seem to be different workspace layout and using local_repository instead of http_archive. I can try adding another child folder and mirroring more of the vendor_local_manifests directory structure, but I would be surprised if that changed behavior.

prestontw avatar May 16 '22 19:05 prestontw

@prestontw can you try using the latest release of rules_rust? https://github.com/bazelbuild/rules_rust/releases/tag/0.4.0

UebelAndre avatar May 16 '22 19:05 UebelAndre

@prestontw can you try using the latest release of rules_rust? https://github.com/bazelbuild/rules_rust/releases/tag/0.4.0

Yes, I will. (Sorry for using an old version, I was copying from https://bazelbuild.github.io/rules_rust/)

Separately, https://github.com/prestontw/bazel-rust-vendored-experiment/tree/compare-to-example works for both vendoring and building targets, so the child directory structure does seem to help!

prestontw avatar May 16 '22 19:05 prestontw

@prestontw can you try using the latest release of rules_rust? https://github.com/bazelbuild/rules_rust/releases/tag/0.4.0

Using that still results in the same error: https://github.com/prestontw/bazel-rust-vendored-experiment/commit/8a7febc8260a7d4b12e23933a1bf369fb56ab599

prestontw avatar May 16 '22 19:05 prestontw

but I would be surprised if that changed behavior.

Separately, https://github.com/prestontw/bazel-rust-vendored-experiment/tree/compare-to-example works for both vendoring and building targets, so the child directory structure does seem to help!

You can indeed count me surprised! 👀

prestontw avatar May 16 '22 19:05 prestontw

Separately, https://github.com/prestontw/bazel-rust-vendored-experiment/tree/compare-to-example works for both vendoring and building targets, so the child directory structure does seem to help!

So a clearer description of the issue is crates_vendor does not work from the root package?

UebelAndre avatar May 16 '22 19:05 UebelAndre

Separately, https://github.com/prestontw/bazel-rust-vendored-experiment/tree/compare-to-example works for both vendoring and building targets, so the child directory structure does seem to help!

So a clearer description of the issue is crates_vendor does not work from the root package?

I think so... That does seem like the biggest difference: the repo example has an empty root BUILD while my worked example tries to vendor from the root BUILD.

prestontw avatar May 16 '22 19:05 prestontw

I finally had time to look into this and found it to be an issue with https://github.com/bazelbuild/rules_rust/blob/0.5.0/crate_universe/src/rendering.rs#L109-L115 creating paths that are prefixed with / (and thus are absolute). My quest to solve this lead me to try to use the builtin Label library but I struggle to integrate it due to https://github.com/bazelbuild/rules_rust/issues/1408. A simple fix is just to do a trim but as of now I feel the shiny-future solution is to delete the Label class from crate_universe and use @rules_rust//util/label... That's just more work than a trim_start_matches

UebelAndre avatar Jun 13 '22 01:06 UebelAndre