How to describe dependencies on local packages with relative path

[NewGyu]

Hello developers,

I'm trying to understand Component Model with cargo-component. To that end I have tried to create a simple example, but I have the following error and would like to discuss it if you allow me to do it here.

$ cargo component build --release
error: failed to create a target world for package `comp2` (/workspaces/wasm-component-example/comp2/Cargo.toml)

Caused by:
    0: failed to decode component dependency `newgyu:comp1`
    1: dependency is not a WebAssembly component

cargo-component version is here.

 $ cargo component --version
cargo-component-component 0.1.0 (6af088a 2023-08-16 wasi:2750b73)

My example and what I expect

https://github.com/NewGyu/wasm-component-example

There are two packages, then each package has a world.

• newgyu:comp1/random-generator
  • rand func is exported.
• newgyu:comp2/hello
  • hello-world func is exported, and that depends on rand func that is provided by newgyu:comp1.

Directory structures

$ tree -L 2.
.
├── comp1
│   ├── Cargo.toml
│   ├── src
│   │      └── lib.rs
│   └── wit
│           └── world.wit
├── comp2
│   ├── Cargo.toml
│   ├── src
│   │      └── lib.rs
│   └── wit
│           └── world.wit

Code

comp2

comp2 is specifying a dependency on comp1 as local package reference.

[package]
name = "comp2"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
cargo-component-bindings = { git = "https://github.com/bytecodealliance/cargo-component" }

[lib]
crate-type = ["cdylib"]

[package.metadata.component]
package = "newgyu:comp2"

[package.metadata.component.target]
path = "wit"

[package.metadata.component.dependencies]
"newgyu:comp1" = { path = "../comp1/wit/world.wit" }

comp1

comp1 could be built with cargo component build --release. The followings have been built.

• target/bindings/comp1/target.wasm
• target/bindings/comp1/world
• target/wasm32-wasi/release/comp1.wasm

[package]
name = "comp1"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
cargo-component-bindings = { git = "https://github.com/bytecodealliance/cargo-component" }

[lib]
crate-type = ["cdylib"]

[package.metadata.component]
package = "newgyu:comp1"

[package.metadata.component.target]
path = "wit"
world = "random-generator"

[package.metadata.component.dependencies]

Points I would like to ask about

• Is it correct tow to describe dependencies on local packages with relative path ?
• Is the dependency type handled as Wit when described as above ?
  • Why does into_component_world respond error?
  • Should I explicitly specify Wasm as the dependency type? How to specify that in Cargo.toml?

https://github.com/bytecodealliance/cargo-component/blob/3f36696db9c39785886b0766b2b81f83cbc3d29e/crates/core/src/registry.rs#L394-L402

[peterhuene]

Hi @NewGyu, thanks for reporting this issue!

Currently cargo-component expects a dependency specified by path to be to a component wasm and not a WIT file (or an binary-encoded WIT package). Thus it should (in theory) work if you specified a path of ../comp1/target/wasm32-wasi/release/comp1.wasm.

However, this developer experience is definitely not a good one and I would like to change it so that you could simply reference path = "../comp1" like you would a local crate dependency and have it do a proper topological ordering to the bindings information generation such that it uses the other project's bindings data rather than the compilation output.

I'll see if I can fix that this coming week, as that's the proper way of doing this.

[NewGyu]

Thank you for quick respoindig @peterhuene .

As following your advice, I have changed Cargo.toml. https://github.com/NewGyu/wasm-component-example/commit/90bf72c4aa73a7540edc52771be4aebe221bc16c

Then the behavior has changed.

$ cargo component build --release
    Encoding target for comp2 (/workspaces/wasm-component-example/comp2/target/bindings/comp2/target.wasm)
thread 'main' panicked at 'no entry found for key', /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/wit-component-0.13.2/src/encoding/wit.rs:294:24

I thought the above might depends on the version of cargo-component, so I tried upgrading the version.

$ cargo component --version
cargo-component-component 0.1.0 (3f36696 2023-08-18 wasi:2750b73)

However, the error is repcoduced now.

Thank you.

[peterhuene]

That definitely is a bug in wit-component (or, more likely, how cargo-component is synthesizing the target world by merging in the other component's world). I'll take a look at that as well.

[peterhuene]

The panic above is caused by an incorrect world definition created by cargo-component for component dependencies that import types.

I've put up #121 to address that issue.

I'll leave this issue open until I implement proper dependency path referencing of other component projects instead of having to reference build outputs.

[NewGyu]

Did that mean that current comp1.wasm binary is incorrect? I guess that it is needed to re-build comp1 after #121 is applied.

By the way, import declaration to newgyu:comp1 is currently commented out.

package newgyu:comp2

world hello {
//    use newgyu:comp1/random-generator.{rand}
//
//    import rand
    export hello-world: func() -> string
}

Because the following error was occurred when cargo component build.

$ cargo component build
error: failed to create a target world for package `comp2` (/workspaces/wasm-component-example/comp2/Cargo.toml)

Caused by:
    0: failed to parse local target from directory `/workspaces/wasm-component-example/comp2/wit`
    1: interface or world `rand` does not exist

I guess that this is due to the failure to resolve the dependency on newgyu:comp1 specified as [package.metadata.component.dependencies], so I'm hoping that this too will be resolved by the fixes you mentioned.

[peterhuene]

comp1.wasm shouldn't need to be rebuilt, the problem lies with how comp2 was generating its bindings information off of the information in comp1.wasm.

If you want to use newgyu:comp1 as part of your target definition, then you don't want it as part of your component dependencies; you'll want it as a target dependency instead.

Component dependencies are for referencing other components (and soon a dependency component project itself) so that cargo-component will automatically generate import bindings for whatever the other component exports; you don't need to do anything with WIT for that to work.

Target dependencies are for importing interfaces and using types in any local WIT files you might have.

Instead of having:

[package.metadata.component.dependencies]
"newgyu:comp1" = { path = "../comp1/target/wasm32-wasi/comp1.wasm" }

You would have (notice the addition of target in the table name and it's referencing the WIT package and not the component):

[package.metadata.component.target.dependencies]
"newgyu:comp1" = { path = "../comp1/wit" }

With that, you'd be able to import the rand interface like so:

package newgyu:comp2

world hello {
    import newgyu:comp1/rand
    export hello-world: func() -> string
}

The interface's methods would be available in the bindings::newgyu::comp1::rand module.

[NewGyu]

Thank you for the details. Information you gave is helpful for me.

I think I didn't identify the difference between component.dependencies and component.target.dependencies. I didn't also get the term of target in this cargo-component context correctly. I'm sorry that my lack of understanding has bothered you.

Component dependencies are for referencing other components (and soon a dependency component project itself) so that cargo-component will automatically generate import bindings for whatever the other component exports; you don't need to do anything with WIT for that to work.

My expectation was for component dependency, not for target dependency. I think I could get the point that cargo-component can generate bindings from only comp1/target/wasm32-wasi/comp1.wasm without comp1/wit.

I realized my big misconception that comp2/wit should not describe dependency on newgyu:comp1. Please forget the followings. ~~If you allow me to continue asking questions, could you let me know if my assumptions below are correct?~~

• ~~comp2/wit/world.wit is necessary for generating bindings bindings::Hello trait when building comp2.~~
• ~~But import declaration to newgyu:comp1 in comp2/wit/world.wit is unnecessary because cargo-component will generate bindings::newgyu::comp1::* from comp1/target/wasm32-wasi/comp1.wasm.~~

[peterhuene]

I think I didn't identify the difference between component.dependencies and component.target.dependencies. I didn't also get the term of target in this cargo-component context correctly. I'm sorry that my lack of understanding has bothered you.

Absolutely no need to apologize! It's actually a difficult distinction to understand and it is sparely documented.

I realized my big misconception that comp2/wit should not describe dependency on newgyu:comp1. Please forget the followings.

Your understanding is correct. Basically, bindings generation is the configured target world for the project, merged with any exports of any configured component dependencies.

When targeting a local wit package, references to foreign packages in the WIT itself (for example, import foo:bar/baz) requires a target dependency with the same name as the foreign package.

But since you already depend on the component, you don't need to also import the interface exported by the component; cargo-component will automatically add that as an import.

Target dependencies are only used when defining a local target WIT package; if you use target a world from a registry (e.g. target = "wasi:http/[email protected]"), when a registry exists in the future, then target dependencies aren't used.

My hope is that once a registry is in existence and heavily used, most users will target worlds that are published and don't need local WIT files to author their components.

[NewGyu]

Thank you for great improvement.

 $ cargo component --version
cargo-component-component 0.1.0 (86ccf3e 2023-08-28 wasi:e250334)

To use the above version, expected bindings are generated,

cargo_component_bindings::generate!();

// Generated binding method for `rand` function that is defined in comp1
use bindings::comp1::rand;
// Generated types that `rand` function depends on
use bindings::newgyu::comp1::types::{Algorithm, Seed};
// Generated for `hello-world` function that is defined in comp2/world.wit
use bindings::Hello;
struct Component;

impl Hello for Component {
    /// Say hello!
    fn hello_world() -> String {
        let seed = Seed {
            algorithm: Algorithm::Goblin,
            value: 9,
        };
        let n = rand(seed);
        format!("Hello, {}", n)
    }
}

$ cargo component build --release
    Encoding target for comp2 (/workspaces/wasm-component-example/comp2/target/bindings/comp2/target.wasm)
   Compiling comp2 v0.1.0 (/workspaces/wasm-component-example/comp2)
    Finished release [optimized] target(s) in 0.20s
    Creating component /workspaces/wasm-component-example/comp2/target/wasm32-wasi/release/comp2.wasm

That was amazing experience!

[nuke-web3]

Thank you for the details in this issue. I would hope this could make it into the https://component-model.bytecodealliance.org/creating-and-consuming/composing.html docs, or a dedicated doc for cargo component more prominently. I spent way too long trying to let the error messages point me in a better direction, but alas, that was mostly in vain.

Specifically it is not clear what best practices are for a repo with multiple components with various crates used as components themselves, or rather just impl of specific interfaces for some world composed of many crates. https://github.com/bytecodealliance/component-docs/tree/main/component-model/examples/tutorial could include more details and perhaps an additional crate that looks more like the example in this issue to contrast the options devs have in composition and when to use them.