humanode icon indicating copy to clipboard operation
humanode copied to clipboard

CI build mode for WASM runtime artifact

Open quasiyoke opened this issue 7 months ago • 9 comments

  1. In the 1st commit, I disable all CI jobs except those necessary for demonstrating the new artifact type.
  2. In the 2nd commit, the new artifact is added.

~~You can also review the draft release: https://github.com/humanode-network/humanode/releases/tag/untagged-c97d2543b7a7af640755~~

I've tested releases as well :grin:

quasiyoke avatar Jun 20 '25 15:06 quasiyoke

I believe we can remove cfg...feature here to include substrate_wasm_builder by default in no-std:

https://github.com/humanode-network/humanode/blob/adab1b49783d96c9195a748d4c6f7283098aad26/crates/humanode-runtime/build.rs#L8

— and then we will be able to build WASM using the following command:

cargo build --locked --workspace --release --package humanode-runtime --no-default-features --features evm-tracing --target wasm32-unknown-unknown

This would reduce the number of dependency crates to build from the current 1270 to just 449. The only thing is, I haven't tested it yet to see if it'll break anything else.

quasiyoke avatar Jun 24 '25 18:06 quasiyoke

I believe we can remove cfg...feature here to include substrate_wasm_builder by default in no-std:

https://github.com/humanode-network/humanode/blob/adab1b49783d96c9195a748d4c6f7283098aad26/crates/humanode-runtime/build.rs#L8

— and then we will be able to build WASM using the following command:

cargo build --locked --workspace --release --package humanode-runtime --no-default-features --features evm-tracing --target wasm32-unknown-unknown

This would reduce the number of dependency crates to build from the current 1270 to just 449. The only thing is, I haven't tested it yet to see if it'll break anything else.

If we remove it it will trigger a wasm build while doing a wasm build - an infinite loop

MOZGIII avatar Jun 24 '25 19:06 MOZGIII

What build command should work just fine though cargo build --locked --workspace --release --package humanode-runtime --no-default-features --features evm-tracing --target wasm32-unknown-unknown

Except it doesn't compress the built artifact

MOZGIII avatar Jun 24 '25 19:06 MOZGIII

After some research, I deem the way of building the runtime via the WasmBuilder preferable; at present that would mean building via build.rs script. It we could avoid building the dependencies of the humanode-runtime before the build script it executed for the usual std mode - then we will have an advantage of not having to pay the price for building them twice (build script will only build the dependencies once, for wasm, and here are the savings)...

MOZGIII avatar Jun 25 '25 01:06 MOZGIII

@MOZGIII

I deem the way of building the runtime via the WasmBuilder preferable; at present that would mean building via build.rs script.

I noticed that Polkadot itself sometime uses wasm-builder from helper scripts. I also believe it's easier to build from build.rs, especially since that was the method envisioned in their template, and we don’t need any specific customization yet.

If we could avoid building the dependencies of the humanode-runtime before the build script it executed for the usual std mode - then we will have an advantage of not having to pay the price for building them twice (build script will only build the dependencies once, for wasm, and here are the savings)...

I don’t yet know how to do this.

quasiyoke avatar Jun 25 '25 13:06 quasiyoke

@dmitrylavrenov

[what about] having a lot of commented code. Is it fully ready for merging or just some kind of checkpoint?

We agreed with Mozgiii right away that launching CI here would require several attempts, and that running too many jobs would consume a significant portion of our CI budget. Therefore, in the first commit, I disabled all CI jobs except those necessary for demonstrating the new artifact type. I think we should first ensure that there are no comments on the Draft PR, and then I'll remove this commit in the final PR.

quasiyoke avatar Jun 25 '25 13:06 quasiyoke

I removed the CLI arguments --package humanode-runtime --lib – the number of crates during compilation remained at 1270. The included std feature likely prevented limiting the scope of compiled packages. It turns out --package only posed a potential risk of breaking features without providing any performance benefits.

I will still see what can be done to avoid building so many crates.

quasiyoke avatar Jun 30 '25 10:06 quasiyoke

Maybe we could just structure the builds in such a way that we also publish a whole node with tracing support? I.e. so that the node would, like, also have the embedded tracing runtime initially, and the tracing variant of the on-chain runtime (code).

MOZGIII avatar Jun 30 '25 13:06 MOZGIII

An experiment confirmed that the cargo flags --package and --workspace work exactly as @Mozgiii initially explained: --package builds the selected package with the minimal possible set of features. --bin builds the selected binary with a set of features for a complete workspace build. --workspace builds the entire workspace in any case. — these options work this way both in 2021 and 2024 edition, with resolver 2 and 3 respectively.

Added Cargo target --bin humanode-runtime-build-selector — the number of crates necessary for building is 673, compared to 1270 when using --workspace.

also publish a whole node with tracing support

I think a separate WASM artifact is more flexible to use. If it’s not needed, I can add node with EVM-tracing here. If another artifact (EVM-tracing node) is required, perhaps it would be best to create it in a separate task?

quasiyoke avatar Jul 01 '25 15:07 quasiyoke