Resolving Node.js packages

[guybedford]

I really like the static nature of source resolution during Wizering as an alternative to bundling.

Further to this, it could be worth implementing the full Node.js resolution algorithm to automatically bundle dependencies from npm as well.

[cataggar]

Checking in. I still really want to expose existing JavaScript libraries in components. For example, I want to be able to use the TypeScript compiler as a component service. With the latest tools, it currently fails:

~/ms/component/packages/cowsay> npx tsc
~/ms/component/packages/cowsay> jco componentize cowsay.js --wit src/cowsay.wit -o cowsay.wasm
Error opening file ./typescript.js (resolved to /var/folders/xm/hr4sq0d567vfs0xdth9t38zc0000gn/T/ad9807ff193c/sources/typescript.js): No such file or directory
Error: the `componentize.wizer` function trapped

Caused by:
    0: error while executing at wasm backtrace:
           0: 0x7a84c5 - <unknown>!<wasm function 12579>
           1: 0x7a798a - <unknown>!<wasm function 12356>
           2: 0x50cf26 - <unknown>!<wasm function 1721>
           3: 0x33853e - <unknown>!<wasm function 485>
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    1: Exited with i32 exit status 1
(jco componentize) Error: Failed to initialize the compiled Wasm binary with Wizer:
Wizering failed to complete
    at componentize (file:///opt/homebrew/lib/node_modules/@bytecodealliance/jco/node_modules/@bytecodealliance/componentize-js/src/componentize.js:260:13)
    at async componentize (file:///opt/homebrew/lib/node_modules/@bytecodealliance/jco/src/cmd/componentize.js:11:25)
    at async file:///opt/homebrew/lib/node_modules/@bytecodealliance/jco/src/jco.js:200:9

The simplest example to get started:

src/cowsay.ts

import * as ts from 'typescript';

export const cow = {

  tsVersion() {
    return ts.version;
  }

};

It is implementing this wit:

src/cowsay.wit

package local:cowsay;
world cowsay {
  export cow: interface {
    ts-version: func() -> string;
  }
}

There are a lot of cool use cases if this could be made to work. The host would be Node.js.

~/ms/component/packages/cowsay> jco --version
1.6.0

[karthik2804]

@cataggar I agree that it would be cool to have full node.js resolution. In the meantime, I think bundling should help unlock the above use case you shared.

[cataggar]

@karthik2804, that would be amazing if there was a way to unlock the above use case! What do you mean by "bundling"?

[karthik2804]

@cataggar using a bundler like webpack or rollup to do the module resolution at build time and output one singular JS file that can be passed as input to jco componentize.

ref: https://webpack.js.org/ or https://rollupjs.org/

[cataggar]

@karthik2804, thanks for the advice. I'll try it.

[cataggar]

I was not able to get it to work with rollup. The stacktrace is something about "async componentize". I put a reproducible project in https://github.com/cataggar/StarlingMonkey15 .

git clone [email protected]:cataggar/StarlingMonkey15.git
cd StarlingMonkey15
npm ci
npm run build

Results in this error:

~/ms/StarlingMonkey15> npm run build

> build
> npm run build --workspace=cowsay


> [email protected] build
> npm run bundle; npm run componentize


> [email protected] bundle
> npx rollup -c


src/cowsay.ts → cowsay.js...
(!) src/cowsay.ts (4:18): Error when using sourcemap for reporting an error: Can't resolve original location of error.
/Users/cataggar/ms/StarlingMonkey15/cowsay/src/cowsay.ts:4:18
(!) Missing exports
https://rollupjs.org/troubleshooting/#error-name-is-not-exported-by-module
src/cowsay.ts
version is not exported by ../node_modules/typescript/lib/typescript.js
2: export const cow = {
3:     tsVersion() {
4:         return ts.version;
                     ^
5:     }
6: };
created cowsay.js in 2.2s

> [email protected] componentize
> npx jco componentize cowsay.js --wit src/cowsay.wit -o cowsay.wasm

Redirecting call to abort() to mozalloc_abort

Error: the `componentize.wizer` function trapped

Caused by:
    0: error while executing at wasm backtrace:
           0: 0x7a3ec5 - <unknown>!<wasm function 11740>
           1: 0x7a7fee - <unknown>!<wasm function 12478>
           2: 0x7a84e0 - <unknown>!<wasm function 12582>
           3: 0x72c8c0 - <unknown>!<wasm function 6861>
           4: 0x352387 - <unknown>!<wasm function 522>
           5: 0x6ad3b2 - <unknown>!<wasm function 4800>
           6: 0x74ed01 - <unknown>!<wasm function 7670>
           7: 0x36bb2f - <unknown>!<wasm function 562>
           8: 0x4dcd55 - <unknown>!<wasm function 1527>
           9: 0xa91cf - <unknown>!<wasm function 79>
          10: 0x3424d4 - <unknown>!<wasm function 498>
          11: 0x66ac73 - <unknown>!<wasm function 4035>
          12: 0x625349 - <unknown>!<wasm function 3385>
          13: 0x422874 - <unknown>!<wasm function 940>
          14: 0x4225c1 - <unknown>!<wasm function 940>
          15: 0x2d00ad - <unknown>!<wasm function 358>
          16: 0x1fc7ba - <unknown>!<wasm function 186>
          17: 0x50ceed - <unknown>!<wasm function 1721>
          18: 0x33853e - <unknown>!<wasm function 485>
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    1: wasm trap: wasm `unreachable` instruction executed
(jco componentize) Error: Failed to initialize the compiled Wasm binary with Wizer:
Wizering failed to complete
    at componentize (file:///Users/cataggar/ms/StarlingMonkey15/node_modules/@bytecodealliance/componentize-js/src/componentize.js:260:13)
    at async componentize (file:///Users/cataggar/ms/StarlingMonkey15/node_modules/@bytecodealliance/jco/src/cmd/componentize.js:11:25)
    at async file:///Users/cataggar/ms/StarlingMonkey15/node_modules/@bytecodealliance/jco/src/jco.js:200:9
npm error Lifecycle script `componentize` failed with error:
npm error code 1
npm error path /Users/cataggar/ms/StarlingMonkey15/cowsay
npm error workspace [email protected]
npm error location /Users/cataggar/ms/StarlingMonkey15/cowsay
npm error command failed
npm error command /opt/homebrew/bin/nu -c npx jco componentize cowsay.js --wit src/cowsay.wit -o cowsay.wasm
npm error Lifecycle script `build` failed with error:
npm error code 1
npm error path /Users/cataggar/ms/StarlingMonkey15/cowsay
npm error workspace [email protected]
npm error location /Users/cataggar/ms/StarlingMonkey15/cowsay
npm error command failed
npm error command /opt/homebrew/bin/nu -c npm run bundle; npm run componentize

[cataggar]

I debugged this for a bit and uploaded the debug sources with details in https://github.com/bytecodealliance/ComponentizeJS/issues/138 .

[vados-cosmonic]

Hey @cataggar a bit late here (I assume you're past this now!), but I did manage to get including external libs working with Rollup -- if you can ignore the wasmCloud specific stuff, I was able to pull in valibot for use in one of our example components.

[joebowbeer]

@vados-cosmonic Your example moved?

I see a few other bundled examples now, too

https://github.com/wasmCloud/typescript/tree/main/examples/

[vados-cosmonic]

Yup, they sure did -- thanks for adding the new link!

Note that upstream Jco now has an example as well that uses rollup:

https://github.com/bytecodealliance/jco/tree/main/examples/components/http-server-hono

Less examples than the wasmCloud tree but glad to finally have something in the upstream!