🐛 BUG: "Module cannot be synchronously required" when using `@cloudflare/vitest-pool-workers`

[nvie]

Which Cloudflare product(s) does this pertain to?

Miniflare, Workers Vitest Integration

What version(s) of the tool(s) are you using?

[email protected], @cloudflare/[email protected], [email protected]

What version of Node are you using?

v18.19.1

What operating system and version are you using?

Mac Sonoma 14.2.1

Describe the Bug

Observed behavior

After switching to using @cloudflare/vitest-pool-workers in the vitest.config.ts file, we're seeing CJS/ESM errors that were not there before. My minimal replication doesn't even use any Miniflare/Wrangler bits, but shows how just changing the defineConfig to defineWorkersConfig breaks the tests.

Expected behavior

The simple test would still pass, despite now using the defineWorkersConfig call.

Steps to reproduce

1. Check out my minimal replication on this branch
2. Check out the second-to-last commit
3. Run npm run test, notice it passes without using the pool workers wrapper
4. Now check out the last commit, which only changes the pool wrapper
5. Run npm run test, notice the tests now fail due to a CJS/ESM issue that wasn't there before

Please provide a link to a minimal reproduction

https://github.com/nvie/clean-project/tree/vitest-jsonwebtoken

Please provide any relevant error logs

 FAIL  test/bug.test.ts [ test/bug.test.ts ]
Error: Module cannot be synchronously required while it is being instantiated or evaluated. This error typically means that a CommonJS or NodeJS-Compat type module has a circular dependency on itself, and that a synchronous require() is being called while the module is being loaded.
 ❯ Users/nvie/Projects/clean-project/node_modules/semver/classes/range.js?mf_vitest_no_cjs_esm_shim:205:20
 ❯ Users/nvie/Projects/clean-project/node_modules/semver/classes/comparator.js?mf_vitest_no_cjs_esm_shim:141:15
 ❯ Users/nvie/Projects/clean-project/node_modules/semver/index.js?mf_vitest_no_cjs_esm_shim:29:20
 ❯ Users/nvie/Projects/clean-project/node_modules/jsonwebtoken/lib/asymmetricKeyDetailsSupported.js?mf_vitest_no_cjs_esm_shim:1:16
 ❯ Users/nvie/Projects/clean-project/node_modules/jsonwebtoken/lib/validateAsymmetricKey.js?mf_vitest_no_cjs_esm_shim:1:42
 ❯ Users/nvie/Projects/clean-project/node_modules/jsonwebtoken/verify.js?mf_vitest_no_cjs_esm_shim:6:31
 ❯ Users/nvie/Projects/clean-project/node_modules/jsonwebtoken/index.js?mf_vitest_no_cjs_esm_shim:3:11

[nvie]

Another, maybe similar, CJS/ESM bug happens when importing from [email protected].

Simply adding:

import { error } from "itty-router-extras" // @0.4.5

will break the test with the following error:

FAIL  test/bug.test.ts [ test/bug.test.ts ]
Error: No such module "Users/nvie/Projects/clean-project/node_modules/itty-router-extras/middleware".
 ❯ Users/nvie/Projects/clean-project/node_modules/itty-router-extras/index.js?mf_vitest_no_cjs_esm_shim:1:20

[nvie]

We worked around this by no longer using these packages, and it's no longer an issue for us.

[JordanFaust]

Is there an actual resolution to this problem other than not using the dependencies? Have the same problem for sermver. Should this issue actually be closed?

[ahmadbilaldev]

Facing the same issue. Is there a solution?

[FlowFlorent]

Getting a similar issue when using the svix package. Should this issue be reopened to hopefully find a solution?

Note: issue still present on 0.4.3

Error: Module cannot be synchronously required while it is being instantiated or evaluated. This error typically means that a CommonJS or NodeJS-Compat type module has a circular dependency on itself, and that a synchronous require() is being called while the module is being loaded.
 ❯ Users/florentlefebvre/dev/liveblocks-cloudflare/node_modules/svix/dist/openapi/http/isomorphic-fetch.js?mf_vitest_no_cjs_esm_shim:13:16
 ❯ Users/florentlefebvre/dev/liveblocks-cloudflare/node_modules/svix/dist/openapi/http/http.js?mf_vitest_no_cjs_esm_shim:25:14
 ❯ Users/florentlefebvre/dev/liveblocks-cloudflare/node_modules/svix/dist/openapi/index.js?mf_vitest_no_cjs_esm_shim:13:14
 ❯ Users/florentlefebvre/dev/liveblocks-cloudflare/node_modules/svix/dist/index.js?mf_vitest_no_cjs_esm_shim:14:17

[esdee]

I'm seeing this issue with Kysely as a dependency in my project. Was wondering if there's been any further news on this.

[ychua]

This issue needs to be look into.

Vitest is the recommended testing framework and this issue is stopping us from using the worker with AWS SDKs, to be specific, awssns.

These days it is common to intetegrate worker with AWS-sdks or sementic versioning, I strongly suggest the worker-sdk team to review the priority of this bug again

[travis]

we're also running into this issue:

https://github.com/travis/fireproof-ucan/pull/2

any updates on an ETA for a fix?

[darrenvechain]

Any update on this? I'm having the same issue with crypto:

import { createHash } from 'crypto';

export default {
    fetch: async () => {
        const hash = createHash('sha256');
        return new Response(hash.digest().toString('hex'));
    }
}

[ducan-ne]

Been waiting for this for months 🥲 My dependencies seem to have errors: zod-error, workos, @microlabs/otel-cf-workers

[jasnell]

This is still on the radar but fixing is very tricky. I do not have an current ETA yet.

[davidbarratt]

Adding another, similar (?) issue to the pile.  Here is the code: https://github.com/davidbarratt/cache-tag/tree/47ad4b65b32791675f455f362f3ffeee6046bf12/workers/watcher

$ pnpm test

> [email protected] test /home/david/sites/davidbarratt/cache-tag/workers/watcher
> vitest run


 RUN  v1.5.3 /home/david/sites/davidbarratt/cache-tag/workers/watcher

Using vars defined in .dev.vars
[vpw:inf] Starting isolated runtimes for vitest.config.mts...
workerd/server/server.c++:2882: error: Fallback service failed to fetch module; payload = ; spec = /?specifier=%2F%40fs%2Fhome%2Fdavid%2Fsites%2Fdavidbarratt%2Fcache-tag%2Fnode_modules%2F.pnpm%2Ftracking-query-params-registry%40https%2B%2B%2Bcodeload.github.com%2Bmpchadwick%2Btracking-query-params-r_7stum53eclswjqfjkg3iectg6a%2Fnode_modules%2Ftracking-query-params-registry%2F_data%2Fparams.csv%3Fmf_vitest_force%3DText&referrer=%2Fhome%2Fdavid%2Fsites%2Fdavidbarratt%2Fcache-tag%2Fnode_modules%2F.pnpm%2Fvite-node%401.5.3_%40types%2Bnode%4022.5.1%2Fnode_modules%2Fvite-node%2Fdist%2Fclient.mjs&rawSpecifier=%2F%40fs%2Fhome%2Fdavid%2Fsites%2Fdavidbarratt%2Fcache-tag%2Fnode_modules%2F.pnpm%2Ftracking-query-params-registry%40https%2B%2B%2Bcodeload.github.com%2Bmpchadwick%2Btracking-query-params-r_7stum53eclswjqfjkg3iectg6a%2Fnode_modules%2Ftracking-query-params-registry%2F_data%2Fparams.csv%3Fmf_vitest_force%3DText
 ❯ test/index.spec.ts (0)

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

 FAIL  test/index.spec.ts [ test/index.spec.ts ]
Error: No such module "@fs/home/david/sites/davidbarratt/cache-tag/node_modules/.pnpm/tracking-query-params-registry@https+++codeload.github.com+mpchadwick+tracking-query-params-r_7stum53eclswjqfjkg3iectg6a/node_modules/tracking-query-params-registry/_data/params.csv?mf_vitest_force=Text".
 ❯ src/index.ts:1:1
      1| import rawParams from "tracking-query-params-registry/_data/params.csv";
       | ^
      2|
      3| const params = new Set<string>(

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

 Test Files  1 failed (1)
      Tests  no tests
   Start at  22:40:52
   Duration  957ms (transform 35ms, setup 0ms, collect 0ms, tests 0ms, environment 0ms, prepare 209ms)

[vpw:dbg] Shutting down runtimes...
 ELIFECYCLE  Test failed. See above for more details.

 

[crisner1978]

I had the same issue, just import whatever package is throwing the error wherever you plan to use it like so

async function myFunc(){
    const package = (await import('yourPackageThatIsThrowingTheError'))
    ...
}

[jsumeracki-navico]

I had the same issue, just import whatever package is throwing the error wherever you plan to use it like so

async function myFunc(){
    const package = (await import('yourPackageThatIsThrowingTheError'))
    ...
}

This allows some code to execute but ultimately just delays the same error later in the runtime for me.

[kyranjamie]

My first intro to writing a worker test and running into this issue. Seems related to the fact I had external deps.

[Grunet]

For anyone using @microlabs/otel-cf-workers, FYI I wasn't able to use vi.mock to mock it or an adapter out, seemingly because the cloudflare:test module was initializing the worker entrypoint module and fetching all the module dependencies before vi.mock was invoked (which I still don't really get because I thought vi.mock was always hoisted before imports). This maybe isn't an issue though for dependencies that aren't used on initial startup afaict, unlike in this case.

My workaround is pretty icky, but I just wrote some scripts to rewrite 'export default instrument(handler, config);' to 'export default handler;' and vice-versa before/after vitest runs, and that seems to avoid this issue.

[edmundhung]

The error should be resolved in Wrangler v3.93.0 and later. I've confirmed the fix using the repo provided by the issue author. If you still see the same error, please let us know. Thank you!

[edmundhung]

Closing this issue as the problem described by the author has been resolved. If you encounter this issue after upgrading to Wrangler v3.93.0 or later, please let us know. Thank you!