vitest-dev
Unexpected Istanbul code coverage report
Describe the bug
In my experience Vitest produces an unexpected code coverage report with Istanbul. In general, when I have only unit tests in my applicaiton, Vitest+ Istanbul outputs a perfect code coverage report. However, if I do anything more complex, the code coverage report is incorrect and unexplainable.
For example, I a have a Fastify application. The moment I do testing with Fastify (i.e. not strict unit tests), the code coverage go haywire. I am comparing Vitest + Istanbul to Node TAP (NYC + Istanbul).
Coverage report with Node TAP:
-----------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------------|---------|----------|---------|---------|-------------------
All files | 92.3 | 75 | 88.88 | 91.66 |
src | 100 | 100 | 100 | 100 |
app.ts | 100 | 100 | 100 | 100 |
src/lib | 87.5 | 75 | 100 | 87.5 |
foo-bar.ts | 100 | 100 | 100 | 100 |
hello-world.ts | 75 | 50 | 100 | 75 | 5
src/routes | 100 | 100 | 100 | 100 |
root.ts | 100 | 100 | 100 | 100 |
src/routes/foo | 80 | 100 | 66.66 | 75 |
index.ts | 80 | 100 | 66.66 | 75 | 6
-----------------|---------|----------|---------|---------|-------------------
Coverage report with Vitest + Istanbul:
-----------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------------|---------|----------|---------|---------|-------------------
All files | 13.63 | 0 | 12.5 | 13.63 |
src | 27.27 | 100 | 50 | 27.27 |
app.ts | 100 | 100 | 100 | 100 |
server.ts | 0 | 100 | 0 | 0 | 2-12
src/lib | 0 | 0 | 0 | 0 |
foo-bar.ts | 0 | 0 | 0 | 0 | 2-5
hello-world.ts | 0 | 0 | 0 | 0 | 2-5
src/routes | 0 | 100 | 0 | 0 |
root.ts | 0 | 100 | 0 | 0 | 3-5
src/routes/foo | 0 | 100 | 0 | 0 |
index.ts | 0 | 100 | 0 | 0 | 3-4
-----------------|---------|----------|---------|---------|-------------------
I am not sure what would be causing such a dramatic difference in coverage report. Any ideas?
Reproduction
I setup a GitHub repo that you can clone to see the above issue: https://github.com/kylerush/vitest-v-tap
System Info
System:
OS: macOS 12.5.1
CPU: (10) arm64 Apple M1 Pro
Memory: 321.05 MB / 16.00 GB
Shell: 5.8.1 - /bin/zsh
Binaries:
Node: 18.7.0 - ~/.nvm/versions/node/v18.7.0/bin/node
npm: 8.15.0 - ~/.nvm/versions/node/v18.7.0/bin/npm
Browsers:
Chrome: 99.0.4844.51
Firefox: 103.0.2
Safari: 15.6.1
npmPackages:
vite: ^3.1.0 => 3.1.0
vitest: ^0.23.1 => 0.23.1
Used Package Manager
npm
Validations
- [X] Follow our Code of Conduct
- [X] Read the Contributing Guidelines.
- [X] Read the docs.
- [X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
- [X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
- [X] The provided reproduction is a minimal reproducible example of the bug.
When @fastify/autoload is loading the routes, these are not loaded through vitest's Vite plugins. nyc has something called istanbul-lib-hook which might be related to this but as it does not have documentation it's a bit tricky to use.
Is the coverage correct when using Jest?
Yes, the coverage is correct with Jest. The uncovered lines in Jest's report matches TAP's report. The overall % discrepancy I suspect has something to do with how TAP and Jest handle TypeScript, but not certain.
Jest:
-----------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------------|---------|----------|---------|---------|-------------------
All files | 85.71 | 75 | 85.71 | 85.71 |
src | 100 | 100 | 100 | 100 |
app.ts | 100 | 100 | 100 | 100 |
src/lib | 83.33 | 75 | 100 | 83.33 |
foo-bar.ts | 100 | 100 | 100 | 100 |
hello-world.ts | 66.66 | 50 | 100 | 66.66 | 5
src/routes | 100 | 100 | 100 | 100 |
root.ts | 100 | 100 | 100 | 100 |
src/routes/foo | 50 | 100 | 50 | 50 |
index.ts | 50 | 100 | 50 | 50 | 6
-----------------|---------|----------|---------|---------|-------------------
TAP:
-----------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------------|---------|----------|---------|---------|-------------------
All files | 92.3 | 75 | 88.88 | 91.66 |
src | 100 | 100 | 100 | 100 |
app.ts | 100 | 100 | 100 | 100 |
src/lib | 87.5 | 75 | 100 | 87.5 |
foo-bar.ts | 100 | 100 | 100 | 100 |
hello-world.ts | 75 | 50 | 100 | 75 | 5
src/routes | 100 | 100 | 100 | 100 |
root.ts | 100 | 100 | 100 | 100 |
src/routes/foo | 80 | 100 | 66.66 | 75 |
index.ts | 80 | 100 | 66.66 | 75 | 6
-----------------|---------|----------|---------|---------|-------------------
Vitest:
-----------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------------|---------|----------|---------|---------|-------------------
All files | 13.63 | 0 | 12.5 | 13.63 |
src | 27.27 | 100 | 50 | 27.27 |
app.ts | 100 | 100 | 100 | 100 |
server.ts | 0 | 100 | 0 | 0 | 2-12
src/lib | 0 | 0 | 0 | 0 |
foo-bar.ts | 0 | 0 | 0 | 0 | 2-5
hello-world.ts | 0 | 0 | 0 | 0 | 2-5
src/routes | 0 | 100 | 0 | 0 |
root.ts | 0 | 100 | 0 | 0 | 3-5
src/routes/foo | 0 | 100 | 0 | 0 |
index.ts | 0 | 100 | 0 | 0 | 3-4
-----------------|---------|----------|---------|---------|-------------------
With istanbul-lib-hook we are able to capture these require calls. But its API is synchronous which makes it difficult to work with asynchronous vitenode.transformRequest.
vitest::node/pool.ts: fetch "/src/app.ts"
vitest::coverage-istanbul/src/provider.ts: onFileTransform "/workspaces/vitest-v-tap/src/app.ts"
vitest::node/pool.ts: fetch "/@fs/workspaces/vitest-v-tap/node_modules/.pnpm/@[email protected]/node_modules/@fastify/autoload/index.js"
vitest::node/pool.ts: fetch "/@fs/workspaces/vitest-v-tap/node_modules/.pnpm/[email protected]/node_modules/fastify/fastify.js"
istanbul-lib-hook::hookRequire captured /workspaces/vitest-v-tap/src/routes/root.ts
istanbul-lib-hook::hookRequire captured /workspaces/vitest-v-tap/src/lib/hello-world.ts
istanbul-lib-hook::hookRequire captured /workspaces/vitest-v-tap/src/routes/foo/index.ts
istanbul-lib-hook::hookRequire captured /workspaces/vitest-v-tap/src/lib/foo-bar.ts
@fastify/autoload seems to be using ts-node, but this is already removed in their master branch. https://github.com/fastify/fastify-autoload/blob/v5.3.0/index.js#L165
I'm not sure whether vitest should already be able to intercept require and import calls done by dependencies. If it should, the ts-node registration is overwriting that.
I added the ts-node register call in autoload in 5.3.0. Since vitest doesn't use ts-node (I think it uses esbuild to compile typescript?) autoload was throwing errors that it couldn't load .ts files because without running the test files with ts-node there was no way to compile the Typescript on the fly. This change was rolled back in 5.3.1 because it broke some non-typescript uses cases of autoload.
➜ vitest-v-tap git:(krush/ts-node) ✗ npm run test:vitest
> test:vitest
> vitest --coverage run
RUN v0.23.1 /Users/kylerush/projects/vitest-v-tap
Coverage enabled with istanbul
❯ test/basic.vitest.test.ts (1)
❯ basic test (1)
× e2e test
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
FAIL test/basic.vitest.test.ts > basic test > e2e test
Error: @fastify/autoload cannot import plugin at '/Users/kylerush/projects/vitest-v-tap/src/routes/root.ts'. To fix this error compile TypeScript to JavaScript or use 'ts-node' to run your app.
❯ findPlugins node_modules/@fastify/autoload/index.js:219:15
❯ autoload node_modules/@fastify/autoload/index.js:30:22
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯
Test Files 1 failed (1)
Tests 1 failed (1)
Start at 13:52:54
Duration 665ms (transform 268ms, setup 0ms, collect 147ms, tests 8ms)
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.
❯ findPlugins node_modules/@fastify/autoload/index.js:166:13
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Rejection ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Error: @fastify/autoload cannot import hooks plugin at '/Users/kylerush/projects/vitest-v-tap/src/routes/foo/index.ts'. To fix this error compile TypeScript to JavaScript or use 'ts-node' to run your app.
% Coverage report from istanbul
-----------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------------|---------|----------|---------|---------|-------------------
All files | 13.63 | 0 | 12.5 | 13.63 |
src | 27.27 | 100 | 50 | 27.27 |
app.ts | 100 | 100 | 100 | 100 |
server.ts | 0 | 100 | 0 | 0 | 2-12
src/lib | 0 | 0 | 0 | 0 |
foo-bar.ts | 0 | 0 | 0 | 0 | 2-5
hello-world.ts | 0 | 0 | 0 | 0 | 2-5
src/routes | 0 | 100 | 0 | 0 |
root.ts | 0 | 100 | 0 | 0 | 3-5
src/routes/foo | 0 | 100 | 0 | 0 |
index.ts | 0 | 100 | 0 | 0 | 3-4
-----------------|---------|----------|---------|---------|-------------------
You can see that in the krush/ts-node branch in the repo I made.
I'm not familiar enough with vitest (or vite) to know what direction to take this in from here. What do you think?
Typescript support is checked with process.env.JEST_WORKER_ID: https://github.com/fastify/fastify-autoload/blob/88447942049db069b89abdb8ac38bdc38a00eeae/index.js#L9
Maybe, if you add Vitest there it will work, but I haven't tried it. Vitest has process.env.VITEST_WORKER_ID.
So does that mean that other tooling can rely on vitest to handle code transforms of require'd files when process.env.VITEST_WORKER_ID is set? The logic on @fastify/autoload seems to expect so for Jest. And for Jest it seems to work.
I tried that locally by modifying the @fastify/[email protected] which has no ts-node. The require'd files are not processed by Vite. This line ends up loading src/routes/foo/index.ts as is - containing ESM and TS.
So does that mean that other tooling can rely on
vitestto handle code transforms ofrequire'd files whenprocess.env.VITEST_WORKER_IDis set? The logic on@fastify/autoloadseems to expect so for Jest. And for Jest it seems to work.
Vitest doesn't process require calls. It cannot process require calls, because Vite's resolution is asynchronous. Non-js file should be imported with await import(), I don't know why this autoload tool uses require at all.
That seems to be the root cause. By manually changing @fastify/[email protected] to use dynamic import() instead of require, the files are picked and instrumented by Vitest correctly.
% Coverage report from istanbul
-----------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-----------------|---------|----------|---------|---------|-------------------
All files | 40.9 | 25 | 62.5 | 40.9 |
src | 27.27 | 100 | 50 | 27.27 |
app.ts | 100 | 100 | 100 | 100 |
server.ts | 0 | 100 | 0 | 0 | 2-12
src/lib | 33.33 | 25 | 50 | 33.33 |
foo-bar.ts | 0 | 0 | 0 | 0 | 2-5
hello-world.ts | 66.66 | 50 | 100 | 66.66 | 5
src/routes | 100 | 100 | 100 | 100 |
root.ts | 100 | 100 | 100 | 100 |
src/routes/foo | 50 | 100 | 50 | 50 |
index.ts | 50 | 100 | 50 | 50 | 6
-----------------|---------|----------|---------|---------|-------------------
@fastify/autoload::loadPlugin::import /workspaces/vitest-v-tap/src/routes/root.ts
@fastify/autoload::loadPlugin::import /workspaces/vitest-v-tap/src/routes/foo/index.ts
vitest::coverage-istanbul/src/provider.ts: onFileTransform "/workspaces/vitest-v-tap/src/routes/root.ts"
vitest::coverage-istanbul/src/provider.ts: onFileTransform "/workspaces/vitest-v-tap/src/routes/foo/index.ts"
vitest::node/pool.ts: fetch "/src/lib/hello-world.ts"
vitest::node/pool.ts: fetch "/src/lib/foo-bar.ts"
vitest::coverage-istanbul/src/provider.ts: onFileTransform "/workspaces/vitest-v-tap/src/lib/hello-world.ts"
vitest::coverage-istanbul/src/provider.ts: onFileTransform "/workspaces/vitest-v-tap/src/lib/foo-bar.ts"
I also had to set deps: { inline: ['@fastify/autoload'] }. Otherwise error below was thrown:
FAIL test/basic.vitest.test.ts > basic test > e2e test
TypeError: Unknown file extension ".ts" for /workspaces/vitest-v-tap/src/routes/root.ts
❯ loadPlugin node_modules/.pnpm/@[email protected]/node_modules/@fastify/autoload/index.js:246:17
files are picked and instrumented by Vitest correctly
By the way, I think if instrumenter should process files under require, maybe we should intercept require for it? I am not against it, there is an open issue for intercepting require to use test.alias. We would need to use Module from 'module' for this, I think?
That would still mean that projects would have to do the possible TS transform themselves, and in require we would only catch the transpiled code, right?
In that case I think we'll have trouble with source maps. Where would we get mappings all the way to source files?
Intercepting require only for coverage instrumentation won't solve everything. It would be simple if we had a way to run the synchronous require'd files through vite, but as it is asynchronous it's not possible.
@kylerush for now, if you could get the @fastify/autoload to use await import() instead of require, Vitest would do the TS transforming and coverage instrumentation correctly, out-of-the-box. As long as you define the package as inline dependency, deps: { inline: ['@fastify/autoload'] }.
That would still mean that projects would have to do the possible TS transform themselves, and in
requirewe would only catch the transpiled code, right?In that case I think we'll have trouble with source maps. Where would we get mappings all the way to source files?
Intercepting
requireonly for coverage instrumentation won't solve everything. It would be simple if we had a way to run the synchronousrequire'd files through vite, but as it is asynchronous it's not possible.
I think it's fine. Right now Vitest only supports original Node require. It won't solve this issue, but will at least be in line with the current trajectory.
@sheremet-va If I try to add the deps: { inline: ['@fastify/autoload'] } line (with v5.4.0 and v5.3.0 of @fastify/autoload), I get the following error:
FAIL test/integration/equipment.test.ts [ test/integration/equipment.test.ts ]
AvvioError: Plugin must be a function or a promise. Received: 'undefined'
❯ assertPlugin node_modules/.pnpm/[email protected]/node_modules/avvio/boot.js:207:11
❯ Boot._addPlugin node_modules/.pnpm/[email protected]/node_modules/avvio/boot.js:240:12
❯ Boot.use node_modules/.pnpm/[email protected]/node_modules/avvio/boot.js:216:25
❯ server.<computed> [as register] node_modules/.pnpm/[email protected]/node_modules/avvio/boot.js:40:14
❯ src/app.ts:15:4
13| });
14|
15| app.register(fastifyAutoload, {
| ^
16| dir: path.join(__dirname, 'plugins'),
17| });
❯ async /Users/jclaessens/dev/github/vitest-alias-repro/packages/api/test/integration/equipment.test.ts:7:31
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: {
"code": "AVV_ERR_PLUGIN_NOT_VALID",
}
I came across this issue when trying to setup a repro for another issue (#2135 ) but suddenly I came across issues that I didn't have with my main app (https://github.com/fastify/fastify-autoload/issues/230#issuecomment-1274259934) 🧐
@jclaessens97 the answer for both of your errors: replace your alias with "@/": path.resolve(__dirname, "./src/"),. Your current one (@) replaces every @ with a path.
I see what you're saying, but it doesn't change anything. I still get the same 2 errors.
I tried both:
'@/': path.resolve(__dirname, './src/'),
'@/*': path.resolve(__dirname, './src/*'),
I see what you're saying, but it doesn't change anything. I still get the same 2 errors.
I tried both:
'@/': path.resolve(__dirname, './src/'),'@/*': path.resolve(__dirname, './src/*'),
The second one will not work, because alias is regex, not a glob.
Are you sure you are getting the same error? When I change alias, I get a new error that fetch is not implemented, which is understandable.
I'm 99% sure. The fetch implementation is probably because you're not using node v18?
This error is not from Vitest, it's from Node. Node doesn't know about your aliases, if you don't have deps.registerNodeLoader enabled.
So you're saying I have to enable deps.registerNodeLoader? I tried just now, but it seems nothing is changing 🥲
Can I send you a dm somewhere to not pollute this issue any further?
So you're saying I have to enable
deps.registerNodeLoader? I tried just now, but it seems nothing is changing 🥲Can I send you a dm somewhere to not pollute this issue any further?
Did you enable deps.inline, as others in the thread pointed out? This is the config I'm using in your repo:
export default defineConfig({
test: {
deps: { inline: ["@fastify/autoload"] },
},
resolve: {
alias: {
"@/": path.resolve(__dirname, "./src/"),
},
},
});
It gives me Environment variable not found: DATABASE_URL. error, which I will not fix myself, since I'm not interested in how this works, but I assume Vitest works fine here.
deps.registerNodeLoader will not help here, since your files are typescript, and this setting only affects aliases, not transforming and running code.
It gives me
Environment variable not found: DATABASE_URL.error, which I will not fix myself, since I'm not interested in how this works, but I assume Vitest works fine here.
For me, that exact setup gives the following (first error I posted):
The error you get is what I would expect since you don't have the .env file. I indeed assume when you get to that point, it works as intended since all the imports are resolved correctly.
My last guess that I can come up is that there is some sort of version mismatch?