workers-sdk icon indicating copy to clipboard operation
workers-sdk copied to clipboard

🐛 BUG: Vitest code coverage does not work when using `@cloudflare/vitest-pool-workers`

Open NuroDev opened this issue 1 year ago • 7 comments

Which Cloudflare product(s) does this pertain to?

Workers Vitest Integration

What version(s) of the tool(s) are you using?

0.1.2

What version of Node are you using?

20.5.1

What operating system and version are you using?

macOS Sonoma 14.1.1

Describe the Bug

Observed behavior

Following the basic example from your announcement blog post I created a basic project with it & attempted to enable code coverage by adding the @vitest/coverage-v8 package & added coverage: { enabled: true } property to my Vitest config.

Expected behavior

When running Vitest you get the following error:

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Error: No such module "node:inspector".
  imported from "Users/nuro/Desktop/vitest-test/node_modules/.pnpm/@[email protected][email protected]/node_modules/@vitest/coverage-v8/dist/index.js"
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

Steps to reproduce

      1 .
      2 ├── src
      3 │   └── index.ts
      4 ├── tests
      5 │   └── basic.test.ts
      6 ├── package.json
      7 ├── pnpm-lock.yaml
      8 ├── tsconfig.json
      9 └── vitest.config.ts
  • Add the @vitest/coverage-v8 package
  • Add the following to the test object to enable code coverage
    coverage: {
      enabled: true,
    },

Please provide a link to a minimal reproduction

No response

Please provide any relevant error logs

No response

NuroDev avatar Mar 15 '24 19:03 NuroDev

Hey! 👋 Thanks for raising this. Native code coverage via @vitest/coverage-v8 is not supported. You must use instrumented code coverage via @vitest/coverage-istanbul instead. Refer to the Vitest Coverage documentation for setup instructions. We've got a PR open to add this to the docs (https://github.com/cloudflare/cloudflare-docs/pull/13465), whilst we work on adding support for native coverage.

mrbbot avatar Mar 15 '24 19:03 mrbbot

Support for native code coverage is tracked by internal ticket DEVX-1137 👍

mrbbot avatar Mar 15 '24 19:03 mrbbot

For whoever ends up implementing this, I think this should be possible by starting an inspector server with Miniflare's inspectorPort option, then polyfilling the node:inspector module to report this URL for the started inspector URL. Something similar would be required for https://github.com/cloudflare/workers-sdk/issues/5391. There's a patch in the DevProd team drive Brendan-handover folder that might be helpful.

mrbbot avatar Mar 26 '24 18:03 mrbbot

Getting error via istanbul as well, for the starter hello world template.

TypeError: this.toSlices is not a function or its return value is not iterable
 ❯ IstanbulCoverageProvider.reportCoverage ../../node_modules/@vitest/coverage-istanbul/dist/provider.js:214:34
 ❯ Vitest.reportCoverage ../../node_modules/vitest/dist/vendor/cli-api.RIYLcWhB.js:6537:7
 ❯ async file:/Users/ashrith/Development/react/feedback-circuit/feedback-circuit/node_modules/vitest/dist/vendor/cli-api.RIYLcWhB.js:6328:7
 ❯ Vitest.runFiles ../../node_modules/vitest/dist/vendor/cli-api.RIYLcWhB.js:6332:12
 ❯ Vitest.start ../../node_modules/vitest/dist/vendor/cli-api.RIYLcWhB.js:6223:7
 ❯ startVitest ../../node_modules/vitest/dist/vendor/cli-api.RIYLcWhB.js:13380:5
 ❯ start ../../node_modules/vitest/dist/cli.js:1386:17
 ❯ CAC.run ../../node_modules/vitest/dist/cli.js:13

Here is my vitest.config.ts

import { defineWorkersConfig } from '@cloudflare/vitest-pool-workers/config';

export default defineWorkersConfig({
  test: {
    globalSetup: ["./setup.ts"],
    coverage: {
      enabled: true,
      provider: 'istanbul',
      reporter: ['text', 'json', 'html'],
    },
    poolOptions: {
      workers: {
        wrangler: { configPath: './wrangler.toml' },
      },
    },
  },
});

Anyone tried the istanbul and has a workaround? Please help

xxcheckmatexx avatar Apr 04 '24 06:04 xxcheckmatexx

Getting error via istanbul as well, for the starter hello world template.

TypeError: this.toSlices is not a function or its return value is not iterable
 ❯ IstanbulCoverageProvider.reportCoverage ../../node_modules/@vitest/coverage-istanbul/dist/provider.js:214:34
 ❯ Vitest.reportCoverage ../../node_modules/vitest/dist/vendor/cli-api.RIYLcWhB.js:6537:7
 ❯ async file:/Users/ashrith/Development/react/feedback-circuit/feedback-circuit/node_modules/vitest/dist/vendor/cli-api.RIYLcWhB.js:6328:7
 ❯ Vitest.runFiles ../../node_modules/vitest/dist/vendor/cli-api.RIYLcWhB.js:6332:12
 ❯ Vitest.start ../../node_modules/vitest/dist/vendor/cli-api.RIYLcWhB.js:6223:7
 ❯ startVitest ../../node_modules/vitest/dist/vendor/cli-api.RIYLcWhB.js:13380:5
 ❯ start ../../node_modules/vitest/dist/cli.js:1386:17
 ❯ CAC.run ../../node_modules/vitest/dist/cli.js:13

Here is my vitest.config.ts

import { defineWorkersConfig } from '@cloudflare/vitest-pool-workers/config';

export default defineWorkersConfig({
  test: {
    globalSetup: ["./setup.ts"],
    coverage: {
      enabled: true,
      provider: 'istanbul',
      reporter: ['text', 'json', 'html'],
    },
    poolOptions: {
      workers: {
        wrangler: { configPath: './wrangler.toml' },
      },
    },
  },
});

Anyone tried the istanbul and has a workaround? Please help

Issue is with vitest 1.4.0, anyone encountering this please use 1.3.0 hopefully they would have fixed it by then.

xxcheckmatexx avatar Apr 04 '24 11:04 xxcheckmatexx

I'm not even able to install @vitest/coverage-istanbul. The instructions (and the comment here) just refer to the vitest docs but trying to do npm install -D @vitest/coverage-instanbul fails due to unable to resolve dependency tree errors about peer dependencies. I've tried installing an older version of the instanbul package but none of them seem to work.

@cloudflare/vitest-pool-workers: 0.4.14 node v20.8.1

Just created the worker a couple days ago with npm create cloudflare

stephensauceda avatar Jul 27 '24 22:07 stephensauceda

I'm not even able to install @vitest/coverage-istanbul. The instructions (and the comment here) just refer to the vitest docs but trying to do npm install -D @vitest/coverage-instanbul fails due to unable to resolve dependency tree errors about peer dependencies. I've tried installing an older version of the instanbul package but none of them seem to work.

@cloudflare/vitest-pool-workers: 0.4.14 node v20.8.1

Just created the worker a couple days ago with npm create cloudflare

I got it working by directly specifying the version of @vitest/coverage-istanbul. With these versions of the dependencies, it works for me

"devDependencies": {
    "@cloudflare/vitest-pool-workers": "^0.4.22",
    "@cloudflare/workers-types": "^4.20240806.0",
    "typescript": "^5.5.4",
    "vitest": "1.5.0",
    "@vitest/coverage-istanbul": "1.5.0",
    "wrangler": "^3.71.0"
  }

carlosds731 avatar Aug 14 '24 13:08 carlosds731

I got it working with newer versions:

	"devDependencies": {
		"@cloudflare/vite-plugin": "^1.8.0",
		"@cloudflare/vitest-pool-workers": "^0.8.49",
		"@cloudflare/workers-types": "^4.20250703.0",
		"@types/node": "^24.0.10",
		"@vitest/coverage-istanbul": "^3.2.4",
		"typescript": "^5.5.2",
		"vite": "^6.3.5",
		"vitest": "3.2.4",
		"wrangler": "^4.22.0"
	}
import { defineWorkersConfig } from "@cloudflare/vitest-pool-workers/config";

export default defineWorkersConfig({
  test: {
    poolOptions: {
      workers: {
        wrangler: { configPath: "./wrangler.toml" },
      },
    },
    coverage: {
      reporter: ["text", "json-summary", "json"],
      reportOnFailure: true,
      provider: "istanbul",
    },
  },
}); 

rachelslurs avatar Jul 06 '25 04:07 rachelslurs