feat: Add support for visual testing in browser mode (WIP)
Description
Adds toMatchScreenshot matcher, only available in browser mode. This matcher takes a screenshot of the passed Element | Locator object, and compares it pixelwise with an existing baseline in the file system.
See here for full proposal and context. Additional discussion here
The approach I chose adds a new browser command __vitest_screenshot_compare, which takes the screenshots and performs the pixelwise comparison via pixelmatch and pngjs. The reasoning for this approach is that the browser command runs in the node environment, as opposed to the browser environment that the test runs in. I previously wrote about the issues with performing this comparison and the filesystem operations directly in browser mode here.
I would like some feedback on the general approach before spending time polishing the details and adding documentation, in case I missed something and a big refactor is needed 😅
TODOs:
- [ ] Documentation
- [ ] Add more test cases
Nice-to-haves:
- [ ] Link with existing
SnapshotStatefunctionality to track written, updated, stale and deleted screenshots - [ ] Handle comparison for images of different dimensions
- [ ] Add baseline and current image to the sides of the generated diff
- [ ] Add support for percentage threshold
- [ ] Somehow save the host OS for the screenshot (maybe in PNG metadata) to avoid comparing screenshots taken on different platforms, which introduce various browser rendering differences that will trigger pixel differences.
Please don't delete this checklist! Before submitting the PR, please make sure you do the following:
- [x] It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
- [ ] Ideally, include a test that fails without this PR but passes with it.
- [ ] Please, don't make changes to
pnpm-lock.yamlunless you introduce a new test example.
Tests
- [ ] Run the tests with
pnpm test:ci.
Documentation
- [ ] If you introduce new functionality, document it. You can run documentation with
pnpm run docscommand.
Changesets
- [ ] Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with
feat:,fix:,perf:,docs:, orchore:.
Deploy Preview for vitest-dev ready!
Built without sensitive environment variables
| Name | Link |
|---|---|
| Latest commit | b95d4be1870d7cc7e605aa9552639c00882ddd56 |
| Latest deploy log | https://app.netlify.com/projects/vitest-dev/deploys/6833a5141520fc00087c4f72 |
| Deploy Preview | https://deploy-preview-8028--vitest-dev.netlify.app |
| Preview on mobile | Toggle QR Code...Use your smartphone camera to open QR code link. |
To edit notification comments on pull requests, go to your Netlify project configuration.
Deploy Preview for vitest-dev ready!
Built without sensitive environment variables
| Name | Link |
|---|---|
| Latest commit | 069ae33b98801e242ccc271bea1c0543deb80bd6 |
| Latest deploy log | https://app.netlify.com/projects/vitest-dev/deploys/6833a571b3e0b40008b3e3fa |
| Deploy Preview | https://deploy-preview-8028--vitest-dev.netlify.app |
| Preview on mobile | Toggle QR Code...Use your smartphone camera to open QR code link. |
To edit notification comments on pull requests, go to your Netlify project configuration.
This feature was released in Vitest 4: https://main.vitest.dev/guide/browser/visual-regression-testing.html