garris
Browser report fails when `node_modules` are read-only
Description of a problem
In environments where node_modules are read-only browser report isn't created.
Examples of such environments: nix based builds, where node_modules land in /nix/store as read-only (555 or 444 rights).
Another one might be some stateless build systems, where node_modules is passed between build steps as an artifact and also might be read-only.
How to reproduce:
mkdir test && cd test
npm init -y && npm install backstopjs
./node_modules/.bin/backstop init
chmod -w ./node_modules/.bin/backstop/compare/output # simulate read-only case
./node_modules/.bin/backstop test
...
report | 2 Failed
report | Writing browser report
report | Failed writing report with error: undefined # <-- this
Potential cause
I dug a little into the internals and it looks that it's caused by https://github.com/garris/BackstopJS/blob/ddfedcbd2d63fd6108922bf0bfabc31d672fd5c1/core/command/report.js#L26
where fs.copy copies output with directory rights. Assuming I'm not wrong about how *nix systems handle permissions, in presented case, there's no write for output directory - thus no new directories html_report directory on the project side can be created as part of copy.
Workarounds
One workaround is to create directory structure before running tests, something like:
rm -rf backstop_data/html_report && mkdir -p backstop_data/html_report/fonts/assets
Summary
Not sure whether it should be considered as a bug, I just wanted to point out that current implementation might be causing some issues.
I’ve stumbled upon the same problem. Additionally when creating the report fails the process exits with 0, which made our CI pipeline silently pass, even though the tests were actually failing.
I am also just running into this problem when packaging in using Electron. The files get packaged in \release\build\win-ia32-unpacked\resources\app.asar\node_modules\backstopjs\compare\output which is a read-only archive.
I am using the code from #1400 for my Electron project and I am passing the comparePath in my config like so:
"comparePath": ${backstopDataDir}/compare/output,
It seems to be working at the moment.