gentest ignores test results when test returns nonzero

[mibzman]

I'm running some mocha tests with please. When a test fails, mocha returns 1. Please interprets this as an error, and doesn't evaluate the results in $RESULTS_FILE.

Is this intended?

I'm currently working around this by running $TEST_COMMAND || true to force the exit code to always be 0. Then please reads $RESULTS_FILE and shows breaking tests.

[peterebden]

No, this isn't intended, a test can return nonzero and produce the results file, which should then get interpreted - generally the expectation is that the exit code matches the success or failure of the test, which is detailed in the results file. What's the message you're getting in that case?

[mibzman]

There's no error message, it just reports the test as having errored. I think what's going on is the test returns nonzero, but I initially cat'ed the test reporting into RESULTS_FILE in a separate command. here's my code if that helps.

I thought that exit_on_error would be false, so if mocha fails it should continue to the next command.