Stop faster on test failures?

[sourcefrog]

The standard cargo test invocation has two layers:

1. cargo test runs various test binaries in series, stopping if any of them fail: starting with lib tests, then binaries, etc, and on to doc tests. https://doc.rust-lang.org/cargo/commands/cargo-test.html
2. the rust test harness internal to each binary runs each test one after the other but does not stop if any of them fail. (This is the default behavior but crates can override it.)

cargo-mutants only wants to know if any tests fail; once one of them has failed it's a waste of time to continue.

I think the cleanest place to fix this would be in the stdlib test harness, but unless/until that's fixed it would be possible for cargo-mutants to do what a human would do: ctrl-c the tests once it notices that anything has failed.

To notice that one test has failed we could

• scrape the text output, though that might be prone to false positives, which would be quite disruptive
• ask for --format json though that apparently is not stable yet
• ...?

Alternatively there is cargo-nextest, https://nexte.st/, which does seem to support --fail-fast and might be faster in general on crates where it works.

See also:

• https://rust-lang.github.io/rfcs/2318-custom-test-frameworks.html