cargo-mutants doesn't work on tarpaulin tree

[sourcefrog]

So tarpaulin tests instrument binaries with breakpoints and do coverage which involves intercepting signals going to the process and handling them in code. There's a possibility I'm intercepting the cargo mutants kill signal and then failing the test because it's got an unexpected termination signal :thinking:

Reproducing is pretty simple, it's just cloning tarpaulin and running cargo mutants on it

git clone --depth 1 https://github.com/xd009642/tarpaulin
cd tarpaulin
cargo mutants --in-place 

Originally posted by @xd009642 in https://github.com/sourcefrog/cargo-mutants/issues/348#issuecomment-2090621779

[sourcefrog]

@xd009642 I can reproduce this; I haven't looked into why yet. I see a bunch of test failures during the baseline test. Since it isn't timing out, it should not be getting signaled. I guess something else about the environment where cargo-mutants runs it might be causing a problem?

I will look into it some more when I get a chance, but if you want to investigate then a good place to start would be to get the command that it runs from mutants.out/debug.log and run that yourself, and see if it causes the same failure.

[sourcefrog]

@xd009642 I came back to this today and started by running cargo t in c493bc1d9e581b0c4bca5e3c07868a5579286cbb from tarpaulin, without involving cargo mutants, on Linux. That fails with

failures:

---- follow_exes_down stdout ----
thread 'follow_exes_down' panicked at /home/mbp/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rusty-fork-0.3.0/src/fork_test.rs:135:9:
child exited unsuccessfully with exit status: 70
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

running 1 test
   Compiling follow_exe v0.1.0 (/home/mbp/src/tarpaulin/tests/data/follow_exe)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.84s

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s


running 3 tests
test launch_hello_world ... ok
test launch_ls ... thread 'follow_exes_down' panicked at tests/mod.rs:103:60:
called `Result::unwrap()` on an `Err` value: NixError(ESRCH)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    follow_exes_down

test result: FAILED. 55 passed; 1 failed; 3 ignored; 0 measured; 0 filtered out; finished in 98.77s

[xd009642]

Hmm that test and the fork ones are unfortunately flaky on some systems (something I'm working on)

[sourcefrog]

My main guess, without having a really crisp reproduction yet, is that it's something to do with cargo-mutants running subprocesses in a new process group. It uses the process group so that it can try to kill off all the processes if they hang. Perhaps that interacts badly with something that Tarpaulin is doing itself in managing its own subprocesses. We could potentially have an option to turn that off.

[xd009642]

Potentially, I have another smaller project that's a debugger and does the standard ptrace+signal shenanigans as tarpaulin and cargo mutants couldn't pass the unmutated baseline

[sourcefrog]

I guess this is your rustybug??

I thought I'd try that as a potentially simpler reproduction but unfortunately cargo test also fails for me there, after building the test programs. (See attached.) I checked and lldb does work on your test binaries on my machine, so it doesn't seem to be something about ptrace being blocked.

I'd like to work out and fix what might be going wrong in cargo-mutants but I don't feel like it's efficient for me to debug the test flakes. If you think you have a flake-free tree that reliably works under cargo test but not under cargo mutants --in-place please let me know! Or try commenting out https://github.com/sourcefrog/cargo-mutants/blob/a154cc7c8c1e598ab3391eec80f662eebd66e3d8/src/process/unix.rs#L35-L38 and see if that helps.