coreutils icon indicating copy to clipboard operation
coreutils copied to clipboard

Run all workspace tests in CI (Windows)

Open drinkcat opened this issue 11 months ago • 8 comments

Similar to #7392, it would be nice to run all tests in CI, on MacOS (basically adding workspace-tests: true to some windows jobs in the matrix).

When I tried in #7386, I hit this issue:

error: failed to run custom build command for `uu_stdbuf_libstdbuf v0.0.29 (D:\a\coreutils\coreutils\src\uu\stdbuf\src\libstdbuf)`
...
  src/libstdbuf.rs(18): error C2065: 'constructor': undeclared identifier
  src/libstdbuf.rs(18): error C2182: '__attribute': this use of 'void' is not valid
  src/libstdbuf.rs(19): error C2146: syntax error: missing ';' before identifier '__stdbuf_init'
  src/libstdbuf.rs(19): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
  src/libstdbuf.rs(21): warning C4508: '__stdbuf_init': function should return a value; 'void' return type assumed

I think libstdbuf.rs should simply not be built on Windows platform -- AFAICT nothing that is supported on Windows uses it.

Of course, there might be some more issues after this.

drinkcat avatar Mar 04 '25 13:03 drinkcat

Looked a bit into this, the main problem is that cargo test --workspace tries to pick up all the modules (i.e. apps) in src/uu, even the ones that make no sense on Windows...

It's possible to add #![cfg(unix)] in these subdirectories. I'm not sure what a full list would look like, but maybe something like this (this might be wider than necessary as I'm hitting other cross-compilation errors):

ls src/uu | xargs -I{} sh -c 'cargo clippy --target=x86_64-pc-windows-gnu -p uu_{} >/dev/null 2>&1 || echo "Failed: {}"'
Failed: chcon
Failed: chgrp
Failed: chmod
Failed: chown
Failed: chroot
Failed: expr
Failed: groups
Failed: hostid
Failed: id
Failed: install
Failed: kill
Failed: mkfifo
Failed: mknod
Failed: nice
Failed: nohup
Failed: pathchk
Failed: pinky
Failed: runcon
Failed: stat
Failed: stdbuf
Failed: stty
Failed: timeout
Failed: tty
Failed: users
Failed: who

Maybe best for somebody with an actual Windows machine to try to fix this. Or maybe we could find a way to be more selective about which packages we actually run tests on.

drinkcat avatar Mar 27 '25 16:03 drinkcat

I have an actual Windows machine an would like to do something. Even the simplest case cargo test is failing on Windows, IMO fixing this can be the first step.

zhw2101024 avatar Apr 21 '25 14:04 zhw2101024

I made some attempt in #7591, not sure if this is the right approach, would be awesome if you can give this a try!

drinkcat avatar Apr 21 '25 16:04 drinkcat

some tests rely on the seq command to produce test data,which is not available on windows。Maybe we need to use powershell instead on windows.

zhw2101024 avatar Apr 30 '25 14:04 zhw2101024

Which tests? I wonder if we can get rid of that dependency and just generate the test data in Rust directly.

drinkcat avatar May 01 '25 02:05 drinkcat

Which tests? I wonder if we can get rid of that dependency and just generate the test data in Rust directly.

One of them is test_all_but_last_bytes_large_file_piped, in file tests/by-util/test_head.rs,. Good idea, maybe we can add a function in uutests for generating the test data?

zhw2101024 avatar May 01 '25 06:05 zhw2101024

I'm a bit confused, test_head::test_all_but_last_bytes_large_file_piped is passing currently in CI: https://github.com/uutils/coreutils/actions/runs/14770493603/job/41469699578 .

Or maybe you are trying to get more tests to pass without all the harness that CI normally sets up? That might not be a bad idea, but it's a bit orthogonal to what I was thinking about here (pass cargo test --workspace on Windows CI)

drinkcat avatar May 01 '25 08:05 drinkcat

I'm a bit confused, test_head::test_all_but_last_bytes_large_file_piped is passing currently in CI: https://github.com/uutils/coreutils/actions/runs/14770493603/job/41469699578 .

Or maybe you are trying to get more tests to pass without all the harness that CI normally sets up? That might not be a bad idea, but it's a bit orthogonal to what I was thinking about here (pass cargo test --workspace on Windows CI)

I'm not familiar with the CI of github, just found something about msys64 in the CICD.yml file. So should i run cargo test in the msys64 shell on an actual Windows machine?

zhw2101024 avatar May 01 '25 22:05 zhw2101024