expr: failing fuzz test
#2 INITED cov: 1780 ft: 1781 corp: 1/1b exec/s: 0 rss: 71Mb
Running test ["expr", "24", "-", "31", "!=", "86", "length", "78", "%", "23", ":", "79", "length", "92", "-", "32"]
Test Type: expr
Input: ["24", "-", "31", "!=", "86", "length", "78", "%", "23", ":", "79", "length", "92", "-", "32"]
Rust stderr: syntax error: unexpected argument 'length'
GNU stderr: syntax error: unexpected argument ‘length’
Diff=
-syntax error: unexpected argument 'length'
+syntax error: unexpected argument ‘length’
Discrepancy detected: stderr differs
Test completed with discrepancies for expr: ["24", "-", "31", "!=", "86", "length", "78", "%", "23", ":", "79", "length", "92", "-", "32"]
NEW_FUNC[1/48]: 0x55e3eb1185c0 (/home/runner/work/coreutils/coreutils/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_expr+0xa855c0) (BuildId: ae0001ee8471c348c8cde61a6763d3bdc84e[235](https://github.com/uutils/coreutils/actions/runs/11405043095/job/31735679459?pr=6791#step:7:236)4)
NEW_FUNC[2/48]: 0x55e3eb11a5d0 (/home/runner/work/coreutils/coreutils/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_expr+0xa875d0) (BuildId: ae0001ee8471c348c8cde61a6763d3bdc84e2354)
https://github.com/uutils/coreutils/actions/runs/11405043095/job/31735679459
I think we need to set LC_ALL=C so that GNU Core Utilities doesn't use "fancy" quotes:
❯ /usr/bin/expr '34' '*' '52' '<' '83' ':' '86(' '&' '9' '!=' '73' 'substr' '64' '&' '67'
/usr/bin/expr: syntax error: unexpected argument ‘substr’
❯ LC_ALL=C /usr/bin/expr '34' '*' '52' '<' '83' ':' '86(' '&' '9' '!=' '73' 'substr' '64' '&' '67'
/usr/bin/expr: syntax error: unexpected argument 'substr'
Okay, I didn't look closely at the failing job run. These discrepancies aren't causing fuzz runs to fail. For instance:
#10 NEW cov: 2042 ft: 3204 corp: 9/14b lim: 4 exec/s: 0 rss: 75Mb L: 2/2 MS: 1 InsertByte-
Running test ["expr", "61bQh(", ">", "75h", "%", "45", "+", "32", "length", "63", "&", "27", "&", "96", "&", "96"]
Test Type: expr
Input: ["61bQh(", ">", "75h", "%", "45", "+", "32", "length", "63", "&", "27", "&", "96", "&", "96"]
Rust stderr: syntax error: unexpected argument 'length'
GNU stderr: non-integer argument
Diff=
-syntax error: unexpected argument 'length'
+non-integer argument
Discrepancy detected: stderr differs
Test completed with discrepancies for expr: ["61bQh(", ">", "75h", "%", "45", "+", "32", "length", "63", "&", "27", "&", "96", "&", "96"]
NEW_FUNC[1/36]: 0x55ae3794e6f0 (/home/runner/work/coreutils/coreutils/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_expr+0xa856f0) (BuildId: 47750ba6bc2d3d32142c7ee5e4a617ecfde4512e)
NEW_FUNC[2/36]: 0x55ae37950700 (/home/runner/work/coreutils/coreutils/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_expr+0xa87700) (BuildId: 47750ba6bc2d3d32142c7ee5e4a617ecfde4512e)
But the run still passed: https://github.com/uutils/coreutils/actions/runs/11405719733/job/31737927995
Regardless, #6795, or something like it, may still be worth doing, so that if there's interest in tightening up the fuzzing rules, some of these issues are taken care of.
compare_result(
"expr",
&format!("{:?}", &args[1..]),
None,
&rust_result,
&gnu_result,
false, // Set to true if you want to fail on stderr diff
);
expr fuzzing is intentionally ignoring stderr differences. This may not be worth messing with, since error messages can typically easily be checked with normal tests, since the number of cases tends to be small. Feel free to close.