gap icon indicating copy to clipboard operation
gap copied to clipboard

Add a way to detect "syntax errors" for CI tests (e.g. by turning them into actual errors)

Open fingolfin opened this issue 4 years ago • 1 comments

I just encountered a case where a package PR was merged because the CI tests were green -- but actually the PR introduced code with syntax errors! But those were not detected by CI, as they "just" triggered syntax warnings, and it so happens that the function containing them was not being tested (of course that's a boo-boo as well, but it happens...).

So it would be nice if CI tests had a reliable way to detect "syntax errors" (i.e., not a hack like parsing the output of GAP for certain strings). E.g. if there was a way to make any syntax error result in a call to exit(1) (aborting GAP immediately with a non-zero exit code).

fingolfin avatar Sep 21 '21 12:09 fingolfin

I have solved this problem in our CI by calling LoadPackage in a testfile, see e.g. https://github.com/homalg-project/FinSetsForCAP/blob/b5772d3fbc182b783a784ac1ebcb71b568d6baa2/tst/100_LoadPackage.tst#L8 Any syntax error triggered in line 8 fails the test.

zickgraf avatar Oct 13 '21 19:10 zickgraf

This would be a candidate for "strict mode" (see #1191).

fingolfin avatar Dec 15 '22 21:12 fingolfin

Duplicate of #1981

fingolfin avatar Jan 09 '24 23:01 fingolfin