add ctest as test driver

[felixschurk]

• add enable_testing() in top level CMakeLists to discover tests
• add add_test for both unit tests as well as integration tests with prefixing the tests with the corresponding test type
• remove custom targets for tests
• remove unneeded warning from https://gitlab.kitware.com/cmake/cmake/-/issues/16062
• adds basic setup for resolving #3445

Basically one can now invoke the whole test suite with running

cmake -S . -B build
cmake --build build --target build_tests
ctest --test-dir build/test 

and one does has a quite decent output of the actual test runs.

Parallel running -j 8, running specific test cases -R cpp or -R variant, having minimal progress bar --progress as well as repeating tests until failure --repeat-until-failure 100 is also possible with CTest.

[felixschurk]

Yes that is true, I realized that yesterday but could not fix it on the spot. My knowledge with Docker is limited.

Would it maybe make sense to move the test invocation out into the test.yaml file? I mean updating this line https://github.com/GothenburgBitFactory/taskwarrior/blob/60575a19675713fd5cc209c77733b8ffc3f442e5/.github/workflows/tests.yaml#L105 to

run: docker-compose run test-$CONTAINER ctest --test-dir build/test -j 8 --rerun-failed --output-on-failure

This way we would not have duplicated code in all the docker files, however it maybe would be a bit less/or maybe even more clear how we do run the tests.

[ryneeverett]

In my opinion reducing the duplication is beneficial and passing the command into the docker run invocation is equally clear to having a CMD in the image.

[felixschurk]

I am a bit sorry for all the failing CI jobs and commit updates, however locally testing it with one image/container it does run. So I am guessing there is a problem with escaping etc. in the tests.yaml file.

I had a similar issue already when setting up another CI. (locally working but in GH Actions not).

[ryneeverett]

I don't think --entrypoint is what you want. It looks like you were close with 18888160c637e4f9df419475e03d229c4c2351e1. CI failed with "No such service: test-fedora40 ctest". Maybe you just need some quotes so it splits the arguments correctly?

        run: docker-compose run 'test-${{ env.CONTAINER }}' 'ctest --test-dir build/test -j 8 --output-on-failure --rerun-failed'

(Not 100% sure if it should be single or double quotes but I assume the template variables are substituted prior to evaluation and thus don't care.)

[felixschurk]

Hm, yes definetely the splitting fixed one problem. I then detected some problem with a whitespace (which as somehow different encoded).

Now it does not find some file/directory. Let's see what the problem ist.

[felixschurk]

In my opinion reducing the duplication is beneficial and passing the command into the docker run invocation is equally clear to having a CMD in the image.

I am sorry, I was not able to work that out. Maybe it makes sense to have this as a new issue (should be an easy one) when someone knows GitHub Actions and Docker.

[felixschurk]

Currently, missing is the update of the test/README.

I also realized now that basically the --repeat or respectively --repeat-until-failure options from ctest do work as a stress test, which we removed in #3447, therefore the functionality is still available, however now a bit more unified.