json icon indicating copy to clipboard operation
json copied to clipboard

Improve unit testing (Part 2)

Open falbrechtskirchinger opened this issue 3 years ago • 4 comments

This is part 2 of the unit test improvements.

The original plan of adding build information output has been scrapped after the code had ballooned into a 4000 line monstrosity.

Instead, this PR addresses some issues that were discussed (remove magic keywords from the unit tests) and adds a very minimal version of what the build information output would have provided by extending JSON meta and printing that at the start of each unit test or via the print_meta target.

For example, most CI targets will now print the following for each enabled C++ standard:

print_meta_cpp11
{
    "version": "3.10.5",
    "platform": "linux",
    "compiler": {
        "family": "clang",
        "version": "15.0.0 ",
        "c++": "201103",
        "libc++": {
            "family": "GNU libstdc++",
            "version": 20210427
        }
    },
    "config": {
        "JSON_DIAGNOSTICS": false,
        "JSON_DISABLE_ENUM_SERIALIZATION": false,
        "JSON_HAS_CPP_11": true,
        "JSON_HAS_CPP_14": false,
        "JSON_HAS_CPP_17": false,
        "JSON_HAS_CPP_20": false,
        "JSON_HAS_CPP_23": false,
        "JSON_HAS_EXPERIMENTAL_FILESYSTEM": false,
        "JSON_HAS_FILESYSTEM": false,
        "JSON_HAS_THREE_WAY_COMPARISON": false,
        "JSON_NOEXCEPTION": false,
        "JSON_NO_IO": false,
        "JSON_USE_IMPLICIT_CONVERSIONS": true,
        "JSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON": false,
        "NLOHMANN_JSON_ABI_STRING": "json_v3_10_5"
    }
}

To Do

  • [ ] Document meta() changes.
  • [ ] Convert boolean macros.

falbrechtskirchinger avatar Mar 09 '22 15:03 falbrechtskirchinger

Coverage Status

Coverage remained the same at 100.0% when pulling 35f6a61dadb7bb2718845272577f592ee788c339 on falbrechtskirchinger:topic/test-improvements-part2 into 98d70d467649828928e299c7c64656d7fed83d41 on nlohmann:develop.

coveralls avatar Mar 09 '22 16:03 coveralls

Note to self: Add ABI string from #3590 to meta().

Edit: Done.

falbrechtskirchinger avatar Jul 19 '22 07:07 falbrechtskirchinger

This starts to become a very large PR. Please consider breaking it into smaller pieces, such as the whole fuzzer part.

nlohmann avatar Aug 01 '22 11:08 nlohmann

This starts to become a very large PR. Please consider breaking it into smaller pieces, such as the whole fuzzer part.

Yes. I'm just using this for CI. Locally I already have a separate branch for the fuzzing stuff.

falbrechtskirchinger avatar Aug 01 '22 11:08 falbrechtskirchinger