ut icon indicating copy to clipboard operation
ut copied to clipboard

Custom reporter to only show errors

Open NelsonEloi opened this issue 3 years ago • 4 comments

Expected Behavior

Ability to customize reporter

Actual Behavior

Summary called twice

Steps to Reproduce the Problem

Using your exemple in https://github.com/boost-ext/ut/blob/master/example/cfg/reporter.cpp I tried to create a reporter that would show only the tests with failures I'm getting a crash because the summay is called twice, first time from my own call to run(), and the second time from the destructor of runner that I can't use because of std containers used in functions beeing tested.

1.https://godbolt.org/z/9Ej6EsKq6

Specifications

  • Version:
  • Platform:
  • Subsystem:

Thank you very much Nelson

NelsonEloi avatar May 27 '22 14:05 NelsonEloi

Check out the implementation of boost::ut::reporter::on(events::summary). It uses a static bool to ensure that the summary is only printed once even if the event is dispatched multiple times. I suggest giving that a try.

80Ltrumpet avatar May 27 '22 14:05 80Ltrumpet

The static bool doesn't appear to work as intended because the static bool is local to a templated class; each templated class gets its own copy of the static bool.

I've been wondering if the runner is trying to be a singleton, but its not really relying on a sound singleton approach, which gives rise to the need for the static bool?

redredlobster avatar Jun 18 '22 00:06 redredlobster

Based on the original godbolt example, changing the definition of Cfg::reporter::on(boost::ut::events::summary) to the following prevents the "summary" message from being printed twice.

  auto on(ut::events::summary) -> void {
    if (static auto once{true}; once) {
      once = false;
      std::cout << "summary\n";
    }
  }

boost::ut::runner invokes its reporter's summary handler in three cases:

  1. A fatal expectation failed.
  2. boost::ut::runner::run() was explicitly invoked with report_errors set to true.
  3. In boost::ut::runner::~runner().

It would be possible to avoid requiring users to remember to put in the "run once" guard if all of the cases above were appropriately conditioned in boost::ut::runner. I can put together a pull request to that effect.

80Ltrumpet avatar Jun 18 '22 04:06 80Ltrumpet

For anyone else who stumbles upon this issue, you may want to take a look at the example here https://github.com/boost-ext/ut/tree/master/test/ft.

redredlobster avatar Jun 19 '22 20:06 redredlobster