google
[FR] state.SkipWithMessage
Is your feature request related to a problem? Please describe.
There are circumstances where a benchmark needs to be skipped, but this is not due to an error. In this case, it might be nice to have a state.SkipWithMessage.
Describe the solution you'd like
Add the following to benchmark.cc:
void State::SkipWithMessage(const char* msg) {
CHECK(msg);
error_occurred_ = false;
{
MutexLock l(manager_->GetBenchmarkMutex());
if (manager_->results.has_error_ == false) {
// uh wait not quick correct; it's not an error it's info:
manager_->results.error_message_ = msg;
manager_->results.has_error_ = false;
}
}
total_iterations_ = 0;
if (timer_->running()) timer_->StopTimer();
}
Of course I'd prepare the PR if this is a desired feature.
It sounds reasonable enough.. do you have any more details on when you'd want to skip a benchmark at runtime dynamically (instead of using the command line filter)?
@dominichamon : This particular case that inspired the issue is when user-supplied datasets are parametrized as inputs to google/benchmarks. The data flows through a lot of potential benchmarks; but not all of these benchmarks are relevant to the data provided.
i have another use case for this.
we are using google benchmark for GPU-based microbenchmarks. very often a specific GPU device does not support a feature we are writing a test for, e.g. raytracing.
today, we use the SkipWithError but with a custom error message that we have to regex in our testing automation to report it as a skip rather than an error.
it "works" but this would be a good QoL fix.
