benchmark icon indicating copy to clipboard operation
benchmark copied to clipboard

JSON files report both CPU time and real time

Open pitrou opened this issue 6 years ago • 2 comments

Each benchmark has a preference for real time or CPU time, depending on what's being tested, but gbenchmark always stores both values in JSON output files. This means it's not possible to programmatically determine, when reading a JSON file, whether CPU time or real time is relevant in a particular benchmark.

It seems that when the benchmark declares a preference for real time, I could parse the benchmark name for the string /real_time. But that seems hackish...

pitrou avatar Feb 20 '19 18:02 pitrou

Each benchmark has a preference for real time or CPU time, depending on what's being tested, but gbenchmark always stores both values in JSON output files.

It displays both in the console, too. This is very intentional.

This means it's not possible to programmatically determine, when reading a JSON file, whether CPU time or real time is relevant in a particular benchmark.

It seems that when the benchmark declares a preference for real time, I could parse the benchmark name for the string /real_time. But that seems hackish...

Yeah, i guess that is the (current) way. What is the use case, why do you want to know that after the benchmark?

LebedevRI avatar Feb 20 '19 19:02 LebedevRI

What is the use case, why do you want to know that after the benchmark?

Because we're going to inject benchmark results in a database and then track results across time, try to detect regressions, etc. For that we need to know which number is authoritative for each benchmark.

pitrou avatar Feb 20 '19 22:02 pitrou