hyperion icon indicating copy to clipboard operation
hyperion copied to clipboard

Implement a way to calculate deviation of measurements.

Open qnikst opened this issue 8 years ago • 2 comments

Currently hyperion returns only raw measurement data and without external tool it's hard to make sense of data because we don't know how accurate those measurements were. It would be nice if we can measure deviation of the measurements.

qnikst avatar Sep 05 '17 09:09 qnikst

Issue not actionable as it stands. "Deviation of the measurements" - do you mean variance? Which quantitative measures (moments) specifically do you want to see?

mboes avatar Sep 05 '17 11:09 mboes

Yes I mean variance, if hyperion is run without --raw flag then it's not possible to understand variance, of the values from report:

{"cycles":null
,"measurements":null
,"garbage_collections":null
,"time_in_nanos":839.5436263736264
,"bench_name":"944"
,"alloc":null,"bench_params":[]}],"metadata":{"location":null,"timestamp":"2017-09-05T11:47:47.633059736Z"}}

P.S. In case of the --raw flag there is enough information in order to implement that test in 3rd party application, but for human readable report this information is missing.

qnikst avatar Sep 05 '17 11:09 qnikst