swift-benchmark icon indicating copy to clipboard operation
swift-benchmark copied to clipboard

Add support for continuous user-defined metrics

Open shabalind opened this issue 5 years ago • 0 comments

Currently only time and warmup time can be recorded as a time series, while counters provide a single aggregate value.

It would be great to add support for recording custom time series data:

benchmark("...") { state in
   while true {
     var v = ...
     state.measure {
       // benchmark body, modifies v
     }
     state.record(metric: "name", value: Double(v))
   }
}

For example, it can be used to report accuracy in ML models, where not only the final result is important but also the shape of the curve towards the converged score.

shabalind avatar Jun 23 '20 19:06 shabalind