BenchmarkTools.jl icon indicating copy to clipboard operation
BenchmarkTools.jl copied to clipboard

How do you save benchmark results?

Open FelixBenning opened this issue 3 years ago • 1 comments

https://juliaci.github.io/BenchmarkTools.jl/dev/manual/#Caching-Parameters

  1. Start a Julia session
  2. Execute a benchmark suite using an old version of your package
  3. Save the results somehow (e.g. in a JSON file)

how? The following description is about saving the parameters to a file, but how do I actually save the Benchmark Results?

FelixBenning avatar Oct 05 '22 10:10 FelixBenning

results = run(suite, verbose = true)

BenchmarkTools.save("output.json", median(results))

cf. https://github.com/benchmark-action/github-action-benchmark/blob/master/examples/julia/fib.jl

seems to work and

BenchmarkTools.load("output.json")

also seems to work. So I guess these are just undocumented

FelixBenning avatar Oct 05 '22 10:10 FelixBenning

I found the BenchmarkTools.save method too verbose for my liking. You can also save the benchmark results as follows:

using BenchmarkTools
using JSON

t = @benchmark fib(n) samples=10
data = Dict("mean"=> mean(t).time, "min"=> minimum(t).time, 
"max"=> maximum(t).time, "med"=>median(t).time)
open("output.json", "w") do f  JSON.print(f, data) end

Note: the time units are stored in nanoseconds. See: #307

vandre avatar May 02 '23 19:05 vandre