[ Feature ] Summary chart with Geometric mean

[omar391]

Is there any plan to implement a feature like this, this would be highly helpful:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/box-plot-summary-charts.html

Ref:

• https://salsa.debian.org/benchmarksgame-team/benchmarksgame/-/blob/master/public/box-plot-summary-charts.html
• https://salsa.debian.org/benchmarksgame-team/benchmarksgame/-/blob/master/mybenchmarks/lib/lib_chartSVG.php

[hanabi1224]

I'm not sure how to represent certain facts with such charts, e.g. nbody/2.py is faster than nbody/1.py with pypy (with JIT) but is slower with python (no JIT), any combination of program selections to represent pypy and python can be misleading in some way, I would only analyze the numbers by reading the source code myself, instead of looking at a chart

[omar391]

hmm..

But, I think we can solve this by considering them as a group just like we have execution matrix in the Github action CI. i.e.

1. python-pypy
2. python-python3
3. go-golang-official
4. go-tinygo
5. javascript-typescript
6. javascript-vanila
7. wasm-wasm3
8. wasm-chorme etc.

Then we can do the geometric mean with box-plot on these datasets. This way we see the performance of of a language and it specific compiler/interprert set.

[igouy]

I would only analyze the numbers by reading the source code myself, instead of looking at a chart

Have you tried to do that? Can you tell us how the Nim programs compare to the Zig programs, or how the Ruby programs compare to the Python programs?

There's an English idiom which describes this situation — "can't see the forest for the trees"

A summary is always less-complete than the detail but we accept that because there can be too much detail to comprehend in the time we have available.

[igouy]

There doesn't seem to be a way to download a file containing all the measurements?

That kind of data file would make it easier for people to do their own analysis and charts (although they probably won't).

[hakanai]

Wouldn't it be nice if you were on the Nim vs Zig page and instead of it being a bunch of numbers, it was actually colourful charts, so you could feel which team was winning by just looking at the page instead of spending an hour reading it.