benchmark-memory icon indicating copy to clipboard operation
benchmark-memory copied to clipboard

Allow comparison against a baseline

Open michaelherold opened this issue 2 years ago • 1 comments

This change introduces a new comparison ordering strategy where the initial report serves as the baseline against which the job compares the other reports.

This is useful in cases where you're testing out different implementations and want to see whether they are better or worse (and the comparative increase or decrease) in an easy-to-decipher order.

Closes #26

michaelherold avatar Oct 13 '23 03:10 michaelherold

Thanks for putting this together! I tried this with a benchmark I had kicking around: https://gist.github.com/technicalpickles/c59d3e59dfc9f965e802045684fb4ebd

The order looks correct, but the relative numbers don't look quite right:

Comparison:
            original:       1680 allocated - baseline
map! with capitalize! & logical or:        760 allocated - same
map! capitalize! & two statements:        760 allocated - same
    each capitalize!:        760 allocated - same
                map!:        760 allocated - same
        string split:        840 allocated - same
         capitalize!:        840 allocated - same
            gsub alt:       1576 allocated - same
                gsub:       1816 allocated - 1.08x more

It's showing same for everything that has less memory allocation.

technicalpickles avatar Oct 13 '23 16:10 technicalpickles