Imaging.Heif icon indicating copy to clipboard operation
Imaging.Heif copied to clipboard

Add Baseline to Benchmarks Report

Open SkyeHoefling opened this issue 4 years ago • 3 comments

Description

Add a baseline benchmark to each printed table. This will show the existing benchmark compared to the PR benchmark side-by-side. The data is saved in the repository, so we just need to extract it and merge it with the existing table

SkyeHoefling avatar Sep 05 '21 05:09 SkyeHoefling

@mitchelsellers @kenny-sellers I have a couple options for us to include baseline benchmarks, which one do you like best? Please vote with the emojis below

👍

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

🚀

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

🎉 - Create 2 tables PR Results

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

Baseline Results

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

Please leave any comments you may have

SkyeHoefling avatar Jan 06 '22 14:01 SkyeHoefling

I voted. One thought and maybe I am an odd duck, what if baseline was first and then the PR result was below? In my mind, that makes it a little easier to compare the two because naturally I expect the baseline to be first.

kenny-sellers avatar Jan 06 '22 14:01 kenny-sellers

Something like this? Baseline Results

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

PR Results

Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

SkyeHoefling avatar Jan 06 '22 14:01 SkyeHoefling