styler icon indicating copy to clipboard operation
styler copied to clipboard

Benchmark / diffview against CRAN release v1.10.0

Open lorenzwalthert opened this issue 4 years ago • 235 comments

DO NOT MERGE, this is for benchmarking only.

You may want to unsubscribe from this PR notifications, as any merge into main will trigger notifications.


Comparision with previous releases:

Releae 1.7.0

  •   :rocket:cache_applying: 29.1ms -> 28.2ms [-5.98%, -0.35%]
  •   :rocket:cache_recording: 1.24s -> 1.22s [-1.91%, -0.46%]
  • :exclamation::snail:without_cache: 3.21s -> 3.3s [+2.1%, +3.94%]

lorenzwalthert avatar Nov 28 '21 18:11 lorenzwalthert

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 7c0acd777d6697aa2af1413362831a290a5e63a1 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 34.2ms -> 34.5ms [-2.41%, +4.22%]
  •   :rocket:cache_recording: 1.64s -> 1.6s [-3.17%, -0.39%]
  •   :ballot_box_with_check:without_cache: 4.24s -> 4.21s [-2.01%, +0.28%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Nov 28 '21 19:11 github-actions[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.07%. Comparing base (8f839e6) to head (8208348).

:exclamation: Current head 8208348 differs from pull request most recent head 4f78252. Consider uploading reports for the commit 4f78252 to get more accurate results

Additional details and impacted files
@@             Coverage Diff             @@
##           v1.10.1     #868      +/-   ##
===========================================
- Coverage    92.33%   91.07%   -1.27%     
===========================================
  Files           46       46              
  Lines         2650     2711      +61     
===========================================
+ Hits          2447     2469      +22     
- Misses         203      242      +39     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Nov 28 '21 20:11 codecov-commenter

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 3c14e96f21427722e1dd7c0d187d901490d71aa4 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 24.5ms -> 24.6ms [-2.03%, +3.34%]
  •   :rocket:cache_recording: 1.07s -> 1.02s [-6.4%, -1.18%]
  •   :ballot_box_with_check:without_cache: 2.76s -> 2.71s [-4.17%, +0.67%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Nov 28 '21 20:11 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 6b8ad76e94b35225cc1b5b054fe821605ef8f29b is merged into v1.6.2:

  •   :rocket:cache_applying: 30.4ms -> 29.3ms [-6.12%, -0.8%]
  •   :rocket:cache_recording: 1.07s -> 1.03s [-4.73%, -2.87%]
  •   :ballot_box_with_check:without_cache: 2.83s -> 2.8s [-2.15%, +0.09%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Nov 30 '21 19:11 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if fcd4c19f81d993a5c0ebea248d032d43a41b74c8 is merged into v1.6.2:

  •   :rocket:cache_applying: 30.3ms -> 29.1ms [-7.04%, -0.98%]
  •   :rocket:cache_recording: 1.1s -> 1.06s [-3.82%, -2.84%]
  •   :rocket:without_cache: 2.83s -> 2.8s [-1.52%, -0.36%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 08 '21 21:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 3ef57c2f968f850dec048ebae271d1381bf40dd7 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 29.8ms -> 30.5ms [-8.12%, +12.32%]
  •   :rocket:cache_recording: 1.08s -> 1.04s [-3.79%, -3.05%]
  •   :rocket:without_cache: 2.82s -> 2.79s [-1.28%, -0.73%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 10 '21 23:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if ea7456ded359ab4337d6c68e776ec4c4e2f78d2e is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 34.3ms -> 34.3ms [-3.76%, +3.95%]
  •   :rocket:cache_recording: 1.42s -> 1.38s [-4.29%, -1.6%]
  •   :ballot_box_with_check:without_cache: 3.74s -> 3.73s [-1.38%, +1.12%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 11 '21 11:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if deff5e98a468703bf50c4e714e1a7a2bcfc26e74 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 35.6ms -> 34.2ms [-8.58%, +0.94%]
  •   :rocket:cache_recording: 1.44s -> 1.39s [-5.06%, -2.4%]
  •   :ballot_box_with_check:without_cache: 3.78s -> 3.75s [-1.8%, +0.41%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 16 '21 23:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if b8f73591b3db1d3097ab5a51c15183142e5f6fb9 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 31.9ms -> 32.1ms [-0.45%, +1.41%]
  •   :rocket:cache_recording: 1.18s -> 1.15s [-3.32%, -2.09%]
  •   :ballot_box_with_check:without_cache: 3.11s -> 3.09s [-0.83%, 0%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 17 '21 09:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 988a9ac6cc181a028fb72cf228db5d0e24c9e5b6 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 36.1ms -> 35.3ms [-5.96%, +1.58%]
  •   :ballot_box_with_check:cache_recording: 1.5s -> 1.47s [-5.48%, +0.67%]
  •   :ballot_box_with_check:without_cache: 3.88s -> 3.88s [-1.7%, +1.2%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 19 '21 18:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 4fd6aee905602323c9f8507fa45b89ab6bd0be63 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 29.2ms -> 29.3ms [-0.33%, +1.46%]
  •   :rocket:cache_recording: 1.06s -> 1.03s [-3.41%, -2.38%]
  •   :ballot_box_with_check:without_cache: 2.75s -> 2.75s [-0.83%, +0.36%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 23 '21 13:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if d0e4f9220ae4fefc064f9b9ece55693ad91d52c1 is merged into v1.6.2:

  •   :rocket:cache_applying: 32.1ms -> 29ms [-11.97%, -7.08%]
  •   :rocket:cache_recording: 1.17s -> 1.15s [-2.22%, -1%]
  •   :ballot_box_with_check:without_cache: 3.04s -> 3.05s [-0.14%, +0.75%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 23 '21 14:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if c1a46e49e62da6b0658a993c4f40155907a64fd6 is merged into v1.6.2:

  •   :rocket:cache_applying: 36.2ms -> 33.4ms [-10.71%, -4.65%]
  •   :rocket:cache_recording: 1.5s -> 1.47s [-2.75%, -1.35%]
  •   :ballot_box_with_check:without_cache: 3.94s -> 3.96s [-0.23%, +1.31%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 24 '21 08:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if ef4216aa89a913034cef091da767c2d58b29c0ca is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 38.7ms -> 38.8ms [-1.95%, +2.75%]
  •   :rocket:cache_recording: 1.62s -> 1.58s [-3.42%, -1.31%]
  •   :ballot_box_with_check:without_cache: 4.25s -> 4.25s [-1.07%, +0.94%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 25 '21 18:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 63a874cabc0ad8c43aa43cf95c2ff7c931242947 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 27.5ms -> 27.4ms [-1.14%, +0.42%]
  •   :rocket:cache_recording: 1.01s -> 989ms [-2.75%, -1.9%]
  •   :ballot_box_with_check:without_cache: 2.68s -> 2.69s [-0.43%, +0.74%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 25 '21 23:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 9a169bb48b14d5cab369e74cf24dc3a575d8b61f is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 28ms -> 28ms [-0.99%, +0.38%]
  •   :rocket:cache_recording: 1.04s -> 1.01s [-2.92%, -2.07%]
  •   :ballot_box_with_check:without_cache: 2.69s -> 2.7s [-0.02%, +0.99%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 26 '21 20:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if c6fc0e32fde782f53c159421d405827d0b8686dd is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 30.4ms -> 30.3ms [-1.59%, +0.72%]
  •   :rocket:cache_recording: 1.1s -> 1.07s [-3.43%, -2.33%]
  • :exclamation::snail:without_cache: 2.85s -> 2.87s [+0.16%, +1.11%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 28 '21 13:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if fac168b6ac20afcc1dc47b7c2876fce556f2869f is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 27.2ms -> 27ms [-1.03%, +0.09%]
  •   :rocket:cache_recording: 1s -> 976ms [-3.16%, -2.22%]
  • :exclamation::snail:without_cache: 2.6s -> 2.62s [+0.2%, +1.05%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 28 '21 22:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if d3388881f6f9f9add3336d7f26dcfabfa7191180 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 37.3ms -> 36.9ms [-3.53%, +1.46%]
  •   :ballot_box_with_check:cache_recording: 1.54s -> 1.52s [-3.35%, +0.33%]
  •   :ballot_box_with_check:without_cache: 4.01s -> 4.03s [-0.3%, +1.56%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 28 '21 22:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if d91d98cec9b66f2bf1fcf7b925a3ee8499115fa9 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 28.3ms -> 28.3ms [-0.8%, +0.84%]
  •   :rocket:cache_recording: 1.02s -> 1s [-2.64%, -1.29%]
  • :exclamation::snail:without_cache: 2.67s -> 2.69s [+0.04%, +1.23%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 29 '21 14:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 5fcc5e53edadf5009399f3fd50e251445e554fcd is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 31.3ms -> 31.2ms [-1.92%, +1.8%]
  •   :rocket:cache_recording: 1.1s -> 1.08s [-3.04%, -1.81%]
  •   :ballot_box_with_check:without_cache: 2.89s -> 2.88s [-0.96%, +0.43%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 29 '21 17:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 14dcc65e33737607615001f1ae4a61e6061097c7 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 32ms -> 32ms [-2.41%, +2.42%]
  •   :rocket:cache_recording: 1.14s -> 1.12s [-3.91%, -0.12%]
  •   :ballot_box_with_check:without_cache: 3.03s -> 3.04s [-1.19%, +2.3%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Dec 29 '21 20:12 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if b1902ece950f8bc052da47d91e7ab87e79f6a4b5 is merged into v1.6.2:

  •   :rocket:cache_applying: 30.4ms -> 26.7ms [-16.18%, -8.41%]
  •   :rocket:cache_recording: 1.09s -> 1.05s [-6.17%, -1.14%]
  •   :ballot_box_with_check:without_cache: 2.8s -> 2.82s [-1.87%, +3.18%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Jan 03 '22 12:01 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 27761d1f9d6b55d5d161dbba5325932f64022bad is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 27.5ms -> 27.4ms [-4.08%, +3.33%]
  •   :rocket:cache_recording: 1.29s -> 1.26s [-3.56%, -0.98%]
  •   :ballot_box_with_check:without_cache: 3.39s -> 3.38s [-1.45%, +1.22%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Jan 15 '22 20:01 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 8f76e8fcdb33d7cb6cb266485b76bb4daa563fdb is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 33ms -> 33.6ms [-3.66%, +6.99%]
  •   :rocket:cache_recording: 1.61s -> 1.56s [-4.67%, -1.12%]
  •   :ballot_box_with_check:without_cache: 4.25s -> 4.23s [-2.17%, +0.97%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Jan 16 '22 09:01 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 59cf703c4646b35fc588eacc26c39a8a75397a92 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 32.6ms -> 31.8ms [-5.58%, +0.98%]
  •   :rocket:cache_recording: 1.55s -> 1.5s [-4.27%, -2.28%]
  •   :ballot_box_with_check:without_cache: 4.09s -> 4.09s [-0.92%, +0.6%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Jan 16 '22 12:01 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 3a9215142ea2022807102a61e8aa74e5d0f3178c is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 25.2ms -> 24.7ms [-4.78%, +0.69%]
  •   :rocket:cache_recording: 1.09s -> 1.05s [-3.96%, -2.79%]
  •   :ballot_box_with_check:without_cache: 2.83s -> 2.83s [-0.75%, +0.41%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Feb 06 '22 13:02 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 185d19282de284e2b04d5586c315ec3fd2104dab is merged into v1.6.2:

  •   :rocket:cache_applying: 25.5ms -> 24.5ms [-6.82%, -0.73%]
  •   :rocket:cache_recording: 1.08s -> 1.05s [-3.75%, -2.65%]
  •   :ballot_box_with_check:without_cache: 2.83s -> 2.83s [-0.44%, +0.73%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Feb 08 '22 14:02 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 26e9a4033e79948aa0915d5321ee3efe1744e6f6 is merged into v1.6.2:

  •   :rocket:cache_applying: 25.1ms -> 24.3ms [-5.92%, -0.92%]
  •   :rocket:cache_recording: 1.06s -> 1.03s [-3.56%, -2.46%]
  •   :ballot_box_with_check:without_cache: 2.74s -> 2.74s [-0.47%, +0.68%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Feb 18 '22 09:02 github-actions[bot]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 73744464e877d7a2a1c9ef08f6a0e5db90abc163 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 25.6ms -> 25ms [-5.69%, +0.66%]
  •   :rocket:cache_recording: 1.06s -> 1.03s [-4.99%, -0.81%]
  •   :ballot_box_with_check:without_cache: 2.8s -> 2.81s [-2.01%, +2.87%]

Further explanation regarding interpretation and methodology can be found in the documentation.

github-actions[bot] avatar Feb 18 '22 21:02 github-actions[bot]