libdatadog icon indicating copy to clipboard operation
libdatadog copied to clipboard

improve crashtracker config deserialization

Open rochdev opened this issue 1 year ago • 7 comments

What does this PR do?

Improve crashtracker config deserialization:

  • A string can now be passed as an endpoint URL when deserializing instead of having to know about Hyper.
  • Fields with a default value are no longer required.

Motivation

When using NAPI-RS, JavaScript types are converted directly to Rust types through serialization/deserialization. This meant that a lot of unneeded options were required to be passed, and the URL had to be passed with the internal structure as well.

Additional Notes

How to test the change?

Good question 😅 I'm not familiar with tests in libdatadog, and I don't know if serialization/deserialization is even tested at all right now.

rochdev avatar Aug 15 '24 21:08 rochdev

Benchmarks

Comparison

Benchmark execution time: 2024-08-16 20:01:22

Comparing candidate commit abdd41e in PR branch rochdev/crashtracker-config-serde with baseline commit 31b6854 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 138.032µs 138.569µs ± 0.502µs 138.527µs ± 0.175µs 138.701µs 138.974µs 139.337µs 144.512µs 4.32% 8.486 97.045 0.36% 0.035µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [138.499µs; 138.638µs] or [-0.050%; +0.050%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.475µs 2.910µs ± 1.412µs 2.748µs ± 0.021µs 2.764µs 2.968µs 12.874µs 15.835µs 476.22% 7.933 62.838 48.40% 0.100µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.714µs; 3.106µs] or [-6.724%; +6.724%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 299.491ns 311.554ns ± 12.905ns 306.230ns ± 3.932ns 314.540ns 339.825ns 349.411ns 350.068ns 14.32% 1.572 1.418 4.13% 0.913ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [309.765ns; 313.342ns] or [-0.574%; +0.574%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 38.763µs 39.446µs ± 1.248µs 38.875µs ± 0.046µs 38.947µs 42.146µs 42.178µs 42.524µs 9.39% 1.696 0.905 3.16% 0.088µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [39.273µs; 39.619µs] or [-0.438%; +0.438%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.700µs 2.746µs ± 0.014µs 2.747µs ± 0.006µs 2.753µs 2.769µs 2.771µs 2.773µs 0.93% -1.081 2.202 0.51% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.744µs; 2.748µs] or [-0.071%; +0.071%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 74.155µs 74.458µs ± 0.154µs 74.455µs ± 0.036µs 74.489µs 74.559µs 75.091µs 75.927µs 1.98% 5.039 43.828 0.21% 0.011µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [74.436µs; 74.479µs] or [-0.029%; +0.029%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 618.408µs 619.178µs ± 0.493µs 619.076µs ± 0.281µs 619.414µs 620.040µs 620.741µs 622.123µs 0.49% 1.804 6.380 0.08% 0.035µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1607400.386op/s 1615044.928op/s ± 1283.759op/s 1615310.647op/s ± 733.787op/s 1615916.432op/s 1616574.545op/s 1616861.728op/s 1617055.930op/s 0.11% -1.791 6.295 0.08% 90.776op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 385.759µs 388.447µs ± 1.072µs 388.400µs ± 0.742µs 389.116µs 390.232µs 391.320µs 392.130µs 0.96% 0.461 0.365 0.28% 0.076µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2550177.643op/s 2574372.310op/s ± 7095.808op/s 2574665.444op/s ± 4917.134op/s 2579720.548op/s 2584595.044op/s 2588356.085op/s 2592289.941op/s 0.68% -0.443 0.339 0.27% 501.749op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 190.661µs 191.151µs ± 0.170µs 191.152µs ± 0.105µs 191.260µs 191.433µs 191.511µs 191.537µs 0.20% -0.227 0.072 0.09% 0.012µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5220924.770op/s 5231469.422op/s ± 4640.005op/s 5231443.065op/s ± 2879.434op/s 5234290.535op/s 5239806.918op/s 5243722.946op/s 5244911.830op/s 0.26% 0.232 0.077 0.09% 328.098op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 45.038µs 45.367µs ± 0.118µs 45.363µs ± 0.077µs 45.437µs 45.557µs 45.639µs 45.727µs 0.80% 0.191 0.000 0.26% 0.008µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 21868712.745op/s 22042821.761op/s ± 57413.905op/s 22044216.925op/s ± 37575.626op/s 22083439.206op/s 22138508.615op/s 22151276.195op/s 22203676.865op/s 0.72% -0.176 -0.011 0.26% 4059.776op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 49.015µs 49.227µs ± 0.134µs 49.254µs ± 0.095µs 49.314µs 49.412µs 49.476µs 50.008µs 1.53% 0.839 4.109 0.27% 0.009µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19996944.653op/s 20314045.383op/s ± 55162.072op/s 20302998.656op/s ± 38888.974op/s 20365581.127op/s 20391169.015op/s 20400678.240op/s 20401980.700op/s 0.49% -0.796 3.826 0.27% 3900.548op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [619.110µs; 619.247µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1614867.011op/s; 1615222.844op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [388.299µs; 388.596µs] or [-0.038%; +0.038%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2573388.899op/s; 2575355.721op/s] or [-0.038%; +0.038%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [191.128µs; 191.175µs] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5230826.362op/s; 5232112.482op/s] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [45.350µs; 45.383µs] or [-0.036%; +0.036%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [22034864.746op/s; 22050778.776op/s] or [-0.036%; +0.036%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [49.209µs; 49.246µs] or [-0.038%; +0.038%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20306400.451op/s; 20321690.316op/s] or [-0.038%; +0.038%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 1.613µs 1.613µs ± 0.000µs 1.613µs ± 0.000µs 1.614µs 1.614µs 1.614µs 1.615µs 0.09% 0.699 0.198 0.02% 0.000µs 1 200
credit_card/is_card_number/ throughput 619317737.278op/s 619845581.811op/s ± 151397.412op/s 619869487.370op/s ± 107776.471op/s 619963608.402op/s 620041797.700op/s 620090775.447op/s 620094668.987op/s 0.04% -0.698 0.194 0.02% 10705.414op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 103.837µs 105.230µs ± 0.783µs 104.998µs ± 0.203µs 105.255µs 107.181µs 107.441µs 107.957µs 2.82% 1.669 2.072 0.74% 0.055µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 9262958.904op/s 9503549.094op/s ± 69860.659op/s 9524032.512op/s ± 18454.459op/s 9539882.135op/s 9573700.145op/s 9594363.650op/s 9630476.606op/s 1.12% -1.641 1.998 0.73% 4939.895op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 96.123µs 96.831µs ± 0.484µs 96.751µs ± 0.258µs 97.029µs 97.469µs 98.589µs 99.762µs 3.11% 2.255 8.733 0.50% 0.034µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 10023867.407op/s 10327501.139op/s ± 51097.291op/s 10335766.313op/s ± 27587.755op/s 10361755.029op/s 10383657.577op/s 10398779.738op/s 10403338.431op/s 0.65% -2.173 8.177 0.49% 3613.124op/s 1 200
credit_card/is_card_number/37828224631 execution_time 1.613µs 1.615µs ± 0.004µs 1.614µs ± 0.001µs 1.616µs 1.618µs 1.621µs 1.658µs 2.71% 9.183 107.376 0.22% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 603228381.720op/s 619261475.208op/s ± 1328155.835op/s 619591542.822op/s ± 339903.528op/s 619860961.468op/s 619990336.617op/s 620080176.449op/s 620114012.505op/s 0.08% -9.020 104.625 0.21% 93914.800op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 94.506µs 95.665µs ± 0.607µs 95.633µs ± 0.406µs 95.999µs 96.684µs 97.284µs 97.753µs 2.22% 0.569 0.374 0.63% 0.043µs 1 200
credit_card/is_card_number/378282246310005 throughput 10229817.228op/s 10453522.160op/s ± 66064.739op/s 10456633.726op/s ± 44276.653op/s 10503285.432op/s 10553225.773op/s 10574915.635op/s 10581335.552op/s 1.19% -0.531 0.298 0.63% 4671.483op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 94.103µs 94.501µs ± 0.380µs 94.457µs ± 0.252µs 94.705µs 94.989µs 95.497µs 97.817µs 3.56% 3.628 27.415 0.40% 0.027µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 10223222.126op/s 10582119.193op/s ± 41976.784op/s 10586856.965op/s ± 28365.264op/s 10615193.527op/s 10624786.831op/s 10626269.509op/s 10626700.648op/s 0.38% -3.439 25.212 0.40% 2968.207op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 22.463µs 22.819µs ± 0.172µs 22.805µs ± 0.104µs 22.902µs 23.170µs 23.273µs 23.409µs 2.65% 0.654 0.451 0.75% 0.012µs 1 200
credit_card/is_card_number/x371413321323331 throughput 42718412.514op/s 43824852.914op/s ± 327977.932op/s 43850873.705op/s ± 199360.847op/s 44057942.734op/s 44287789.413op/s 44428520.131op/s 44518605.982op/s 1.52% -0.609 0.372 0.75% 23191.542op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 1.613µs 1.614µs ± 0.001µs 1.614µs ± 0.000µs 1.614µs 1.615µs 1.615µs 1.616µs 0.16% 0.938 1.127 0.04% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 618791555.402op/s 619729399.600op/s ± 237064.983op/s 619758755.058op/s ± 158182.557op/s 619898827.388op/s 620033439.491op/s 620067197.497op/s 620145296.901op/s 0.06% -0.935 1.119 0.04% 16763.026op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 85.812µs 86.663µs ± 0.301µs 86.656µs ± 0.213µs 86.851µs 87.124µs 87.485µs 87.901µs 1.44% 0.408 0.953 0.35% 0.021µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 11376408.076op/s 11539069.109op/s ± 40024.746op/s 11539861.743op/s ± 28324.453op/s 11569635.205op/s 11596904.556op/s 11615519.927op/s 11653317.423op/s 0.98% -0.379 0.888 0.35% 2830.177op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 79.113µs 80.178µs ± 0.496µs 80.155µs ± 0.307µs 80.454µs 80.986µs 81.250µs 82.495µs 2.92% 0.680 1.784 0.62% 0.035µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 12121964.418op/s 12472777.351op/s ± 76865.986op/s 12475778.901op/s ± 47666.787op/s 12525749.312op/s 12585395.757op/s 12630354.436op/s 12640086.284op/s 1.32% -0.620 1.570 0.61% 5435.246op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 1.613µs 1.613µs ± 0.000µs 1.613µs ± 0.000µs 1.614µs 1.614µs 1.614µs 1.615µs 0.09% 0.406 -0.503 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 619276971.900op/s 619820741.683op/s ± 171840.795op/s 619837820.939op/s ± 129885.260op/s 619962740.210op/s 620060183.661op/s 620129689.723op/s 620142844.005op/s 0.05% -0.405 -0.505 0.03% 12150.979op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 77.332µs 78.402µs ± 0.470µs 78.374µs ± 0.285µs 78.660µs 79.278µs 79.723µs 80.117µs 2.22% 0.605 1.066 0.60% 0.033µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 12481697.979op/s 12755279.686op/s ± 76181.108op/s 12759264.832op/s ± 46449.949op/s 12805344.320op/s 12870742.946op/s 12915397.897op/s 12931206.487op/s 1.35% -0.557 0.976 0.60% 5386.818op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 94.086µs 94.673µs ± 0.359µs 94.638µs ± 0.235µs 94.879µs 95.337µs 95.532µs 96.172µs 1.62% 0.711 0.711 0.38% 0.025µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 10398051.131op/s 10562832.779op/s ± 39905.907op/s 10566577.915op/s ± 26316.372op/s 10592284.324op/s 10623362.367op/s 10626779.559op/s 10628600.501op/s 0.59% -0.687 0.633 0.38% 2821.774op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 22.434µs 22.799µs ± 0.227µs 22.765µs ± 0.146µs 22.933µs 23.268µs 23.334µs 23.493µs 3.20% 0.635 -0.095 0.99% 0.016µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 42566444.377op/s 43865186.741op/s ± 434042.970op/s 43927107.422op/s ± 281538.627op/s 44193209.975op/s 44465349.459op/s 44574602.600op/s 44575474.452op/s 1.48% -0.590 -0.168 0.99% 30691.473op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [1.613µs; 1.613µs] or [-0.003%; +0.003%] None None None
credit_card/is_card_number/ throughput [619824599.586op/s; 619866564.036op/s] or [-0.003%; +0.003%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [105.121µs; 105.338µs] or [-0.103%; +0.103%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [9493867.078op/s; 9513231.109op/s] or [-0.102%; +0.102%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [96.764µs; 96.898µs] or [-0.069%; +0.069%] None None None
credit_card/is_card_number/ 378282246310005 throughput [10320419.546op/s; 10334582.732op/s] or [-0.069%; +0.069%] None None None
credit_card/is_card_number/37828224631 execution_time [1.614µs; 1.615µs] or [-0.030%; +0.030%] None None None
credit_card/is_card_number/37828224631 throughput [619077405.582op/s; 619445544.833op/s] or [-0.030%; +0.030%] None None None
credit_card/is_card_number/378282246310005 execution_time [95.581µs; 95.749µs] or [-0.088%; +0.088%] None None None
credit_card/is_card_number/378282246310005 throughput [10444366.223op/s; 10462678.098op/s] or [-0.088%; +0.088%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [94.448µs; 94.553µs] or [-0.056%; +0.056%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [10576301.614op/s; 10587936.771op/s] or [-0.055%; +0.055%] None None None
credit_card/is_card_number/x371413321323331 execution_time [22.796µs; 22.843µs] or [-0.104%; +0.104%] None None None
credit_card/is_card_number/x371413321323331 throughput [43779398.327op/s; 43870307.501op/s] or [-0.104%; +0.104%] None None None
credit_card/is_card_number_no_luhn/ execution_time [1.614µs; 1.614µs] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/ throughput [619696544.674op/s; 619762254.527op/s] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [86.621µs; 86.705µs] or [-0.048%; +0.048%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [11533522.064op/s; 11544616.153op/s] or [-0.048%; +0.048%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [80.109µs; 80.246µs] or [-0.086%; +0.086%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [12462124.465op/s; 12483430.237op/s] or [-0.085%; +0.085%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [1.613µs; 1.613µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [619796926.202op/s; 619844557.165op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [78.337µs; 78.467µs] or [-0.083%; +0.083%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [12744721.717op/s; 12765837.655op/s] or [-0.083%; +0.083%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [94.623µs; 94.723µs] or [-0.052%; +0.052%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [10557302.204op/s; 10568363.354op/s] or [-0.052%; +0.052%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [22.768µs; 22.831µs] or [-0.138%; +0.138%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [43805032.560op/s; 43925340.922op/s] or [-0.137%; +0.137%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.992µs 26.704µs ± 16.469µs 20.395µs ± 0.300µs 20.650µs 58.929µs 66.191µs 153.821µs 654.21% 3.982 22.228 61.52% 1.165µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.422µs; 28.987µs] or [-8.547%; +8.547%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 310.133µs 313.765µs ± 0.972µs 313.853µs ± 0.624µs 314.456µs 315.204µs 315.719µs 315.875µs 0.64% -0.531 0.554 0.31% 0.069µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3165812.344op/s 3187133.694op/s ± 9892.751op/s 3186206.184op/s ± 6320.089op/s 3193918.585op/s 3203151.271op/s 3213152.911op/s 3224421.071op/s 1.20% 0.552 0.600 0.31% 699.523op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 27.967µs 28.047µs ± 0.062µs 28.030µs ± 0.030µs 28.081µs 28.138µs 28.181µs 28.507µs 1.70% 2.494 13.971 0.22% 0.004µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 35078570.939op/s 35655169.717op/s ± 78561.773op/s 35676575.380op/s ± 38292.909op/s 35706720.108op/s 35745486.296op/s 35753141.226op/s 35756218.230op/s 0.22% -2.430 13.351 0.22% 5555.156op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 16.723µs 16.763µs ± 0.034µs 16.756µs ± 0.018µs 16.774µs 16.835µs 16.889µs 16.896µs 0.83% 1.907 4.097 0.20% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 59184923.293op/s 59655745.960op/s ± 119219.704op/s 59678469.601op/s ± 62960.081op/s 59741163.814op/s 59764377.423op/s 59783018.548op/s 59796246.823op/s 0.20% -1.892 4.032 0.20% 8430.106op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [313.630µs; 313.899µs] or [-0.043%; +0.043%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3185762.653op/s; 3188504.734op/s] or [-0.043%; +0.043%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [28.038µs; 28.055µs] or [-0.031%; +0.031%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [35644281.811op/s; 35666057.623op/s] or [-0.031%; +0.031%] None None None
normalization/normalize_name/normalize_name/good execution_time [16.758µs; 16.768µs] or [-0.028%; +0.028%] None None None
normalization/normalize_name/normalize_name/good throughput [59639223.256op/s; 59672268.664op/s] or [-0.028%; +0.028%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz abdd41e 1723837895 rochdev/crashtracker-config-serde
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.154µs 1.350µs ± 0.061µs 1.357µs ± 0.049µs 1.398µs 1.436µs 1.444µs 1.444µs 6.45% -0.631 0.479 4.47% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.341µs; 1.358µs] or [-0.622%; +0.622%] None None None

Baseline

Omitted due to size.

pr-commenter[bot] avatar Aug 15 '24 21:08 pr-commenter[bot]

Codecov Report

Attention: Patch coverage is 0% with 18 lines in your changes missing coverage. Please review.

Project coverage is 71.84%. Comparing base (31b6854) to head (abdd41e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #588      +/-   ##
==========================================
+ Coverage   71.77%   71.84%   +0.07%     
==========================================
  Files         237      237              
  Lines       32871    32873       +2     
==========================================
+ Hits        23594    23619      +25     
+ Misses       9277     9254      -23     
Components Coverage Δ
crashtracker 21.25% <0.00%> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 50.00% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 82.03% <0.00%> (-0.08%) :arrow_down:
ddcommon-ffi 68.11% <ø> (ø)
ddtelemetry 59.02% <ø> (ø)
ipc 84.29% <ø> (ø)
profiling 84.26% <ø> (+0.69%) :arrow_up:
profiling-ffi 77.42% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 40.47% <ø> (ø)
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 54.87% <ø> (ø)
trace-mini-agent 70.88% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 92.97% <ø> (ø)

codecov-commenter avatar Aug 15 '24 22:08 codecov-commenter

Most of the testing would be through unit tests. Though there are some end to end FFI tests that could be leveraged for crashtracking.

r1viollet avatar Aug 20 '24 15:08 r1viollet

Most of the testing would be through unit tests. Though there are some end to end FFI tests that could be leveraged for crashtracking.

It doesn't look like there are existing tests for crashtracking to begin with. And FFI tests are something else completely as Node cannot use FFI.

rochdev avatar Aug 20 '24 16:08 rochdev

It doesn't look like there are existing tests for crashtracking to begin with. And FFI tests are something else completely as Node cannot use FFI.

If adding a rust unit test is difficult, ping me, I can approve to unblock. I agree the ffi tests won't help. If you are able to add a test just to cover the new parts, that is great.

r1viollet avatar Aug 21 '24 08:08 r1viollet

If adding a rust unit test is difficult, ping me, I can approve to unblock. I agree the ffi tests won't help.

Is there an example anywhere in the project where serialization/deserialization is tested that I can look at?

rochdev avatar Aug 27 '24 02:08 rochdev

If adding a rust unit test is difficult, ping me, I can approve to unblock. I agree the ffi tests won't help.

Is there an example anywhere in the project where serialization/deserialization is tested that I can look at?

Yes, I think this is an example https://github.com/DataDog/libdatadog/blob/bc99dfcd68addb8c7676c927c605382ed29a432a/profiling/src/internal/profile/mod.rs#L945

r1viollet avatar Aug 27 '24 12:08 r1viollet

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. To override this behavior, add the keep-open label or update the PR.

github-actions[bot] avatar Nov 27 '24 00:11 github-actions[bot]